Development and Preliminary Evaluation of a Technology for Assessing Hedonic Aspects of UX in Text-Based Chatbots

Pamella A. de L. Mariano, Ana Paula Chaves, Natasha Valentim

2025

Abstract

For text-based chatbots to achieve a desired level of quality, it is essential to evaluate their performance, particularly focusing on the hedonic aspects of User Experience (UX), which is a crucial quality attribute. A comprehensive evaluation must consider the specifics of the context being assessed. A Systematic Mapping Study (SMS) revealed that no existing UX evaluation technologies address the hedonic aspects of UX in text-based chatbots. The Guidelines to Assess Hedonic Aspects in Chatbots (GAHAC) was developed to address this gap. The guidelines were formulated by selecting and evaluating the hedonic aspects of UX and the evaluation technologies identified in the SMS. Relevant questions from these technologies were filtered and adapted to the context of text-based chatbots. GAHAC aims to provide a context-specific evaluation technology in the form of guidelines encompassing the hedonic aspects of UX. Its primary contribution is providing a structured and accessible method for evaluating hedonic aspects, which have been largely overlooked in UX studies of text-based chatbots. This enables developers and researchers to qualitatively identify opportunities to improve UX in chatbot interactions. A preliminary evaluation conducted with two Human-Computer Interaction experts led to refinements in the guidelines. By offering a dedicated UX evaluation technology for text-based chatbots, GAHAC contributes to improving the quality of such systems.

Paper Citation

in Harvard Style

Mariano P., Chaves A. and Valentim N. (2025). Development and Preliminary Evaluation of a Technology for Assessing Hedonic Aspects of UX in Text-Based Chatbots. In Proceedings of the 27th International Conference on Enterprise Information Systems - Volume 2: ICEIS; ISBN 978-989-758-749-8, SciTePress, pages 544-551. DOI: 10.5220/0013289200003929

in Bibtex Style

@conference{iceis25,
author={Pamella Mariano and Ana Chaves and Natasha Valentim},
title={Development and Preliminary Evaluation of a Technology for Assessing Hedonic Aspects of UX in Text-Based Chatbots},
booktitle={Proceedings of the 27th International Conference on Enterprise Information Systems - Volume 2: ICEIS},
year={2025},
pages={544-551},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0013289200003929},
isbn={978-989-758-749-8},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 27th International Conference on Enterprise Information Systems - Volume 2: ICEIS
TI - Development and Preliminary Evaluation of a Technology for Assessing Hedonic Aspects of UX in Text-Based Chatbots
SN - 978-989-758-749-8
AU - Mariano P.
AU - Chaves A.
AU - Valentim N.
PY - 2025
SP - 544
EP - 551
DO - 10.5220/0013289200003929
PB - SciTePress