Hedonic and Pragmatic Qualities of AAL Pilot Region Technologies:

Evaluation with the AttrakDiff Survey

Daniela Elisabeth Ströckl

, Elena Oberrauner

, Daniela Krainer

, Lukas Wohofsky

and Johannes Oberzaucher

Institute for Applied Research on Ageing, Carinthia University of Applied Sciences, Villach, Austria

Research Unit Active & Assisted Living, Carinthia University of Applied Sciences, Klagenfurt, Austria

Keywords: User Experience, Acceptance, AAL, Active and Assisted Living, Attractiveness, Pilot Region.

Abstract: Uniform structuring, clear presentation, simple navigation - terms that are associated with good usability these

days. However, purely pragmatic quality features are often not sufficient to gain an insight into how the

product, technologies or software are assessed by the user. In order to be able to evaluate the user experience

with the parameter “Joy of Use”, hedonic quality features should be taken into account. Pragmatic and hedonic

characteristics result in the degree of attractiveness, which plays an essential role in the presented work.

Within the Carinthian pilot region Smart VitAALity, the attractiveness of the provided AAL technologies was

evaluated with a subsample of ten participants using the AttrakDiff questionnaire as well as interviews. By

combining the two methodologies, the system could be assessed without much effort and first impressions of

the users could be collected. As the results show, the overall experience was good and positive. However, in

general, the interviews showed that there is still room for the improvement of the system.

1 INTRODUCTION

The evaluation of products or software is becoming

more and more important in user-centered design

driven implementation. Products should be easy to

use in order to be able to utilize them in an efficiently,

effectively and satisfactorily way for a certain context

on the one hand, and to increase technology

acceptance and usage behavior on the other. A

frequently associated term with this characteristic is

usability. Thus, it still plays a crucial role in

technology development for senior citizens. Attention

must be paid to any restrictions associated with age,

since special needs and criteria influence the use of

technologies. Older people are sometimes rather

reserved when it comes to the use of new technologies

(Raymundo et al., 2014). This is why it is especially

important to stress which benefits they can gain by

using new technologies and services. However, in

order to make a product appear particularly attractive,

not only the goal and task-oriented orientation but

also the joy and enjoyment ("Joy of Use") must be

Project Smart VitAALity - https://www.smart-vitaality.at/

(04.02.2020)

taken into account when using it, since perceived

enjoyment is associated with usage behavior and its

extent (Igbaria et al., 1994).

Active and Assisted Living (AAL) technologies

and services are developed and designed especially

for older people and aim to be adapted to the needs of

the future users with the aim to support daily activities

at home in the best possible way for a longer

independent life. In some pilot regions of Austria,

AAL systems have been developed and subsequently

evaluated in terms of technology acceptance,

usability and user behavior. In the Carinthian pilot

region Smart VitAALity

, the evaluation domains

should be expanded to consider also the user

experience in order to measure the attractiveness of

AAL technologies and services in pilot regions for the

first time.

The evaluation was based on a one-year test phase

in which 102 participants (between 60 and 85 years)

in the intervention group used the Smart VitAALity

system, consisting of a bundle of technologies and

services with the main focus on health management

and support of social participation.

Ströckl, D., Oberrauner, E., Krainer, D., Wohofsky, L. and Oberzaucher, J.

Hedonic and Pragmatic Qualities of AAL Pilot Region Technologies: Evaluation with the AttrakDiff Survey.

DOI: 10.5220/0009575701870193

In Proceedings of the 6th International Conference on Information and Communication Technologies for Ageing Well and e-Health (ICT4AWE 2020), pages 187-193

ISBN: 978-989-758-420-6

187

2 METHODS

In this chapter the used AttrakDiff survey is explained

with the different measured components.

Furthermore, the evaluation schedule and participant

selection are displayed.

2.1 Qualities

The underlying two-component model of this

questionnaire distinguishes between two types of

subjective quality perception: pragmatic (PQ) and

hedonic (HQ). Human needs for safety, control and

trust, which refer to the perceived usefulness of users

of interactive products, are described by the

pragmatic quality perception. These quality

characteristics help to evaluate products or software

e.g. as clear, supporting or controllable. HQ, on the

other hand, is not a goal-oriented or task-related

quality characteristic, but rather emphasizes the

human need for curiosity, social comparison and

desired identity, e.g. by making the software appear

professional, striking, modern, exciting or simply

different. However, this quality feature addresses our

needs much more directly, because we love beautiful

things or want to learn something new (Hassenzahl,

Burmester, Koller, 2008). (Burmester et al., 2002)

Both qualities are independent of each other and

users can assess them separately (Hassenzahl,

Burmester, Koller, 2008). I.e. products with a high

pragmatic quality do not necessarily have to have a

high hedonic quality. However, it would be desirable

if both quality features were equally pronounced in a

product or software (Hassenzahl, et al., 2003).

The attractiveness as an overall assessment can be

formed from both quality characteristics. An

attractiveness judgement corresponds in the true

sense of the word to a global subjective emotional

categorization between good or bad. For a user, the

opinion on the attractiveness of a product or software

is made within seconds, a decision from the heart

without thinking about the word as such, and depends

on the situation in which the particular software is

used. It is assumed that pragmatic or hedonic

perceptions remain relatively stable in different

situations, although the global evaluation can still

change and is more likely to be formed situational.

(Hassenzahl, et al., 2003)

The model itself separates three essential aspects,

as shown in figure 1.

Objective Product Quality: When creating a

product or software, the designer aims at a certain

AttrakDiff Sience - www.attrakdiff.de (04.02.2020)

product quality, formed by pragmatic and hedonic

features. For example, the decision of designing a

clear layout (PQ) to make the product look

professional (HQ).

Subjective Quality Perception and Evaluation:

The quality is perceived by the user and evaluated on

this basis. Perceived pragmatic and hedonic quality is

summarized within the evaluation as attractiveness.

Behavioral and Emotional Consequences:

The evaluation ultimately leads to two possible

consequences. On the one hand, the evaluated

attractiveness influences the behavior or the handling

of the software, such as avoidance or increased use.

On the other hand, the emotional consequence is

expressed by emotions such as joy, satisfaction or

anger.

Figure 1: Basic model

of the AttrakDiff questionnaire -

Interaction of hedonic and pragmatic quality.

2.2 AttrakDiff

In order to assess the attractiveness of Smart

VitAALity technologies and services, the AttrakDiff2

questionnaire (Hassenzahl, et al., 2003), which is

based on the two-component model of pragmatic and

hedonic subjective quality perception, was used as a

measuring instrument. For the assessment of the HQ,

the questionnaire focused on the two dimensions

stimulation and identity. The questionnaire belongs to

the class of semantic differentials and is used to

record attractiveness as an overall assessment in a

standardized way. For this purpose, 28 bipolar items

with seven gradations are mapped. The total number

of items is again summarized in four scales, each with

seven items and therefore the respective mean is

calculated. A distinction is made between PQ,

ICT4AWE 2020 - 6th International Conference on Information and Communication Technologies for Ageing Well and e-Health

188

hedonic quality stimulation (HQ-S), hedonic quality

identity (HQ-I) and attractiveness (ATT).

PQ: When using the product or software, action

objectives should be provided by useful and usable

functions.

HQ-S: This category focuses on the need to improve

one's own knowledge and skills as well as natural

curiosity.

HQ-I: This includes the communication of relevant

other self-serving messages.

ATT: Is the global positive or negative evaluation of

a product or software, such as good or bad.

The questionnaire was empirically tested in a pilot

study and can be used online free of charge since 2002.

For the survey of the attractiveness of Smart

VitAALity, the selected participants filled out the

questionnaire on paper. For the evaluation, the

AttrakDiff

Online Tool was used.

2.3 Evaluation of UX and Integration

in Daily Routine

The Smart VitAALity System was evaluated in the

four domains subjective quality of life, socio-

economical potential analysis, usage frequency as

well as acceptance with user experience as one

essential influencing factor, and effects on integration

in daily routine (Oberzaucher, et al., 2020) Within

acceptance studies, additional to a survey of the

whole intervention group, the UX questionnaire

AttrakDiff as well as interviews have been conducted

(Krainer, et al., 2020).

To gain a first idea about the user experience,

usability and usage opinions, persons have been

interviewed with a pre-defined interview guide and

the AttrakDiff survey.

The evaluation of the interviews was based on the

method of content structuring (qualitative content

analysis) according to Mayring (Mayring, 2010).

2.3.1 User Selection

To get a sample out of the Smart VitAALity

participants it was necessary to have a look at the

categories the people are in. For the project the main

categories sex, age, geographical location, Care

Center membership and the usage behavior were

relevant. Each of them got an internal ranking, how

important it is for the selection of the ten participants

of the UX analysis and interviews. Most important

were the Care Center membership and the usage

behavior; the Care Center membership, because some

AttrakDiff Online Tool - www.attrakdiff.de (04.02.2020)

questions in the interview guide were directly related

to that function and the usage behavior, because it

should reflect the variety of users as in the project at

that time. As shown in table 1 there were some

differences regarding the distribution of the main

categories to the total participant group of Smart

VitAALity. An example is the main category age, it

was not possible to get people out of the age group

55-64 years for the interview. This fact was tolerated,

because the category age was previously weighted as

secondary. This means for the user selection it is less

important, because it is not directly related to the

usage behavior – as pre-evaluation showed.

Table 1: Distribution of the interview participants

compared to the total Smart VitAALity distribution.

Main

Category

Interview

Total

total

(100%)

102

(100%)

sex

(secondary)

6 (60%)

68 (66.6%)

4 (40%)

34 (33.3%)

age

(secondary)

55-64

0 (0%)

30 (29.4%)

65-74

6 (60%)

46 (45.1%)

75-80

4 (40%)

26 (25.5%)

geographical

location

(secondary)

Klagenfurt

3 (30%)

48 (47%)

Villach

4 (40%)

32 (31.4%)

Ferlach

3 (30%)

22 (21.6%)

Care Center

membership

(primary)

yes

6 (60%)

69 (67.6%)

4 (40%)

33 (32.4%)

usage

behavior

(primary)

High

usage

5 (50%)

30 (29.4%)

Moderate

usage

3 (30%)

54 (52.9%)

Low usage

2 (20%)

18 (17.6%)

2.3.2 UX Analysis and Interview Schedule

Before starting with the questionnaire and the

interview, the participants had to sign an informed

consent to agree with the processing of the gathered

Hedonic and Pragmatic Qualities of AAL Pilot Region Technologies: Evaluation with the AttrakDiff Survey

189

data and, furthermore, that they allow to audio record

the interview. Then the one-hour session started with

the AttrakDiff survey. The participants were

instructed to choose between the word-pairs what

they think will fit the best for them. They should not

think about it too long. On average, they needed 107.8

seconds to fulfill the survey on their own. Afterwards

the recording started and the interview part began.

The interview guide contains questions to describe

daily situations with Smart VitAALity, usage,

handling and some ranking of particular services e.g.

the Care Center. In total, the interview session lasted

about one hour.

3 RESULTS

In the following subchapters the results of the

AttrakDiff evaluation of the pilot region Smart

VitAALity is shown. The evaluation was done in

November 2018, five months after the participants

got their technologies.

3.1 Portfolio of Results

The results of the AttrakDiff survey of Smart

VitAALity is first shown in figure 2 in the graphical

representation of the confidence rectangle (light blue

rectangle) and the medium value (tiny dark blue

rectangle). The Portfolio view displays on the vertical

axis the hedonic quality and on the horizontal axis the

pragmatic quality. Depending on the answers /

dimension values the blue rectangle, that represents

the rating of the product / technologies, lies in one or

more character regions – as in figure 2, it lies over

four regions, self-oriented, desired, neutral and task-

oriented. If the confidence rectangle is small the

investigation results are more valid and less

coincidental. Bigger rectangles are less significant

and it is not possible to assign them to a certain region

(e.g. self-oriented).

Furthermore, if the confidence rectangle is bigger

it means, that the evaluation ratings of each

participant are very different and there is no

unambiguous opinion. (Hassenzahl, et al., 2003). As

shown in figure 2, the result shows a relatively small

confidence rectangle with a little more spreading at

the PQ, which indicates that the participants were

more divided in PQ. It is located in kind a natural

position with a slightly tendency to self-oriented but

has no extrema into “too self-oriented” or “too task-

Figure 2: AttrakDiff evaluation result: Portfolio-

presentation.

oriented”, which is positive. Overall, the participants

think, it is more self-oriented than task-oriented.

Assistive technology should support during daily

living and not burden the users with further tasks that

could be hard to do. Furthermore, the confidence

rectangle shows that the feeling is slightly more

neutral than the participants really desired such

technology. Often people think that elderly people are

afraid of state of the art assistive technologies

(Raymundo et al., 2014) but this result reflects that

they are in a neutral rather positive (desired) mood.

According to the shape of the confidence rectangle it

can be said, that the Smart VitAALity system has

more hedonic than pragmatic quality.

3.2 Diagram of Average Values

The diagram of average values represented in figure

3 shows that the Smart VitAALity technologies and

services are attractive for the users. (Hassenzahl, et al.,

2003) Furthermore, the hedonic quality is

differentiated into the two parts identity (HQ-I) and

stimulation (HQ-S). Both factors are nearly on the

same value and are relatively high (HQ-I = 1, HQ-S

= 1.1). On the one hand, that means that the

technologies and services help the user empower

themselves in their own development because it

provides e.g. interesting functionalities. On the other

hand, the HQ-I value shows that the users can identify

themselves with Smart VitAALity, so they do not feel

any shame of using such technologies and services.

ICT4AWE 2020 - 6th International Conference on Information and Communication Technologies for Ageing Well and e-Health

190

Figure 3: AttrakDiff evaluation result: Diagram of average

values.

As mentioned in the description of the portfolio-

presentation (figure 2) the PQ is slightly lower (0.7)

than the HQ (1.07) but it is still in the positive sector

as shown in figure 3. Furthermore, it depends on the

fact, that Smart VitAALity is a prototype (market

readiness level 6-7), which is tested in a field study.

Efficiency and effectiveness should be improved

when the products are on the market, in the first run,

the aim of the project was that the technologies and

services are running stable. All three qualities

together lead to the attractiveness level of 1.41, which

represents the statement, that Smart VitAALity is

attractive for the users with some room for

improvement.

3.3 Description of Word-pairs

In figure 4, the mean values of the word-pairs are

presented. This shows a deeper level of the evaluation,

which means, that here not just the upper categories

PQ, HG-I, HQ-S and ATT are represented, but also

the word-pair level. Extreme values show

whichcharacteristics are particularly critical or

particularly well-resolved (Hassenzahl et al., 2003).

Furthermore, the standard deviation is shown (light

blue bar). This additional value provides the

information on the scattering of the answers.

The results show that just one parameter is in front

of the neutral line (0) and is thus some kind of

negative - the word-pair “technical – human” (-0.2 in

the ranking) is not an extreme value but the only

negative evaluated word-pair. For the project Smart

VitAALity it was an expected value for two reasons:

the participants are not used to such assistive systems

and, therefore, they categorize it as more technical.

Secondly, Smart VitAALity was introduced as

technical system to support the daily living of the

participants. The word-pair “unprofessional –

professional” (0.2 in the ranking) indicates that Smart

VitAALity is a prototype and not a finished product.

Figure 4: AttrakDiff evaluation result: Description of word-

pairs.

Therefore, of course it seems to be not as professional

as the users expected it to be.

Especially the result for the word-pair

“unpredictable – predictable” (0.4 in the ranking) has

to be considered as noticeable. Assistive systems for

seniors have the aim to work in a way that no

surprises come up for the users. That means they

should know after some time of usage what to expect

from the system and how it works. On the one hand,

maybe the users connected this word-pair with the

issue, that especially in the beginning of the test phase

some problems and errors occur. On the other hand,

it cannot be ruled out, that there are unpredictable

features in the software that should be avoided.

The word-pair “bad-good” was mainly chosen in

a very positive way (1.8 in the ranking) and is after

the word-pairs “cheap-premium”, “unimaginative-

creative”, “ordinary-novel”, all three with 1.6 in the

ranking and “conservative-innovative” with 1.5 in the

ranking the most positive pair. All of them represent

that Smart VitAALity is a new, innovative and overall

good selection of technologies, functions and services.

Hedonic and Pragmatic Qualities of AAL Pilot Region Technologies: Evaluation with the AttrakDiff Survey

191

3.4 Results Interviews

In the qualitative interviews, participants reported

about positive feelings caused by the Smart

VitAALity system and integration in the daily life,

that underlines the hedonistic quality.

P08: “I often thought about it and it is actually

very pleasant. You also prepared it very well with the

tablet and the whole thing”

P03: “I am happy that I can participate as a test

person, probably the oldest one in the study”

P01: “The first thing in the morning is I measure

my weight and blood pressure. That’s always the

same before I get dressed.”

Participants reflected the pragmatic quality in the

interviews in both ways – positive and negative.

P03: “The most important for me is to check on

my health data”

P06: “I would like to use it but it seems so

inconvenient”

Not all of the participants agreed with the

premium look of the Smart VitAALity system like a

quote about the smartwatch shows.

P07: „It is not much to look at. It’s pretty chunky”

The issue with the unpredictability of the system

reflects in following statement of an interview

participant.

P01: „I have to say, I often walk around with my

watch for half a day before I notice that it doesn’t

count my steps at all”

One participant gives insights into her preference

to rather have a human person for support than a

technical product.

P06: “Of course I still prefer a human being

because I can simply communicate better this way”

The quotations of interview participants are

originally in German and were translated for this

paper. The original quotes can be found in (Krainer,

et al., 2020).

4 CONCLUSIONS

The AttrakDiff survey as well as the qualitative

interviews were realized to get an atmospheric picture

from the Smart VitAALity participants about their

experience with the system.

As the results displayed, the overall experience is

positive, both the pragmatic and hedonic quality are

ranked positively. This indicates that the participants

of Smart VitAALity could reach their goals with the

system in an efficient and effective way as well as

they felt a joy in using the technologies and services.

The Smart VitAALity system was tested in the

project on a market readiness level of 6-7, which

means, that is a prototype. Therefore, of course,

during the testing phase problems, errors and

misconduct of the system occurs. This can lead to the

fact, that the results of the AttrakDiff and the quotes

of the participants show a clear tendency for room of

improvement. The system should have a more

professional look and feel if it will be developed to a

market ready product. As a matter of course, it is

important to improve the system according to the

predictability. As one of user experience principles

foreseeability is characteristic for good usability.

Users should always know where they are, how they

get there, what they can do and how to go back in a

system. The evaluation result of 0.4 was just slightly

positive and, therefore, it should get more attention to

optimize the overall user experience.

Collectively said, Smart VitAALity was ranked as

good which is reflected in the results; on the one hand

shown in the word-pair ranking good-bad and on the

other hand on the category results of PQ and HQ. It

may be sometimes to technical for the participants,

because of the non-experience in using technical

support systems or other reasons, but the combination

of human-driven and technology-driven aspects lead

to an accepted and joyful system that help people

doing their tasks efficiently.

ACKNOWLEDGEMENTS

The pilot region Smart VitAALity (grant no. 858380)

is supported in the framework of the FFG program

benefit and co-financed by bmvit.

REFERENCES

Raymundo T., Santana C. C., 2014. Fear and the use of

technological devices by older people. In Official

Journal of the International Society for

Gerontechnology, 13(2), 260-260.

Igbaria, M., Schiffman, S. J., Wieckowski, T. J., 1994. The

respective roles of perceived usefulness and perceived

fun in the acceptance of microcomputer technology.

Behaviour & Information Technology, 13(6), 349-361.

Hassenzahl M., Burmester M., Koller F., 2008. Der User

Experience (UX) auf der Spur: Zum Einsatz von

www.attrakdiff.de. In Brau, H., et al., (Hrsg): Usability

Professionals 2008, Stuttgart: German Chapter der

Usability Professionals Association.

Burmester M., Hassenzahl M., Koller F., 2002. Usability ist

nicht alles – Wege zu attraktiven Produkten*. In i-com-

ICT4AWE 2020 - 6th International Conference on Information and Communication Technologies for Ageing Well and e-Health

192

Zeitschrift für interaktive und kooperative Medien, 1,

S.32-40.

Hassenzahl M., Burmester M., Koller F., 2003. AttrakDiff:

Ein Fragebogen zur Messung wahrgenommener

hedonischer und pragmatischer Qualität. In J. Ziegler &

G. Szwillus (Eds) Mensch & Computer 2003.

Interaktion in Bewegung, Stuttgart, Leipzig: B.G.

Teubner, pp. 187-196.

Mayring P., 2010. Qualitative Inhaltsanalyse. In Mey G.,

Mruck K. (eds) Handbuch Qualitative Forschung in der

Psychologie. VS Verlag für Sozialwissenschaften, pp

601-613.

Oberzaucher J., Kada O., Krainer D., 2020.

Evaluierungsdesign im Rahmen der Pilotregion Smart

VitAALity. In Smart VitAALity – Einblicke, Ergebnisse

und Befunde aus einer AAL Pilotregion, Hrsg.

Oberzaucher J., et al., BoD – Books on Demand,

Nordstedt, 2020. Work in Progress.

Krainer D., Wohofsky L., Scharf P., Lattacher S.L., 2020.

Akzeptanz und Alltagsintegration von Smart

VitAALity. In Smart VitAALity – Einblicke, Ergebnisse

und Befunde aus einer AAL Pilotregion, Hrsg.

Oberzaucher J., et al., BoD – Books on Demand,

Nordstedt, 2020. Work in Progress.

Hedonic and Pragmatic Qualities of AAL Pilot Region Technologies: Evaluation with the AttrakDiff Survey

193