Inﬂuence of UX Factors on User Behavior: The Critical Incident

Technique

Jessica Kollmorgen

1,2 a

, Yaprak Turhan

2 b

, Mar

ıa Jos

e Escalona

1 c

and J

org Thomaschewski

2 d

University of Sevilla, Seville, Spain

University of Applied Sciences Emden/Leer, Emden, Germany

Keywords:

User Experience, UX, Holistic UX Factors, Usage Frequency, Critical Incident Technique.

Abstract:

Measuring user experience (UX) is essential for strengthening user loyalty and product success. Usage fre-

quency plays a central role, as it both can inﬂuence and be inﬂuenced by UX. Standard measurement methods

like questionnaires can assess UX factors and calculate their impact on usage frequency. Alongside UX factors,

socio-demographic data like gender are also collected in practice, as they can affect usage patterns depending

on the product. However, the question remains whether additional holistic UX factors exist that are not yet

captured in standard UX questionnaires. Understanding these could improve UX and raise long-term usage.

To investigate the possibility of new factors, we applied the Critical Incident Technique (CIT), a method from

psychology, to UX research. Using Netﬂix as an example, we employed CIT in a questionnaire to capture very

positive/negative (“critical”) user experiences and conducted 12 interviews to assess the incidents’ inﬂuence

on usage frequency. The study found that beyond the known UX factors, additional holistic factors such as

Nostalgia and Anticipation were identiﬁed. These newly identiﬁed factors were also shown to impact usage

frequency. Overall, CIT proves to be a promising method for capturing holistic UX factors, providing a foun-

dation for future research into the context of use.

1 INTRODUCTION

User loyalty plays an important role in supporting the

long-term success of products. To this end, it is ad-

visable to record the expectations and needs of users

and fulﬁll them. In practice, user experience (UX)

measurement methods can be used for this purpose,

such as questionnaires like the User Experience Ques-

tionnaire Plus (UEQ+) (Schrepp and Thomaschewski,

2019), which can measure relevant UX factors de-

pending on the individual product and use case such

as Efﬁciency, Quality of Content or Trust. New UX

factors are also constantly being identiﬁed and made

measurable (e.g., for voice user interfaces (Klein

et al., 2020) or social factors (Mortazavi et al., 2023)).

However, it can be assumed that it is not clear

whether, in addition to standard UX factors, there

are other factors that have not yet been recorded and

which can be relevant for a positive UX. According to

https://orcid.org/0000-0003-0649-3750

https://orcid.org/0009-0002-4877-7372

https://orcid.org/0000-0002-6435-1497

https://orcid.org/0000-0001-6364-5808

ISO 9241-210:2019, UX is deﬁned as “a consequence

of brand image, presentation, functionality, system

performance, interactive behavior, and assistive capa-

bilities of a system, product or service. It also results

from the user’s internal and physical state resulting

from prior experiences, attitudes, skills, abilities and

personality; and from the context of use” (ISO9241-

210, 2020). The ISO deﬁnition shows that more UX

factors are possible than we have previously consid-

ered in questionnaires and other UX methods. This is

why we refer to all factors as holistic UX factors in

the following. A more detailed deﬁnition is given in

Section 2.1.

In this article, we use a method from the ﬁeld

of psychology to investigate holistic UX factors: the

Critical Incident Technique (CIT) (Flanagan, 1954).

The CIT is used to record very positive or very nega-

tive (i.e. “critical”) experiences that have had an im-

pact on the perception of the product. The method

has already been applied in the past in customer ex-

perience research to identify factors that have an in-

ﬂuence on purchasing and recommendation behavior

(e.g., (Oh and Oh, 2017), see Section 2.2). The CIT

is therefore transferred to user experience research in

Kollmorgen, J., Turhan, Y., Escalona, M. J. and Thomaschewski, J.

Inﬂuence of UX Factors on User Behavior: The Critical Incident Technique.

DOI: 10.5220/0013669600003985

Paper published under CC license (CC BY-NC-ND 4.0)

In Proceedings of the 21st International Conference on Web Information Systems and Technologies (WEBIST 2025), pages 121-132

ISBN: 978-989-758-772-6; ISSN: 2184-3252

121

this study with the help of a questionnaire and 12 in-

terviews to identify factors that can have an inﬂuence

on user behavior. The well-known and internationally

popular streaming platform Netﬂix is being examined

for this purpose. Accordingly, the following ﬁrst re-

search question is to be answered:

RQ1: Which holistic UX factors are men-

tioned most for the streaming platform Net-

ﬂix?

In addition to UX factors, socio-demographic and

behavioral data (e.g. frequency of use or place of

use) are often recorded in questionnaires to obtain an

holistic overview of the target group. In the past, it

has already been shown that age, gender or culture can

have an inﬂuence on the perception of products (Koll-

morgen et al., 2024; Kollmorgen et al., 2023; Santoso

and Schrepp, 2019). This already shows that there are

other factors apart from the UX factors that can inﬂu-

ence user perception. For example, a high frequency

of use can also inﬂuence the perceived UX, and con-

versely, a good UX can ensure a high frequency of use

(Kollmorgen et al., 2022). A good UX is described by

a high rating of the UX factors in the questionnaires.

However, to the best of our knowledge, there is no es-

tablished UX method that can capture the factors in-

ﬂuencing the frequency of use in a holistic way. This

leads to the following second question:

RQ2: Which holistic UX factors can have

an inﬂuence on the frequency of use, using

Netﬂix as an example?

By answering the two research questions, this

study aims to methodologically advance UX research

by exploring the applicability of the CIT for identi-

fying holistic UX factors that inﬂuence usage behav-

ior. Using Netﬂix as a case example, the study pro-

vides new insights into usage-related factors beyond

those covered by standard UX instruments, thereby

contributing both to the development of UX methods

and to the understanding of user experience in real-

world contexts.

The article structures as follows: in Section 2,

background on the Critical Incident Technique and

their application in related studies is given, followed

by the description of this study including participants

and approach in Section 3. Afterwards, in Section

4, the results of the study are presented and then dis-

cussed in Section 5 to answer both research questions.

Finally, a conclusion is drawn and an outlook given in

Section 6.

2 RELATED WORK

As the ﬁeld of research is constantly evolving and new

UX factors are being identiﬁed (e.g., see (Klein et al.,

2020)), this section will ﬁrst provide an overview of

UX factors and their signiﬁcance for different expe-

riences. This will be differentiated from the holistic

UX factors. The Critical Incident Technique is then

described in its functionality and application in order

to clarify the research gap and its added value for the

identiﬁcation of holistic UX factors.

2.1 UX and Holistic UX Factors

To ensure that users can interact with the products sat-

isfactorily, they should be able to use the products ef-

ﬁciently and with little effort. The intentions of use

are individual and subjective, which raises the ques-

tion of how generalized conclusions can be drawn

(Hinderks et al., 2020; Kollmorgen et al., 2024).

To do this, it is ﬁrst necessary to measure the user

experience. An established method here is standard-

ized questionnaires such as the UEQ+, which makes

it possible to measure various real UX aspects in a

modular way depending on the use case (Schrepp and

Thomaschewski, 2019). The UX factors measured in

the questionnaire can relate on the one hand to classic

usability aspects such as Efﬁciency and Usefulness,

for example. On the other hand, the UEQ+ can also

measure UX factors that go beyond usefulness and are

intended to give the user pleasure, such as Stimulation

or Novelty (Schrepp and Thomaschewski, 2025). An

overview of the 27 UX factors that can currently be

measured with the UEQ+ can be found on the web-

site (Schrepp and Thomaschewski, 2025).

The UEQ+ has been further developed over the

years and did not include all 27 UX factors from the

start. With the further development of technology,

such as the establishment of social media and voice

user interfaces, additional factors have been added

over time (e.g., see (Mortazavi et al., 2023; Klein

et al., 2020)). It has also been shown that not all of

the 27 factors are equally relevant for every product.

Different factors should be measured depending on

the investigated product and product category. While

factors such as Stimulation and Novelty are important

for social networks and games, factors such as Trust

and Dependability are important for booking systems

and online banking (Kollmorgen et al., 2024). Fur-

thermore, other experiences also have factors that are

important for consideration, such as Image or Authen-

ticity for the brand experience (Alﬁkry et al., 2024)

or Price and Personalization for the customer experi-

ence (Garg et al., 2011).

WEBIST 2025 - 21st International Conference on Web Information Systems and Technologies

122

Against this background, it becomes clear that

UX factors are not only diverse and dynamic, but

also have different relevance depending on the con-

text of use and product type. At the same time, there

are other inﬂuencing factors in other areas of expe-

rience, such as brand experience or customer expe-

rience, which also shape the overall experience of a

product. In order to bring these different dimensions

together, holistic UX factors are becoming increas-

ingly important. In this study, holistic UX factors

refer to overarching, context-independent factors that

are not limited to individual areas of experience, but

rather shape the overall perception of a product. The

identiﬁcation and measurement of such holistic fac-

tors is crucial in order to capture the user experience

in its entirety and to adequately take into account the

ongoing changes brought about by new technologies

and usage contexts.

2.2 The Critical Incident Technique

The Critical Incident Technique (Flanagan, 1954),

which comes from the ﬁeld of psychology, opens

up new possibilities here by focusing on individual

user experiences. It collects particularly positive or

negative (i.e. “critical”) experiences, which make it

clear which moments have a particularly strong im-

pact on the UX and which factors are most important

for users.

Psychological methods are often very valid and

robustly developed. Their transfer to UX research

brings a plurality of methods and strengthens the sci-

entiﬁc foundation of UX studies. It helps to identify

new, emergent factors because the users themselves

prioritize what was critical for them and thus unex-

pected user needs can also be made visible. This often

reveals authentic, unﬁltered UX problems or success

factors that would be overlooked in traditional ques-

tionnaires or usability tests.

This subsection thus presents the Critical Incident

Technique as well as scientiﬁc papers that have ap-

plied the CIT in different contexts. These papers

aim to illustrate the diverse applications of the CIT

method and serve as a foundation for this work.

To design a product or service tailored to the pref-

erences and needs of the end user or customer, the

challenge lies not only in the precise analysis of their

characteristics and abilities but also in understand-

ing the relationships between these factors and the

end users. Thus, it is crucial to identify which par-

ticularly positive attributes of the product experience

contribute to its success, while also adopting a holis-

tic perspective to thoroughly examine its particularly

negative attributes. In this way, a comprehensive un-

derstanding can be developed of how a product can be

improved and aligned with the needs of the end user.

The methodological approach of the CIT (Flanagan,

1954) is suitable for this purpose. It focuses on ana-

lyzing both the signiﬁcantly positive and signiﬁcantly

negative critical incidents to draw conclusions for im-

provements. In this context, a Critical Incident (CI) is

an event related to the product or service investigated

that is outside the normal expectations of the interac-

tion and therefore easier to recall than everyday events

(Oh and Oh, 2017).

Grison et al. (2013) investigated in 2013 routes

using public transportation using CIT and identiﬁed

key insights. Interviews were conducted with 19 par-

ticipants and a questionnaire was completed by them.

The authors identiﬁed 94 critical incidents (35 posi-

tive and 59 negative). The most important ﬁndings

were that route choice depended on different fac-

tors, such as contextual/personal factors (e.g., time or

personal comfort), (un)satisfying factors (e.g., unex-

pected events or delays), alternatives or different per-

ceptions (e.g., age and knowledge). Thus, the authors

stated that the decision-making was inﬂuenced by in-

dividual and contextual variables.

Oh and Oh (2017) describe in their study from

2016 the behavior of smartphone users across the cat-

egories of devices, device-related services, network

services, and content services. Using the CIT method,

the authors conducted a two-staged survey, whereby

the responses of 144 participants to a questionnaire

were ﬁrst evaluated to identify CIs, which were di-

vided into 13 categories (factors). Subsequently, 651

responses were analyzed, whereby the inﬂuence of

the 13 factors on future intentions was examined. The

authors found that positive critical incidents improved

user intentions and increased willingness to recom-

mend, whereas negative critical incidents weakened

these aspects. This study also highlights that the CIT

method is not only ﬂexible in its application but also

capable of identifying both positive and negative ex-

periences.

Overall, it becomes clear that all four studies

use the CIT method to analyze speciﬁc challenges

in different application areas and derive certain fac-

tors. The related work has shown that, in addition to

the speciﬁc product and use case-related factors, also

contextual and personal factors can be relevant for a

good experience. The CIT considers a wide range of

factors, leading to more differentiated results. This is

achieved through the indicators of positively critical

and negatively critical factors. Thus, it can be said

that the versatility of CIT enables detailed examina-

tion of human behavior in various usage contexts. It

can be assumed that it is possible to transfer the CIT to

Inﬂuence of UX Factors on User Behavior: The Critical Incident Technique

123

the ﬁeld of UX research and thus identify holistic fac-

tors that have an inﬂuence on the perception of prod-

ucts (RQ1). Furthermore, Oh and Oh (Oh and Oh,

2017) have also already investigated the inﬂuence of

the factors on the future intentions of customers based

on the CIT, so that an investigation of the inﬂuence on

the frequency of use is also possible (RQ2).

3 STUDY DESCRIPTION

The procedure for this study is based on the approach

already used in other human factors research, includ-

ing research on public transport modes (Grison et al.,

2013) and the customer experience (Oh and Oh, 2017)

(see Section 2). The streaming platform Netﬂix was

chosen as the product to be investigated, as the prod-

uct is used internationally by different target groups in

a wide variety of contexts and can therefore generate

ﬁndings that are as heterogeneous as possible.

In the ﬁrst step, a qualitative questionnaire was

created to obtain as many answers as possible and

to derive UX factors from the critical incidents. In

the second step, ﬁve speciﬁc UX factors from the

ﬁrst step (Subscription Price, Bingewatching, Recom-

mendations, Nostalgia, Amount of Time until Goal is

Reached, see Section 4.1) were selected as a sample

and their inﬂuence on the frequency of use was exam-

ined in interviews. The structure of the questionnaire

and the interviews are described in the following sub-

sections and an overview of the participants is given.

3.1 Questionnaire

In the ﬁrst part of the questionnaire, socio-

demographic information on age, gender and country

of residence was requested (see Figure 1). In the sec-

ond part, the duration of use, frequency of use, place

of use and self-assessed knowledge were recorded. In

the third part, ﬁrstly the positive “critical” incidents

were recorded using the following three questions:

• What has happened that you remember as partic-

ularly positive in relation to Netﬂix? Please also

explain when/where/with whom it happened.

• What makes this event critical / particularly posi-

tive for you?

• What were the short, medium and long-term con-

sequences of this event?

In this way, the scope for interpretation could be

limited. The users were also asked to rate on a scale

from 1 (not signiﬁcant) to 5 (signiﬁcant) how critical

they considered the incident described to be. Subse-

quently, up to two further critically positive incidents

could be explained. Afterwards, the procedure was

repeated for up to three critically negative incidents.

The critical incidents form the basis for identifying

the holistic UX factors and thus for answering RQ1.

This is where the CIT differs from simple open-

ended questions. While the latter often capture gen-

eral impressions or opinions that could be inﬂuenced

by the current mood, the CIT is not about average

experiences, but about outstanding, memorable mo-

ments that have made a difference. This forces re-

spondents to focus on speciﬁc details, actions and

consequences rather than sticking to generalities (e.g.,

“It was okay”, “I found it user-friendly”). The CIT is

exploratory and designed to draw conclusions about

overarching success or failure factors from the criti-

cal incidents (Flanagan, 1954).

Next, based on the 90 UX factors generated in

this CIT questionnaire, the 5 UX factors Subscrip-

tion Price, Bingewatching, Recommendations, Nos-

talgia and Amount of Time until Goal is Reached were

then selected as random samples in order to subse-

quently investigate their inﬂuence on the frequency of

use with the help of interviews. These 5 factors were

determined based on the number of mentions, the al-

location of both positive and negative characteristics

as well as the signiﬁcance of the associated CIs indi-

cated by the participants (see Section 5.1 for further

explanations on the selection).

3.2 Interviews

In the ﬁrst part of the interviews (see Figure 2), socio-

demographic data (age, gender, country of origin) and

information on usage behavior (duration of use, fre-

quency of use, place of use and self-assessed knowl-

edge) were again recorded.

In the second part, a UX factor was used as an ex-

ample to explain what a critical incident is and how

the interview will proceed. To ensure a common un-

derstanding, the meaning of the ﬁve UX factors was

explained with an example at the beginning of the

interview. Questions were asked by the participants

in case of uncertainty (e.g., regarding nostalgia: “Do

you mean nostalgia through content or through inter-

face design?”).

Subsequently, the number of positive and negative

critical incidents for the ﬁve UX factors that the par-

ticipant can speciﬁcally remember for each UX factor

was recorded, and explanations of the respective CIs

were queried. An example of this is:

“How many situations can you remember in which

the factor ’Nostalgia’ made you think critically posi-

tive about Netﬂix?”

If anything was unclear, follow-up questions were

WEBIST 2025 - 21st International Conference on Web Information Systems and Technologies

124

Figure 1: Structure of the questionnaire to query the critical (positive/negative) incidents in order to derive the UX factors.

Figure 2: Structure of the questionnaire to query the critical (positive/negative) incidents according to the 5 chosen UX

factors in order to investigate their inﬂuence on the frequency of use.

asked that were similar to the open-ended questions

in the questionnaire (see Section 3.1).

In the third part, we then recorded how a change

in the UX factors affects the frequency of use, both

positively and negatively. Participants were given a

7-point Likert scale to choose from, including the fol-

lowing labels:

• 1 (“I would use Netﬂix signiﬁcantly less”)

• 4 (“I would use Netﬂix just as often”)

• 7 (“I would use Netﬂix signiﬁcantly more often”)

They were asked the according questions in the

following format:

“If the ’Nostalgia’ factor improved according to

your needs, how would your frequency of Netﬂix use

change?”

Finally, respondents were asked whether they

would recommend the product in general, whether

they would only recommend content/functions and

whether they would subscribe to another streaming

service (possibly additionally).

Based on the interviews, the inﬂuence of the 5

holistic UX factors on the frequency of use can be

investigated in order to answer RQ2.

3.3 Participants

The questionnaire ran from November 11 to Decem-

ber 5, 2024. The participants were recruited via dif-

ferent channels in order to ensure a sample that was as

diverse and relevant to the study as possible. On the

one hand, LinkedIn posts were created and shared by

the authors, and on the other hand, people from the

extended professional network (e.g. acquaintances

of colleagues or former work colleagues) were also

contacted to share the questionnaire. In addition, the

questionnaire was linked to a survey platform where

interested people worldwide could answer it free of

charge to reach as many people as possible. In this

way, a total of 49 participants were recruited who

completed the questionnaire in full. All participants

answered the quality assurance question correctly and

apart from that had no anomalies in their answers or

processing time, which is why all data sets could be

used. The participants were between 19 and 65 years

old (33.29 years on average). There were 28 male, 18

female and 3 diverse participants.

Subsequently, UX factors were derived from the

questionnaire and the interview guideline was created

so that the interviews could be conducted between

Inﬂuence of UX Factors on User Behavior: The Critical Incident Technique

125

January 3 and 10, 2025. Overall 12 users from the

extended professional network (e.g. acquaintances

of colleagues or former work colleagues) were inter-

viewed, who came from different professional areas

and had different socio-demographic characteristics

to gain the most heterogeneous insights possible. The

participants (6 men and 6 women) came from differ-

ent backgrounds, such as software development, ac-

count management, media design or planning in the

ﬁeld of ﬁber optics. The participants were between

19 and 35 years old (29.17 years on average).

4 RESULTS

This section ﬁrst describes the usage behavior of the

study participants, followed by the holistic UX factors

identiﬁed in the questionnaire and the inﬂuence of 5

UX factors on the frequency of use.

4.1 Usage Behavior

The usage behavior of the participants for the

questionnaire (N=49) and the interviews (N=12) is

described in the following. The questionnaire showed

that the majority of participants rated their knowledge

of Netﬂix as medium (n=29) or high (n=13) and

only a few as low (n=5) or excellent (n=2). This is

also reﬂected in the duration of use, as most of the

participants had been using Netﬂix for between 3 and

7 years (see Table 1). Around 16% of participants

were frequent users and used Netﬂix once a day or

more (see Table 2).

Table 1: Number of participants for questionnaire (N=49)

and interviews (N=12) with respective usage duration.

Duration of Use Participants

Questionnaire

Participants

Interviews

Less than 1 year 1 0

1-3 years 7 1

3-5 years 18 0

5-7 years 15 4

More than 7 years 8 7

Overall 49 12

The sample for the interviews was slightly differ-

ent. Two persons rated their knowledge as medium,

nine as high and one as excellent. 11 out of 12 peo-

ple had been using Netﬂix for more than 5 years (see

Table 1). Over 91% were frequent users (see Table 2).

Table 2: Number of participants for questionnaire (N=49)

and interviews (N=12) with respective usage frequency.

Frequency of Use Participants

Questionnaire

Participants

Interviews

1x a month/less 4 0

>1x a month 11 1

1-3x a week 11 0

>3x a week 12 0

1x a day 8 7

>3x a week 3 4

Overall 49 12

4.2 Questionnaire: RQ1 and Holistic

UX Factors

In the ﬁrst step, the holistic UX factors were deter-

mined based on the critically positive and negative

incidents of the N=49 participants. As described in

Section 2, these UX factors are all factors inﬂuencing

the positive/negative perception of Netﬂix.

All 49 participants described at least one positive

as well as one negative critical incident. In addition,

5 participants described another critically positive in-

cident and 6 participants described another negative

incident.

The critical incidents were summarized into holis-

tic UX factors based on recurring terms. One example

of this is the statement “It gives me a particularly nos-

talgic feeling, like being a child at home on the sofa

again”, which was assigned to the UX factor Nos-

talgia. The explanatory questions concerning the im-

pression and consequences of the incident (see Figure

1) were used as an additional limitation of the inter-

pretation. In this way, also multiple UX factors could

be identiﬁed in one critical incident. The assignment

was carried out and compared by two researchers to

achieve a higher quality of identifying the factors.

A total of 55 UX factors with positive incidents

(e.g., Trust, Fun or Nostalgia) and 35 UX factors with

negative incidents (e.g., Time Wasting, Frustration or

Addiction Potential) were formed. Several UX fac-

tors could be formed by one participant, and one UX

factor could be named by several participants. The

most frequently mentioned UX factors are shown in

Table 3. Some of the identiﬁed UX factors were rated

as positive by some participants, while others found

them to be negative. These included, for example,

Versatility (“Lots of Choice” vs. “Too much con-

tent suddenly dropped, leading to little choice”), Sub-

scription Price or Nostalgia.

At this point, it becomes clear that only one of the

most frequently mentioned positive and negative UX

WEBIST 2025 - 21st International Conference on Web Information Systems and Technologies

126

factors is included in standard questionnaires such as

the UEQ+ (Social Interaction). Moreover, this factor

was not included in the UEQ+ until 2023 (Mortazavi

et al., 2023). Furthermore, factors from other experi-

ences were identiﬁed (e.g., Subscription Price of the

customer experience), which are included in the holis-

tic UX consideration.

Table 3: Most frequently mentioned holistic UX factors

using Netﬂix as an example based on the questionnaire

(N=49). One factor could be named by multiple partici-

pants, and one participant could name multiple factors.

Number of Mentions UX Factor (positive)

27 Versatility

13 Social Interaction

12 Bingewatching

12 Comfortability

7 Self-Improvement

Number of Mentions UX Factor (negative)

17 Subscription Price

10 Patience

7 Consistency

6 Transparency

6 Recommendations

Afterwards, 5 of the 90 identiﬁed holistic UX fac-

tors should be selected to be examined in interviews

for their inﬂuence on the frequency of use. For this

purpose, the 10 most frequently mentioned positive

and negative UX factors (20 factors overall) were ﬁrst

taken into consideration to ensure that the selected

factors have a broad relevance for the sample. The se-

lection was then reduced to 8 factors that caused both

positive and negative events. Subsequently, 3 factors

that had lower signiﬁcance values (<3) in relation to

the critical incidents were removed to further limit the

amount of UX factors to be investigated. As a result,

the following 5 holistic factors remained for an inves-

tigation of their inﬂuence on the frequency of use in

the interviews:

• Subscription Price: Perception of the pricing

model in relation to beneﬁts

e.g., ”Netﬂix’s subscription model has become far

too expensive.”

• Bingewatching: Ability to watch multiple

episodes in a row

e.g., ”I ended up watching the entire season in

one night. I didn’t plan to, but Netﬂix kept auto-

playing – it was addictive.”

• Recommendations: Quality and relevance of sug-

gestions in the interface

e.g., ”The recommendations were completely off.

I kept seeing shows I’d never watch, which made

browsing frustrating.”

• Nostalgia: Emotional recollection of earlier times

through content

e.g., ”Netﬂix offers many old series that remind

me of my childhood, which I think is great.”

• Amount of Time until Goal is Reached: Time until

desired content is found or experience is achieved

e.g., ”It sometimes takes me 20 minutes just to

ﬁnd something I actually want to watch. There’s

too much irrelevant content.”

4.3 Interviews: RQ2 and Frequency of

Use

The ﬁrst aim of the interviews was to encourage par-

ticipants to think about the UX factors and gain an

impression of the positive and negative connections

between critical incidents and the factors. To this

end, the participants were asked for each factor how

many and which critical incidents they could remem-

ber. The number of CIs per factor is shown in Table

4. For the 12 participants, the Subscription Price and

Amount of Time until Goal is Reached are linked more

to negative impressions overall (63.6% and 60.5% re-

spectively), while Nostalgia and Recommendations

are more linked to positive impressions (86.7% for

both factors).

Table 4: Number of Positive Incidents (pCI) and Negative

Critical Incidents (nCI) for the ﬁve investigated holistic UX

factors.

UX Factor pCI nCI Overall

Subscription Price 12 21 33

Bingewatching 18 11 29

Recommendations 23 6 29

Nostalgia 26 4 30

Amount of Time un-

til Goal is Reached

15 23 38

Regarding the frequency of use subsequently sur-

veyed, it should ﬁrst be noted that participants gen-

erally stated that the frequency of use would remain

the same or increase if the UX factor improved. If the

respective factor deteriorated, a constant or lower fre-

quency of use was expected. Thus, positive (negative)

experiences in relation to the 5 factors do not appear

to result in a reduction (increase) in the frequency of

use on the example of the streaming platform Netﬂix.

Related to the Subscription Price factor, 11 of the

12 participants stated that they would still watch Net-

ﬂix just as often if the price were better, one per-

son slightly more often. The most common argument

Inﬂuence of UX Factors on User Behavior: The Critical Incident Technique

127

given here was that this would be limited by the time

available. Conversely, 7 participants stated that they

would use Netﬂix less or signiﬁcantly less if the price

got worse, while 5 would still watch just as often.

Here, the lack of alternatives or general satisfaction

with Netﬂix was cited as an argument.

The Recommendations factor showed that only

one person would watch just as often if the sugges-

tions within Netﬂix improved, while 11 people would

watch it slightly to signiﬁcantly more often. However,

if the Recommendations worsened, 8 people would

watch less, 1 person would watch slightly less and

3 people would watch as often. The available time

and the switch to alternative streaming platforms were

also cited as arguments for this factor.

Regarding Bingewatching, half of the people

stated that they would watch Netﬂix just as often if the

factor improved, while 2 participants would watch it

slightly more often, more often or signiﬁcantly more

often. If the factor worsened, half of the people would

watch Netﬂix signiﬁcantly less, related to the addic-

tion risk.

With regard to the Amount of Time until Goal is

Reached, 3 people would watch signiﬁcantly more of-

ten if less time was required, 5 people would watch

slightly more often and 3 participants would watch

just as often. If it worsened, half of the people would

watch Netﬂix less.

Related to the Nostalgia factor, it also became

clear that 11 of the 12 people would watch Netﬂix

slightly to signiﬁcantly more often if the factor im-

proved. If it worsened, half would still watch just as

often and only one person would watch signiﬁcantly

less.

Figure 3 provides a visual overview of how the

ﬁve holistic UX factors inﬂuence the frequency of

Netﬂix usage, depending on whether the factor im-

proves or deteriorates. The vertical positioning of the

factors in the graph is for illustrative purposes only

and does not represent a quantitative Y-axis. The

focus lies solely on the comparison of mean values

along the X-axis.

The sample shows that the holistic UX factor of

Nostalgia has the greatest inﬂuence on increasing the

frequency of use in the case of an improvement of the

factor. The argument put forward by the participants

was that they would then be able to watch movies etc.

without other streaming services, leaving more time

to use Netﬂix. The UX factor Subscription Price,

on the other hand, has the least inﬂuence on the fre-

quency of use, which was argued with the available

free time.

If the UX factor deteriorates, however, the Binge-

watching factor has the greatest inﬂuence on the re-

duction in usage frequency, which was linked to a

higher risk of addiction and corresponding action

measures. The Nostalgia factor has the least inﬂu-

ence in case of deterioration, as streaming new con-

tent would be preferred.

In summary, the 5 holistic UX factors using Net-

ﬂix as an example showed that the factors have an

inﬂuence on the frequency of use.

5 DISCUSSION

In this section, the research questions

• RQ1: Which holistic UX factors are mentioned

most for the streaming platform Netﬂix? (Sec-

tion 5.1) and

• RQ2: Which holistic UX factors can have an

inﬂuence on the frequency of use, using Netﬂix

as an example? (Section 5.2)

are answered based on the results. The key ﬁnd-

ings are summarized in Figure 4.

5.1 RQ1: Most Mentioned Holistic UX

Factors

Based on the responses of 49 participants to critical

incidents in connection with the streaming platform

Netﬂix, a total of 90 holistic UX factors were iden-

tiﬁed. These were 55 positive and 35 negative UX

factors. The identiﬁed factors provided a better im-

pression of the target group and their needs. Some

of the factors were both positive and negative, such

as Versatility or Nostalgia. The ﬁrst thing that be-

comes clear is that only one (Social Interaction) of

the 10 most frequently mentioned factors can already

be measured using established UX methods such as

the UEQ+ (see Table 3 in Section 4.2).

This also shows that within the 90 holistic UX fac-

tors, UX factors were found that can already be mea-

sured using standard UX methods such as the UEQ+

questionnaire. These include Trust, Efﬁciency, Clar-

ity and Novelty (see Table 5).

Furthermore, factors were identiﬁed that can al-

ready be measured using standard methods from other

experiences, such as Subscription Price of the cus-

tomer experience or Image and Exclusivity from the

brand experience.

Additional holistic UX factors were also identi-

ﬁed, which make it clear that even more factors are

relevant for the positive perception and expectations

of users than can currently be measured using es-

tablished UX methods. With regard to the Netﬂix

WEBIST 2025 - 21st International Conference on Web Information Systems and Technologies

128

Figure 3: Mean rating of the inﬂuence of the 5 holistic UX factors on the frequency of use using the streaming platform

Netﬂix as an example (N=12).

streaming platform, these include, for example, Nos-

talgia, Frustration, Internationality and Anticipation.

It became clear that the Critical Incident Technique

is also suitable for UX measurement to identify user

expectations and needs and form a basis for a good

UX.

Table 5: Examples of the 5 most frequently mentioned

established UX factors, factors of other experiences and

newly identiﬁed holistic UX factors.

Existing UX

Factors

Factors of Other

Experiences

Identiﬁed Holis-

tic UX Factors

Social

Interaction

Subscription

Price

Versatility

Trust Image Bingewatching

Efﬁciency Exclusivity Patience

Clarity Personalization Consistency

Novelty Social

Responsibility

Self-

Improvement

5.2 RQ2: Inﬂuence on Frequency of Use

The holistic UX factors that can have an inﬂuence on

the frequency of use were also investigated. This was

done using the example of the 5 factors Subscription

Price, Bingewatching, Recommendations, Nostalgia

and Amount of Time until Goal is Reached with the

help of 12 interviews.

It was shown that all of the factors examined have

an inﬂuence on the frequency of use. In the case of an

improvement in the respective factor, the frequency

of use increased or remained the same, while in the

case of a deterioration, the factor remained the same

or decreased.

It should be noted that some of the factors are lim-

ited by the context of use. For example, the frequency

of use changes little or not at all when the Subscrip-

tion Price improves, as according to the participants,

the available free time is decisive and the subscription

simply continues as normal.

In contrast, improving the Nostalgia factor would

signiﬁcantly increase the frequency of use, as in this

case there would be no need to subscribe to addi-

tional streaming platforms. This shows that not only

product-related but also emotional factors can have an

inﬂuence on the perceived user experience. With the

targeted use of these factors, it is also possible to in-

ﬂuence user behavior.

By using the Critical Incident Technique when

evaluating the product, it is possible to identify sig-

niﬁcant events and factors that have a long-term inﬂu-

ence on the perceived UX. Factors that are associated

with positive memories (positive CIs) can increase the

frequency of use. By using these ﬁndings, it is possi-

ble to increase the quality of products, strengthen user

loyalty and create products that are more memorable

and stand out from the competition.

5.3 Limitations

As with any empirical study, several limitations

should be acknowledged, even though steps were

taken to mitigate their impact.

First, the number of participants in the interview

phase (N=12) is relatively small, which may affect the

generalizability of the ﬁndings. However, this sample

size aligns with common practice in qualitative UX

research using the CIT, where depth and richness of

Inﬂuence of UX Factors on User Behavior: The Critical Incident Technique

129

Figure 4: Summary of key ﬁndings of the research questions.

individual responses are prioritized over quantity. The

participants were selected to represent diverse profes-

sional and demographic backgrounds in order to cap-

ture a wide range of experiences.

Second, while the ﬁve UX factors examined in the

interviews do not represent an exhaustive list, they

were selected systematically based on the most fre-

quently mentioned and critically rated factors from

the questionnaire phase. This approach was used to

ensure relevance and to capture factors with both pos-

itive and negative connotations. Additionally, the se-

lection process considered diversity across emotional,

functional, and contextual dimensions.

Third, the context of the study, with using Net-

ﬂix as a single application example, may limit the

applicability of the ﬁndings to other domains. How-

ever, Netﬂix was deliberately chosen as a widely used

and context-rich platform that offers both utilitarian

and emotional UX touchpoints. The goal was not to

generalize to all platforms, but to demonstrate how

CIT can uncover holistic UX factors in a concrete and

well-understood use case.

Finally, while CIT provides valuable qualitative

insights, it is also resource-intensive in both data col-

lection and analysis. This study followed a two-stage

process to balance exploratory depth with feasibility.

Future work could explore hybrid approaches (e.g.,

combining CIT with standardized questionnaires) to

scale the method for broader use.

Despite these limitations, the study provides

a solid methodological foundation and contributes

meaningful insights into holistic UX measurement us-

ing CIT.

6 CONCLUSION

That UX factors can have an inﬂuence on user behav-

ior has been demonstrated in the past. However, it

was unclear whether, in addition to the UX factors of

established UX methods such as questionnaires, there

are other holistic factors that can inﬂuence user be-

havior (Kollmorgen et al., 2024; Kollmorgen et al.,

2023; Santoso and Schrepp, 2019; Kollmorgen et al.,

2022). For this reason, this article uses the stream-

ing platform Netﬂix as an example to examine which

holistic UX factors are mentioned most frequently

and what inﬂuence they have on the frequency of use.

For this purpose, the Critical Incident Technique

(Flanagan, 1954) was used, which has been used to in-

vestigate inﬂuences on future intentions in other areas

such as customer experience. In the ﬁrst part of the

study, critical incidents were recorded from 49 people

using a questionnaire. A total of 90 holistic UX fac-

tors (55 positive and 35 negative) were derived from

these incidents. This showed that only one (Social In-

teraction of the 10 most frequently mentioned holis-

tic UX factors can already be measured using stan-

dard questionnaires such as the UEQ+. Furthermore,

WEBIST 2025 - 21st International Conference on Web Information Systems and Technologies

130

factors from other experiences such as Subscription

Price of the customer experience or Image and So-

cial Responsibility of the brand experience were also

among the factors identiﬁed. It is clear that a more

holistic picture of the user experience may be neces-

sary to support product success in the long term, for

which the transfer of CIT to UX research is suitable.

A sample was selected in the form of the ﬁve

most frequently mentioned factors to investigate their

inﬂuence on the frequency of use: Subscription

Price, Bingewatching, Recommendations, Nostalgia

and Amount of Time until Goal is Reached. It was

found that each of the factors had an inﬂuence on the

frequency of use. When the corresponding factor im-

proved, the frequency of use also increased on aver-

age, and when it deteriorated, it decreased. The holis-

tic UX factor Nostalgia stood out, as it had the great-

est inﬂuence on the frequency of use when improved,

with the absence of additional competitor products

being cited as an argument. The Subscription Price

factor had the greatest inﬂuence on the frequency of

use when the factor deteriorated, which was justiﬁed

by the increased use of competitor products. It can

therefore be seen that the consideration of critical in-

cidents and the holistic UX factors derived from them

can inﬂuence usage behavior. By strengthening the

positive characteristics, it may also be possible to in-

crease long-term customer loyalty and create products

that stand out from the competition.

The critical incidents were formed based on par-

ticipants with different frequencies of use. However,

it should be noted that, with one exception, the in-

terview participants were frequent users (they used

Netﬂix at least once a day). Furthermore, the actual

strength of the inﬂuence cannot be determined due to

the small sample size. A future investigation of the

strength of the inﬂuence of the holistic UX factors for

different participant groups can therefore be an inter-

esting prospect.

In summary, the implication for researchers is

that the example of the streaming platform Netﬂix

demonstrates how CIT can effectively uncover in-

dividual, subjectively signiﬁcant experiences, even

within narrowly deﬁned product categories. It makes

it possible to identify key experiences from the user’s

perspective that are often not depicted in traditional

models or questionnaires.

At the same time, it becomes clear that the CIT

works depending on the context and product. The

type of factors identiﬁed depends heavily on the prod-

uct, its use and target group. However, this does

not speak against the method, but rather shows the

strength of the CIT, as it can be used ﬂexibly and

adapted to different contexts.

The holistic factors found are initially speciﬁc

to the streaming platform Netﬂix and similar, but

provide indications of general categories that could

also be relevant in other areas (e.g., Social Interac-

tion,Personalization or Consistency). Such factors

can serve as a starting point for comparative studies

to ﬁnd out which holistic UX factors actually apply

across products and which are context-speciﬁc. They

also help to expand and reﬁne existing UX models.

The results of this study also offer several impli-

cations for practitioners. First, the identiﬁed holistic

UX factors, such asNostalgia or Amount of Time until

Goal is Reached, highlight that emotional and contex-

tual aspects can signiﬁcantly inﬂuence usage behav-

ior. These factors are not integrated in standardized

questionnaires, yet they may play a role in product en-

gagement and long-term user retention. Practitioners

should consider incorporating such dimensions into

their UX strategies, especially when aiming to design

for memorable and emotionally resonant experiences.

Furthermore, although the CIT is not yet widely

used in UX practice, our study demonstrates its po-

tential to uncover rich, context-sensitive insights that

might remain hidden through conventional methods.

CIT is particularly suited for early-stage product eval-

uations, where understanding real user experiences

and expectations is essential. However, due to its

resource-intensive nature by requiring detailed qual-

itative data collection and analysis, its use in large-

scale evaluations is limited unless the method is

adapted.

To enhance practical usability, a two-stage pro-

cess could be applied: ﬁrst using CIT to explore

unknown or product-speciﬁc UX factors, followed

by a standardized survey to quantify their relevance.

Alternatively, CIT elements may be integrated into

lightweight methods such as diary studies, post-test

interviews, or in-situ feedback prompts. In this way,

practitioners can beneﬁt from the depth of CIT in-

sights while maintaining efﬁciency in applied set-

tings.

Overall, the results show that UX cannot be de-

scribed in purely functional or rational terms, but is

strongly characterized by individual and emotional

experiences. They suggest that holistic UX factors

provide added value in understanding the overall per-

ception of a product. Even if the generalizability is

limited, the results exemplify how important it is to

consider usage contexts in UX research.

Inﬂuence of UX Factors on User Behavior: The Critical Incident Technique

131

ACKNOWLEDGEMENTS

This research was supported by the EQUAVEL

project PID2022-137646OB-C31, funded by MI-

CIU/AEI/10.13039/501100011033 and by ERDF,

EU.

REFERENCES

Alﬁkry, A., Widodo, A., Silvianita, A., and Rubiyanti, N.

(2024). The effect of sensory marketing, brand ex-

perience, brand image, and perceived service qual-

ity on brand loyalty with customer satisfaction as

an intervening variable (study conducted on luxury

brand product consumers): A conceptual paper. In-

ternational Journal of Scientiﬁc Multidisciplinary Re-

search, 2(5):523–538.

Flanagan, J. C. (1954). The critical incident technique. Psy-

chological Bulletin, 51(4):327–358.

Garg, R., Rahman, Z., Kumar, I., and Qureshi, M. (2011).

Identifying and modelling the factors of customer ex-

perience towards customers’ satisfaction. Interna-

tional Journal of Modelling in Operations Manage-

ment (IJMOM), 1(4):359–381.

Grison, E., Gyselinck, V., and Burkhardt, J.-M. (2013). Us-

ing the critical incidents technique to explore vari-

ables related to users’ experience of public transport

modes. In Proceedings of the 31st European Confer-

ence on Cognitive Ergonomics, ECCE ’13, New York,

NY, USA. Association for Computing Machinery.

Hinderks, A., Winter, D., Schrepp, M., and

Thomaschewski, J. (2020). Applicability of user

experience and usability questionnaires. Journal of

Universal Computer Science, 25:1717–1735.

ISO9241-210 (2020). Ergonomics of human-system inter-

action - part 210: Human-centred design for interac-

tive systems.

Klein, A. M., Hinderks, A., Schrepp, M., and

Thomaschewski, J. (2020). Construction of ueq+

scales for voice quality: measuring user experience

quality of voice interaction. In Mensch und Computer

2020 - Tagungsband, page 1–5. ACM, New York.

Kollmorgen, J., Hinderks, A., and Thomaschewski, J.

(2024). Selecting the Appropriate User Experience

Questionnaire and Guidance for Interpretation: The

UEQ Family. In International Journal of Interactive

Multimedia and Artiﬁcial Intelligence, pages 1–14.

Kollmorgen, J., Schrepp, M., and Thomaschewski, J.

(2022). Impact of usage behaviour on the user expe-

rience of netﬂix, microsoft powerpoint, bigbluebutton

and zoom. In International Conference on Web In-

formation Systems and Technologies, pages 297–406.

doi: 10.5220/0011380100003318.

Kollmorgen, J., Schrepp, M., and Thomaschewski, J.

(2023). Inﬂuence of demographic variables and us-

age behaviour on the perceived user experience. In

Marchiori, M., Dom

ınguez Mayo, F. J., and Filipe, J.,

editors, Web Information Systems and Technologies,

pages 186–208, Cham. Springer Nature Switzerland.

Mortazavi, E., Doyon-Poulin, P., Imbeau, D., and Robert,

J.-M. (2023). Development and validation of four so-

cial scales for the ux evaluation of interactive prod-

ucts. International Journal of Human-Computer In-

teraction, pages 1–14.

Oh, Y. and Oh, J. (2017). A critical incident approach to

consumer response in the smartphone market: prod-

uct, service and contents. Information Systems and

e-Business Management, 15(3):577–597.

Santoso, H. B. and Schrepp, M. (2019). The impact of cul-

ture and product on the subjective importance of user

experience aspects. Heliyon, 5(9):e02434.

Schrepp, M. and Thomaschewski, J. (2019). Design and

validation of a framework for the creation of user ex-

perience questionnaires. International Journal of In-

teractive Multimedia and Artiﬁcial Intelligence, 5(7).

Schrepp, M. and Thomaschewski, J. (2025). UEQ+ Web-

site. [Online; accessed 09-May-2025].

WEBIST 2025 - 21st International Conference on Web Information Systems and Technologies

132