On the Double Interpretation of the Description of the “Consistency and

Standards” Heuristic in the Heuristic Evaluation Method

Bruno Thom

e J

unior, Lucas de Souza Tim

oteo,

Wagner Ferreira da Silva J

unior and Celmar Guimar

aes da Silva

School of Technology, University of Campinas, Limeira, Brazil

Keywords:

Heuristic Evaluation, Consistency.

Abstract:

Heuristic Evaluation (HE) is a well-known method for assessing usability in Human-Computer Interaction.

Among Nielsen’s usability heuristics for HE, ”Consistency and standards” heuristic (H4) is related to con-

sistency, a relevant concept for usability. However, the description of this heuristic does not deal with some

relevant aspects of the concept of consistency. Therefore, the literal interpretation of this description can force

some evaluators to search for another heuristic to associate with some inconsistency problems – a search that

may or may not be successful. This paper is part of a research that questions if the description of this heuristic

must be improved. In particular, this paper aims to understand how evaluators may behave when they do this

literal interpretation during the execution of a HE. Our ﬁrst step towards these goals was to analyze Nielsen’s

heuristics in order to search if other heuristics than H4 also have consistency aspects. We also hypothesized

how some evaluators may associate these heuristics with speciﬁc inconsistency problems. We noted that at

least 5 heuristics (in addition to H4) have some aspect of consistency. Besides, we observed that it is possible

to construct a logical reasoning that may associate inconsistency problems to one of these heuristics. We sug-

gest that it is worth doing a discussion about extending the current description of the consistency heuristic to

include more aspects of consistency, instead of allowing this literal interpretation to deviate evaluators from

the expected interpretation of H4 heuristic.

1 INTRODUCTION

Heuristic Evaluation (HE) is a well-known user inter-

face evaluation method in Human-Computer Interac-

tion (HCI) area. It is an inspection method introduced

by Nielsen and Molich (1990) that aims to reveal us-

ability problems without the need of user participa-

tion in the evaluation process. Its authors based their

research on an extensive set of guidelines. They sum-

marized them into nine heuristics, which were further

reﬁned to ten heuristics (Nielsen and Molich, 1990;

Nielsen, 1994) (which we will refer to as “Nielsen’s

heuristics”, for short).

Despite its widespread acceptation and use for

more than 25 years by the HCI community, we ob-

served that the Nielsen’s heuristics hide some sub-

tleties that may confuse novice evaluators (as we

observed in HCI classes about HE), and that may

be invisible to many of the expert ones. In this

paper we focus on the fourth heuristic (“Consistency

and standards”, or H4 for short), which is related

https://orcid.org/0000-0001-6112-892X

to the concept of consistency. It is known that de-

signers must strive for consistency (Shneiderman and

Plaisant, 2004) in many aspects, such as consistency

among screens of the same software, or between the

designers’ and the users’ mental models. However,

the H4 heuristic states only that “Users should not

have to wonder whether different words, situations, or

actions mean the same thing” and that they must “Fol-

low platform conventions” (Nielsen, 1994). In other

words, this description is only related to parts of the

concept of consistency.

We hypothesize that this conciseness may be a

source of ambiguity for evaluators. On the one hand,

we believe that some evaluators may link every incon-

sistency problem to H4, even when this problem is not

related to the description of this heuristic. This may

happen due to two situations at least. First, it may

be an oversimpliﬁcation of the problem by the novice

evaluator, to which every inconsistency undoubtedly

matches the heuristic “Consistency and standards”

(regardless its description), as we observed in some

classes about HE. Second, the expert evaluator con-

222

Thomé Júnior, B., Timóteo, L., Silva Júnior, W. and Guimarães da Silva, C.

On the Double Interpretation of the Description of the “Consistency and Standards” Heuristic in the Heuristic Evaluation Method.

DOI: 10.5220/0010313002220228

In Proceedings of the 16th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2021) - Volume 2: HUCAPP, pages

222-228

ISBN: 978-989-758-488-6

sciously extends the meaning of the H4’s description

to cover complementary aspects of consistency due to

his experience with HCI (as in the detailed version

of H4’s description presented by Abulfaraj and Steele

(2020)).

On the other hand, we think that part of the evalu-

ators will try to follow literally the description of the

heuristic. This may happen when the novice evalua-

tor is executing a heuristic evaluation by the ﬁrst time

and follows exactly the heuristics’ descriptions (as we

also observed in HCI classes), or because the evalua-

tors simply believe that the description of H4 deﬁnes

all possibilities covered by the heuristic. When con-

sidering this literal meaning of the H4’s description,

they may discover that a given inconsistency problem

does not ﬁt this description. In this case, they will

probably try to ﬁnd another heuristic that can cap-

ture some aspects of the inconsistency problem under

evaluation.

This research focus on this double interpretation

of H4: the generalized interpretation of the H4’s de-

scription versus its literal meaning. In a broad sense,

given this duplicity, we want to discover how evalu-

ators perceive the meaning of H4 and of its descrip-

tion, and if it needs changes to be better understood

and used.

As a preliminary step towards this direction, this

paper aims to explore situations in which an evaluator,

focused on the literal meaning of H4’s description,

faces an inconsistency problem that does not ﬁt this

description. First of all, we argue that some consis-

tency aspects are also present in other of the Nielsen’s

heuristics. Considering this argument, we hypothe-

size that this kind of evaluator will try to ﬁnd another

heuristic with complementary consistency aspects to

associate with the inconsistency problem under anal-

ysis. We suggest that both interpretations enable eval-

uators to detect inconsistency problems through HE.

However, we argue that a better description of H4

would help evaluators to group all the consistency-

related problems they found, instead of distributing

them over a set of heuristics. Besides, this improved

description may reduce the learning curve of HE by

novice evaluators and also may enable a consistent

use of the “Consistency and standards” heuristic.

This paper is organized as follows. Section 2

presents some works related to the concept of con-

sistency, including mismatches related to the clas-

siﬁcation of inconsistency problems in a HE. Sec-

tion 3 points out some consistency aspects we found

in Nielsen’s heuristics other than H4. It also illustrates

how inconsistency problems may be associated with

these heuristics. Besides, this section argues about the

need to change the H4’s description to embrace more

aspects of consistency. Section 4 concludes the paper

and points out future work.

2 RELATED WORK

This section brieﬂy discusses some aspects of the con-

cept of consistency and presents some classiﬁcations

of these aspects. Besides, it presents some works that

sought to broaden the understanding or coverage of

Nielsen’s heuristics. It also makes some considera-

tions about the execution of HE by novice evaluators

and their expected failures during this evaluation.

The term “consistency” has many embedded as-

pects. Some of them are registered in the works

of Nielsen and Molich (Molich and Nielsen, 1990;

Nielsen and Molich, 1990; Nielsen, 1994) and Shnei-

derman and Plaisant (2004). When Molich and

Nielsen published their ﬁrst nine heuristics (Molich

and Nielsen, 1990), they called evaluators’ attention

for consistency in their HE method, with a heuristic

called “Be consistent”. According to its description,

“Users should not have to wonder whether different

words, situations, or actions mean the same thing. A

particular system action – when appropriate – should

always be achievable by one particular user action.

Consistency also means coordination between sub-

systems and between major independent systems with

common user populations”.

Four years after the publication of these heuristics,

Nielsen (1994) changed the set of heuristics and re-

placed this heuristic by a new one called “Consistency

and standards” (which is the focus of this paper, and

was already presented at the Introduction section). Its

description became more concise than the previous

one, and part of the description related to the com-

parison among systems was lost.

The eight golden rules of interface design from

Shneiderman and Plaisant (2004) also include a rule

related to consistency: “Strive for consistency”. It is

related to some forms of consistency, such as using

consistent sequences of actions for similar situations,

and using identical terminology and formatting (col-

ors, layout, fonts etc.) throughout the system.

Rocha and Baranauskas (2003) goes even further

by indicating the relevance of achieving consistency

throughout all the media that composes the interface:

screens, help, training material and so on.

Consistency is also related to accessibility.

When the Nielsen’s heuristics are compared with

the WCAG (Web-Content Accessibility Guidelines)

(W3C, 2008), consistency is partially related to the

principles “Operable” and “Understandable” (Casare

et al., 2019). In other words, a consistent interface

On the Double Interpretation of the Description of the “Consistency and Standards” Heuristic in the Heuristic Evaluation Method

223

may be easier to understand and operate.

Some authors tried to classify some aspects of the

concept of consistency. Tanaka et al. (1991) (apud

AlTaboli and Abou-Zeid (2007)) present two possible

classiﬁcations of consistency. In the ﬁrst one, internal

consistency refers to consistency within a task, and

external consistency is related to consistency among

distinct tasks (such as in the heuristic “Be consis-

tent”). A second classiﬁcation deﬁnes cognitive con-

sistency as “the consistency in what the user knows”,

and display consistency as “the consistency in the

layout of screen displays” (AlTaboli and Abou-Zeid,

2007).

Other authors (Kellogg, 1989; Schlatter and

Levinson, 2013; Moran, 2020) deﬁne the terms “in-

ternal” and “external” in a different way. They call

internal consistency the consistency between screens

of an application or a family of products, and external

consistency the consistency between applications. We

will use these deﬁnitions along this paper.

Some expert evaluators recommend to verify these

last two aspects of consistency during a HE. Salgado

et al. (2016) surveyed a set of 38 tactics suggested by

experts to improve the performance of novice evalu-

ators when they are executing a Heuristic Evaluation.

Three of these tactics are directly related to H4. First,

the tactic T12 is related to the analysis of whether

there is a consistent use of terms (or other interface

elements) when referring to a same content. The tac-

tic T13 states that evaluators should verify the con-

sistency about causes and effects, i.e., if similar ac-

tions cause similar effects in the interface. Finally,

the tactic T14 suggests that evaluators must analyze if

the application is consistent with other similar appli-

cations already known by the users. In other words,

T14 is a tactic related to external consistency, and T12

and T13 refer to internal consistency.

Some works focus on enhancing the comprehen-

sion (Abulfaraj and Steele, 2020) or the coverage

(Granollers, 2018a,b; Pribeanu, 2017) of the Nielsen’s

heuristics. Abulfaraj and Steele (2020) provided a de-

tailed version of these heuristics, based on descrip-

tions of each heuristic provided by experts. They

split H4 into two subheuristics: “Consistency” and

“Standards”. “Consistency” is related to internal con-

sistency and is divided into consistency of meaning,

function, effort, organization, and feeling. In its turn,

“Standards” is related to external consistency.

Granollers (2018a,b) proposed a combination be-

tween Nielsen’s heuristics and Tognazzini’s princi-

ples of interaction design. This combination also has

a ”Consistency and standards” heuristic that is de-

tailed in the following questions to be answered by

the evaluator:

• Do link labels have the same names as their desti-

nations?

• Do the same actions always have the same results?

• Do the icons have the same meaning everywhere?

• Is the information displayed consistently on every

page?

• Are the colours of the links standard? If not, are

they suitable for its use?

• Do navigation elements follow the standards?

(Buttons, check box, ...)

Pribeanu (2017) merged Nielsen’s heuristics and

ergonomic criteria. His new list of heuristics has 14

heuristics that were grouped into the following gen-

eral ergonomic criteria: user guidance, user effort,

user control and freedom, and user support. He also

provided detailed versions of each heuristic. In partic-

ular, his consistency heuristic considered the follow-

ing items:

• Provide similar phrasing, text justiﬁcation, color,

and punctuation.

• Display similar objects (windows, menus, exit

buttons, etc.) in the same way and at the same

location.

• Provide similar procedures for similar functions

and tasks.

• Follow platform conventions.

It is worth noting that the works of Abulfaraj

and Steele (2020), Granollers (2018b,a), and Pribeanu

(2017) did not use the Nielsen’s deﬁnition of H4.

They provided their own deﬁnitions with more details

about their consistency heuristics, aiming to guide

evaluators’ comprehension about them.

Regarding classiﬁcation mismatches related to

H4, a relevant work from Salgado and Fortes (2016)

surveyed the number of expert evaluators (among 13

ones) that believed that novice evaluators would asso-

ciate with a heuristic b a problem that should be asso-

ciated with heuristic a. For the problems in which a is

H4, 10 of the experts supposed that novice evaluators

could mistakenly associate a given consistency prob-

lem with the heuristic “Match between system and the

real world”. Besides, 5, 4, and 4 experts, respectively,

supposed that novice evaluators can erroneously re-

late the problem to the heuristics “Recognition rather

than recall”, “Aesthetic and minimalist design”, and

“Help users recognize, diagnose, and recover from

errors”. For each one of the other heuristics, 2 or 3

experts believe that the problem is related to that par-

ticular heuristic and not to H4. This research indicates

that expert evaluators expect some kind of confusion

regarding the classiﬁcation of problems related to the

HUCAPP 2021 - 5th International Conference on Human Computer Interaction Theory and Applications

224

H4 heuristic. However, the paper does not discuss

why this mismatching may happen.

In summary, consistency is a relevant concept of

usability and has some facets that should be consid-

ered by novice and expert evaluators. Particularly,

it is possible that novice and expert evaluators link

inconsistency problems to different heuristics (which

may not be an illogical choice, as we will explain

later). Regarding the consistency heuristic, the sug-

gested tactics for novice evaluators to avoid wrong

classiﬁcations are to pay attention to both internal

and external consistency concepts during the execu-

tion of the HE. This suggests that these aspects could

be added to the description of H4.

3 METHODOLOGY AND

RESULTS

In the current work, our ﬁrst step was to identify

consistency concepts that we observed inside the

Nielsen’s heuristics (except H4). Next, we analyzed

examples of inconsistencies related to these concepts

and that are not related to the literal description of

H4. Each example was associated with a heuristic that

could capture the consistency aspect related to the us-

ability problem under analysis.

3.1 Consistency Concepts inside

Heuristics

Despite the existence of the heuristic “Consistency

and standards”, which is clearly related to consis-

tency, we noted that some of the other Nielsen’s

heuristics also have aspects related to consistency, and

that these aspects are not covered by the literal de-

scription of H4. We list some of these heuristics in

this section.

First of all, the heuristic “Match between system

and the real world” (H2) has the following descrip-

tion: “The system should speak the users’ language,

with words, phrases and concepts familiar to the user,

rather than system-oriented terms. Follow real-world

conventions, making information appear in a natu-

ral and logical order” (Nielsen, 1994). However, if

we rephrase this description, its consistency-related

aspects become evident: “The system should speak

the users’ language: it should use words, phrases

and concepts that are consistent with those known by

users, rather than system-oriented terms. Be consis-

tent with real-world conventions, making information

appear in a natural and logical order”. In other words,

this heuristic could be renamed to “Consistency be-

tween system and the real world”.

Other heuristics cannot be easily rephrased to in-

clude the concept of consistency, but are also related

to it. As a ﬁrst example, the heuristic “Error preven-

tion” (H5) states that designers must prevent a prob-

lem from occurring instead of presenting good error

messages. A careful design that prevents an error to

occur is consistent with designer’s and user’s mental

models.

“Recognition rather than recall” (H6) is another

example of a heuristic that is not easily related to con-

sistency. However, suppose a situation in which a sys-

tem has two parts, and a same button in both. The

button has distinct functions in each part. Users must

remember that the button in part 1 has a distinct mean-

ing from the button in part 2. This may also be seen

as an inconsistency problem to be solved.

As a third case, the heuristic “Aesthetic and min-

imalist design” (H8) states that “Dialogues should

not contain information which is irrelevant or rarely

needed. Every extra unit of information in a dialogue

competes with the relevant units of information and

diminishes their relative visibility”. We understand

that inconsistencies such as wrong formatting, unnec-

essary repetition of objects, blank screens or screens

without information are problems that compete with

the content of a site, and then they may be associated

with this heuristic.

Last but not least, the heuristic “Help and docu-

mentation” (H10) deﬁnes that the system and its help

should be consistent to each other.

All of these situations focused on internal consis-

tency. They are not an exhaustive list, but they point

out that some inconsistency problems may be associ-

ated with H4 (in the generalized interpretation of H4’s

description) or to one of the other ﬁve heuristics dis-

cussed in this section (if the literal description of H4 is

considered). That is, there is a chance of association

between an inconsistency problem unrelated to H4’s

description and another heuristic. However, it is not

possible to ensure that the heuristics other than H4

can be associated with every inconsistency problem

not related to H4’s description. This is an argument

in favour of improving the description of H4.

It is worth noting that 3 of our situations (H2, H6,

and H8) are also cited by Salgado and Fortes (2016)

as part of the “top 4” problems regarding classiﬁca-

tion of consistency problems. Given that our analysis

was made without prior knowledge about their work,

this intersection signalizes that the literal interpreta-

tion of H4’s description may be a cause for some of

the problems they cite.

On the Double Interpretation of the Description of the “Consistency and Standards” Heuristic in the Heuristic Evaluation Method

225

Figure 1: Distinct menus in Portuguese (top) and English

(bottom) versions of the same site.

3.2 HE Considering H4’s Literal

Meaning

In this section we exemplify some real-world usabil-

ity problems that are related to consistency and that

do not ﬁt the H4’s description. We associate each of

these problems with another heuristic than H4, and

present explanations about the choice we made.

• Distinct versions of the same website, for dis-

tinct languages, do not have the same con-

tent. As an example, a website in Portuguese

has a simpliﬁed English version of its content

for foreigners, with less content and a menu

with less options. In Figure 1 (top), the blue

menu has the options ”Apresentac¸

ao” (Presenta-

tion), ”Coordenac¸

ao de Curso” (Course Coordi-

nation), Comiss

oes (”Commissions”), ”Cursos”

(Courses), and ”Disciplinas e hor

arios” (Subjects

and schedules). In Figure 1 (bottom), the options

have names of courses. These versions are there-

fore inconsistent with each other. If we consider

that in the real world a translation does not re-

move parts of the original text or change its struc-

ture, we can classify this problem as disregarding

the heuristic “Match between system and the real

world” (H2).

• Distinct links are so close that they seem to be a

single one. Figure 2 illustrates this situation. The

ﬁgure shows a list of ﬁve bullets. In each bul-

Figure 2: Each of the last three bullets have two links. Each

of the two ﬁrst bullets also have two links, but they are con-

catenated and seems to be just one link.

let we have two distinct links, but in the ﬁrst two

these links are concatenated. Probably this situa-

tion happened due to an incorrect link reuse. The

use of a “2-in-1 link” is not consistent with the

link usage in web pages, and this may lead users

to select a wrong link. Therefore, the heuristic

“Error prevention” ﬁts this situation.

• Using two languages in a same screen without a

clear reason. Some websites whose content is

written in a given language present contents in

other languages. This may be due to the use of

interface components that are not correctly con-

ﬁgured or translated. As an example, web pages

from laboratories of an institute have content in

Portuguese, except one that is in English. It is a

consistency problem, but it is not related to H4’s

description. If a user wants to navigate in this

page and has no knowledge about English lan-

guage, he may need to rely on his previous ex-

perience regarding the interaction with the pages

of other laboratories in Portuguese. Therefore, the

heuristic “Recognition rather than recall” may be

associated with this problem.

• A text has a format that is inconsistent with the

format of other texts with similar function in the

screen, without a clear reason. For example, in

Figure 3, the description of a course is in bold,

and the description of other courses is not. Given

that the description of H4’s does not cover this

case, evaluators may consider this situation as an

aesthetic problem. In this case, the bold font is

an extra unit of information that may diminish the

relative visibility of the other course descriptions.

Therefore, it violates the heuristic “Aesthetic and

minimalist design”.

• Showing contentless or incomplete web pages. A

typical example of this situation is a page that has

only the message “coming soon” or “under con-

struction” (Figure 4). When a web page points

to this kind of problematic pages, an evaluator

may deﬁne this case as an inconsistency problem.

HUCAPP 2021 - 5th International Conference on Human Computer Interaction Theory and Applications

226

Figure 3: A web site using bold for the description of

one course (Transport Engineering) and normal font for the

other descriptions.

Figure 4: A web site with a menu, a title “TEAM” and a

message “Coming soon”.

In other words, it would be consistent to provide

links that point to pages with useful information.

Again, H4’s description does not cover this case.

We may consider that this problem disrespect the

heuristic “Aesthetic and minimalist design”, given

that the interface provides a useless link that could

be removed.

• Using symbols with unknown meaning for the

user. The use of a symbol whose meaning is not

clear for the target users may be explained as an

inconsistency between the symbols that the sys-

tem uses and the ones that the users know. For ex-

ample, a button with the symbol “@” that sends

an e-mail to a person in his proﬁle page is use-

less if the user does not understand the “@” as a

kind of abbreviation to the concept of e-mail (Fig-

ure 5). The heuristic “Match between system and

the real world” (H2) is directly related to this kind

of usability problem. Note that, as we already ar-

gued, the proposed match is a kind of consistency,

but with a particular heuristic that should be used

instead of H4.

Figure 5: The char “@” as a link to a personal e-mail.

It is worth reinforcing that in all these cases the

concept of consistency is present, but the description

of the consistency heuristic is not related to them.

3.3 Discussion

The last two subsections presented some situations

that violate the concept of consistency but that are not

related to the description of H4. They revealed how

restrict is this description, and why some evaluators,

with focus on the description, may try to select other

heuristics than H4 for an inconsistency problem.

We presented in Section 3.2 a reasoning that

elected other heuristics instead of the consistency

heuristic when its description is not applicable to a

given inconsistency problem. However, considering

that the heuristics should be easy to learn, we believe

that it would be relevant to the HCI community to

think about expanding the description of H4 to in-

clude other inconsistency situations. As a matter of

fact, we believe that the inclusion of the concepts of

internal and external consistency – even that quickly

explained inside the heuristic’s description – would

be of great help for evaluators, and mainly for novice

ones.

4 CONCLUSION

In this paper, we argued that a literal interpreta-

tion of the description of ”Consistency and stan-

dards” heuristic may force evaluators to ﬁnd another

Nielsen’s heuristic to associate with some types of in-

consistency problems. We observed that some heuris-

tics other that H4 also have aspects related to consis-

tency, and presented how some evaluators may resort

to these heuristics to reveal inconsistency problems.

Supposing that the original intention of the heuristics’

authors was that the fourth heuristic should be asso-

ciated with most of the inconsistency problems, we

argue that HCI community can expend some effort on

reﬁning the description of this heuristic. The efforts

On the Double Interpretation of the Description of the “Consistency and Standards” Heuristic in the Heuristic Evaluation Method

227

of some works on providing detailed versions of their

heuristic sets may help to avoid some of the ambigui-

ties regarding H4 that were presented in this paper.

In any way we want to propose the creation of a

very detailed description of the fourth heuristic. How-

ever, we believe that generalizing and enhancing this

description would avoid some negative effects of its

literal interpretation, e.g. the detaching of inconsis-

tency problems in the HE report, or some situation in

which no other heuristic could be associated with the

inconsistency problem at hand.

This is an initial work regarding this theme. We

know that Nielsen’s heuristics were accepted and have

been adopted by the HCI community for decades, and

that they are general enough to be used to evaluate

the usability of many kinds of interfaces. Therefore,

our purpose here is to do a constructive criticism that

may enhance the comprehension of these heuristics

and reduce ambiguous interpretations.

We also believe that proposing changes to this

consolidated set of heuristics is a work that should

not be done according to the opinion of a small group

of researchers. Therefore, future work includes re-

searching how the HCI community classiﬁes incon-

sistency problems in the Heuristic Evaluation, how

often the literal interpretation of the “Consistency and

standards” heuristic occurs, and promoting a discus-

sion of the need to change that description to over-

come interpretation problems.

ACKNOWLEDGEMENTS

We thank the grant #2015/21 from

FAEPEX/PRP/Unicamp.

REFERENCES

Abulfaraj, A. and Steele, A. (2020). Detailed usabil-

ity heuristics: A breakdown of usability heuristics

to enhance comprehension for novice evaluators. In

Stephanidis, C., Marcus, A., Rosenzweig, E., Rau, P.-

L. P., Moallem, A., and Rauterberg, M., editors, HCI

International 2020 - Late Breaking Papers: User Ex-

perience Design and Case Studies, pages 3–18, Cham.

Springer International Publishing.

AlTaboli, A. and Abou-Zeid, M. R. (2007). Effect of physi-

cal consistency of web interface design on users’ per-

formance and satisfaction. In Jacko, J. A., editor,

Human-Computer Interaction. HCI Applications and

Services, pages 849–858, Berlin, Heidelberg. Springer

Berlin Heidelberg.

Casare, A. R., Silva, C. G., Martins, P. S., and Moraes,

R. (2019). Mapping WCAG guidelines to Nielsen’s

heuristics. In Proceedings of the 16th International

Conference on Applied Computing, pages 79–88.

Granollers, T. (2018a). Usability Evaluation with Heuris-

tics, Beyond Nielsen’s List. In Proceedings of the

Eleventh International Conference on Advances in

Computer-Human Interactions (ACHI), pages 60–65.

Granollers, T. (2018b). Usability Evaluation with Heuris-

tics. New Proposal from Integrating Two Trusted

Sources. In Marcus, A. and Wang, W., editors, Design,

User Experience, and Usability: Theory and Practice,

pages 396–405. Springer International Publishing.

Kellogg, W. A. (1989). The dimensions of consistency. In

Coordinating User Interfaces for Consistency. Aca-

demic Press.

Molich, R. and Nielsen, J. (1990). Improving a human-

computer dialogue. Communications of the ACM,

33(3):338–348.

Moran, K. (2020). Usability Heuristic 4: Consis-

tency and Standards. Nielsen Norman Group.

https://www.nngroup.com/videos/usability-heuristic-

consistency-standards/ (video; accessed October 29,

2020).

Nielsen, J. (1994). Usability inspection methods. In Con-

ference Companion on Human Factors in Comput-

ing Systems, CHI ’94, page 413–414, New York, NY,

USA. Association for Computing Machinery.

Nielsen, J. and Molich, R. (1990). Heuristic evaluation of

user interfaces. In Proceedings of the SIGCHI Confer-

ence on Human Factors in Computing Systems, pages

249 – 256.

Pribeanu, C. (2017). A revised set of usability heuristics

for the evaluation of interactive systems. Informatica

Economica, 21:31–38.

Rocha, H. V. and Baranauskas, M. C. C. (2003). De-

sign e Avaliac¸

ao de Interfaces Humano-Computador.

NIED/UNICAMP.

Salgado, A. L. and Fortes, R. P. M. (2016). Heuristic evalu-

ation for novice evaluators. In Marcus, A., editor, De-

sign, User Experience, and Usability: Design Think-

ing and Methods, pages 387–398. Springer Interna-

tional Publishing.

Salgado, A. L., Lara, S., Freire, A., and Fortes, R. (2016).

What is hidden in a heuristic evaluation: tactics from

the experts. In CONTECSI USP - International Con-

ference on Information Systems and Technology Man-

agement - ISSN 2448-1041, pages 2931 – 2946.

Schlatter, T. and Levinson, D. (2013). Visual Usability Prin-

ciples and Practices for Designing Digital Applica-

tions. Elsevier.

Shneiderman, B. and Plaisant, C. (2004). Designing

the User Interface: Strategies for Effective Human-

Computer Interaction (4th Edition). Pearson Addison

Wesley.

Tanaka, T., Eberts, R. E., and Salvendy, G. (1991). Con-

sistency of human-computer interface design: Quan-

tiﬁcation and validation. Human Factors, 33(6):653–

676.

W3C (2008). Web Content Accessibility

World Wide Web Consortium (WCAG) 2.0.

https://www.w3.org/TR/WCAG20/ (accessed October

29, 2020).

HUCAPP 2021 - 5th International Conference on Human Computer Interaction Theory and Applications

228