Prioritization of Mobile Accessibility Guidelines for Visual Impaired

Users

Fiamma E. M. Quispe, Lilian P. Scatalon and Marcelo M. Eler

University of S

ao Paulo (EACH-USP), S

ao Paulo - SP, Brazil

Keywords:

Prioritization, Accessibility Guidelines, eMAG, m-Government, Mobile Applications, Visual Impairment.

Abstract:

m-Government involves delivering public services to citizens through mobile applications. Naturally, such

apps should beneﬁt everyone, including citizens with disabilities. However, several Brazilian government

apps still present accessibility issues, given the high demand of development teams, which end up working on

functional requirements in detriment of other aspects such as usability. In this scenario, we investigate the pri-

oritization of mobile accessibility guidelines extracted from e-MAG (the Brazilian Government Accessibility

Model) to help dealing with limited resources while also addressing accessibility. We surveyed people with

some kind of disability by asking them to rate the relevance of each guideline. They were supposed to consider

the severity of problems faced by them due to non-compliance. Most respondents were visual impaired users

(66 out of 103 overall responses), so we considered this group of respondents in our analysis. Results show

variations on the perception of relevance among subgroups with different levels of visual impairment. More-

over, results allowed us to propose a priority list of the evaluated mobile accessibility guidelines. Ultimately,

the idea is to help developers with a strategy to remediate accessibility issues on existing apps and also to

avoid them during the early development phases of new ones.

1 INTRODUCTION

With the increasing use of smartphones, mobile appli-

cations have been gaining ground for e-government

services, which are also referred as m-government

(Kushchu and Kuscu, 2003; Trimi and Sheng, 2008).

In the Brazilian scenario, a population survey of 2018

shows that many households (56%) use only smart-

phones to access the Internet

Ideally, m-government applications should be us-

able by everyone, guaranteeing broad access by cit-

izens. However, users can face many problems

while interacting with mobile apps, especially people

with disabilities (Carvalho et al., 2018; Acosta-Vargas

et al., 2020).

Aiming to address these problems, some standards

have been established to help developers avoid in-

serting barriers in digital content, such as WCAG

and BBC

accessibility guidelines. Still considering

the Brazilian scenario, the eMAG model

(based on

https://www.cetic.br/pesquisa/domicilios/indicadores

https://www.w3.org/TR/WCAG

https://www.bbc.co.uk/guidelines/futuremedia/accessi

bility/

http://emag.governoeletronico.gov.br

WCAG) was created for the context of e-government.

Both accessibility and e-MAG compliance are en-

forced by law in Brazil for government websites (Law

No. 13.146/2015, Ordinance No. 3/2007). Despite

that, many of these websites still present accessibility

issues (Oliveira and Eler, 2015; Oliveira et al., 2017).

Reasons may include the high demand faced by devel-

opment teams, which end up implementing functional

requirements ﬁrst, in detriment of other aspects such

as accessibility.

Similarly, Brazilian m-government apps also

present many accessibility issues (Serra et al., 2015).

For this reason, in a previous work we identiﬁed a set

of mobile accessibility guidelines, based on eMAG,

aiming to address government mobile applications as

well (Quispe and Eler, 2018). However, there is still a

lack of methods and time that support the application

of accessibility standards endorsed by the government

(Oliveira and Eler, 2017).

In this sense, prioritization is a good strategy for

dealing with limited resources and decision-making

(Berander and Andrews, 2005). Hence, we further

investigate accessibility in the context of Brazilian

m-government, but aiming to help mobile app devel-

opers in terms of method to incorporate accessibility

Quispe, F., Scatalon, L. and Eler, M.

Prioritization of Mobile Accessibility Guidelines for Visual Impaired Users.

DOI: 10.5220/0009393405630570

In Proceedings of the 22nd International Conference on Enterprise Information Systems (ICEIS 2020) - Volume 2, pages 563-570

ISBN: 978-989-758-423-7

563

guidelines into their projects.

In this paper, we propose a prioritization of the

mobile accessibility guidelines in Quispe and Eler

(2018). To establish the priority ranking, we sur-

veyed mobile device users with some kind of disabil-

ity. We focused the analysis on results from people

with visual impairment, which was our most signif-

icant group of respondents (66 out of 103 total re-

sponses).

The remainder of this paper is organized as fol-

lows. Section 2 describes the accessibility model in-

vestigated in this study. Section 3 presents the survey

that we conducted with disabled people to ﬁnd out

their opinion on the mobile accessibility guidelines.

Section 4 discusses the obtained results, i.e. the rele-

vance rating of each guideline. Section 5 presents our

proposed prioritization, based on survey results. Sec-

tion 6 indicates related work and, ﬁnally, Section 7

presents conclusions, limitations and directions for

future work.

2 BACKGROUND

There are many m-government apps in the Brazilian

scenario, which cover several different areas, such as

education, economy, citizenship and justice, science

and technology, tourism, health, among others (Lima,

2017; Dutra and Soares, 2019).

In general, it is possible to observe user interface

problems in such apps, especially related to accessi-

bility (Serra et al., 2015; Isagah and Wimmer, 2017).

In face of these problems, there is a need of guide-

lines concerning the quality of the content targeted at

citizens (Lima, 2017).

The set of guidelines in Quispe and Eler (2018)

identify accessibility problems that hinder the use of

mobile applications. They are based on eMAG, which

is the model established for Brazilian e-government.

Since eMAG is focused on websites, the authors

also considered other standards on mobile applica-

tions as input, such as BBC Mobile Accessibility

Guidelines

, recommendations of the Android plat-

form

and the WCAG draft for mobile devices

The guidelines are outlined in tables 1 to 6, sorted

by the following aspects: structure (G1-G5), behav-

ior (G6-G11), content/information (G12-G21), pre-

sentation/design (G22-G24), multimedia (G25-G29)

https://www.bbc.co.uk/guidelines/futuremedia/

accessibility/mobile

http://developer.android.com/design/patterns/

accessibility.html

https://www.w3.org/TR/mobile-accessibility-mapping

and forms (G30-G35). There is an id and a brief de-

scription for each guideline.

Table 1: Mobile Accessibility Guidelines – Structure.

ID Description

G1 Screen elements should be organized in a logical and seman-

tic way.

G2 Screens should present a logical sequence of reading to navi-

gate between links, form controls and other elements.

G3 Links on the screen should be organized to avoid confusion.

G4 Information should be divided into speciﬁc groups to ease

ﬁnding and reading content.

G5 Users should be informed if links open new screens so they

can decide whether or not they want to leave current screen.

Table 2: Mobile Accessibility Guidelines – Behavior.

ID Description

G6 All functionalities on screen should be available from the key-

board.

G7 All interface elements on screen should be accessible.

G8 There should not be automatic redirect between screens.

G9 In a time-limited screen there should be options to turn off,

adjust or extend such limit.

G10 There should not be any blinking, intermittent or sparkling

visual effects on the screen.

G11 User control over time-related content changes should be en-

sured. Animated content should not start automatically.

Table 3: Mobile Accessibility Guidelines – Con-

tent/information.

ID Description

G12 The language used on the screen should be speciﬁed.

G13 Changes in the content language should be always speciﬁed.

G14 Screen titles should be descriptive, informative and represen-

tative of the main content.

G15 There should be a mechanism to indicate users where they are

at the moment in a set of screens.

G16 Link targets should be identiﬁed clearly, including informa-

tion on whether they are working or direct to other screen.

G17 All images on screen should have a text description.

G18 Documents in accessible formats should be available.

G19 When a table is used in a screen, an appropriate title and sum-

mary should be provided.

G20 The text on the screen should be easy to read and understand.

G21 All acronyms, abbreviations and unusual words on screen

should have an explanation.

All eMAG guidelines (45 in total) were considered to

establish the mobile guidelines. Each one was ana-

lyzed to check whether it dealt with web-speciﬁc as-

pects, thus indicating a need to adapt it to the mo-

bile context. For example, keyboard-related guide-

lines need to reﬂect the way users interact with mo-

ICEIS 2020 - 22nd International Conference on Enterprise Information Systems

564

Table 4: Mobile Accessibility Guidelines – Presenta-

tion/design.

ID Description

G22 There should be a minimum contrast ratio between fore-

ground and background.

G23 Sensory characteristics (e.g color, shape, sound) cannot be

the only means to distinguish screen elements.

G24 The element or area on focus should be visually evident.

Table 5: Mobile Accessibility Guidelines – Multimedia.

ID Description

G25 Videos that do not include audio should provide alternatives,

such as subtitles.

G26 There should be alternatives to audio, such as a transcript and

sign language version.

G27 Visual content that is not available as audio should be audio

described.

G28 There should be mechanisms to control any audio that plays

on the application.

G29 There should be mechanisms to control any animation that

starts automatically on the screen.

Table 6: Mobile Accessibility Guidelines – Forms.

ID Description

G30 Image buttons or audio content in forms should have text al-

ternative or description.

G31 Each form ﬁeld should be identiﬁed.

G32 A logical order of form navigation should be ensured.

G33 There should not be automatic changes when a form element

receives focus, so user does not get confused or disoriented.

G34 Forms should have ﬁlling instructions.

G35 Input errors should always be described and data submission

conﬁrmed.

bile devices, allowing them to also use gestures, such

as swipe to left or right, click on an item or the use of

assistive technologies.

As a result, 26 eMAG guidelines did not require

any adjustment, since they were generic enough to

guide both web and mobile development. Namely, the

unchanged guidelines were: G6-G15, G17, G19-G30,

G32, G34 and G35.

A total of nine guidelines were adjusted to com-

prise the mobile platform characteristics, i.e. G1-G5,

G16, G18, G31 and G33. Hence, the original guide-

lines were rewritten to reﬂect the context of mobile

applications. For example, the G1 counterpart stated

originally: “Organize HTML code in a logical and

semantic way”. It was adjusted to: “Screen elements

should be organized in a logical and semantic way”.

Still, 10 eMAG guidelines were not included nor

adjusted, since there was no relation with mobile ap-

plications. For example, “comply with web stan-

dards” and “provide other speciﬁc security strategies

rather than CAPTCHA”.

3 SURVEY DESIGN

The proposed prioritization involves using ranks to

gauge the relevance of accessibility guidelines. We

assessed the relevance of each guideline based on the

opinion of potential end users with disabilities, since

they are the most affected people by non-compliance.

To that end, we designed a survey targeted at

people with disabilities. We recruited respondents

through emails to our academic community and posts

on social media groups for people with disabilities.

The message included the informed consent and a link

to a web-based questionnaire.

The questionnaire was composed of three sec-

tions. In the ﬁrst section we aimed to identify

the respondent proﬁle, by asking their age and type

of disability, with the following options: vision

impairment, hearing loss, physical disability, men-

tal/intellectual disability, multiple, other or none.

These questions were presented equally for all re-

spondents.

Next, we asked the disability level with appropri-

ate options according to the informed type of disabil-

ity. For example, vision impairment had the follow-

ing level options: partially sighted, low vision, legally

blind and totally blind.

The second section had the purpose to identify ac-

cessibility problems. Firstly, we asked what assistive

technologies they use in their mobile devices, such as

screen reader, braille terminal, speech synthesizer and

so on.

Then, we presented the model discussed in Sec-

tion 2 and asked respondents to rate each guideline in

a 5-point Likert scale, from “minor relevance” (1) to

“major relevance” (5). We transcribed the guidelines

into statements with simpler wording, so respondents

could understand the purpose of each guideline.

Depending on the informed type of disability, a

different subset of the mobile accessibility guidelines

was presented to the respondent. This distinction had

the purpose to narrow respondents’ rating to aspects

that indeed are impacted by their type of disability.

For example, a blind user did not have to rate guide-

lines related to text color or the use of subtitles in

videos.

Finally, the third section was composed of one op-

tional question where respondents could include ad-

ditional information on mobile accessibility deemed

necessary.

We calculated the overall relevance for each

Prioritization of Mobile Accessibility Guidelines for Visual Impaired Users

565

guideline using a weighted average, similarly to

Oliveira (2005):

∑

i=1

where:

= relevance rating for guideline g

i = traverses the Likert scale options

= value of Likert option i

= frequency of i in responses for guideline g

N = total number of respondents

If the resulting value is close to ﬁve, it means that

users ﬁnd the respective guideline of the most im-

portant degree and, therefore, that such guideline pre-

vents more severe barriers to visual impairment. On

the other hand, if it is close to one, although still im-

portant, the guideline addresses less severe barriers.

4 RESULTS

In total, 103 people answered the survey. More than

half (64.08% – 66) of respondents declared having vi-

sual impairment. The other kinds of disability had a

minimal representation in results: there was one re-

spondent with hearing impairment and one with mul-

tiple disability (less than 1% each). Surprisingly, a

signiﬁcant number of respondents (34% – 35) de-

clared not having any kind of disability.

When considering only respondents that indeed

have some disability (a total of 68 people), the great

majority of respondents (97.06% – 66) are visually

impaired. Hence, we limited the analysis of results

to this group. It is important to mention that visual

impaired respondents rated the following guidelines:

G1, G6-G11, G15, G22-G25, G27 and G28.

Still, we also considered the informed levels of

visual impairment to further divide respondents into

subgroups: Partially Sighted (40 – 60.6%), Low Vi-

sion (15 – 22.7%), Legally Blind (6 – 9.1%) and To-

tally Blind (5 – 7.6%). We refer to these subgroups

from now on by using their initials, respectively PS,

LV, LB and TB.

Table 7 shows the resulting relevance (R

), in av-

erage, for each guideline (rows) rated by respondents

with visual impairment. The average relevance rat-

ings are given both for separate subgroups and for all

visually impaired respondents (columns).

Note that, in general, the relevance rating in-

creases with the severity of visual impairment. It is

possible to observe that most guidelines presented a

high relevance rating, with an average above 3. Also,

Table 7: Relevance of Guidelines (R

g PS LV LB TB All visual impaired

G1 2.8 3.4 4.0 4.8 3.75

G6 3.1 3.4 3.2 4.8 3.6

G7 3.8 3.8 4.2 4.6 4.1

G8 3.2 3.3 3.4 5.0 3.7

G9 3.5 4.3 3.9 4.6 4.1

G10 3.3 3.7 3.7 3.4 3.5

G11 3.4 4.2 4.2 4.2 4

G15 3.2 3.5 2.9 4.6 3.5

G22 3.5 3.8 4.2 5.0 4.1

G23 2.7 3.2 3.4 5.0 3.6

G24 3.6 3.2 4.5 4.6 4

G25 3.7 4.0 4.2 4.8 4.2

G27 3.3 3.4 4.0 4.6 3.8

G28 3.8 4.2 4.5 4.8 4.3

a subset of guidelines stands out with values above 4,

namely G7, G9, G22, G25 and G28.

4.1 Partially Sighted (PS)

There were 40 partially sighted respondents, 32 of

them with ages between 18 and 29 years and eight be-

tween 30 and 59 years. Table 8 shows, for each guide-

line, the frequency of each level of relevance (i.e. how

many respondents chose each Likert option) and the

resulting relevance rating for that guideline in the last

column. For example, considering guideline G1, ten

respondents chose option 1, nine chose 2, 11 chose 3,

and so on, resulting in a weighted average relevance

of 2.8.

Table 8: Results for Partially Sighted Respondents.

Frequency

g (by level of relevance) R

1 2 3 4 5

G1 10 9 11 2 8 2.8

G6 7 10 8 3 12 3.1

G7 0 7 12 4 17 3.8

G8 2 8 19 4 7 3.2

G9 2 7 16 2 13 3.5

G10 5 7 11 5 12 3.3

G11 2 8 14 4 12 3.4

G15 4 11 9 8 8 3.2

G22 2 7 11 9 11 3.5

G23 6 12 16 2 4 2.7

G24 2 6 12 7 13 3.6

G25 1 7 12 5 15 3.7

G27 6 5 13 5 11 3.3

G28 1 4 15 3 17 3.8

Note that most values are between 2 and 4, indicating

a medium level of relevance for users. Also, it is inter-

esting to note that, for partially sighted users, there is

ICEIS 2020 - 22nd International Conference on Enterprise Information Systems

566

not much difference in the level of relevance of acces-

sibility issues. The guidelines with higher relevance

in this subgroup are G7 and G28, both with 2.8. On

the other hand, the guideline G23 had the lower rele-

vance rating (2.3).

Most partially sighted users (37) stated that do

not use or feel the need to use assistive technology.

Nonetheless, two users stated that sometimes use a

screen reader and another one use the zoom feature of

mobile devices.

In the free text question, one user left the follow-

ing comments on mobile accessibility:

“Large fonts or options to increase font size are very

useful, although still little used. Mobile devices

present a bad resizing.”

4.2 Low Vision (LV)

There were 15 users with low vision, 11 of them with

ages between 18 and 29 years e four between 30 and

59 years. Table 9 shows the frequencies of each level

and the respective relevance rating for this user sub-

group.

Table 9: Results for Respondents with Low Vision.

Frequency

g (by level of relevance) R

1 2 3 4 5

G1 3 1 3 3 5 3.4

G6 0 5 5 0 5 3.4

G7 1 3 2 1 8 3.8

G8 3 1 4 3 4 3.3

G9 1 0 3 1 10 4.3

G10 1 2 4 2 6 3.7

G11 1 0 2 5 7 4.2

G15 0 2 7 3 3 3.5

G22 1 1 4 4 5 3.8

G23 3 2 3 3 4 3.2

G24 4 1 2 4 4 3.2

G25 2 1 0 5 7 4.0

G27 1 3 5 2 4 3.4

G28 1 1 2 2 9 4.2

Note that, unlike partially sighted users, there is

more variation among the relevance ratings of dif-

ferent guidelines, demonstrating that some accessi-

bility problems have greater impact than others on

users with low vision. Namely, guidelines G9, G11

and G28 are the most relevant ones according to users

> 4), while G23 and G24 had the lower values,

but still above three.

In this subgroup, 10 users informed not using any

assistive technology, while two indicated using it, but

did not specify which one, and two users informed us-

ing speech synthesizer and screen reader. In the last

question, one respondent of this subgroup said the fol-

lowing:

“I think it is very important that mobile applications

offer ease of use, with a simple interface that allows

users to navigate and understand all functionalities

easily. Nowadays there are apps whose features are

difﬁcult to understand and, consequently, use.”

4.3 Legally Blind (LB)

There were six legally blind users that responded the

questionnaire, ﬁve of which with ages between 18 and

29 years, and one between 30 and 59 years. Table 10

shows results for this subgroup.

In particular, it is possible to observe that most

guidelines presented values above four (R

> 4), thus

indicating that most accessibility problems are very

important for this subgroup. Still, guidelines G24 and

G28 presented the higher relevance ratings (R

> 4.5),

while G15 had the lowest value (2.9).

Table 10: Results for Legally Blind Respondents.

Frequency

g (by level of relevance) R

1 2 3 4 5

G1 1 0 0 2 3 4.0

G6 0 1 4 0 1 3.2

G7 0 0 2 1 3 4.2

G8 0 1 3 1 1 3.4

G9 0 0 2 3 1 3.9

G10 0 1 2 1 2 3.7

G11 0 0 2 1 3 4.2

G15 1 1 2 2 0 2.9

G22 0 0 2 1 3 4.2

G23 1 0 2 2 1 3.4

G24 0 0 1 1 4 4.5

G25 0 0 2 1 3 4.2

G27 0 0 3 0 3 4.0

G28 0 0 1 1 4 4.5

About assistive technologies, ﬁve legally blind users

informed not using any aid, while one indicated using

large fonts and invert colors features.

4.4 Totally Blind (TB)

There were ﬁve totally blind users that responded the

questionnaire, one of which with age between 18 and

29 years, and four between 30 and 59 years. Table 11

shows frequencies of options and respective relevance

ratings of each guideline for this particular subgroup.

Results clearly demonstrate that totally blind users

are the most impacted subgroup by the lack of mobile

accessibility. Note that most guidelines present values

above 4.5 and many are close to 5.

Prioritization of Mobile Accessibility Guidelines for Visual Impaired Users

567

Guidelines G8, G22 and G23 presented relevance

rating of ﬁve, which is the maximum value, while G10

presented the lower value (3.4), but still above aver-

age. Four users informed using assistive technologies

for interacting with mobile applications, but did not

specify which ones.

Table 11: Results for Totally Blind Respondents.

Frequency

g (by level of relevance) R

1 2 3 4 5

G1 0 0 0 1 4 4.8

G6 0 0 0 1 4 4.8

G7 0 0 1 0 4 4.6

G8 0 0 0 0 5 5.0

G9 0 0 1 0 4 4.6

G10 1 0 2 0 2 3.4

G11 1 0 0 0 4 4.2

G15 0 0 1 0 4 4.6

G22 0 0 0 0 5 5.0

G23 0 0 0 0 5 5.0

G24 0 0 0 2 3 4.6

G25 0 0 0 1 4 4.8

G27 0 0 1 0 4 4.6

G28 0 0 0 1 4 4.8

5 PRIORITIZATION OF MOBILE

ACCESSIBILITY GUIDELINES

Table 12 presents our prioritization of guidelines,

which is based on the survey results. Positions in

the ﬁrst column reﬂect values of relevance rating, i.e.

guidelines with the higher value are in the ﬁrst posi-

tion, therefore indicating highest priority of the un-

derlying accessibility problem, and so on.

We deﬁned ﬁve different lists in the subsequent

columns. There is one list for each level of visual

impairment (identiﬁed by the abbreviations PS, LV,

LB and TB) in the intermediate columns. Also, there

is one overall list for visual impaired users in the last

column. Note that some guidelines are in the same

position of a given priority list (i.e. some table cells

have more than one guideline). This overlap indicates

that the corresponding guidelines resulted in the same

relevance rating. For example, considering subgroup

PS, guidelines G7 and G28 are in the ﬁrst position,

because both present the same relevance value.

These lists of prioritization can be used by devel-

opers as an implementation strategy, so that guide-

lines with higher priority can be considered earlier in

the mobile app development process. If developers

address accessibility in an orderly way, they ensure

that barriers causing more impact to users are avoided

before.

Developers can create strategies for the different

levels of visual impairment, according to the poten-

tial users of the mobile application. Also, the overall

list can be used to develop an app that comply with

guidelines related to visual impairment as a whole.

6 RELATED WORK

There are studies addressing prioritization of usabil-

ity problems to deal with limited resources, such as

Hassenzahl (2000), Hertzum (2006) and Harrison and

Petrie (2007). Our approach is similar to the ones

adopted in these studies, which are described as, re-

spectively, “judgment-driven priorities”, “user-made

severity assignments” and “severity ratings”. In other

words, ratings based on users’ opinions. Eler et al.

(2019) gathered information from user reviews on the

Google Play Store to understand which accessibility

violation types result in higher or lower scores, but

there is no systematic study on the severity of each

violation type found withing different user reviews.

Severity of usability problems and relevance of ac-

cessibility guidelines are interrelated concepts, since

the guidelines aim to address usability problems.

Hence, it is possible to say that the relevance of

a given guideline increases with the severity of the

problem that it addresses.

Other studies, such as Carvalho et al. (2018),

Damaceno et al. (2018) and Acosta-Vargas et al.

(2020), focus on mobile accessibility problems for vi-

sual impaired users, which are in line with the user

group and platform that we investigated. Still, both

Serra et al. (2015) and Lara et al. (2018) are studies

targeted at Brazilian m-government apps. All these

studies present similar ﬁndings in the sense of en-

countering many recurrent accessibility problems in

the evaluated mobile apps.

Prioritization of accessibility guidelines can also

be related to WCAG

conformance levels, i.e. A,

AA and AAA. Ultimately, both prioritization and con-

formance levels stratify accessibility requirements.

However, there is a difference in how we stratiﬁed

the accessibility guidelines. A WCAG criterion is as-

signed to a given level according to the number of

users it beneﬁts and ease of implementation (Holder,

2019). On the other hand, we assigned guidelines to

different ranks according to relevance ratings deter-

mined by a speciﬁc group of people with disabilities,

i.e. visual impaired users.

In another perspective, the prioritization of ac-

cessibility problems has been discussed in the con-

text of “remediation” (Groves, 2011; Byrne-Haber,

https://www.w3.org/TR/WCAG/

ICEIS 2020 - 22nd International Conference on Enterprise Information Systems

568

Table 12: Prioritization of Accessibility Guidelines for Visual Impaired Users.

Subgroups

Priority PS LV LB TB All visual impaired

G7, G28 G9 G24, G28 G8, G22, G23 G8

G25 G11, G28 G7, G11, G22, G25 G1, G6, G25, G28 G25

G24 G25 G1, G27 G7, G9, G15, G24, G27 G7, G9, G22

G9, G22 G7, G22 G9 G11 G11, G24

G11 G10 G10 G10 G27

G10, G27 G15 G8, G23 G1

G8, G15 G1, G6, G27 G6 G8

G6 G8 G15 G6, G23

G1 G23, G24 G10, G15

G23

2018). Basically, remediation is ﬁxing accessibility

issues/problems, which can be prioritized and tackled

more efﬁciently.

Prioritization is done according to the impact of

accessibility problems. There are several ways to ac-

count such impact

, such as impact on users with dis-

abilities, volume of repeated issues and trafﬁc parts of

the website/application. Again, our study considered

the impact on users.

7 CONCLUSIONS

M-government applications present several recurrent

accessibility problems, increasing the digital divide

for citizens with disabilities. Factors include the high

demand on development teams, which may focus on

functional requirements, in detriment of other quality

aspects such as accessibility. In this sense, prioriti-

zation is a way to deal with scarce resources while

addressing accessibility at the same time.

In this paper we presented a prioritization pro-

posal of mobile accessibility guidelines. We consid-

ered the model presented by Quispe and Eler (2018),

which we adapted from eMAG in a previous work,

since it is the standard enforced by law in Brazil for

government websites.

To establish the priority ranks, we surveyed peo-

ple with disabilities to consult their opinion on the

relevance of each guideline. There were 103 respon-

dents in total, but we limited the analysis for re-

sponses from visual impaired users, since it was the

most signiﬁcant group of respondents (66).

Results show that users indeed think there is a

difference on the relevance of the mobile guidelines,

thus enabling the prioritization. Also, it is possible to

https://web.accessibility.duke.edu/get-started/

remediation/prioritization

note a difference on the relevance perception among

groups of different visual impairment levels.

From the obtained results we were able to create

a priority list of the mobile accessibility guidelines,

using as criterion the relevance perceived by visual

impaired users. We also created additional priority

lists for different levels of visual impairment. Ul-

timately, the proposed priority lists are supposed to

help developers to deﬁne strategies to follow accessi-

bility guidelines.

However, our study presents limitations. We used

convenience sampling, which may introduce selection

bias in the results. Also, some subgroups had a con-

siderable low number of respondents, hence their re-

sponses may not be representative.

Still, we did not check beforehand whether re-

spondents had trouble understanding the terms used

in the questions, neither assessed their proﬁciency on

the use of mobile devices or assistive technologies.

In addition, users responded the questionnaire based

on their personal experience with mobile applications

and assumptions, rather than using a set of apps des-

ignated for the study.

As future work, we intend to expand the number

of respondents, aiming to have a representative par-

ticipation of people with other kinds of disabilities as

well. Still, we intend to gather more details on respon-

dents’ proﬁle to investigate whether the proﬁciency

on mobile devices and assistive technologies affects

relevance ratings.

ACKNOWLEDGEMENTS

Lilian Passos Scatalon and Marcelo Medeiros Eler are

ﬁnancially supported by the grants 19/15367-3 and

18/12287-6, respectively, of the S

ao Paulo Research

Foundation (FAPESP).

Prioritization of Mobile Accessibility Guidelines for Visual Impaired Users

569

REFERENCES

Acosta-Vargas, P., Salvador-Ullauri, L., Jad

an-Guerrero, J.,

Guevara, C., Sanchez-Gordon, S., Calle-Jimenez, T.,

Lara-Alvarez, P., Medina, A., and Nunes, I. L. (2020).

Accessibility assessment in mobile applications for

android. In Nunes, I. L., editor, Advances in Hu-

man Factors and Systems Interaction, pages 279–288,

Cham. Springer International Publishing.

Berander, P. and Andrews, A. (2005). Requirements Pri-

oritization, pages 69–94. Springer Berlin Heidelberg,

Berlin, Heidelberg.

Byrne-Haber, S. (2018). Prioritizing accessibil-

ity defects for remediation. Available at

https://medium.com/@sheribyrnehaber/prioritizing-

accessibility-defects-for-remediation-

7e56893a6f40.

Carvalho, M. C. N., Dias, F. S., Reis, A. G. S., and Freire,

A. P. (2018). Accessibility and usability problems en-

countered on websites and applications in mobile de-

vices by blind and normal-vision users. In Proceed-

ings of the 33rd Annual ACM Symposium on Applied

Computing, SAC ’18, page 2022–2029, New York,

NY, USA. Association for Computing Machinery.

Damaceno, R. J. P., Braga, J. C., and Mena-Chalco, J. P.

(2018). Mobile device accessibility for the visu-

ally impaired: problems mapping and recommenda-

tions. Universal Access in the Information Society,

17(2):421–435.

Dutra, D. and Soares, D. (2019). Mobile applications

in central government of Brazil and Portugal. In

Proceedings of the 12th International Conference on

Theory and Practice of Electronic Governance, ICE-

GOV2019, pages 86–92, New York, NY, USA. ACM.

Eler, M. M., Orlandin, L., and Oliveira, A. D. A. (2019). Do

android app users care about accessibility? an analy-

sis of user reviews on the google play store. In Pro-

ceedings of the 18th Brazilian Symposium on Human

Factors in Computing Systems, IHC ’19, New York,

NY, USA. Association for Computing Machinery.

Groves, K. (2011). Prioritizing remediation of web ac-

cessibility issues. Available at https://karlgroves.com/

2011/06/12/prioritizing-remediation.

Harrison, C. and Petrie, H. (2007). Severity of usability and

accessibility problems in ecommerce and egovern-

ment websites. In Bryan-Kinns, N., Blanford, A., Cur-

zon, P., and Nigay, L., editors, People and Computers

XX – Engage, pages 255–262, London. Springer Lon-

don.

Hassenzahl, M. (2000). Prioritizing usability problems:

Data-driven and judgement-driven severity estimates.

Behaviour & Information Technology, 19(1):29–42.

Hertzum, M. (2006). Problem prioritization in usability

evaluation: From severity assessments toward impact

on design. International Journal of Human–Computer

Interaction, 21(2):125–146.

Holder, F. (2019). WCAG levels: What’s the differ-

ence? Available at https://myaccessible.website/blog/

wcaglevels.

Isagah, T. and Wimmer, M. A. (2017). Mobile govern-

ment applications: Challenges and needs for a com-

prehensive design approach. In Proceedings of the

10th International Conference on Theory and Prac-

tice of Electronic Governance, ICEGOV ’17, pages

423–432, New York, NY, USA. ACM.

Kushchu, I. and Kuscu, M. (2003). From e-government to

m-government: Facing the inevitable. In The 3rd Eu-

ropean Conference on e-Government.

Lara, R., Gosling, M., and Rodrigues, I. (2018). Mo-

bile government: uma analise dos aplicativos estad-

uais como mediadores do relacionamento entre os

cidadaos e os governos estaduais. Revista Do Servico

Publico, 69(2):63–89.

Lima, C. C. B. (2017). m-government: Limits and possi-

bilities for the use of mobile apps as a citizenship tool

in Brazil. In Proc. of the 10th Int. Conf. on Theory

and Practice of Electronic Governance, ICEGOV ’17,

pages 245–252, New York, NY, USA. ACM.

Oliveira, A. D. A. and Eler, M. M. (2015). Acessibilidade

em governo eletr

onico: um estudo sobre a aplicac¸

de padr

oes web em s

ıtios gov. br. XI Simp

osio

Brasileiro de Sistemas de Informac¸

ao, pages 683–690.

Oliveira, A. D. A. and Eler, M. M. (2017). Strategies and

challenges on the accessibility and interoperability of

e-government web portals: A case study on brazilian

federal universities. In Computer Software and Ap-

plications Conference (COMPSAC), 2017 IEEE 41st

Annual, volume 1, pages 737–742. IEEE.

Oliveira, A. D. d., Souza, E. M. d., and Eler, M. M. (2017).

Accessibility model in electronic government: Evalu-

ation of brazilian web portals. In Proceedings of the

Brazilian Symposium on Information Systems (SBSI),

pages 332–339. SBSI.

Oliveira, L. H. (2005). Exemplo de calculo de

ranking medio para likert. Available at

https://administradores.com.br/producao-academica/

ranking-medio-para-escala-de-likert. Lecture notes

on Research Methods in Business Administration.

Quispe, F. E. M. and Eler, M. M. (2018). Accessibility rec-

ommendations for mobile applications: A contribu-

tion to the brazilian digital government standards. In

Proceedings of the XIV Brazilian Symposium on Infor-

mation Systems, SBSI’18, New York, NY, USA. As-

sociation for Computing Machinery.

Serra, L. C., Carvalho, L. P., Ferreira, L. P., Vaz, J. B. S.,

and Freire, A. P. (2015). Accessibility evaluation of

e-government mobile applications in Brazil. Proce-

dia Computer Science, 67:348 – 357. Proc. of the 6th

Int. Conf. on Sw Development and Technologies for

Enhancing Accessibility and Fighting Info-exclusion.

Trimi, S. and Sheng, H. (2008). Emerging trends in m-

government. Communications of the ACM, 51(5):53–

58.

ICEIS 2020 - 22nd International Conference on Enterprise Information Systems

570