Usability Evaluation of a Community-led Innovation Mobile App
André Castello Branco
, Eveline Sacramento
, Eliza Oliveira
, Oksana Tymoshchuk
Maria Antunes
, Margarida Almeida
, Luís Pedro
, Fernando Ramos
and Daniel Carvalho
Department of Communication and Arts, University of Aveiro, DigiMedia, Aveiro, Portugal
Department of Communication and Arts, University of Aveiro, INET-md, Institute of Ethnomusicology, Aveiro, Portugal
Keywords: Mobile Application, Usability, Heuristic Evaluation, Community-led Innovation.
Abstract: Digital media can facilitate collaborative processes among local agents, value endogenous resources, and
promote assets associated with territory. This article presents the results of a study concerning the
development and validation of a mobile app for promoting the relationship among agents of the Portuguese
Centro region’s communities/entities. This paper focuses on the results of a heuristic evaluation of the mobile
app carried out with two groups of experts in Digital Technologies, Tourism, Health, and Well-Being, besides
providing an overview of the mobile app that was developed and a theoretical background regarding
community-led innovation, usability, and heuristics. For the CeNTER app prototype evaluation itself, the use
of Nielsen's heuristics, a MATCH-MED scale, together with a Think-Aloud Protocol allowed us to improve
its usability. This article contributes to a reflection about the evaluation of mobile apps in the scope of
territorial-based innovation initiatives, engaging its stakeholders in the process.
Community-led initiatives have great potential in the
development of cultural, tourism, and environmental
projects, enabling valuing endogenous resources and
promoting innovation in the territory. Digital
technologies can be particularly useful in community-
led initiatives, allowing to recreate a "virtual
proximity" between the different actors involved in
the territory's development process (Martínez-Rolán
et al., 2019). They can offer several advantages to
community-led initiatives, as, for example,
facilitating innovation, because of access to
information and experiences; sharing practices,
leading to knowledge co-creation and the emergence
of new ideas; and enabling their members to improve
their practices through a continuous engagement in a
meaningful participatory environment (Saint-Onge &
Wallace, 2012; Snyder & Wenger, 2010).
However, several studies indicate that
community-led initiatives face several challenges,
especially in rural areas (Brown & Nylander, 2009;
Marré & Weber, 2010). These challenges are related
to the fact that local participants, namely elderly
people, have difficulties accessing the Internet,
present limited digital skills, and lack access to
technological equipment. To overcome these
challenges, new political measures are required to
empower local communities to the social, cultural,
and economic valorisation of territories.
Branco, A., Sacramento, E., Oliveira, E., Tymoshchuk, O., Antunes, M., Almeida, M., Pedro, L., Ramos, F. and Carvalho, D.
Usability Evaluation of a Community-led Innovation Mobile App.
DOI: 10.5220/0011263000003323
In Proceedings of the 6th International Conference on Computer-Human Interaction Research and Applications (CHIRA 2022), pages 81-88
ISBN: 978-989-758-609-5; ISSN: 2184-3244
2022 by SCITEPRESS Science and Technology Publications, Lda. All rights reserved
This study was developed under the CeNTER
Program, which has as one of its objectives to
understand the active role those digital technologies
can play in the process of territorial innovation (Silva
et al., 2021). In this context, a digital platform (mobile
app) is being designed, whose primary focus is to
promote collaboration between the various agents
involved in territorial-based innovation processes.
The CeNTER mobile app intends to be a tool that
encourages internal and external interactions and
facilitates communication and collaboration
processes, to strengthen existing mediation strategies
and stimulate the creation of new ones, as well as new
ideas/activities (Tymoshchuk et al., 2019). However,
the design of a mobile app for community mediation
is still a great challenge due to the variety of
initiatives and the diversity of stakeholders (public
institutions, companies, and communities) involved
in territorial-based innovation processes.
This paper presents the results of a heuristic
evaluation of the CeNTER app prototype, which will
provide the development cycle with crucial usability
data, required to converge to a final solution that will
offer a coherent and effective experience about the
distinguishing features of this app to its users. The
study is supported by a User-Centered Design
approach, which focuses on satisfying the user desires
and needs (Hartson & Pyla, 2012). Also, it is framed
by the User Experience (UX) theoretical basis,
guiding the prototype evaluation in the CeNTER
Program scope, which will crucial the development
of the CeNTER mobile app.
2.1 Territorial Innovation
Innovation can be translated as the systematization of
knowledge shared between actors in the spaces in
new products and services. Systematized knowledge,
when interacting with the social, historical, and
cultural constructs for which they are intended,
change the behaviour of social groups in a systemic
way, enabling the development by promoting
territorial innovation (Ferreira, 2020). In
geographically close groups, these historical-socio-
cultural constructs are perceived in a systemic way,
which characterizes the territories (Keating, 2020).
Territorial innovation is a complex process that
results from an intentional collective action promoted
by a set of actors and organizations (companies,
universities, local government bodies, communities)
that interact to develop innovation (Morgan, 1997).
So, the initiatives led by groups coordinated from the
bottom up respond to the development needs of
territories, as they address ecological, social,
economic, and political problems of global resonance
(Seyfang & Smith, 2007, p. 585).
Furthermore, the sharing of information to solve
local problems has become more effective as a result
of the integration provided by the dissemination of
information and telecommunication technologies,
especially mobile apps (Diniz et al., 2019), further
enhancing the importance of mobile apps in the
territorial-based innovation domain. These digital
solutions must meet the community's needs to foster
the local economy. To ensure that a product meets
users' needs, such mobile digital solutions must be
developed under the User Experience (UX)
theoretical basis, which is crucial to gather relevant
information regarding the interaction with the
2.2 User Experience
An experience is a complex event, which is created in
the mind of the user and is influenced by many
factors, being a completely personal issue (Knight,
2019). Therefore, User experience is subjective and is
related to how the user feels regarding a created
product. Further, the current concept of UX is strictly
related to emotional results that emerged from the
user when interacting with a product, such as pride,
joy, and fun (Bernhaupt & Pirker, 2013). Therefore,
creating an experience is not just about how the
product is designed, which structures were
implemented or whether state-of-the-art technology
is used (Knight, 2019). It is also about how the
specific product can help the user to accomplish their
tasks successfully and how the user feels when
getting involved with the product.
UX is composed of three factors: usability,
usefulness, and emotional impact, and generates a
memory related to the product interaction. Emotional
impact is the affective component of the user
experience, focusing on the system as a means that
affects the user's feelings. Usefulness focuses on the
use of a system to achieve goals and accomplish
specific tasks. Usability is constituted by
effectiveness, efficiency, ease-of-use, learnability,
and the user satisfaction, being the practical
component of user experience (Hartson & Pyla,
The usability factor contributes to the elaboration
of a product that is easy to use and is pivotal in
creating digital products since it provides products
with a low level of difficulty, reducing the platform's
CHIRA 2022 - 6th International Conference on Computer-Human Interaction Research and Applications
disuse (Partridge, 2017). Since ensuring high
usability for a mobile app is of great importance, as a
predictor of its acceptance and of its success, is
crucial in all phases of app testing (Muchagata &
Ferreira, 2019), including early testing with experts.
2.3 Usability
Usability is an essential concept in developing digital
interfaces, focusing on users and contexts of use, and
ensuring they can achieve their goals with efficiency,
effectiveness, and satisfaction. The usability concept
was defined by Nielsen (2012) as an attribute with
qualitative nature that assesses the ease of use of user
interfaces. Usability directly relates to methods for
enhancing this attribute during the design process
phase that includes the following five main
components: (1) the ability to learn - the ease with
which users complete basic tasks during the primary
interaction with the interface; (2) efficiency - the
speed with which users reach their goals after they
have learned to interact with the platform; (3)
memory - the easiness with which users interact with
the interface after a period without using it; (4) errors
- the number of errors that are made by the user and
the ease with which it is possible to correct them; (5)
satisfaction - a component related to the desire and
pleasure expressed by the user during and after the
interaction with a specific product (Nielsen, 2012).
Due to several specificities related to the small
screen, storage capacity, and energy consumption of
mobile phones, it is essential that these apps meet a
set of basic requirements, such as: being easy to use,
being flexible, having a simple and intuitive interface,
allowing the user to easily adapt to different contexts
of usage, etc. (Feijó et al., 2013; Kumar & Mohite,
In this sense, usability evaluation of mobile apps
is a mandatory process to ensure that such apps are
practical, effective, and easy to use (Kumar &
Mohite, 2018). Usability evaluation is the generic
designation for a set of methods aiming at interface
inspection. The final objective is to identify usability
issues, through the indication of the severities’ level
of each issue (Nielsen, 1994). These methods include
heuristic evaluation, cognitive pathway, consistency,
pattern inspection, among others. One of the
commonly available and employed methods for
assessing and improving interfaces is heuristic
evaluation (Nielsen, 1994).
2.4 Heuristic Evaluation
Nielsen's heuristics (1994) is a systematic evaluation
method allowing the identification of problems in
user interface design that involves interface analysis.
As the author states, heuristic evaluation requires
fewer resources than other methods to detect usability
issues. As Nielsen (1994) states, “basically, a set of
evaluators inspects the interface with respect to a
small set of fairly broad principles, which are referred
to as the ‘heuristics” (p. 152).
This specific method is based on a 10 “heuristic”
items checklist, which can be used in interface
specifications, prototypes, or complete systems.
Nielsen's 10 heuristics include the following
guidelines (Nielsen, 1994): (1) Visibility of the
system; (2) Match between system and real-world; (3)
User control and freedom; (4) Consistency and
standards; (5) Error prevention; (6) Minimise the
user’s memory load; (7) Efficiency of use and
performance; (8) Aesthetic and minimalist design; (9)
Help and documentation; (10) Help users identify,
analyse, and improve from errors.
According to Nielsen and Landauer (1993),
between three to five evaluators are recommended to
perform a heuristic evaluation, which can identify
around 75% to 95% of the problems, considering that
an individual evaluator usually finds around 35% of
2.4.1 MATCH-MED Scale
Testing the usability of mobile apps can be done using
several sets of checklists and protocols available to
carry out heuristic evaluations. However, the vast
majority of these scales are geared towards generic
systems, demanding adaptations to fulfil specific
requirements of mobile apps (Hashim & Isse, 2019;
Inostroza et al., 2016). Considering that the CeNTER
application is aimed at Tourism, Health, and Well-
being clusters, a set of MATCH-MED usability
heuristics was used.
The MATCH-MED scale aims to evaluate the
usability of mHealth systems on smartphones and was
developed based on Nielsen's generic heuristics, with
the addition of three heuristics specifical for mobile
devices (Lacerda et al., 2015; Salazar et al., 2012),
which consist of: (1) Minimization of human-
computer interaction: Considering that typing on
mobile touch screen keyboards is more error-prone
than on conventional keyboards, it is essential to
minimise user interaction with the mobile app; (2)
Physical interaction and ergonomics: Given the
limited screen size of a mobile device, the action
Usability Evaluation of a Community-led Innovation Mobile App
controls must present suitable sizes and minimal
distance from each other, ensuring the user does not
press a button by mistake; (3) Readability and Quick
View: Considering that mobile apps are generally
used in dynamic contexts, it is essential to ensure
quick access to system information by the user (e.g.,
at a glance).
According to Zhang and Adipat (2005), a
comprehensive usability study for a mobile
application should assess a variety of issues such as
interface design, ease of use, and perceived attitude
by the user and measures related to the application's
The team researchers developed a CeNTER mobile
application prototype based on an in-depth study
under this project. That research included a
systematic review of the literature, a benchmarking of
the communities' digital platforms, a series of
interviews with local stakeholders, and two focus
groups with representatives of community-led
initiatives (Oliveira et al., 2021). The results of these
methodological procedures made it possible to
identify the main difficulties found on community-led
initiatives and led to the proposed digital solution that
aimed to fill these gaps.
The researchers chose a card-based user interface
design that is both simple and innovative in
appearance. This type of interface design tries to
provide information in a readable format, easy to
navigate, allowing an overview of the application's
content and quick access to all categories (Seifi,
2015). Considering the requirements and
functionalities defined as important in the proposed
digital solution, the medium-fidelity prototype was
developed with the support of “Principle” software,
an interactive user interface, and animation design
The prototype’s content evaluated by experts was
organised according to the categories defined by the
team as being essential in the mobile application:
Initiatives; Events; Entities; Volunteers; Resources
and Highlights. These categories were defined during
the Focus Group sessions with the local initiatives,
through a User-Centered Design approach. In the
CeNTER app context, these categories are called
"Tabs" and represent the main relevant elements in
the community dynamics. Such “Tabs” are organised
in horizontal lines on the screen and have secondary
content that is modified according to the preferences
defined by the user in the app settings (Branco et al.,
By touching each one of the Tabs, a vertical
opening occurs on the screen, where it is possible to
view suggestions (of local Initiatives, for example)
through cards displayed on the screen. Each card
presents a single content and appears ordered in a
carousel layout.
Figure 1: Home screen of the version of the medium-
fidelity prototype “CeNTER” evaluated by experts.
The evaluation process for the CeNTER mobile
application prototype is outlined according to an
iterative design methodology (Lorenz, 2010). It
provides for the initial testing of app mockups with
team members, evaluating medium-fidelity
prototypes by experts and laboratory tests with end-
users. This article presents the second stage of
evaluation of the prototype, which consisted of
heuristic evaluation by experts to detect and correct
issues related to the prototype's usability.
Through dissemination seminars, organized by
the CeNTER team, it was possible to recruit local
stakeholders to actively participate in the project as
experts. To accomplish the assessment, two panels of
experts were organized according to the evaluators
expertise. In this sense, the first group consisted of
five experts in the field of digital technologies, having
deep knowledge and experience in developing
interfaces. The second group consisted of five experts
in the fields of Tourism, Health, and Well-being, who
have knowledge of their specific domain and
involvement in local community projects.
To determine the severity of usability issues for
this study, a heuristic checklist was developed to
evaluate the application prototype, based mainly on
Nielsen's 10 heuristics (1994). Besides that, three
heuristics of MATCH-MED scale (Salgado et al.,
CHIRA 2022 - 6th International Conference on Computer-Human Interaction Research and Applications
2019) were added to allow the evaluation of mobile
apps in health and well-being areas. It is important to
note that the researchers chose to add these items on
the MATCH-MED scale since Nielson's heuristics do
not consider specific characteristics and limitations of
mobile devices (Hashim & Isse, 2019), as it was
pointed out before.
A total of 44 items were identified in this
checklist, which was employed to evaluate the
prototype usability. The heuristic evaluation process
took place in three stages, according to the proposal
of Nielsen (1994): (i) preparation phase, in which the
prototype screens for evaluation and the list of
heuristics were defined and organised; (ii) evaluation
phase, which consisted of collecting data from each
expert, individually. The evaluators tested the
prototype by identifying the guidelines that were
violated and the degree of severity of the problem.
In addition, the Think-Aloud Protocol (Jaspers,
2009) was also employed to obtain immediate
feedback from experts about their experience of
interacting with the prototype. The application of this
method allowed the qualitative evaluation of the
prototype based on the experts' verbal comments.
The tests took place at the facilities of the
University of Aveiro, with the first group on October
28-31, 2019, and the second group on November 11-
29, 2019. In total, 78 screens of the developed
application prototype were evaluated. This evaluation
consisted of two phases: in the first phase, the experts
freely explored the prototype and commented on their
doubts about the CeNTER program, and in the
second, the experts filled a table of design-oriented
heuristics for mobile phone interfaces and apps.
User-Centered Design approach
was used as an
interactive design process to get users feedbacks
and their needs in each phase of design evaluation
This section presents the results obtained from the
usability test, through a heuristic evaluation, of the
CeNTER application prototype with the two groups
of experts (five users in each group). To consolidate
the heuristic validation results performed by both
groups, a grid was created for each group to gather all
of the obtained results.
A total of 174 usability problems were identified.
After eliminating the duplicate issues, 155 unique
usability problems remained. Table 1 shows the
number of problems identified by each group of
evaluators, the severity of such problems and their
average severity.
Regarding the severity of problems, it is important
to refer that Group 1 (G1) identified 46 (42,2%)
cosmetic problems, 39 (35,8%) small usability
problems, 21 (19,3%) main usability problems and 2
(1,8%) usability “catastrophes”. The group of digital
technologies experts identified two problems with
score level four. These problems referred to the
second heuristic: “Correspondence between the
system and the real world” and were related to the
following items: “The proposed interactions in the
application are similar to real actions”; and
“Information appears in a logical and natural order”.
Those are relevant feedback because the prototype is
applied to promoting community-led initiatives and
those participants may present different levels of
digital literacies. In this sense, a simple and intuitive
interface, easily adaptable to different contexts of
usage (Feijó et al., 2013; Kumar & Mohite, 2018) is
important to make sure individuals use the application
properly, which in this case is represented by different
territorial agents (communities, entities, networks
and citizens).
Table 1: Number of Problems and Average Severities
identified by each group of evaluators.
Digital Technologies
109 1,78
Tourism, Health and
Well-being Experts
46 1,53
Regarding the severity of the found problems, it is
important to note that Group 2 identified 14 (30,4%)
cosmetic problems, 26 (56,5%) small usability
problems, and 6 (13%) main usability problems. It
should be noted that no problems with a score of level
4 (usability catastrophes) were reported by the second
Tables 2 and 3 show the type of usability
problems and average severities identified by each
group of evaluators. With the highest Average
Severity, Group 1 identified the heuristics: Help and
Documentation (2.75), Recognition rather than
Reminder (2.5), and Interaction between person and
application (2.33). Group 2 reported the following
Heuristic Severities Averages: Flexibility and
Efficiency (2.25), Help and Documentation (2.2)
Recognition rather than Reminder (2), and Interaction
between person and application (2). Therefore, both
Usability Evaluation of a Community-led Innovation Mobile App
groups consistently assessed the severity of the
heuristic violation. The Mean Average Severity in
these two groups was low, with a score close to 1.78
(G1) and 1.53 (G2).
The difference in results among both groups also
occurred in the type of the identified problems. The
problems identified by Group 2 were more related to
the suitability of the prototype for each application
domain, as well as the efficiency and type of
functionality available to users (Table 3). For
example, experts of Group 2 did not observe
inconsistencies rectified after the first round of tests,
such as different icons representing the same
function, or the absence of the return icon on some
Table 2: Type of usability problems and average severities
identified by Group 1 of evaluators.
G 1 - Digital
Technologies Experts
Average of
Visibility of system status 11 2.18
Match between system and the
real world
11 2.27
User control and freedom 13 1.54
Consistency and standards 7 1.57
Error prevention 5 1.6
Recognition rather than recall 4 2.5
Flexibility and efficiency of use 7 1.85
Aesthetic and minimalist design 12 1.66
Help users recognize, diagnose,
and recover from errors
10 1.1
Help and documentation 4 2.75
Interaction between person and
3 2.33
Physical interaction and
8 1.75
Readability and layout 14 1.5
As it is possible to see in Table 2, the maximum
number of usability problems, identified by Group 1,
included: Readability and layout (14, mean severity
1.5), followed by User controls and free will (13,
mean severity 1, 54) and Aesthetics and minimalist
design (12, medium severity 1.66). In Table 3, Group
2 identified the following number of heuristic
violations: User controls and exercises free will (11,
mean severity 1.45), Avoid errors (7, mean severity
1.86), and Matching the system to the real world (6,
mean severity 1.5). Identifying these heuristic
violations will make it easier to identify and prioritise
issues that need urgent attention before the final
deployment of the application.
The application of the three heuristics proposed
by the MATCH-MED scale allowed the identification
of 28 usability problems related to the different
specificities of mobile devices, namely "Readability
and layout" (16) and "Physical interaction and
ergonomics" (8).
Table 3: Type of usability problems and average severities
identified by Group 2 of evaluators.
G 2 - Tourism, Health,
and Well-being Experts
Average of
Visibility of system status 1 1
Match between system and the
real world
6 1.5
User control and freedom 11 1.45
Consistency and standards 3 1.66
Error prevention 7 1.86
Recognition rather than recall 1 2
Flexibility and efficiency of use 4 2.25
Aesthetic and minimalist design 1 1
Help users recognize, diagnose,
and recover from errors
4 1.9
Help and documentation 5 2.2
Interaction between person and
1 2
Physical interaction and
0 0
Readability and layout 2 1
The Think-Aloud Protocol application allowed
the collection of 124 suggestions for improving the
prototype under evaluation, which will be considered
in the following stages of the mobile application
development. The first group of experts made 83
suggestions for improvement, which were very
focused on technical aspects and improving
interactions. For example, “standardise the ‘save’
icons on the right side of the ideas screen with the rest
of the application” (G1E1) and “Drag and drop visual
feedback is required” (G1E2), respectively, for the
evaluators E1 and E2 of Group 1.
In contrast, the second group of experts mainly
commented on the concept of the CeNTER
application, functionalities, and effectiveness and did
not mention technical aspects. This group made
several suggestions regarding the lexical inaccuracies
CHIRA 2022 - 6th International Conference on Computer-Human Interaction Research and Applications
used in the application. It is important to note that
both experts' groups highlighted the "Resources" and
"Volunteer" components as being the most innovative
features of this application: "Resources and
volunteers are the differentiating elements of the
platform" (G2E5). Overall, the prototype was
described as having a simple interface, with an
appropriate layout, easy to use and not requiring
much effort from the user.
The study's results demonstrate that usability
testing is an effective way to significantly improve
the interface of a future mobile app, favouring the
user experience, as mentioned by other authors
(Lacerda et al., 2016; Salazar et al., 2012). The
methods and techniques of application production
were supported by methodologies oriented to the
user’s requirements and were anchored in a spiral of
evaluation of the prototype, resulting in improved
versions of the pilot app, attending to users'
statements, and developing an application with a
more attractive look & feel.
As the development of the CeNTER prototype
requires an iterative design process, performing early
tests is a crucial part of the system design to detect
usability problems and make essential improvements.
Through the techniques used for the presented
evaluation, it was possible to achieve value data, as
the experts are from different areas of knowledge.
Therefore, according to the current study findings,
the identification of usability problems facilitated the
identification and prioritisation of problems that need
urgent attention before the final implementation of
the application. In this case, the heuristic "control and
freedom" deserves special attention, since it received
the major severity score by both groups. Furthermore,
a decrease in the heuristic values between the two
groups was verified, which leads to the conclusion
that the changes made after the first round of tests
helped for the improvement of the prototype.
This study also showed the importance of
developing heuristic usability evaluation scales. As
already mentioned, the researchers applied three
items proposed by the MATCH-MED scale, which
allowed us to identify usability problems related to
the specificities of mobile devices. The evaluation of
the prototype, not only by professionals in the field of
digital technologies but also by professionals in areas
related to the application, such as tourism, health, and
well-being is also important. It is essential to mention
that the professionals in the areas of tourism, health,
and well-being, who participated in this study, have a
strong connection with different community
initiatives and deeply know the local associative
dynamics, which was very advantageous for
evaluating the prototype.
Future work includes performing laboratory tests
with end-users, which are of most importance in this
project, requiring accommodation of the experts’
suggestions for improvements. Some limitations of
this work include the fact that Principle software does
not allow to test some features and interactions of the
prototype, such as writing on the interface and
changing the letters’ size for accessibility. Besides,
the Principle software only works with iPhone, which
limits the evaluation using Android devices.
Specifically, it is hoped that this app can make a
difference in everyday life, promoting community-
led initiatives, towards adding value to local
resources and, therefore, the increment of the region’s
social, cultural, and economic levels.
This work is financially supported by national funds
through FCT – Foundation for Science and
Technology, I.P., under the project UIDB/05460/
Bernhaupt, M. & Pirker, R. (2013). Evaluating User
Experience for Interactive Television: Towards the
Development of a Domain-Specific User Experience
Questionnaire, Human-computer Interact. - Lect. Notes
Comput. Sci. 8118, 642–659.
Brown, R. & Nylander, A. (2009). Community Leadership
Structure: Differences Between Rural Community
Leaders' and Residents' Informational Networks.
Journal of the Community Development Society, 29, p.
Branco, A.C., Carvalho, D., Sacramento, E., Tymoshchuk,
O., Oliveira, E., Antunes, M.J., Pedro, L., Almeida, M.,
Ramos, F. (2021). Prototyping and evaluating a mobile
app to promote territorial innovation. In Proceedings of
CeNTER Conference, (pp. 73-81).
Diniz, E., Cernev, A., Bastos, T. P., Pinheiro, A., La
Rovere, R., Querette, E., Morel, L., & Arroio, A.
(2019). Inovação em Serviços na Economia do
Compartilhamento. Publisher: Saraiva.
Feijó, V. C., Gonçalves, B. S. & Gomez, L. S. (2013).
Heurística para avaliação de usabilidade em interfaces
de aplicativos smartphones: Utilidade, produtividade e
imersão. Design e Tecnologia, 3(06), 33-42.
Usability Evaluation of a Community-led Innovation Mobile App
Ferreira, L.M. (2020). Sustainable generation of innovative
enterprises in territories: a proposal of an ecosystem
management model. In Proceedings of CeNTER, 55-62.
Hartson, R. & Pyla, P. S. (2012). The UX Book: Process
and guidelines for ensuring a quality user experience.
Hashim, N. & Isse, A. (2019). Usability Evaluation Metrics
of Tourism Mobile Ap-plications. Journal of Software
Engineering and Applications, 12, 267-277.
Inostroza, R., Rusu, C., Roncagliolo, S., Rusu, V., &
Collazos, C. A. (2016). Developing SMASH: A set of
SMArtphone's uSability Heuristics. Computer
Standards & Interfaces, 43, 40-52.
Jaspers, M. (2009). A comparison of usability methods for
testing interactive health technologies: methodological
aspects and empirical evidence. International journal of
medical informatics, 78(5), 340-353.
Keating, M. (2020) Beyond the nation-state: territory,
solidarity, and welfare in a multiscalar Europe.
Territory, Politics, Governance, 9, 2021 - Issue 3:
Inequalities, territorial politics, nationalism.
Knight, W., 2019. UX for Developers. Northampton.
Kumar, B. A. & Mohite, P. (2018). Usability of mobile
learning applications: a systematic literature review.
Journal of Computers in Education, 5(1), 1-17.
Lacerda, T. C., Nunes, J. V. & von Wangenheim, C. G.
(2015). Usability heuristics for mobile phone
applications: a literature review. In Emerging
Perspectives on the De-sign, Use, and Evaluation of
Mobile and Handheld Devices (pp. 143-157). IGI
Lorenz, T., Richter, A., Köster, F. & Rataj, J. (2010, April).
Iterative design process for the development and testing
of cooperative applications. In Proceedings of the 2010
Spring Simulation Multiconference, (pp. 1-6).
Marré, A. & Weber, B. (2010). Assessing community
capacity and social capital in rural America: lessons
from two rural observatories. Community Development,
41(1), p. 92-107,
Martínez-Rolán, X., Tymoshchuk, O., Piñero-Otero, T. &
Renó, D. (2019). Instagram como red de promoción e
hipermediación del turismo rural: el caso de Aldeias
Históricas. Revista Latina de Comunicacion Social, 74,
1610 - 1632
Morgan, K. (1997). The learning region: institutions,
innovation and regional renewal. Regional Studies, 31,
Muchagata, J. & Ferreira, A. (2019). Visual Schedule: A
Mobile Application for Autistic Children-Preliminary
Study. In Proceedings of ICEIS 2, (pp. 452-459).
Nielsen, J. (1993). Usability Engineering. Morgan
Kaufmann Publishers Inc. San Francisco, CA, USA.
Nielsen, J. (1994). Enhancing the explanatory power of
usability heuristics. In Conference on Human Factors
in Computing Systems - Proceedings, 152–158.
Nielsen, J. (2012). Usability 101: Introduction to usability
(2012). Retrieved from URL: http://www. nngroup.
Nielsen, J. & Landauer, T. K. (1993). A mathematical
model of the finding of us-ability problems. In
Proceedings of the INTERACT'93 and CHI'93
conference on Human factors in computing systems
(pp. 206-213). ACM.
Nielsen, J. (15 de Novembro de 2020). 10 Usability
Heuristics for User Interface Design. Retrieved from
Partridge, D. (2017). Launch Your Dream: A 30-day Plan
for Turning Your Passion In-to Your Profession. Harper
Collins Leadership.
Oliveira, E., Branco, A. C., Carvalho, D., Sacramento, E.
R., Tymoshchuk, O., Pedro, L., & Ramos, F. (2021).
End-user Evaluation of a Mobile Application Prototype
for Territorial Innovation. In ICEIS (2), (pp. 495-504).
Saint-Onge, H., & Wallace, D. (2012). Leveraging
communities of practice for strategic advantage.
Salazar, L. H., Lacerda, T., von Wangenheim, C. G. &
Barbalho, R. A. (2012). Customizando heurísticas de
usabilidade para celulares. In Companion Proceedings
of the 11th BSHFCS, (pp. 37-38).
Salgado, A. D., Fortes, R. P. M., Hung, P. C. & Moreira, D.
D. A. (2019). A Method for Classifying Usability
Findings to Enhance Validation of New Heuristics.
Revista de Sistemas e Computação-RSC, 9(1), 96-118.
Seifi, S. (2015). Developing an Engaging Local Community
Application as a Tool for Promoting Social Integration.
(Doctoral dissertation, Malmö University). Retrieved
Seyfang, G. & Smith, A. (2007). Grassroots innovations for
sustainable development: Towards a new research and
policy agenda. Environmental politics, 16(4), 584-603.
Silva, A., Antunes, M. J., Tymoshchuk, O., Pedro, L.,
Almeida, M., & Ramos, F. (2021). Barriers and
incentives to territory-based innovation processes: from
technology to interaction among actors. In Handbook of
Research on Cultural Heritage and Its Impact on
Territory Innovation and Development, (pp. 44-61). IGI
Snyder, W. M. & Wenger, E. (2010). Our world as a
learning system: A communities-of-practice approach.
Social learning systems and communities of practice
(pp. 107-124). Springer, London.
Tymoshchuk, O., Almeida, A.M., Ramos, F., Pedro, L.,
Silva, P., & Renó, D. (2019). Mediação digital para a
inovação territorial: um estudo de caso múltiplo em
redes sociais digitais. Observatorio (OBS*), 13(4), 70-
Zhang, D. & Adipat, B. (2005). Challenges, methodologies,
and issues in the usability testing of mobile
applications. International journal of human-computer
interaction, 18(3), 293-308.
CHIRA 2022 - 6th International Conference on Computer-Human Interaction Research and Applications