Satisfaction, Self-management and Usability: Assessment of Two
Novel IT Solutions for Type 2 Diabetes Patients’ Empowerment
Vincenzo De Luca
1a
, Lutgarda Bozzetto
2b
, Clemente Giglio
2c
, Giovanni Tramontano
2d
,
Carlos Juan Chiatti
3e
, Fotis Gonidis
4f
, Strahil Birov
5g
, Ozan Beyhan
6h
, Simon Robinson
5i
,
Gorka Sanchez-Nanclares
7j
, Maria del Pilar López-Acuña
8k
, Adriano Fernandes
9l
,
Maria Triassi
1m
, Giovanni Annuzzi
2n
, Guido Iaccarino
10 o
and Maddalena Illario
1p
1
Dipartimento di Sanità Pubblica, Università degli Studi di Napoli “Federico II”, via S. Pansini 5, Naples, Italy
2
Azienda Ospedaliera Universitaria Federico II, via S. Pansini 5, Naples, Italy
3
Tech4Care Srl, Via Guglielmo Marconi 31, 60015 Falconara Marittima AN, Italy
4
Gnomon Informatics SA, Antoni Tritsi 21, 57001, Thessaloniki, Greece
5
empirica Gesellschaft für Kommunikations- und Technologieforschung mbH, Oxfordstr. 2, Bonn, Germany
6
Ministry of Health Turkey, Üniversiteler Mah. 6001. Cad. No. 9, Ankara, Turkey
7
Servicio Murciano de Salud, Central, 7 Edificio “Habitamia”- 5ª, 30100 Murcia, Spain
8
FFIS, Luis Fontes Pagán, 9. 1ª, 30003 Murcia, Spain
9
Misericordia of Amadora, Innovation Department, Estrada da Portela-Quinta das Torres, Amadora, Portugal
10
Dipartimento di Scienze Biomediche Avanzate, Università degli Studi di Napoli “Federico II”,
via S. Pansini 5, Naples, Italy
Keywords: ICT, Diabetes Mellitus, Management, Satisfaction, Usability, mHealth.
Abstract: The growing digitalization of health and care calls for the development of ICT tools and mHealth solutions
to monitor and control the patient’s health parameters and lifestyles. ProEmpower is a Pre Commercial
Procurement project aimed at procuring research and development services to develop innovative solutions
for patient empowerment and self-management of Type 2 Diabetes Mellitus. The project consortium
launched a call for tenders, articulated in 3 phases to select solutions. During Phase III, two solutions have
been selected to be tested by end-users: DM4All and DiaWatch. A pilot study has been carried out to
evaluate direct and indirect outcomes linked to the use of the novel solutions. Among these, we assessed the
post-intervention satisfaction, self-management and usability of the two novel solutions, using a 5-point
a
https://orcid.org/0000-0002-6115-931X
b
https://orcid.org/0000-0001-6549-4476
c
https://orcid.org/0000-0003-3121-7504
d
https://orcid.org/0000-0002-0441-699X
e
https://orcid.org/0000-0003-4810-9630
f
https://orcid.org/0000-0002-5605-4249
g
https://orcid.org/0000-0002-4575-0492
h
https://orcid.org/0000-0002-4337-3232
i
https://orcid.org/0000-0002-8572-3595
j
https://orcid.org/0000-0002-1797-9660
k
https://orcid.org/0000-0002-0986-3633
l
https://orcid.org/0000-0003-3644-7544
m
https://orcid.org/0000-0002-9420-9571
n
https://orcid.org/0000-0002-9324-6047
o
https://orcid.org/0000-0002-8997-835X
p
https://orcid.org/0000-0001-9834-6517
130
De Luca, V., Bozzetto, L., Giglio, C., Tramontano, G., Chiatti, C., Gonidis, F., Birov, S., Beyhan, O., Robinson, S., Sanchez-Nanclares, G., López-Acuña, M., Fernandes, A., Triassi, M.,
Annuzzi, G., Iaccarino, G. and Illario, M.
Satisfaction, Self-management and Usability: Assessment of Two Novel IT Solutions for Type 2 Diabetes Patients’ Empowerment.
DOI: 10.5220/0010395901300136
In Proceedings of the 7th International Conference on Information and Communication Technologies for Ageing Well and e-Health (ICT4AWE 2021), pages 130-136
ISBN: 978-989-758-506-7
Copyright
c
 2021 by SCITEPRESS – Science and Technology Publications, Lda. All rights reserved

Likert scale questionnaire. Users expressed a favourable opinion on both solutions, evaluating the
experience during the pilot phase as positive. DM4All results are better, however, the questionnaire
completion rate was higher in DiaWatch. Users appreciated DM4All for its usefulness in managing their
conditions.
1 INTRODUCTION
The growing digitalization of health and care,
combined with an increasing Information and
Communication Technologies (ICT) literacy of
citizens and patients, calls for the development of
tools and mobile-health (mHealth) solutions to
monitor and control the patient’s status and, in
general, health parameters and lifestyles. A new
generation of software, apps and algorithms for
managing patient’s data, has paved the way for the
implementation of new ways to collect and process
health-related data, helping both professionals and
patients to manage disease, improving the quality of
healthcare.
“Procuring innovative ICT for patient
empowerment and self-management of type 2
diabetes mellitus” (ProEmpower) is a Pre
Commercial Procurement (PCP) project, financed by
European Commission (EC)’s Horizon 2020
Programme (European Commission, 2019), aimed at
procuring research and development (R&D) services
to develop innovative ICT solutions for patient
empowerment and self-management of Type 2
Diabetes Mellitus (T2DM). The project involved
four public procurers across Europe (Turkey,
Portugal, Campania - Italy and Murcia - Spain) that
cooperated to develop detailed specifications, based
on user-centred approach (De Luca V, et al., 2019),
for new diabetes management processes supported
by fully integrated ICT solutions. As part of
ProEmpower PCP, the project consortium launched
a call for tenders, articulated in 3 phases to select
solutions. During Phase I, the technical, economic
and organizational feasibility of five alternative
solutions has been assessed. Phase II aimed to verify
the main characteristics of three prototypes. During
phase III two solutions have been tested by end-
users (patients and health professionals) enrolled by
healthcare organisations of the four procurers:
DM4ALL and DiaWatch (De Luca, V., et al., 2020).
DM4ALL digital platform includes web and
mobile interfaces along with intelligent medical
devices, able to support all the diverse needs of the
T2DM care pathway. Patients, Informal Caregivers,
and Healthcare professionals are able to manage,
communicate, and monitor the disease progression
through the system. Thus, this multi-pronged and
integrated approach promotes self-care practices and
continuous monitoring. DM4ALL is developed
based on the Shared Care Plan (SCP), a “document”
including information about lifestyles, treatment
plan, and disease-related markers. Furthermore, it
collects information and feedback from the patients
through validated questionnaires aiming at
increasing impact and personalization.
DiaWatch is a mHealth and telemedicine
solution to provide a more effective and
personalized T2DM management. DiaWatch
presents a sensing system platform, that operates
using a smartphone optionally integrated with other
devices such as a wristband, a glucose monitoring
sensor, a blood pressure meter and a scale. The
DiaWatch's Virtual Coach based on an artificial
intelligent system to profile the patient and make
appropriate recommendations for diabetes treatment,
exercises and healthy lifestyles. A patient personal
profile and related data-entry functions are
embedded in a SCP progressively updated with new
data from different sources. The desktop and mobile
interface for clinicians allows professionals to
monitor compliance to treatment and goals, to
communicate with patients (via textual messages,
audio and video features) directly from the
healthcare facility, and toidentify people at risk of
developing diabetes or acute conditions. DiaWatch
presents a social community tool for interaction,
communication and peer training. A cloud-based
platform ensures data exploitation for risk
prediction.
Here we show the results of a questionnaire
built to capture the patients’ opinions about
satisfaction, usability and acceptance of the
solutions.
2 METHODS
The aim of the pilot study was to test the feasibility,
effectiveness and usability of incorporating the two
solutions into the current care pathway for patients
with type 2 diabetes. Study objectives were to
evaluate direct and indirect outcomes linked to the
use of the novel solutions, including:
Satisfaction, Self-management and Usability: Assessment of Two Novel IT Solutions for Type 2 Diabetes Patients’ Empowerment
131

a) behavioural changes:
i. smoking habits;
ii. physical activity;
iii. steps;
iv. meals;
v. medication adherence;
b) clinical and quality of life (QoL) outcomes:
i. HbA1c;
ii. weight;
iii. blood pressure (BP);
iv. blood lipids;
v. cholesterol;
vi. quality of life;
c) satisfaction, self-management and usability.
The pilot study took place in all four pilot sites,
involving 50 participants for each solution per pilot
site, with a total of 400 participants. All participants
were adults with a previous diagnosis of type 2
diabetes mellitus (from recently diagnosed to long-
standing diabetes), that received insulin therapy
and/or diabetes medication. During the enrolment of
patients, the criteria in the Table 1 were taken into
consideration.
Table 1: Patients inclusion and exclusion criteria.
Inclusion criteria
Type 2 Diabetes diagnosis
Aged between 45 and 79
Ability to provide written
informed consent
Exclusion criteria
Chronic renal replacement
therapy (Haemodialysis,
peritoneal dialysis or
trans
p
lantation
)
History of active malignancy
within the last 12 months
Pre
g
nanc
y
Chronic viral he
p
atitis
HIV infection
Eligible participants were offered the opportunity to
use one solution as part of their care pathway and
underwent evaluations by a research assistant at
baseline (enrollment) and post intervention.
The phase III of ProEmpower was carried out from
July 2019 to July 2020(Figure 1).
Figure 1: ProEmpower Phase III timeline.
An eleven-questions survey was administered post-
intervention to patients for the assessment of
satisfaction, self-management and usability (Nielsen,
J., 1994). In order to build the questionnaire we used
a 5-point Likert scale as a psychometric scale to
assess patients’ opinions regarding the two novel
solutions (Joshi, A., 2015)(Sullivan, G.M., 2013) for
the T2DM self-management. The consortium
developed the questionnaire based on the
Questionnaire for User Interaction Satisfaction
(QUIS7) instrument (Harper, B.D., 1993). The
questionnaire was implemented as an online survey
and administered to the patients taking part in the
four pilots. The following elements have been
considered for each category:
a) Satisfaction
 Experience with the solution during the
pilot;
 Satisfaction with the overall solution;
 Ability to get along with diabetes;
 Worth the effort involved.
b) Self-management
 Usefulness in managing the patient’s
condition;
 Self-management quality improvement;
 Overall fit with the way of life.
c) Usability
 Usability from terrible to wonderful;
 Usability from frustrating to satisfying;
 Usability form dull to stimulating;
 Usability from difficult to easy.
3 RESULTS
3.1 DM4all
The DM4all local pilot managers distributed the
questionnaire to the patients participating in the pilot
study. During July 2020, they were able to collect
the answers of n=56 patients (from all the 4 pilot
sites).
The experience with the DM4All during the testing
period is considered positive by 46.4% of the
patients and 33.9 % of them considered that it as was
very positive (Figure 2).
ICT4AWE 2021 - 7th International Conference on Information and Communication Technologies for Ageing Well and e-Health
132

Figure 2: DM4All Satisfaction survey, Question 1.
The vast majority of interviewed patients were fairly
satisfied (58.9%) or very satisfied (25%) with it
(Figure 3).
Figure 3: DM4All Satisfaction survey, Question 2.
The results regarding the way that the participation
in the project affected the patients ability to manage
their condition on a day-to-day basis were also very
encouraging: 46.4% of the patients considered that
by participating in the study they increased a little
bit their ability to manage their condition; and 26.8%
considered that that by participating in the study
they increased a lot their ability to manage their
condition (Figure 4).
Figure 4: DM4All Satisfaction survey, Question 3.
Using the DM4All system, 50% of the patients
considered that effort is mostly worth it and, more
importantly, 33.9% of the patients considered that
effort as totally worth it (Figure 5).
Figure 5: DM4All Satisfaction survey, Question 4.
When questioned about the usefulness of the
DM4All system in the management of their clinical
condition, 47.3% of the patients agreed that it is
useful while 36.4% of the patients strongly agreed
that is useful (Figure 6).
Figure 6: DM4All Self-management survey, Question 1.
The patients agreed with the fact that the DM4All
system increased the quality of the self-management
of their condition, 55.4% of them agreed and 28.5%
strongly agreed with that sentence (Figure 7).
Figure 7: DM4All Self-management survey, Question 2.
Satisfaction, Self-management and Usability: Assessment of Two Novel IT Solutions for Type 2 Diabetes Patients’ Empowerment
133

The patients agreed with the fact that the DM4All
system fitted with their way of living (78.6 %)
(Figure 8).
Figure 8: DM4All Self-management survey, Question 3.
Regarding the way that the patients perceive the
DM4All system: 30.4 % considered it as being
wonderful; 36.4 % considered it as being satisfying;
33.9% considered it as being stimulating; more than
39% considered it as being easy (Figure 9).
Figure 9: DM4All Usability survey.
Overall, we can say that the patients that were
inquired and answered the questionnaire were very
happy with the DM4all system in terms of usability,
acceptance and satisfaction.
3.2 DiaWatch
The DiaWatch local pilot managers collected the
answers of n=158 patients (from all the 4 pilot sites).
A good majority of patients (69.0%) reported a
positive or very positive experience with DiaWatch,
whereas only few people (3.8%) indicated a negative
experience (Figure 10).
Figure 10: DiaWatch Satisfaction survey, Question 1.
The general satisfaction with the DiaWatch system
was high, with 61.2% of patients indicating to be
fairly or very satisfied with it. Dissatisfaction with
the solution (i.e. being fairly or very dissatisfied)
was reported only in few cases (Figure 11).
Figure 11: DiaWatch Satisfaction survey, Question 2.
Answers from users emphasized a very positive
perception of the DiaWatch impact on self-
management of health condition. In total, 61.2% of
patients reported a little or big increase of individual
ability, whereas 38.6% said that the system did not
have a substantial impact. No negative answers were
given (Figure 12).
Figure 12: DiaWatch Satisfaction survey, Question 3.
ICT4AWE 2021 - 7th International Conference on Information and Communication Technologies for Ageing Well and e-Health
134

In line with previous results, the general feeling by
patients was that the DiaWatch solution is certainly
worth to be used. 67.7% of all respondents said that
the solution is either mostly or very much worth to
be used, whereas only 5% indicated a negative
answer (Figure 13).
Figure 13: DiaWatch Satisfaction survey, Question 4.
More than two thirds of users (70.0%) confirmed
that they found DiaWatch useful for the
management of their condition (Figure 14).
Figure 14: DiaWatch Self-management survey, Question 1.
In terms of quality of self-management, overall
DiaWatch helped patients to increase it. 64.7% of
users agreed or strongly agreed with this statement,
whereas a minority (14.6%) disagreed or strongly
disagreed (Figure 15).
Figure 15: DiaWatch Self-management survey, Question 2.
Finally, a good majority of users (64.9%) reported
that DiaWatch fits well their way of living (agree or
strongly agree) (Figure 16).
Figure 16: DiaWatch Self-management survey, Question 3.
The perceived usability of the DiaWatch system is
fair: 22.9% considered it as being wonderful; 25.8%
considered it as being satisfying; 15.4% considered
it as being stimulating; 19.4% considered it as being
easy (Figure 17).
Figure 17: DiaWatch Usability survey, Question 1.
4 CONCLUSIONS
The collected data regarding each question for each
Satisfaction, Self-management and Usability: Assessment of Two Novel IT Solutions for Type 2 Diabetes Patients’ Empowerment
135

solution are quite similar but DM4ALL is generally
represented better. However, it must be considered
that the rate of questionnaire completion was higher
in DiaWatch. Both solutions created a positive
experience during the testing phase and increased
patients’ ability to manage their condition. For
DM4All, the scores varied between 4.04 and 4.18,
while for DiaWatch, the scores varied between 3.5
and 4.01. DM4All was appreciated mostly for its
usefulness in managing patients’ condition, and
users considered that the effort involved in using
DiaWatch was worth it (Figure 18).
Figure 18: Summary graph of the results of the patient’s
questionnaire.
ProEmpower solutions were developed
iteratively, taking into account user-centered design.
User-friendliness of the interfaces for digital health
solutions play a key role in ensuring adherence. The
two solutions developed in ProEmpower are not yet
mature for large-scale adoption. Building the key
elements for usability since early stages of solutions
design may be related to moderate adaptations,
including meanings and formats, or extensive
adaptations with changes such as the removal of
items or the addition of more sophisticated
functions. Adequate further testing needs to be
ensured for any changes, to collect patient’s
feedback.
This is particularly important considering the
burden of the disease these patients face every day
dealing with a chronic and severe condition such as
type 2 diabetes. These solutions may help reducing
this burden and improving quality of life. Our results
are encouraging, although further study will be
required, to assess correlations between specific
features and outcomes.
ACKNOWLEDGEMENTS
The ProEmpower project has received funding from
the European Union’s Horizon 2020 research and
innovation programme under the Grant Agreement
No. 727409.
REFERENCES
European Commission, 2019. Pre-commercial Procure-
ment: Driving innovation to ensure sustainable high
quality public services in Europe. COM(2007) 799
final.
European Commission, 2019. Horizon 2020 Work
Programme 2018-2020. European Commission
Decision C(2019)4575.
De Luca, V., et al., 2019. European Specifications for
Value-based Pre-Commercial Procurement of
Innovative ICT for Empowerment and Self-
management of Diabetes Mellitus Patients.
Proceedings of the 5th International Conf. on
Information and Communication Technologies for
Ageing Well and e-Health (ICT4AWE 2019), pages
19-27.
De Luca, V., et al., 2020. Developing a Digital
Environment for the Management of Chronic
Conditions: The ProEmpower Experience of a Horizon
2020 PCP for Type 2 Diabetes. In: Ziefle M.,
Maciaszek L. (eds) Information and Communication
Technologies for Ageing Well and e-Health.
ICT4AWE 2019. Communications in Computer and
Information Science, vol 1219. Springer, Cham.
https://doi.org/10.1007/978-3-030-52677-1_1.
Nielsen, J., 1994. Usability engineering. Morgan
Kaufmann.
Joshi, A., et al., 2015. Likert Scale: Explored and
Explained. https://doi.org/10.9734/bjast/2015/14975.
Sullivan, G.M., et al., 2013. Analyzing and Interpreting
Data from Likert-Type Scales. J. Grad. Med. Educ. 5,
541–542. https://doi.org/10.4300/JGME-5-4-18.
Harper, B.D. and Norman, K.L. 1993. Improving User
Satisfaction: The Questionnaire for User Interaction
Satisfaction Version 5.5. Proceedings of the 1st
Annual Mid-Atlantic Human Factors Conference, (pp.
224-228), Virginia Beach, VA.
Aiyegbusi, O.L., 2020. Key methodological considerations
for usability testing of electronic patient-reported
outcome (ePRO) systems. Qual Life Res 29, 325–333.
https://doi.org/10.1007/s11136-019-02329-z.
ICT4AWE 2021 - 7th International Conference on Information and Communication Technologies for Ageing Well and e-Health
136