An Ontology based Game Platform for Mild Cognitive Impairment
Rehabilitation
Christos Goumopoulos and Ioannis Igoumenakis
Information and Communication Systems Engineering Department, University of the Aegean, Greece
Keywords: Serious Games, Elderly, Mild Cognitive Impairment, Ontology, Knowledge Engineering, Rehabilitation,
Usability Study.
Abstract: In this paper a new ontology based game platform for maintaining and recovering cognitive functions in the
case of mild cognitive impairment is discussed. Leveraging on a knowledge base that specifies game rules
and organizes training resources such as words, images and sounds in terms of categories, relationships and
properties, a system is developed for the efficient creation of new exercises. The ontology model by
organizing the material required in the patient rehabilitation process, reduces the workload of domain experts
in terms of developing the practices to be applied for a subject. Employing an ontology, the main requirement
is to create a model for each exercise type, whereas the platform will be responsible to synthesize the exercises
by instantiating the models with the suitable resource objects automatically. In addition, the system can use
the ontology to create and manage exercises by selecting the appropriate level of difficulty based on the
patient's previous performance, skills and preferences. The design of the ontology and the architecture of the
overall game platform are presented while experiences from a pilot evaluation study assessing the perceived
usability of the game platform by elderly users are reported.
1 INTRODUCTION
Mild cognitive impairment (MCI) is a state of a
cognitive performance below of what is expected for
an age and an educational level, but above a
pathological level (APA, 2011). MCI is one of the
early symptoms of Alzheimer’s disease characterized
by significant memory impairment that does not,
however, meet the criteria for dementia (Sperling et
al., 2011). MCI patients may forget important
information previously recalled, such as meetings,
conversations, or recent events but continue to exhibit
normal functional activities. MCI is also linked to the
impairment of other aspects of cognitive function,
such as attention, language, visual or executive
function, including the inability of a person to make
the right decisions, judge the time or sequence of
steps required to fully execute a complex task (Langa
and Levine, 2014).
Long-term studies show that on a global scale 15-
20% of people aged 65 and over may develop MCI
(Hu et al., 2017). Besides ageing, a wide spectrum of
diseases and clinical conditions are related to MCI. A
review of 41 cohort studies with a maximum ten-year
follow-up showed that, on average, 32% of people
with MCI will develop dementia (Mitchell and Shiri-
Feshki, 2009). In the case of cardiovascular diseases
(stroke, heart attack, etc.), in addition to motor
deficiency, patients develop cognitive and affective
disorders (Abete et al., 2014). The relation of MCI to
Parkinson's disease has been similarly examined
(Janvin et al., 2006). Aphasia, a non-amnestic single
cognitive domain MCI, does not affect memory and
patients still reason as normal but are unable to
communicate their thoughts (Seniów et al., 2009).
Given the increase of the proportion of older
people in the world population as well as the sharp
increase in the survival rate of patients with accute
diseases which, however, affect their cognitive
functions, the importance of developing MCI
prevention and rehabilitation tools is obvious. In
recent years the interest in cognitive rehabiliation has
led to the discovery of pathogenetic mechanisms of
cognitive impairment and the development of new
approaches to the recovery of neurons of the brain
(Carelli et al., 2017).
MCI causes cognitive changes that are severe
enough to be perceived by the ailing individuals or
their relatives. However, the neurological symptoms
complexity makes it difficult to choose appropriate
130
Goumopoulos, C. and Igoumenakis, I.
An Ontology based Game Platform for Mild Cognitive Impairment Rehabilitation.
DOI: 10.5220/0009793501300141
In Proceedings of the 6th International Conference on Information and Communication Technologies for Ageing Well and e-Health (ICT4AWE 2020), pages 130-141
ISBN: 978-989-758-420-6
Copyright
c
2020 by SCITEPRESS – Science and Technology Publications, Lda. All rights reserved
rehabilitation tools and requires a multidisciplinary
approach to address them. Traditionally, the most
common approach to support MCI rehabilitation is to
give patients written exercises that they have to work
out of - almost as with school work. This on the one
hand makes it quite difficult for the specialist to keep
records of each patient individually and on the other
hand, it makes the procedure quite tedious for the
patients, who receive similar exercises, causing their
displeasure their reluctance to continue the process.
To this end, neuropsychologists are now
increasingly using information and communication
technologies, including serious games (Rodríguez-
Fórtiz et al., 2016). Digital applications can provide a
great alternative with respect to traditional methods,
as they allow the patient to practice cognitive
functions with rich media, such as sounds and images.
A rehabiliation tool capable of managing such types
of training material can generate different and
operative experiences supporting the recovery
process of the patient, while also recommending
novel exercises that are difficult to realize with hard
copy approaches.
In this paper a new ontology based game platform
for MCI recovery is presented. Leveraging on a
knowledge base that specifies game rules and
organizes training resources such as words, images
and sounds in terms of categories, relationships and
properties allows for the automatic creation of new
exercises. By organizing the material required in the
patient rehabilitation process, the ontology model
reduces the workload of domain experts in terms of
developing the exercises to be applied for a subject.
Employing an ontology, the main requirement is to
create a model for each exercise type, whereas the
platform will be responsible to synthesize the
exercises by instantiating the models with the suitable
resource objects automatically. In addition, via the
ontology the system can adapt exercises by selecting
the appropriate difficulty level based on the patient's
previous performance, skills and preferences.
The rest of the paper is organized as follows.
Section 2 presents related work. Section 3 discusses
the game platform and ontology design and the details
of system development. Section 4 presents evaluation
in terms of assessing system usability and presents
future work. Finally, our conclusions are given.
2 RELATED WORK
Experimental results show that cognitive as well as
physical exercise help develop new brain neurons,
which reduce the impact of dementia (Carelli et al.,
2017). Research has shown that brain training should
target specific brain activities (Tong et al., 2017).
Specifically these activities are short-term (new
information) and long-term (information retention)
memory, switching between different tasks, word or
object recognition and finally chronological and
spatial placement.
In recent years, several serious games have been
proposed focusing on various stages of cognitive
damage (McCallum and Boletsis, 2013; Rodríguez-
Fórtiz et al., 2016). The main purpose of these games
is to delay the onset of symptoms of the disease. The
use of these games can also lead to improved quality
of life for patients, as it helps them to maintain their
autonomy and social relationships. Although MCI is
characterized as a cognitive dysfunction, support for
both physical and social activities through games has
been shown to contribute to delayed cognitive
impairment and / or recovery, especially when
combined with cognitive activities (Karp et al., 2006;
Chartomatsidis and Goumopoulos, 2019).
A game suite, called Tapbrain, was developed for
smart mobile devices (Kang et al., 2016). The game
tool includes 17 mini-games, 13 to stimulate brain
activity and 4 games to induce physical activity.
Games are divided into six categories: four targeting
brain exercise and two targeting physical activity.
The games that stimulate brain exercise are divided in
the following cognitive areas: memory, attention,
problem solving and decision making. All the games
have 5 levels of difficulty, while the first level
operates as the educational level.
A similar tool developed for cognitive
rehabilitation in tablet devices is Padua Rehabilitation
(Cardullo et al., 2015). It consists of 35 exercises
divided into 7 cognitive areas: attention, memory,
language, logic, identification, orientation and motor
control. The shapes of the objects used in the
application are the basic geometric shapes avoiding
complicated images and vivid colors. In order to
minimize mistakes, each exercise starts from a very
easy level and with the increase of playing the degree
of difficulty increases. In this way, participants can
easily finish the first level by gaining confidence and
a positive attitude to continue using the application.
Garcia-Betances et al. (2015) presented effective
strategies with virtual reality applications and their
use in cognitive rehabilitation or in the daily life of
people with MCI or dementia. They presented a
detailed overview of strategies in cognitive
rehabilitation and proposed methodologies and
development phases either from a psychologist or an
application developer perspective in order to achieve
the maximum rehabilitation results.
An Ontology based Game Platform for Mild Cognitive Impairment Rehabilitation
131
Quaglini et al. (2009) suggested the use of a
database and a graphical user interface to support the
creation of new games and their objects with a focus
on customizing them to the user needs as cognitive
exercises. A follow up study by Leonardi et al. (2011)
proposed that such games can be automatically
created using an ontology, which models the games
and their objects. A software tool that creates games
and their corresponding objects which are specified in
an ontology and are targeted to patients afflicted by
Parkinson’s disease was presented by Alloni et al.
(2015). The ontology, however, in this case encodes
only relationships between the game objects and does
not support modeling of game rules or inference
operations at the runtime (porting of the ontology to
OWL specification is suggested as a future work).
The proposed approach shares similar goals with
the related work and embraces the perspective of
developing personalized serious games for the
rehabilitation of impaired cognitive functions. It
follows, however, a different approach by modelling
not only the static relationships of game resources but
also game rules that help to combine the knowledge
of the ontology with the corresponding dynamic
knowledge contained in these rules. Therefore,
personalized game activities can be defined via rules
according to user conditions and preferences. Finally,
the platform functionality is accessible through HTTP
protocol for client applications that can start at any
place (home, care center) and on any device.
3 THE COGNIPLAT PLATFORM
3.1 General Objectives
The COGNIPLAT platform is an innovative
cognitive impairment rehabilitation tool that is
developed for assisting elderly who have MCI but
have not yet developed dementia. It is built based on
a multi-disciplinary approach combining theories of
neuropsychology, cognitive linguistics and speech
therapy organized in six domains, one diagnostic and
five training domains focused on enhancing cognitive
functions through different game exercises. In
addition, the platform has been designed to
automatically adjust the complexity and type of
exercises by adapting the cognitive requirements of
the games to the characteristics of each patient.
The diagnostic domain aims to evaluate, through
a specific set of exercises, the degree of impairment
of the patient's cognitive functions in order to
determine the difficulty level of the exercises to start.
The remaining areas focus on the reinforcement
of different cognitive functions through exercises.
The first cognitive domain is focusing on memory and
includes exercises that focus on recalling the position
of objects and patterns, a sequence of numbers,
elements of associative knowledge etc. The second
cognitive domain focuses on attention with the use of
exercises that request, for example, to replace words
or phrases with images or image sequences and
identifying objects displayed on the screen. The third
cognitive domain focuses on enhancing the
perception ability with the use of exercises that
require some kind of orientation or determining the
location of an object with respect to another. The
fourth cognitive domain focuses on reasoning and
problem solving with exercises such as solving
arithmetic crosswords and selecting the right pattern
to reasonably complete a given sequence. The fifth
cognitive domain (not currently available) is focusing
on utterance and includes exercises that concentrate
on the mobilization of organs supporting utterance
(i.e. lips, tongue, cheeks and jaw).
An innovation of the proposed platform is the use
of the propositional-frame method, which aims at
restoring the impaired relational cognitive
connections that help a person navigate the
surrounding world. The basis of this method is the
assertion made by cognitive linguistics that one
person thinks reasonably, associating one
phenomenon with another. An effective tool in this
regard is the use of derivative words that constitute
the majority of vocabulary in any language.
Derivative words are constructed by analogy from a
verbal basis, and it is this property that facilitates their
preservation in a person's memory. The use of an
ontology that semantically describes this word
formation process is foreseen in the COGNIPLAT
platform to restore speech in older adults with MCI.
3.2 MCI Rehab Games
Currently the following games have been realized
(the targeted cognitive domain is also indicated):
1. Anagram: solving a word puzzle (reasoning)
2. Antonyms&Synonyms: finding word
antonyms/synonyms (memory)
3. Calculation: solving arithmetic crosswords
(reasoning)
4. ChronologicalOrder: placing shuffled images in
chronological order in order to create a brief
story (reasoning).
5. FindThePattern: remembering a pattern of
highlighted boxes appeared shortly in the context
ICT4AWE 2020 - 6th International Conference on Information and Communication Technologies for Ageing Well and e-Health
132
of a background screen of several square boxes
(visual memory).
6. FindTheSound: listening to sounds and selecting
the corresponding image (acoustic memory).
7. Labyrinth: finding the exit from a labyrinth
(perception).
8. LogicalOrder: selecting the right pattern to
reasonably complete the given sequence
(reasoning).
9. MemoryCards: revealing pairs of alike pictures
(memory).
10. NumberOrder: ordering a random sequence of
numbers in increasing sequence (reasoning).
11. Observation: counting specific types of objects
given a set of discrete images shown on the
screen (attention).
12. Outsider: finding an object that does not match
with the rest (attention).
13. Puzzle: solving a photo puzzle (attention).
14. Quiz: recalling knowledge in various categories
such as history, geography, food, etc. (memory).
15. Suitcase: placing a new object in the correct
position so that the suitcase closes without
collisions with existing objects (perception).
The patients may use the platform at home either
independently or together with a family member or a
caregiver in case of severe cognitive impairment. The
user interface design has been developed taking into
account the characteristics of the elderly which call
for simplicity, clarity, consistency and adaptability to
the skills of each individual (Gerling et al., 2012). Fig.
1 shows sample screens regarding two of the games.
Figure 1: Snapshots from FindTheSound (top) and Outsider
(bottom) games.
Fig. 2 shows, as another example, the user
interface for the ChronologicalOrder game. The user
has to drag picture pieces to the correct time space to
create a meaningful story. The game has three
difficulty levels, i.e. easy, medium and advanced
levels which are associated to a story with 3, 4 and 6
picture pieces. The time sequence of a 3-pieces story
is defined from the top of the screen downwards. The
time sequence of 4-pieces or 6-pieces story is defined
from left to right and top to bottom.
Figure 2: An instance of the ChronologicalOrder game.
Figure 3: Example of a 3D scene used in the games.
An Ontology based Game Platform for Mild Cognitive Impairment Rehabilitation
133
Some games are planned to use 3D
representations of physical objects and scenes (Fig.
3). The human brain is used to managing 3D images,
so it is more natural to interact with 3D objects than
flat images or text. So using a more natural
representation of objects seems a better idea and more
efficient in terms of rehabilitation. In addition, 3D
models give the patient the ability to interact more
fully with them, but also allow the rehabilitation
specialist to control every aspect of their appearance.
3.3 Ontologies
Ontologies are used in Knowledge Engineering for
modelling a domain in a structured representation
form containing its clear definition (Gomez-Perez et
al., 2006). To make this modeling correctly, all
entities, properties, restrictions or other assumptions
that exist in the domain of interest are specified. This
specification must be modeled in a language
comprehendible by both humans and machines
(Baader et al., 2004).
There are many formal languages that can be used
to define and construct ontologies, with the aim of
codifying the targeted knowledge in a simple and
formal way. However, the most popular approach
involves the use of the RDF (Resource Description
Framework) standard and the OWL (Ontology Web
Language) language (Welty et al., 2004). An
ontology can, therefore, be formally described in
OWL by using classes, objects, and properties as key
attributes. These elements are used to describe
concepts, instances or members of a class, as well as
relationships between objects of two different classes
(object properties), and relationships that link objects
to data types (data properties). OWL supports also
the expression of equivalences or assumptions in the
form of axioms that combine entities and
relationships. In this work, the OWL 2 DL
(Description Logic) profile was used to specify the
ontology models.
On the other hand, the existence of rules helps to
combine the knowledge of an ontology with
corresponding dynamic knowledge contained in these
Rules. a Rule-based System Usually Contains a Set of
"if-then" Rules That Indicate the next Action to Be
Made, Depending on the Current Situation, and Also
a Rule Mechanism to Implement Them. Thus, using
a Set of Rules Makes It Possible to Express the
Behavior of Individuals within a Domain, Thereby
Providing New Insights for These Individuals and,
Consequently, Personalized Services (Skillen Et Al.,
2014).
in This Context, the SPARQL (SPARQL Protocol
and RDF Query Language) Language Was Used to
Query the Knowledge Base of the Ontology and to
Describe Rules That Encode Expressions over the
User Profile Instances. despite Being a Query
Language, SPARQL Provides Extensive Power to
Guide the Provision of Personalized Services by
filtering Persons with Certain Characteristics
(Applying the FILTER NOT EXISTS Construct),
Asserting New Facts (Applying the CONSTRUCT
Expression), Updating Data in the Ontology
(Applying the UPDATE Expression), Etc. Therefore,
Personalized Game Activities Can Be Defined via
SPARQL Rules According to User Conditions and
Preferences.
Protégé (V. 5.5.0) Ontology Editor Was Used to
Create the Ontologies. Protégé Is a Software Tool
That Enables the Creation of an OWL Ontology and
Supports the Capabilities of the OWL Language.
Protégé Supports the Definition of a Hierarchy of
Entities-Classes as Well as the Ability to Create
Relationships-Restrictions via a Handy Graphical
User Interface Instead of Directly using the OWL
Language. It Also Integrates Various Reasoners to
Control the Consistency of the Ontology as Well as
for Inference Purposes. in This Work the Pellet
Reasoner, an Open Source Java OWL-DL Reasoner,
Was Chosen. Finally, It Provides Useful Ontology
Information Such as the Set of Axioms, I.E. the Self-
Proven Values, Which Exist in the Ontology.
for a Selected Entity, Protégé Lists the
Relationships with the following Syntax Which Is
Adopted in the next Sections:
<Relationhip-Name> <Cardinality>
<Cardinality Value (Optional)> <Range of
Relationship>
Where <Cardinality> May Be One of the
following Keywords: Min, Max, Exactly, Only.
the following Notation Is Also Used for
Expressing Relationships: OP Denotes
Objectproperty Relationships, DP Denotes
Dataproperty Relationships and Obj Denotes All
Entities That Are Objects.
for Example, the following Restriction:
Hasmathoperation (OP) Max 5 Mathoperator (Obj)
,
Is Interpreted as the Hasmathoperation Relationship
Is an Objectproperty with Max Cardinality the
Number 5 of the Mathoperator Entity-Object.
3.3.1 User Profile
In COGNIPLAT platform, there is a need for
describing, in addition to the static user traits,
dynamic aspects that relate to specific game activities
ICT4AWE 2020 - 6th International Conference on Information and Communication Technologies for Ageing Well and e-Health
134
in which the user engages. The User Profile (UP)
uses temporary sub-profiles of a user profile in order
to encode game activity related user preferences.
In UP ontology, a user’s Profile has a permanent
sub-profile which contains static information about
the user and a set of temporary sub-profiles. The
PermanentSubProfile contains GeneralInformation
(e.g. name, age, gender, height and weight), Likes,
Dislikes, Disabilities, ContactInformation,
SocialInformation (e.g. friend and family_member),
Possessions (e.g. object, and living_thing), and
HealthInformation (e.g. medication, diseases,
allergies, etc.). Each TemporarySubProfile contains
Preferences (e.g. privacy_preference,
interaction_preference,
environmental_condition_preference, etc.) which are
associated with the GameActivity that a User carries
out. Moreover, each temporary sub-profile depends
on the UserContext which contains the Location of
the user, the Time in which the activity is carried out,
the State of the user, which means whether the user is
“alone” or “with-friends”, and the user’s Mood. Fig.
4 depicts the relations between Person, Profile,
TemporarySubProfile and PermanentSubprofile and
the properties of each sub-profile. In particular, a
Person has exactly one permanent sub-profile and
several temporary sub-profiles.
Figure 4: Visualisation of a part of the UP Ontology.
Leveraging on UP data the game platform can
select favourably game resources that are aligned to
the stored preferences of each patient (e.g. favourite
sports, food, hobbies, animals, music, etc.). This
personalization of the exercises can increase user
engagement and platform acceptability. Similarly, the
declared educational level can adjust the exercise
parameters (e.g. completion time, performance
threshold to change the difficulty level) and exercise
plans (e.g., minimize the use of certain games)
according to the patient capabilities.
3.3.2 Game Rules
A basic goal of this ontology is to represent also the
knowledge of the game rules and their constituent
resources so that game instances can be automatically
created by an application server. Also, the ontology
should be easily extensible in order to specify new
games following a specific pattern. The ontology also
enables the correlation between resources, resulting
in a greater variety of entities that can be associated
to the rules of the game. The ability to associate
entities with one another can lead to new games, such
as finding the sound that can characterize a word.
After analyzing the logic of the fifteen MCI rehab
games some basic properties have been identified:
There are different difficulty levels for a game;
There is a time limit to play a game;
There is a maximum number of repetitions for a
game;
After a number of correct answers the difficulty
level of a game increases;
A game is associated with resources that can be
words, operators, blocks, puzzle pieces,
questions, images, videos and sounds
.
Therefore, the two main entities of the Games
ontology are:
1. Game, this is the basic entity associated with
the basic properties of a game such as the difficulty
level. Every game that is modeled must have this
entity as a base class.
2. Resource, this is the base class entity of each
resource that is modeled. The only common feature
among different resources is a unique id.
These entities are independent of each other, i.e.
a Game entity does not intersect with a Resource
entity and consequently an individual Game cannot
be a Resource.
The Game entity has the following data
properties:
1. maxCompletionTime (DP) value n, specifying
the time allowed to complete the exercise which can
vary depending on the difficulty level of the game.
The value is an integer corresponding to a time
measured in seconds.
2. completedDate (DP) value xsd: string,
specifying the end date of the current game.
3. hasDifficulty (DP) value {"EASY",
"MEDIUM", "HARD"}, specifying the difficulty of
the game.
4. hasGameId (DP) value xsd: string, specifying
a unique identifier of the individual game. The syntax
of such an identifier is: <game name>_<difficulty
level>_<player name>_<round number>.
An Ontology based Game Platform for Mild Cognitive Impairment Rehabilitation
135
5. hasLevel (DP) value xsd: string, specifying the
current level of the game.
6. hasPlayer (DP) value xsd: string, specifying
the player's alias.
7. isCompletedIn (DP) value xsd: positiveInteger,
specifying the time in secs for actually completing the
current game.
For each developed game an entity is defined in
the ontology. As an example, the entity in Fig. 5
describes the game of associating images to sounds
(FindTheSound). The number of images displayed in
this game increases with respect to the difficulty
level. The restrictions (object properties) defined in
the ontology for the FindTheSound Game entity are
as follows:
1. hasImage (OP) exactly n ImageSound (obj),
specifying the number of images to be used as options
with the associated sound.
2. hasSound (OP) exactly 1 Sound (obj),
specifying the sound the user hears with the
associated image.
Figure 5: FindTheSound Game entity in the ontology.
Similarly, for each resource used in the games an
entity is defined in the ontology. Fig. 6 describes, as
an example, the image class. Each image is associated
via the OP relationships hasTitle and hasSubject with
a couple of Word entities. In this way, an image can
be categorized based on the hasSubject relationship
and consequently rules can be defined that require
images from a specific category. Also, in this manner,
games that focus on a Word entity can contain rules
with corresponding images, such as finding opposite
emotions. The Image entity is analysed into two
subcategories:
1. OrderedImage, describing one image and its
association with another image via the OP
relationship hasPreviousImage. It was observed in
the developed games that each image may have at
most one precursor, which actually characterizes the
property as Functional (i.e. any entity that belongs to
the domain of this relationship can be associated at
most with one entity, which belongs to the range of
the relationship).
2. ImageSound, describing the association of an
image with a sound via the OP relationship
hasAssociatedSound.
Figure 6: Image Resource entity in the ontology.
The ontology contains also relationships that
associate games with resources and resources with
other resources. A subset of these relationships is
provided next in the following format:
relationship(D: Domain, R: Range):
hasAntonym(D:Word, R:Word)
hasSynonym (D:Word, R:Word)
hasAssociatedSound(D:Image, R:Sound)
hasAssociatedImage(D:Sound, R:Image)
hasBlock(D:BlockSet or Calculation or Labyrinth
or HidingBlocks or LogicalOrder, R:Block)
hasBlockSet(D:LogicalOrder, R:BlockSet)
hasCategory(D:Question, R:Word)
hasChoice(D:Question, R:Word)
hasConnectingPiece(D:Piece, R:Piece)
hasOperator(D:Calculation, R: OperatorBlock)
hasObservation(D:Observation,
R:ObservationObj)
hasPiece(D:Puzzle, R:Piece)
hasPreviousImage(D:OrderedImage,
R:OrderedImage)
hasSubject(D:Image or Sound, R:Word)
hasTitle(D:Image or Sound, R:Word)
The Games ontology developed currently
contains 103 entities with 36 OP relationships and 46
DP relationships. Fig. 7 shows the reported metrics of
the developed ontology. The complete ontology is
available in the Github repository
(https://github.com/Binarios/MciOntology).
Figure 7: MCI Rehab Games Ontology metrics.
ICT4AWE 2020 - 6th International Conference on Information and Communication Technologies for Ageing Well and e-Health
136
3.4 Game Generator
In this section the back end of the COGNIPLAT
platform which generates game instances for MCI
rehab client applications is described. The ontology is
accessed by a server, which reads the stored rules and
then automatically generates game instances that are
appropriate for the specific clients. One of the
difficulties was the discovery of the available games
by a MCI rehab client application as the latter has a
different life cycle than the ontology. This discovery
is achieved as all the games are available under the
provision of a server. This makes it possible to
communicate with the client application from any
device to access the available games at any time,
without the intervention of a third party, such as
medical personnel. The interaction sequence to
discover the games is shown graphically in Fig. 8.
Figure 8: Interaction for the discovery of available games.
Fig. 9 illustrates the system architecture. The
COGNIPLAT platform consists of one Tomcat server
for running game services and a second Tomcat-
based server hosting: a) the ontology model, which
must be loaded at startup into the memory of the
application server; b) resource files, such as images
and sounds, which are needed by the games; and c)
the knowledge base with the game rules. For the
latter, the Apache Jena Fuseki server is employed.
This server supports the SPARQL Protocol and
serves as an ontology provider hosting the knowledge
base. It enables SPARQL queries to be executed on
the knowledge base, thereby enabling immediate
access to the ontology. The accompanied Apache
Jena, provides three different APIs. The RDF API
supports the creation and reading of RDF triplets.
Another API deals with Apache Fuseki functionality
as well as the dynamic management of the knowledge
base. Finally, the OWL API offers tools and methods
to create and access rules in the ontology. The OWL
API also provides access to the Pellet reasoner
enabling flexibility to the game application by
exploiting inferred knowledge on demand.
The COGNIPLAT platform embraces the
microservices architectural style to deploy game
applications as lightweight services that can be
independently developed, tested, deployed, operated,
scaled, and upgraded (Newman, 2015). Typically,
microservices communicate with each other via
HTTP, but in this work the dependency injection (DI)
design pattern was used in an attempt to mimic
microservices. This design pattern increases the
decoupling between classes of an application thus
achieving greater isolation between them, easier
testing of their functionality as well as support for the
microservices architecture (Prasanna, 2009).
Figure 9: COGNIPLAT platform architecture.
Therefore, in COGNIPLAT platform each game
is implemented as a separate service (Fig. 10). Each
service contains the logic that describes which
resources will be used for the game as well as whether
an object needs to be created, and what its value will
be according to the constraints specified in the
ontology. Also every service is accessible remotely
through an API gateway.
Besides game services there is the ontology
service that is responsible for reading the rules of each
game and communicating with the knowledge base to
create the requested resources. The ontology service
hides from the game services the ontology structure
and storage. In this way it is possible to change the
ontology technology without disturbing the rest of the
system.
During the platform bootstrap, the owl schema is
downloaded from the Apache Fuseki server and is
loaded into the memory. Following the ontology
model load, each game-microservice has access to the
rules of the schema and can generate an instance of
the game which is then saved in the common
knowledge base. In this way each microservice has
access to the same rules and to the same knowledge
base.
The client application (e.g. a health application,
an agent, a healthcare expert through a front-end tool)
An Ontology based Game Platform for Mild Cognitive Impairment Rehabilitation
137
can only access the games services and not the
ontology service. Because the services have their own
API gateways for communicating with each other, it
is necessary to create a master API gateway that only
exports the gaming services gateways (Fig. 10).
Figure 10: COGNIPLAT platform as a service architecture.
All COGNILAT platform services accept
requests in the JSON (JavaScript Object Notation)
format and respond to the same formatting. Table 1
lists the supported endpoints of COGNILAT platform
services where {resource} denotes one of the
available games.
Table 1: Endpoints of COGNILAT platform services.
HTTP
Verb
Resource URL Response
GET /mci/{resource}
All available game
instances
POST /mci/{resource}
A created game
instance
GET
/mci/{resource}
/{id}
A specific game
instance
PUT
/mci/{resource}
/
{
id
}
Resolution of a specific
g
ame instance
Fig. 11 and Fig. 12 show as an example the use of
the HTTP POST message to request the creation of a
FindTheSound game instance and the corresponding
response message payload.
POST/mci/FindTheSoundHTTP/1.1
Host:localhost:8443
Content‐Type:application/json
X‐INFO‐PLAYER:Postman
cache‐control:no‐cache
{
"difficulty":"easy"
}
Figure 11: Request message for creating an instance of the
FindTheSound game.
{"payload":{
"images":[{
"iId":"Image_9c90b4a254c142edb29c9bfca7287a1c",
"iPath":"\\WB\\mci\\resources\\images\\cat.jpg"
},{
"iId":"Image_16ba6b21974f411f9b3931b0cafa360c",
"iPath":"\\WB\\mci\\resources\\images\\dog.jpg"
},{
"iId":"Image_6442d418cd584e969a644c87671fdea2",
"iPath":"\\WB\\mci\\resources\\images\\donkey.jpg"
}],
"sId":"Sound_2ec5c2314c2b4748a8a7fb40c8e7fe83",
"sPath":"\\WB\\mci\\resources\\sounds\\donkey.mp3",
"game":{
"id":"FindTheSound_EASY_Postman_1",
"difficulty":"EASY",
"playerName":"Postman",
"level":1,
"maxCompletionTime":180
},
"resolved":false
}}
Figure 12: Response message for the FindTheSound game.
Fig. 13 shows as an example the request message
for resolving the choice of the user Postman for a
specific FindTheSound game instance. Fig. 14 gives
the SPARQL query for finding the triplet that
associates the image selected by the user with the
sound via the hasAssociatedSound relationship. The
game service would then be able to check the
correctness of the user choice.
PUT/mci/FindTheSound/FindTheSound_EASY_Postman_1
HTTP/1.1
{
"completionTime":18,
"resolution":{
"sId":"Sound_2ec5c2314c2b4748a8a7fb40c8e7fe83",
"iId":"Image_6442d418cd584e969a644c87671fdea2"
}
}
Figure 13: Request message for resolving user’s choice.
PREFIXmci:<http://localhost:3030/mci/ontology/mci#>
SELECT?subject?predicate?object
WHERE{
?subject<mci:hasId>
"Image_6442d418cd584e969a644c87671fdea2";
<mci:hasAssociatedSound>?object.
}
Figure 14: SPARQL query to find the association between
resources via the hasAssociatedSound relationship.
ICT4AWE 2020 - 6th International Conference on Information and Communication Technologies for Ageing Well and e-Health
138
4 EVALUATION – DISCUSSION
An experimental evaluation was conducted in terms
of technology acceptance of the game platform
involving 15 elderly volunteers by using both
quantitative and qualitative data. All participants
were healthy and over 65 years of age (9 male and 6
female, mean 68.7±3.8 years). The evaluation process
comprised the following steps that performed by each
participant: i) read and sign the consent form; ii)
complete the demographic questionnaire; iii)
complete the Montreal Cognitive Assessment
(MoCA) test (Nasreddine et al., 2005); iv) make
thorough use of the game platform by mimicking a
rehabilitation session; v) complete the SUS (System
Usability Scale) questionnaire (Brooke, 1996); vi)
attend a brief semi-structured interview session.
All participants had a good cognitive functioning
as assessed by the MoCA test (score above 26) except
from two cases with a score between 18 and 26 which
is considered average. The education level varied as
primary (13%), secondary (67%) and higher (20%).
By employing the SUS questionnaire the
perceived usability of the game platform can be
assessed. The questionnaire includes ten statements
(Fig. 15) with five scale responses from 1 (strongly
disagree) to 5 (strongly agree).
Figure 15: SUS questionnaire statements.
The participant’s grades for each item were
processed so that the original scores of 0-40 are
converted to 0-100. The average SUS score was 80
out of 100 (Median=85, Fig. 16), suggesting a good
user acceptance (Bangor et al., 2008). Fig. 17
summarizes the SUS questionnaire results indicating
for each statement the distribution of the 5-scale
ratings. The responses in S2, S6, S8 and partly in S10
indicate that the participants have a firm judgement
that the use of the game platform is simple,
consistent and handy while the learnability effort is
low. On the other hand, a more attentive response is
provided in terms of S4 and S9 as their median ratings
are stronger than the extreme and more definite
ratings. These statements actually express the
capacity of the participants to handle the system on
their own without the support of an expert. The
interpretation of the above variation is attributed to
the less familiarity with technology for some
participants as well as to the general fear for the
technology that the elderly often feel.
Figure 16: Perceived usability as conveyed by SUS score.
Figure 17: SUS questionnaire ratings.
In addition, a brief interview with each one
participant was conducted to acquire qualitative data.
The analysis of this feedback affirms the previous
results indicating that participants believe that the
system complexity is low, the game platform is easy
to understand on its operations and doesn’t require
much effort to use. Particularly noteworthy was the
feedback from several participants who believe that
their memory will positively improve over time via
practicing with the application. On the other hand,
20% of the participants (3/15) responded that they
would definitely need support to complete some
games. The majority of the participants also
expressed a preference towards games that use image
and sound resources than text-based exercises, which
opens possibilities for improvements in the games.
The cognitive level, the familiarity with the
technology and the educational level are factors that
affect the acceptability of any computerized system.
The design of the games and their user interface
50
60
70
80
90
100
SUSScore
An Ontology based Game Platform for Mild Cognitive Impairment Rehabilitation
139
followed a user-centered design approach from the
beginning of the development process targeting a
high usability and efficiency. According to this pilot
evaluation that goal was achieved to a very good
degree.
An evaluation of the game platform regarding its
MCI rehabilitation objective is underway. The
methodology includes a control group and an
intervention group applying a randomized controlled
trial. For all users, measurements of cognitive
functions are recorded before using the COGNIPLAT
platform, while after the intervention the same
measurements will be recorded only for the
intervention team. The intervention team will use the
COGNIPLAT platform 1-2 times a week until each
participant completes 24 sessions of use. The Mini-
Mental State Examination (MMSE) and Montreal
Cognitive Assessment (MoCA) scales will be used
for assessing cognitive improvements.
The following performance data are collected by
the COGNIPLAT platform for each user and for each
game: correct answers, wrong answers, quits (i.e. the
EXIT button was selected before the game round was
completed), the total time the game was played in
minutes, how many different days the game was
played, the points earned in accordance with the point
system and the overall accuracy percentage. The data
collection process is in compliance with the
requirements of the General Data Protection
Regulation and patient information is anonymized by
removing their identifiable and personal data.
The performance data is analysed by the platform
in order to adapt the game difficulty to the user
improvement. For example, if the performance of the
user during a specified time-frame (e.g. 2 weeks) or
over a number of successive sessions, is higher than a
specified threshold (e.g. 75%), the game level will be
increased automatically. Moreover, the stored data
can be processed by machine learning algorithms to
determine more effective exercise plans to improve
the required cognitive skills, whereas the discovery of
patterns can assist the classification of patients to
diagnostic levels. This functionality is expected that
will increase more the engagement of the users and
their willingness for continuous practicing with the
platform.
5 CONCLUSIONS
The field of rehabilitation combined with
technological innovation (wearables, serious games,
robotic systems) opens up new avenues for
telemedicine and home care research. It is now
possible to transfer treatment from care centers to
home and to use computers for rehabilitation.
Statistics show that most patients prefer the use of
new technologies for their recovery, and the results of
these methods are better than the corresponding
traditional ones. In addition, the per capita cost of
rehabilitation of these patients is significantly
reduced, and the quality of services provided is
significantly increased. Consequently, older patients
will be able to receive services for a long time without
a significant burden on the health system.
As for the use of ontologies in MCI health care
applications, their benefits are obvious. It is possible
to personalize intervention practice exercises and also
to significantly differentiate between game instances.
All of these play a crucial role in the end result
because they allow the patient to engage in the
exercises for prolonged time periods without getting
tired. More persistence in exercises yields better
recovery results.
ACKNOWLEDGEMENTS
This work is developed in the framework of ERA.Net
RUS PLUS project COGNIPLAT (RUS_INNO2017-
102) and the MIS 5041669 grant supported by the
General Secretariat for Research and Technology in
Greece. The authors wish to thank the volunteers that
took part in the evaluation for their contribution in
this study.
REFERENCES
Abete, P., Della-Morte, D., Gargiulo, G., Basile, C.,
Langellotto, A., Galizia, G., Testa, G., Canonico, V.,
Bonaduce, D., Cacciatore, F. 2014. Cognitive
impairment and cardiovascular diseases in the elderly.
A heart–brain continuum hypothesis. Ageing research
reviews, vol. 18, pp. 41-52.
Alloni, A., Sinforiani, E., Zucchella, C., Sandrini, G.,
Bernini, S., Cattani, B., Pardell, D. T., Quaglini S.
Pistarini, C. 2015. Computer-based cognitive
rehabilitation: the CoRe system. Disability and
rehabilitation, vol. 39, no 4, pp. 407-417.
American Psychiatric Association (APA). 2011. DSM-5:
the future of psychiatric diagnosis. Washington (DC):
APA.
Baader, F., Horrocks, I., Sattler, U. 2004. Description
logics. In Handbook on ontologies (pp. 3-28). Springer,
Berlin, Heidelberg.
Bangor, A., Kortum, P. T., Miller, J. T. 2008. An empirical
evaluation of the system usability scale. Intl. Journal of
ICT4AWE 2020 - 6th International Conference on Information and Communication Technologies for Ageing Well and e-Health
140
Human–Computer Interaction, vol. 24, no. 6, pp. 574-
594.
Brooke, J. (1996). SUS-A quick and dirty usability scale.
Usability evaluation in industry, vol. 189, no. 194, pp.
4-7.
Cardullo, S., Valeria, P. M., Ilaria, T., Ambra, P.,
Gamberini, L., Mapelli, D. 2015. Padua Rehabilitation
Tool: A Pilot Study on Patients with Dementia. In
International Conference on Games and Learning
Alliance (pp. 292-301).
Carelli, L., Solca, F., Faini, A., Meriggi, P., Sangalli, D.,
Cipresso, P., Riva, G., Ticozzi, N., Ciammola, A.,
Silani, V., Poletti, B. 2017. Brain-computer interface
for clinical purposes: cognitive assessment and
rehabilitation. BioMed research international, vol.
2017, pp. 1-12.
Chartomatsidis, M., Goumopoulos, C. 2019. A Balance
Training Game Tool for Seniors Using Microsoft
Kinect and 3D Worlds. In International Conference on
Information and Communication Technologies for
Ageing Well and e-Health (pp. 135-145).
Garcia-Betances, R. I., Jiménez-Mixco, V., Arredondo, M.
T., Cabrera-Umpiérrez, M. F. 2015. Using virtual
reality for cognitive training of the elderly. American
Journal of Alzheimer's Disease & Other Dementias,
vol. 30, no. 1, pp. 49-54.
Gerling, K. M., Schulte, F. P., Smeddinck, J., Masuch, M.
2012. Game design for older adults: effects of age-
related changes on structural elements of digital games.
In International Conference on Entertainment
Computing (pp. 235-242).
Gomez-Perez, A., Fernández-López, M., Corcho, O. 2006.
Ontological Engineering: with examples from the areas
of Knowledge Management, e-Commerce and the
Semantic Web. Springer Science & Business Media.
Hu, C., Yu, D., Sun, X., Zhang, M., Wang, L., Qin, H. 2017.
The prevalence and progression of mild cognitive
impairment among clinic and community populations:
a systematic review and meta-analysis. International
psychogeriatrics, vol. 29, no. 10, pp. 1595-1608.
Janvin, C. C., Larsen, J. P., Aarsland, D., Hugdahl, K. 2006.
Subtypes of mild cognitive impairment in Parkinson's
disease: progression to dementia. Movement disorders,
vol. 21, no. 9, pp. 1343-1349.
Kang, K., Choi, E. J., Lee, Y. S. 2016. Proposal of a serious
game to help prevent dementia. In International
Conference on Games and Learning Alliance (pp. 415-
424).
Karp, A., Paillard-Borg, S., Wang, H. X., Silverstein, M.,
Winblad, B., & Fratiglioni, L. 2006. Mental, physical
and social components in leisure activities equally
contribute to decrease dementia risk. Dementia and
geriatric cognitive disorders, vol. 21, no. 2, pp. 65-73.
Langa, K. M., Levine, D. A. 2014. The diagnosis and
management of mild cognitive impairment: a clinical
review. Jama, vol. 312, no. 23, pp. 2551-2561.
Leonardi, G., Panzarasa, S., Quaglini, S. 2011. Ontology-
based automatic generation of computerized cognitive
exercises. Studies in health technology and informatics,
vol. 169, pp. 779-783.
McCallum, S., Boletsis, C. 2013. A taxonomy of serious
games for dementia. In Games for Health (pp. 219-
232). Springer Vieweg, Wiesbaden.
Mitchell, A. J., Shiri‐Feshki, M. 2009. Rate of progression
of mild cognitive impairment to dementia–meta‐
analysis of 41 robust inception cohort studies. Acta
Psychiatrica Scandinavica, vol. 119, no. 4, pp. 252-
265.
Nasreddine, Z. S., Phillips, N. A., Bédirian, V.,
Charbonneau, S., Whitehead, V., Collin, I., Cummings,
J. L., Chertkow, H. 2005. The Montreal Cognitive
Assessment, MoCA: a brief screening tool for mild
cognitive impairment. Journal of the American
Geriatrics Society, vol. 53, no. 4, pp. 695-699.
Newman, S. 2015. Building microservices: designing fine-
grained systems. O'Reilly Media, Inc.
Prasanna, D. R. 2009. Dependency injection. Manning
Publications.
Quaglini, S., Panzarasa, S., Giorgiani, T., Zucchella, C.,
Bartolo, M., Sinforiani, E., Sandrini, G. 2009.
Ontology-Based Personalization and Modulation of
Computerized Cognitive Exercises. In Artificial
Intelligence in Medicine in Europe (pp. 240-244).
Rodríguez-Fórtiz, M. J., Rodríguez-Domínguez, C., Cano,
P., Revelles, J., Rodríguez-Almendros, M. L., Hurtado-
Torres, M. V., Rute-Pérez, S. 2016. Serious games for
the cognitive stimulation of elderly people. In Serious
Games and Applications for Health (SeGAH) (pp. 1-7).
Seniów, J., Litwin, M., Leśniak, M. 2009. The relationship
between non-linguistic cognitive deficits and language
recovery in patients with aphasia. Journal of the
Neurological Sciences, vol. 283, no. 1-2, pp. 91-94.
Skillen, K. L., Chen, L., Nugent, C. D., Donnelly, M. P.,
Burns, W., Solheim, I. 2014. Ontological user
modelling and semantic rule-based reasoning for
personalisation of Help-On-Demand services in
pervasive environments. Future Generation Computer
Systems, vol. 34, pp. 97-109.
Sperling, R. A., Aisen, P. S., Beckett, L. A., Bennett, D. A.,
Craft, S., Fagan, A. M., ... & Park, D. C. (2011). Toward
defining the preclinical stages of Alzheimer’s disease:
Recommendations from the National Institute on
Aging-Alzheimer's Association workgroups on
diagnostic guidelines for Alzheimer's disease.
Alzheimer's & dementia, vol. 7, no. 3, pp. 280-292.
Tong, T., Chan, J. H., Chignell, M. 2017. Serious games for
dementia. In International Conference on World Wide
Web Companion (pp. 1111-1115).
Welty, C., McGuinness, D. L., Smith, M. K. 2004. Owl web
ontology language guide. W3C recommendation, W3C,
https://www.w3.org/TR/2004/REC-owl-guide-
20040210/.
An Ontology based Game Platform for Mild Cognitive Impairment Rehabilitation
141
