Analysis of Serious Games based Learning Requirements using

Feedback and Traces of Users

Afef Ghannem

, Karim Sehaba

, Raoudha Khcherif

and Henda Ben Ghezala

Ecole Nationale des Sciences de l’Informatique, Université de la Manouba, RIADI, Tunis, Tunisia

Université de Lyon, CNRS, Université Lyon 2, LIRIS UMR5205 F-69676, France

Keywords: Content-extraction, Educational Objectives, Evaluation, Serious Game.

Abstract: Identify the games that best meet the needs and expectations of teachers and objectives of their courses

remains a necessity about the integration of serious games among active teaching methods. Indeed, several

serious games have developed in recent years, and it is often difficult for a teacher, not a computer scientist

in particular, to find a game that meets these specific needs. Our aim is to develop models and tools enabling

the teacher to find serious games adapted to his needs, considering user feedback and their traces of interaction

with the game. To this end, we have explored the evaluation methods of serious games as well as methods of

extracting knowledge from traces and texts. In this paper, we present our method of knowledge extraction of

educational objectives. Thus, our proposal is assisting and supporting teachers/trainers to choose serious

games and easily integrate them into their learning processes and devices.

1 INTRODUCTION

According to the definition of (Alvarez, 2007), a

Serious Game (SG) is “a software that combines a

serious intention, educational kind, informative,

communicative, marketing, and ideological or work-

out with fun spring”. This fun part of the game allows

motivating and maintaining the player in a dynamic

learning. Currently, thousands of students invest a

tremendous amount of time in playing computer and

the Internet. This generation of the game, strong in

technology, has difficulty in cognitive learning,

methodological and social. The serious game can

therefore make an important place and establish itself

as a complement to traditional training methods.

Moreover, its usability in most activity sectors

gives him a definite advantage for its future. Besides,

the use of serious game is growing exponentially and

dramatically to reach several fields such as education,

learning, continuing training, health care, military,

etc. The serious game has been the objective of

various research studies (Zyda, 2005). Indeed, there

has been increasing interest of using SG for promote

learning in carries serious objectives and learning

outcomes. In this respect, several research works have

been developed to highlight the specific place and the

original role of play in the learning process. This

research specifically aimed at enhancing the

effectiveness of the use of games in learning. The

teachers have been attracted by these games because

they facilitate the manner of receiving the information

by the learners. Therefore, the evaluation of their effi-

ciency and effectiveness according to the course

objectives becomes necessary. In fact, the increasing

use of SGs in teaching laying the problem of their use

by the objectives and content (Paraskeva et al, 2010).

The lack of reliable methods of evaluation and

characterization of SGs constitutes a research gap

linked to a real need for teachers. Indeed, selecting

the most appropriate game at a given learning

objective appears to be insufficiently treated in litera-

ture and the teachers/trainers whose specialties and

knowledge are far from serious games and computer

science in general have difficulty locating them-

selves. In a context of strong growth in the use of SGs,

it becomes necessary to help teachers to identify the

most suitable SG according to the defined educational

objective and their own pedagogical needs. Through

an evaluation and characterization approach of SGs

and based on the extraction and analysis of the

objectives treated by the game, we are trying to solve

the research question addressed by this paper. Indeed,

the extraction of information is a new discipline of

analyzing an automatic manner to extract text a set of

information considered relevant (Poibeau, 2003).

280

Ghannem, A., Sehaba, K., Khcherif, R. and Ben Ghezala, H.

Analysis of Serious Games based Learning Requirements using Feedback and Traces of Users.

DOI: 10.5220/0006713702800287

In Proceedings of the 10th International Conference on Computer Supported Education (CSEDU 2018), pages 280-287

ISBN: 978-989-758-291-2

Our proposal is to collect all the descriptions, the

feedbacks, the possible traces of the players of a given

game in a bank of games. Then, analyze each one

according to the predefined ontology to extract the

criteria and the key concepts of the game from the

texts already stored in the corpus. After having had

the criteria in result of the extraction, we will have

ontologies relative to the games in quest of analysis

and characterization. We then align these ontologies

with the ontology of the needs of the teacher/ trainer

in order to obtain, through automatic calculations, the

precision rates of the information contained in the

first ontologies in relation to the last ontology

(ontology of the needs of the teachers).

Finally, and through the values of adequacy

obtained during the previous stage, it is automatically

recommended to the teacher/trainer the game best

suited to his pedagogical needs. In order to meet our

objective, we first carried out a state of the art on the

methods of evaluation and analysis existing in the

literature to identify the common characteristics cited

and to consider and judge their usefulness according

to our approach analysis of SGs. We also explore the

techniques and tools used in related work to decide

which analytical techniques to adopt.

In this paper, we proceed as follows: Firstly, we

will present in section 2 the back-ground of our

research work. In section 3, we will present our

method of research. Section 4, describes in detail our

methodology and used techniques. In section 5, we

present the field application of our research. In

section 6, we expose a part of preliminary results.

Finally, we conclude by summarizing our work and

the proposed perspectives.

2 BACKGROUND

Serious games’ users still have difficulty with the

choice of the most appropriate game for their needs.

In this section, we present attempts to establish

evaluation systems existing in the literature as well as

the terminology associated with the characterization

and modelling of Sgs.

2.1 Serious Games Evaluation:

Methods and Used Tools

Several studies are currently devoted to evaluating the

contribution of games in learning. The evaluation and

analysis of serious games has affected several

aspects. Some of these works evaluated the design

component of the games such as the evaluation of the

playability of the gameplay experience (Nacke et al,

2010), the user-friendliness of the interface of the

game which tends to keep the attention of the player

(Pinelle et al, 2008), the verification of the

compatibility of the objective of the game with the

content. For example, (Mitgutsch and Alvarado, 2012)

proposed an evaluation of the content of the game

through a questionnaire which offers a potential for a

critical discourse on the strengths and weaknesses of

a serious game and Emphasize cohesion between the

essential elements of design and consistency in

relation to the objective of the games (Calderón and

Ruiz, 2015) (Lameras et al, 2016). Others have evaluated

the amount of information acquired by the

learner/player (Oulhaci et al, 2013), and follow his

actions during playing the game and therefore assess

the assimilation's degree of the knowledge provided

by the game.

On his part, (Molnar and Kostkova, 2013) suggests

an evaluation of the gain offered by a serious game.

Nevertheless, these works are most dedicated to game

designers and applies during the creating and

developing games process. Numerous research

advocates the use of methodologies and theoretical

approaches such as grids, questionnaires, logs,

interviews, monitoring, etc. to help teachers/trainers

to analyze an educational game and evaluate its

pedagogical profitability (Boughzala, 2014).

In relation to our research question, few studies

have tried to associate Natural Language Processing

techniques (NLP) with serious games (Picca et al,

2015) and has talked about the importance of such

association. They admit that with the use of NLP, they

can collect information without destroying the game

and more accurately interpret the users' behaviour.

2.2 Serious Game Criteria

Works related to ours, tried to evaluate various

aspects of the game. The frequently evaluated criteria

are the usefulness of the game, the domain, the

understandability, the motivation, the kind of the

game, the feedback and the objective of the game

(Bellotti et al, 2013). The learning outcomes in turn

are a very selective criterion of SG (Mayer, 2012) (Ra

et al, 2016) (Arnab et al, 2015). Other studies have

proposed tools and databases of games where we find

an educational games collected and analyzed by

certain number of criteria.

In fact, our approach differs from above works

and our objective is to extract the content of the SG,

especially the educational goals through the

descriptions accompanying SGs, feedbacks, to beable

to compare and evaluate the course objectives, in

order to make the right choice of the game to

Analysis of Serious Games based Learning Requirements using Feedback and Traces of Users

281

Table 1: Serious Game Criteria.

Category

Criteria

Definition

Game

Gameplay/Playability

Evaluate the

gameplay and the

pleasure it offers to

the player.

Usability/interface

Evaluate the user-

friendlness of the

interface and the

learning of the game

by the players.

Usefulness

Measure the interest

of the serious game

in relation to the

field.

Domain

Thefield and

discipline of the

game.

Understandability/

Degree of difficulty

The ability of

serious game to be

understood.

Game type

Competitive or

cooperative

Game genre

Role game, strategy,

action, reflection,

simulation, etc.

Timing

The duration oft he

game.

Feedback

Clear information

on how the partici-

pants are doing.

Pedagogy

Objective

Define and describe

the skill to be

acquired via the

game.

Learning outcomes

Describe the desired

learning that

students should have

acquired at the end

of a game: skills and

attitudes.

Learning style

Informations and

indicators that how

students learn and

interact with the

game.

Target

Audience

Age

The age range of

learners whose

game is dedicated.

Prerequisites

Determine basic

knowledge levels.

Engagement

A generic indicator

of involvement in

the game.

incorporate it into the learning process by any

instructional designer, we propose our terminology

associated to characterizing and modelling SGs.

Table 1 define and describe these criteria.

2.3 Serious Game Knowledge Modeling

In spite of the hopeful results of the use of SGs in

teaching and learning presented in the literature, their

analysis and evaluation still require metrics to

characterize games in an educational context. In our

context, ontologies allows to model a knowledge

formally. Thus, they make it possible to represent the

learner/player profile, the context of the game and the

learning offered and integrated into the SG.

This section identify ontologies and meta-models

of SG available in the literature. We proceeded as

follows :

 Search for available ontologies of serious

games.

 Analyze and compare the concepts used in

each of them in relation to our needs.

In (Tang and Hanneghan, 2011), authors

introduce a SG ontology that aims to develop a high-

level creation environment to facilitate the

development of SGs for teachers. We see that this

ontology defines technical concepts and aspects of the

game more than the pedagogical concepts. It is an

ontology dedicated to the development of SGs. Other

searchers define ontology of SG (Prayaga and

Rasmussen, 2008), it is an ontology that describe and

define the essential elements of games, the essential

elements of the learning environment and the

essential elements of SGs.

Moreover, a meta-model for SGss in higher

education (Longstreet and Cooper, 2012), consists of

three basic parts namely external entities, educational

game elements and traditional game components.

This model focuses mainly on the educational

elements of the game. The knowledge of the domain

is defined in an ambiguous way which requires an

imprecise communication of the domain knowledge.

A new ontology has appeared in the work of (Rocha

and Faron-Zucker, 2015). It aims to enable the

modelling and creation of SGs that use Linked Data

datasets as a knowledge base to represent resources in

the game and considering the profile and context of

the player.

3 RESEARCH METHOD

To help and support teachers in choosing the right

SG, we are trying to invent and create an evaluation

model of SG content. Design methods, development

of SGs and how they integrate pedagogy unwittingly

are diverse. These differences should be taken into

consideration when designing our automatic

evaluation model (see Figure 1).

CSEDU 2018 - 10th International Conference on Computer Supported Education

282

The goal is to extract the objectives and the

description content that accompanies SGs, feedbacks

and interaction of users of such SG. This research

implements the technologies of semantic web and

information retrieval. It fits in the field of Natural

Language Processing (NLP) and specifically in that

of the information extraction. It accepts input in plain

text with domain ontology. The extraction process

used to identify entities and aims to extract and

generate annotations for each text feature.

Figure 1: Architecture of our proposed.

To this end, we are trying to implement a

systematic approach for teachers and trainers. The

information extraction is a technology that aims to

meet a user needs, it seeks to gain knowledge from

text. Discover information of interest that we help us

to take decision to adopt or not such SG in such

learning process, is often our approach in this work.

We use ontologies for the description and

formalization of game knowledge and the needs of

the teachers/trainers. After extracting the SG content

and save it, we obtain in result ontologies of

characterization of the SGs. These latter’s instantiate

the categories of the criteria of a given game already

mentioned above in the previous section. On the

other hand, we have a teacher/ trainer who is looking

for a game that matches his or her educational needs.

After having formulated the needs of the

teachers/trainers in an ontology, we make the

matching between the two (or more) ontologies

corresponding to obtain a list of the most suitable

games in percentage in relation to the needs of the

end user.

4 METHODOLOGY

4.1 Serious Game Ontology

As we stated above, ontologies represent and deal

with information at the semantic level effectively.

Their use continues to cover various areas such as

technical knowledge, research and indexing of

information. They promote sharing, knowledge

organization, interoperability between systems and

facilitate communication between experts in software

development as they establish a common vocabulary

and semantic interpretation of terms.

The SGs evaluation process for teaching is very

important for their adoption in the learning process. It

is directly related to the pedagogical needs of

teachers/trainers. Each serious game is specific in

terms of design, modelling, how pedagogy has been

integrated, and so on. It is for this reason that we

propose an ontology to model the knowledge offered

by such game. It is generic and has a reasonable size.

We have developed our top-down ontology starting

from the most general to the most specific concepts.

In order to model all the information of interest to

extract from the SG, we will update and exploit a

domain SG ontology already proposed in (Ghannem

and Khemaja, 2011). The main purposes of our SG

ontology are: (i) a formal modelling to provide an

automatic interpretation of the SG to solve problems

related to standardization and interoperability. (ii)

Promote the sharing of knowledge associated with SG

in the educational field. (iii) Favour reuse of

knowledge related to SG analysis and

characterization.

The purpose of our SG ontology is to facilitate

characterizing and the extraction of the contents of

such game through the criteria mentioned in the

previous sections. Indeed, a SG is characterized by

one or more objectives, actions that compose it, the

skills to develop among players, etc. A pedagogical

objective can be affective, cognitive or psychomotor

type. Pedagogical Objective accomplishes many

skills such as knowledge, know-how and know-be.

Actions that can accomplish the objectives, also used

to describe and develop the skills covered by the

game. The ontology will guide us to our knowledge

extraction approach and to our evaluation system.

Effectively, our ontology should answer the

following questions:

 What are the different criteria of the serious

game?

 How can they be characterized?

In order to answer to these two competency

questions, we referred to the related works which

tried to characterize the SGs and we then classify

them by category (previously mentioned in section 2).

The Figure 2, presents the SG ontology formalized

under the Protégé ontology editor.

Analysis of Serious Games based Learning Requirements using Feedback and Traces of Users

283

Figure 2: Partial view of our serious game ontology.

To validate our ontology, we create several

instances of serious games belonging to different

fields of education. The instantiated concepts are used

for characterization and extracting the main

knowledge of SGs.

4.2 Definition and Modeling of

Teacher’s Needs

We booked more time for modelling the needs of

teachers/trainers in terms of the criteria and

functionality offered by such a game and more

particularly the objective class given the importance

of these entities for the adoption of such game in

learning. Considered as essential element in the

context of teaching and learning, the Oxford

Advanced Learner’s Dictionary defines an objective

as an act of kicking or hitting. Moreover, (Guilbert,

1984) defines an objective as the result sought by the

learner at the end of the educational program, i.e what

the students should be able to do at the end of a

learning period that they could not to do beforehand.

These are the statements which express specifically

and in measurable terms, an attitude that will be

developed cognitive or psychomotor skills that the

students would be able to do because of prescribed

treatment method or mode of instruction.

From these two definitions, we can shoot a

statement of a learning objective contains a verb (an

action) and an object (usually a noun). The verb

generally refers to the intended cognitive process.

The object generally describes the knowledge

students are expected to acquire or construct

(Anderson and Krathwohl, 2001). Thus, to represent

an objective formally, we rely on the definition and

Bloom's taxonomy. Bloom's taxonomy is a research

tool in education and evaluation. It represents an

objective classification system of the educational

process. It is used to categorize the level of

abstraction skills to be acquired during the learning

process. The taxonomy consists of three domains

cognitive (about knowing), affective (about attitudes,

feelings) and psychomotor (about doing).

Defining the way whose users will express their

pedagogical needs regarding the SG will allow us to

exploit the information collected from

teachers/trainers. Then, we align them with the

knowledge extracted during the previous section to

decide one or more adequacy between users’ needs

and our annotated games corpus.

5 APPLICATION FIELD

It is essential when we want to practice active

learning to set goals to determine what the participant

will get of training. But, the most common tasks in

information extraction are the extraction of named

entities (Nadeau and Sekine, 2007), and relationships.

Therefore, extraction of objectives can be conceived

as a form of relationship extraction where an action

(verb) is related to other entities such that a player can

perform in game. The development of such system

first requires defining the nature of that in-formation

to share with other information processing services.

Ontology is the mode of representation most used for

this purpose. Comes then the central phase of textual

analysis to extract these types of information.

Our method of automatic identification of

objectives and criteria is based on the description

accompanying each game, the user's feedback and the

user's interactions with the game such as logs, player's

traces, etc. These descriptions can then be exploited

to extract the concepts, relations and rules of the game

ontology. Through the semantic annotation, we

provide To users the useful knowledges to

characterize the serious game. This extraction process

follows the following steps: (1) The first concerns the

definition and construction of the two ontologies. (2)

The second is related to the identification of

concepts/relationships in ontology, locating the

corresponding terms and synonyms in the corpus. (3)

The third concerns the export of the obtained result in

RDF format, (4) in the fourth step, we align the

obtained ontologies through the exploration of the

corpus previously stored in our serious gaming bank

with that of the users’ needs and calculate the

performance evaluation. To begin with, we define a

CSEDU 2018 - 10th International Conference on Computer Supported Education

284

set of criteria considered as advertiser’s criteria and

objectives likely to realize it in the text. These

advertisers are divided into different lists that each of

which is associated with a specific class. For

example, a type of objective can be cognitive,

affective, or psychomotor. In other words, identify

the different types of objectives defined in our

domain ontology. Thereafter, it is specified the class

of the ontology associated to advertiser’s verbs

present in the text to be analyzed.

Then, we need to associate their circumnutates

involved to achieve the objective. For that, we must

identify the relationship between the advertiser and

other entities of the sentence. It is useful to use a

parser defining dependencies between the different

elements of the sentence. For each game analyzed, we

could extract if they are present, the different types of

objectives of the game, the type of the game, its

domain, the level of difficulty, etc.

The tool presented here, uses semantic extractors

suitable for text engineering platform named General

Architecture of Text Engineering (GATE)

(Cunningham et al, 2003) and aims to extract the goals,

relationships, etc. The recognition component is

based on the Gate Transducer component that uses

JAPE to manually define the models. JAPE provides

a layer between the user and regular expressions that

are used internally. A JAPE rule consists of a pattern

that must be identified LHS (regular expression for

annotation pattern), followed by the code to be

executed RHS (manipulation of the annotation patter

from LHS) when this model is compared. The result

of JAPE rule is stored in an annotation property. In

GATE, each goal (verb) to be with his entry in the

ontology. We opted for an automatic extraction of

criteria from the descriptive text accompanying the

SGs as a first filter in our overall evaluation approach.

To get there, we developed a new resource type JAPE

Transducer which adds to the Gate processing

resources. We defined a JAPE rule to identify

objectives. To avoid having false results and for the

extraction engine does not consider any verb as an

educational objective, we developed heuristics that

eliminate noise that can be generated.

 The verb should be an infinitive.

 The verb must be followed by an object.

 The verb must belong to a predefined area.

These three heuristics are translated into rules to

filter the results.

To conclude, we did not find any research work to

automatically extract knowledge from serious games,

and to characterize this knowledge through a concrete

and semantic model. Moreover, to our knowledge

there are no works based on semantic web

technologies, specifically NLPs to automatically

extract the content of serious games from texts and

therefore give a formal and structured

characterization to this knowledge and

understandable by machine.

6 EXPERIMENT AND

VALIDATION

With the aim of helping and supporting future SGs

users for learning, and more specifically teachers /

trainers, to adopt new teaching methods, ie teaching

and training through games, We try to propose a

generic and formal referential. Integrating SGs in a

meaningful way into learning processes requires a

well defined and precise protocol.

We adopted an approach based on the collection

of data through the descriptions contained on the net,

user feedback from the forums, collection of

formalized and non-formalized traces of users and

exploit them as input of our system of extraction and

characterization of SGs. The output and the result of

our system will be an ontology of criteria followed by

statistics of the correspondence rates of one or several

games with the criteria sought by the teacher / trainer.

Next, our SGs evaluation protocol provides useful

recommendations for the game user to adopt and

favour one game over another based on the suitability

values derived from the previous step.

The implementation of our content extraction

method and the objectives of serious games are

currently underway. But, we can expose some

preliminary results. To test the effectiveness of our

system and to validate our proposal, we have built our

own repository through a range of serious games

collected from the web, based on that they are open

source and having educational objectives. The

advantage of using NLPs to solve the problem of

finding and characterizing SGs in favour of learning

lies in its potential to develop a semantic network of

knowledge related to the description of the game. Be

exploited to find the adequacy between the learning

objectives of a learning process and the learning

outcomes that such a game can offer. Unlike existing

work, we try to provide a complete solution while

defining a generic tool for evaluating and analyzing

SGs. We provide teachers/trainers especially non-

computer scientists and non-connoisseurs of

advanced technologies help and support to find the

most appropriate game in relation to their pedagogical

needs and therefore adopt it in their learning

processes.

Analysis of Serious Games based Learning Requirements using Feedback and Traces of Users

285

Figure 3: Annotation of ontology concepts.

Figure 4: Objectives extraction.

Figure 3 and Figure 4, show the output from the

extraction of some objectives present in the

description of the Colobot and the stanrbank game.

Through this extraction, w can get information on:

 What do we teach to the learner/player?

 What type of knowledge to be acquired?

 The abstraction level of educational

objectives?

In fact, this is the first filter of the first lines of

extraction results in our approach adopted for

evaluation of serious games through their educational

objectives.

7 CONCLUSION AND FUTURE

WORK

The principal aim of the work presented in this paper

is to provide a method and model for extracting

(semi-) automatically the serious game content. This

model extracts automatically the objectives and

content of the game through the description that

accompanies it as first filter. This responds to the

serious game users’ needs to help and support

teachers in their choice of the SG and to facilitate its

evaluation.

Our future research consists of fully automating

the content-extraction and evaluation process of

serious games.

REFERENCES

Alvarez, J., 2007. Du jeu vidéo au serious game: approches

culturelle, pragmatique et formelle, PhD thesis,

University Toulouse II.

Anderson, A., Krathwohl, D. A., 2001. Taxonomy for

Learning, Teaching and Assessing: A Revision of

Bloom's Taxonomy of Educational Objectives,

Longman, New York.

Arnab, S., Lim, T., Carvalho, M. B., Bellotti, F., Freitas, S.,

Louchart, S., Suttie, N., Berta, R., De Gloria, 2015. A.,

Mapping learning and game mechanics for serious

games analysis, British Journal of Educational

Technology 46 (2), 391-411.

Bellotti, F., Kapralos, B., Lee, K., Moreno-Ger, P., Berta,

R., 2013. Assessment in and of Serious Games: An

Overview, Advances in Human-Computer Interaction

(11). 2013.

Boughzala, I., 2014. Characterizing the serious game and

assessing learning goals, Systèmes d'information &

management 17 (3).

Calderón, A., Ruiz, M., 2015. A systematic literature

review on serious games evaluation: An application to

software project management, Computers & Education

(87), 396-422.

Cunningham, H., Tablan, V., Bontcheva, K., Dimitrov, M.,

2003. Language engineering tools for collaborative

corpus annotation, Proceedings of Corpus Linguistics.

Ghannem, A., Khemaja, M., 2011. Defining a game

ontology for helping games and learning processes

integration. In eLSE Conference, 7th edn.

Guilbert, J. J., 1984. How to Devise Educational Objectives

Medical Education, MED Educ 18 (3), 134-41.

CSEDU 2018 - 10th International Conference on Computer Supported Education

286

Lameras, P., Arnab, S., Dunwell, I., Stewart, C., Clarke, S.,

Petridis, P., 2016. Essential features of serious games

design in higher education: Linking learning attributes

to game mechanics, British Journal of Educational

Technology.

Longstreet, C.S., Cooper, K.M.L., 2012. Developing a

metamodel for serious games in higher education. In

Proceedings of the IEEE 12th International Conference

on Advanced Learning Technologies (ICALT), Rome,

Italy.

Mayer, I. S., 2012. Towards a comprehensive methodology

for the research and evaluation of serious games,VS-

Games (15), 1–15.

Mitgutsch, K., Alvarado, N., 2012. Purposeful by Design?

A Serious Game Design Assessment Framework.

Proceedings of the International Conference on the

Foundations of Digital Games, pp. 121-128.

Molnar, A., Kostkova, P., 2013. On effective integration of

educational content in serious games: Text vs. Game

mechanics. IEEE International Conference Advanced

Learning Technologies, pp. 299-303.

Nacke, L., Drachen, A., Göbel, S., 2010. Methods for

Evaluating Gameplay Experience in a Serious Gaming

Context, International Journal of Computer Science in

Sport 9 (2), Germany.

Nadeau, D., Sekine, S., 2007. A survey of named entity

recognition and classification, Linguistic Investigations

30 (1), 3–26.

Oulhaci, A., Tranvouez, E., Fournier, S., Espinasse, B.,

2013. A Multi-Agent System for Learner Assessment

in Serious Games: application to learning processes in

Crisis Management. IEEE Int. Conf. On Research

Challenges in Information Science.

Paraskeva, F., Mysirlaki, S., Papagianni, 2010. Multiplayer

online games as educational tools: Facing new

challenges in learning, Computers & Education (54),

498–505.

Picca, P., Jaccard, D., Eberlé, G., 2015. Natural Language

Processing in Serious Games: A state of the art,

International Journal of Serious Games 2 (3).

Pinelle, D., Wong, N., Stach, T., 2008. Heuristic Evaluation

for Games: Usability Principles for Video Game

Design. CHI Proceedings Game Zone, Italy.

Poibeau, T., 2003. The multilingual named entity

recognition framework. In Proceedings of the tenth

conference on European chapter of the association for

computational linguistics, Vol.2, pp. 155-158.

Prayaga, L., Rasmussen, K., 2008. Ontology of serious

games, L. iManager's Journal of Educational

Technology 5 (2), 10- 21.

Ra, G., Sung, J., Yu, H., Min, H., 2016. Analysis of serious

games based on pedagogical features and proposal of

civil defence training game, International Journal of

Computational Vision and Robotics 6 (3), 235-243.

Rocha, O.R., Faron-Zucker, C., 2015. Ludo: An Ontology

to Create Linked Data Driven Serious Games. ISWC

Workshop on LINKed EDucation, Bethlehem,

Pennsylvania, United States.

Tang, S., Hanneghan, M., 2011. Game content model: an

ontology for documenting serious game design. In

Proceedings of the IEEE 4th International Conference

on Developments in E-systems Engineering (DeSE),

Dubai, UAE.

Zyda, M., 2005. From Visual Simulation to Virtual Reality

to games, IEEE Computer 38(9), 25-32.

Corpus Linguistics, Lancaster, UK.

Analysis of Serious Games based Learning Requirements using Feedback and Traces of Users

287