A Survey of Possibilities and Challenges with AR/VR/MR and

Gamiﬁcation Usage in Healthcare

Yu Fu

1 a

, Yan Hu

1 b

, Veronica Sundstedt

1 c

and Cecilia Fagerstr

2 d

Department of Computer Science, Blekinge Institute of Technology, Karlskrona, Sweden

Department of Health and Caring Sciences, Linnaeus University, Kalmar, Sweden

Keywords:

Virtual Reality, Augmented Reality, Mixed Reality, Healthcare, Game, Gamiﬁcation.

Abstract:

Software and applications of augmented reality (AR), virtual reality (VR), and mixed reality (MR) technol-

ogy combined with game/gamiﬁcation techniques in healthcare have increasingly been studied in academia.

However, there is a need to explore the usage, challenges and opportunities of AR/VR/MR game/gamiﬁcation

software/applications in the healthcare system. To explore this, we present an online survey conducted in the

healthcare-relevant system (including hospital-based system, homecare-based system, institute and university,

and industry). Based on the answers, we found examples of digital games and AR/VR/MR applications used

in healthcare, as well as some general information (name and feature, purpose, target user, and use occasion),

usage situation, and user experience. This presented survey is beneﬁcial for both researchers and developers

in computer science and medical science. It can familiarise them with existing products and their current

use, advantages and potential issues of AR/VR/MR and game applications in healthcare. In future work,

the survey would be extended to obtain other user experiences and feedback of AR/VR/MR techniques and

game/gamiﬁcation technology applied to healthcare, as well as to study how to overcome the challenges, and

develop the opportunities further.

1 INTRODUCTION

Recent research has started to focus on combining

augmented, virtual and mixed reality (AR/VR/MR)

technology with game techniques in healthcare ap-

plications. These applications have been developed

to match the requirements of healthcare issues or to

enhance traditional solutions in healthcare. This pa-

per presents a user survey with relevant staff, students,

and professionals related to the healthcare area, with

the aim to explore the possibilities and challenges

with AR/VR/MR and game/gamiﬁcation applied in

healthcare. The research objectives focused on the

use, opportunities, and challenges, with AR/VR/MR

techniques and game/gamiﬁcation technology.

The rest of the paper is organised as follows. Sec-

tion 1.1 describes the relevant background concepts

including VR/AR/MR and game/gamiﬁcation. Then,

Section 2 illustrates the related work in both medi-

https://orcid.org/0000-0003-3520-3302

https://orcid.org/0000-0002-3283-2819

https://orcid.org/0000-0003-3639-9327

https://orcid.org/0000-0002-4257-282x

cal science and computer science. Next, Section 3

explains the aim and objectives, research questions,

and the research method, including the ethics descrip-

tion of this research, questionnaire structure and logic,

questions topic content, and pilot study. Section 4

shows the result based on 30 questionnaire answers,

including the perspective of application purpose and

target user, use time and frequency, advantages and

disadvantages, obstacles to mass adoption, and re-

quirements and concerns. Based on the results pre-

sented in Section 4, Section 5 tries to explore the

possible reasons behind the results. Finally, Section6

gives a summary of the study and highlights some rel-

evant future work.

1.1 Background

Games are not a concept only in the ﬁeld of computer

science, but they are increasingly familiar and used

in other areas because they are often combined with

computer technology. A game is a system based on a

rule, which has a variable and quantiﬁable outcome,

with assigned different values (Lima et al., 2017). The

effort of the player will inﬂuence the result (Lima

Fu, Y., Hu, Y., Sundstedt, V. and Fagerström, C.

A Survey of Possibilities and Challenges with AR/VR/MR and Gamiﬁcation Usage in Healthcare.

DOI: 10.5220/0010386207330740

In Proceedings of the 14th International Joint Conference on Biomedical Engineering Systems and Technologies (BIOSTEC 2021) - Volume 5: HEALTHINF, pages 733-740

ISBN: 978-989-758-490-9

733

et al., 2017). The challenge and fun in the game are

the primary motivations for people to play it. Gami-

ﬁcation is using game-design elements and principles

in non-game contexts (Deterding et al., 2011), and it

aims to bring the beneﬁts of games to non-game con-

texts. AR systems add or remove virtual (computer-

generated) objects, which coexist in the same space

as the real world (Blusi, 2014). VR systems are dif-

ferent. They generate 3D objects by the computer,

which simulates a virtual world, and cut off users’

perception of the real world by using peripherals such

as immersion glasses and interaction devices (Okeil,

2010). MR is sometimes referred to as ’hybrid re-

ality’, and is the merging of real worlds and virtual

environments. MR provides new environments and

visualisations, where physical and digital objects can

coexist and interact in real-time (de Lima et al., 2016).

According to the immersion level, Li et al. clas-

siﬁed AR/VR techniques into seven categories: (1)

least-immersive VR, (2) semi-immersive VR, (3) im-

mersive VR, (4) tangible AR, (5) collaborative AR,

(6) distributed VR/AR, and (7) other VR/AR (Li et al.,

2018). They claimed that technology selection leads

to different immersion and interaction and should

adapt to the application requirements (Li et al., 2018).

According to equipment, Okeil grouped VR by

two immersive levels: (1) immersive VR (using HMD

or CAVE) and (2) non-immersive VR (using screen

including laptop, computer, mobile, tablet PC, TV,

and large screen) (Okeil, 2010). Desktop-based VR

was deﬁned as operating on a simple computer moni-

tor, controlled by mouse and keyboard, and without

tracking equipment (Wang et al., 2018). Although

desktop-based VR provided lower-level immersion,

compared with other kinds of VR techniques, the

device also has a lower cost. Immersive VR relies

on specialised hardware such as HMDs and sensor

gloves (Wang et al., 2018).

Adding sensors into the hardware made them pro-

vide real-time feedback. Because of the real-time

capabilities, Sacks and Pikas believed that immer-

sive VR is better than non-immersive VR (Sacks and

Pikas, 2013). Additionally, Wang et al. highlight that

by integrating visual and interactive multi-user oper-

ation technologies, VR training games could reduce

the complexity of task processes, thereby enhancing

user interactions (Wang et al., 2018).

2 RELATED WORK

Ma et al. summarised three keywords to the fu-

ture trends of VR/AR and game in healthcare: the

location-based exergaming, mobile apps, and social

media gaming for public health (Ma et al., 2014).

They claimed that the approach of healthcare and

medical education might be changed by the research

and development of mobile devices combined with

wearable devices over the next decade (Ma et al.,

2014). A social media effect on games in health-

care will be growing due to social games’ popular-

ity and social interactions’ effectiveness (Ma et al.,

2014). AR/VR techniques were proven effective to

some healthcare issues, as a new solution or to en-

hance the conventional solutions. Jorge et al. re-

viewed the approaches and challenges to the VR/AR

technique in healthcare and rehabilitation (Jorge et al.,

2019). In the review, they illustrated several exciting

VR/AR systems, such as VR for radiologists in the

reading room, and in the operations room, augmented

surgery, and laparoscopic, as well as exergames for

locomotion rehabilitation by VR (Jorge et al., 2019).

Perkins Coie LLP and the XR Association have

surveyed AR/VR yearly, since 2017. They inter-

viewed 200 startup founders in 2019, executives with

established technology companies, investors and con-

sultants in the AR/VR/MR related area, in the past

four years. In their new survey report in 2019 (LLP

and the XR Association, 2019), they pointed out that

90% of the respondents believed immersive technolo-

gies including AR/VR/MR would be ubiquitous like

mobile devices, by 2025.

They also claimed that the quality of user expe-

riences, available content offerings, and the pace of

adoption were the concerns and similar to the past sur-

veys’ result. For AR technology to be mass adopted,

the largest obstacle was the lower user experience,

including bulky hardware or technical glitches. The

second one was the content offerings, including the

quantity and quality of content. Similarly with the

survey results of AR technology, user experience and

offering content were the two most considerable ob-

stacles to getting VR technology to be mass adopted.

Respondents also pointed out that the most common

concern was the slow adoption among consumers.

Some respondents believed hardware should be

more user-friendly, and the cost, comfort and utility

should be addressed. However, they have still strong

conﬁdence with them when it comes to the future

of the AR/VR/MR technologies. One of the respon-

dents said that immersive technologies could connect

further to people in an organisation, such as in doc-

tors’ ofﬁces or schools, instead of only facing indi-

vidual consumers. In their report, the game technique

again, as in their previous surveys, led the combina-

tion with AR/VR/MR applications. Not far behind

was the healthcare and medical devices.

HEALTHINF 2021 - 14th International Conference on Health Informatics

734

Figure 1: Example of AR/VR/MR technology.

3 METHOD

The survey used an online questionnaire to eval-

uate the usage of AR/VR/MR applications and

game/gamiﬁcation technology in healthcare. The re-

search questions are shown below:

• RQ1: How can digital game/gamiﬁcation and

AR/VR/MR technology be applied in healthcare?

• RQ2: What are the opportunities and challenges

of game/gamiﬁcation and AR/VR/MR applica-

tions in healthcare?

The link to the questionnaire was either emailed to

target organisations, or posted on social media such

as LinkedIn and Twitter. Based on the principles of

Singer and Vinson’s practical guideline for ethical is-

sues on the conduct of empirical studies (Singer and

Vinson, 2002), and the General Data Protection Reg-

ulation (GDPR) (Voigt and Von dem Bussche, 2017),

we created a detailed ethics description of our survey

in the informed consent, and illustrated it in the invita-

tion letter, the consent form, and the beginning of the

online questionnaire. Here it was highlighted that no

sensitive data was gathered and it was all conﬁdential.

In the questionnaire, there were three sections

with 85 questions. Out of these, 17 were required

and mandatory marked with a star. The structure

and logic of the questionnaire are shown in Fig. 2,

Except for the informed consent, the other ques-

tions included two aspects: (1) general information

and (2) topic-related questions. Before the topic-

related questions, there was an introduction to the

survey, informed consent, background concepts intro-

duced, and data collection of the respondents’ gen-

eral information (including the location, and occu-

pation and the work content of the participant). In

the topic-related questions, we tried to obtain in-

formation about the game/gamiﬁcation application(s)

and AR/VR/MR application(s) in healthcare. To ad-

dress RQ1, the application(s) information questions

Yes

Q2-Q5

Q7-Q14

Q15

Q42-Q43

Q44

Q84-Q85

Q45-Q53

Q54

Q16-Q23

Q24

Q25-Q32

Q33

Q34-Q41

Q55-Q63

Q64

Q65-Q73

Q74

Q75-Q83

Required question

Repeat 1

Repeat 2

Repeat 3

Repeat 2

Repeat 1

Part 2. Game/ gamication in general healthcare

software/applications

Part 3. Healthcare software/applications

with AR/VR/MR technology

Part 1.

Consent form

Start

End

Figure 2: The questionnaire structure and logic.

(Game/gamiﬁcation part: Q7-Q14; AR/VR/MR part:

Q45 to Q53) covered application name and feature,

purpose, target user, use environment, use frequency,

use time, advantages and disadvantages, and whether

it was using a game/gamiﬁcation technique (only in

Part 3). To answer RQ2, we generated questions

(Game/gamiﬁcation part: Q42-Q43; AR/VR/MR

part: Q84-Q85) to better know the requirements and

obstacles to mass adoption of AR/VR/MR techniques

and game technology.

In part two and three, the questions for collect-

ing application information were repeated three times

(Game/gamiﬁcation part: Q16-Q23, Q25-Q32, Q34-

41; AR/VR/MR part: Q55-Q63, Q65-73, Q75-83),

to obtain further data if the interviewee used more

than one software/application. Before each repetition,

there was a required question (Game/gamiﬁcation

A Survey of Possibilities and Challenges with AR/VR/MR and Gamiﬁcation Usage in Healthcare

735

0 5 10 15 20 25 30

User experience

Content oﬀerings

Consumer and business reluctance to

embrace game/gamiﬁca�on technique

Regula�on and legal risks

Financing and investment

Cost to consumers

Government oversigh

Other

0 5 10 15 20 25 30

User experience

Content oﬀerings

Consumer and business reluctance to…

Regula�on and legal risks

Financing and investment

Cost to consumers

Government oversigh

Other

Game/Gamiﬁca�on AR/VR/MR

0 5 10 15 20 25 30

User experience

Content oﬀerings

Consumer and business reluctance to…

Regula�on and legal risks

Financing and investment

Cost to consumers

Government oversigh

Other

Game/Gamiﬁca�on AR/VR/MR

0 5 10 15 20 25 30

User experience

Content oﬀerings

Consumer and business reluctance

Regula�on and legal risks

Financing and investment

Cost to consumers

Government oversigh

Other

Game/Gamiﬁca�on AR/VR/MR

0 5 10 15 20 25 30

User experience

Content oﬀerings

Consumer and business reluctance

Regula�on and legal risks

Financing and investment

Cost to consumers

Government oversigh

Other

Game/Gamiﬁca�on AR/VR/MR

0 5 10 15 20 25 30

User experience

Content oﬀerings

Consumer and business reluctance

Regula�on and legal risks

Financing and investment

Cost to consumers

Government oversigh

Other

Game/Gamiﬁca�on AR/VR/MR

0 5 10 15 20 25 30

User experience

Content oﬀerings

Consumer and business reluctance

Regula�on and legal risks

Financing and investment

Cost to consumers

Government oversigh

Other

Game/Gamiﬁca�on AR/VR/MR

0 5 10 15 20 25 30

User experience

Content oﬀerings

Consumer and business reluctance

Regula�on and legal risks

Financing and investment

Cost to consumers

Government oversigh

Other

Game/Gamiﬁca�on AR/VR/MR

0 5 10 15 20 25 30

User experience

Content oﬀerings

Consumer/business reluctance

Regula�on and legal risks

Financing and investment

Cost to consumers

Government oversigh

Other

Game/Gamiﬁca�on AR/VR/MR

0 5 10 15 20 25 30

User experience

Content oﬀerings

Consumer/business reluctance

Regula�on and legal risks

Financing and investment

Cost to consumers

Government oversigh

Other

Game/Gamiﬁca�on AR/VR/MR

0 5 10 15 20 25 30

User experience

Content oﬀerings

Consumer/business reluctance

Regula�on and legal risks

Financing and investment

Cost to consumers

Government oversigh

Other

Game/Gamiﬁca�on AR/VR/MR

0 5 10 15 20 25 30

User experience

Content oﬀerings

Consumer/business reluctance

Regula�on and legal risks

Financing and investment

Cost to consumers

Government oversight

Other

Game/Gamiﬁca�on AR/VR/MR

0 1 2 3 4 5 6 7 8 9

Doctor

Nurse

Assistant nurse

Pharmacise

Den�st

Psychologist

Physiotherapist

Therapist

Anaesthe�st

Researcher and developer

Research student

Others

0 1 2 3 4 5 6 7 8 9

Doctor

Nurse

Assistant nurse

Pharmacise

Den�st

Psychologist

Physiotherapist

Therapist

Anaesthe�st

Researcher and developer

Research student

Others

0 1 2 3 4 5 6 7 8 9

Doctor

Nurse

Assistant nurse

Pharmacist

Den�st

Psychologist

Physiotherapist

Therapist

Anaesthe�st

Researcher and developer

Research student

Others

Figure 3: Participant’s position.

part: Q15, Q24, Q33; AR/VR/MR part: Q54, Q64,

Q74) to determine whether to enter the follow-up re-

peat questions at this stage or directly jump to the next

section. If the respondents did not have the user ex-

perience of another software/application, they could

skip these questions. There were instructions at the

beginning of each part, and after the questions, to re-

duce any misunderstanding.

The questions were designed as a single choice,

multiple-choice, and open questions. For the single-

choice or multiple-choice questions, we quantiﬁed the

options and performed descriptive tables to analyse

them. Before the main questionnaire survey was car-

ried out, ﬁve researchers from the internal organisa-

tion were invited to participate in a brief pilot study

to estimate the answering time, test the validity of the

questionnaire, and obtain any potential comments or

feedback on the main questionnaire.

4 RESULTS

By mid-November (2020-11-15), there were 31 an-

swers from ten countries (Sweden, Australia, China,

USA, Netherlands, Thailand, the United Kingdom,

Spain, Japan, and Germany). One answer was

excluded due to the participant position being a

babysitter (not necessarily related to the medical

background requirement). In this work, healthcare-

relevant knowledge and skill are not mandatory, de-

spite useful. Hence, this was classiﬁed to belong to

non-healthcare relevant work in the analysis. Thus,

the following analysis and results were based on 30

individual participants.

The interviewees mainly came from academia (ten

people), hospital (eight people), and primary health

care organisations (six people). As shown in Fig. 3,

the participants were working more as research stu-

dents, researchers, and developers in their organisa-

tion. Their main tasks were research and studying

in their ﬁelds, such as eHealth, information systems,

telecommunication systems, palliative care, dental

hygienists, psychological counselling, and nursing.

Classiﬁcation of the doctors and nurses in the partici-

pants were various, such as being an orthopedist, pae-

diatrician, pharmacist, family doctor, as well as mid-

wife, or in elderly home care.

From the 30 answers, we found six people

having experience of game/gamiﬁcation soft-

ware/applications, and 16 people had experience

with AR/VR/MR software/applications, which

involved six games, and nine AR/VR/MR soft-

ware/applications amongst them. Based on the

used techniques, these software/applications were

classiﬁed into three groups: (1) game/gamiﬁcation

(without AR/VR/MR), (2) game/gamiﬁcation

and AR/VR/MR, and (3) AR/VR/MR (non-

game/gamiﬁcation). Table 1 shows the general

information (name and feature, purpose, target user,

and use occasion) of the software/applications.

4.1 Application Purpose and Target

User

Game/gamiﬁcation and AR/VR/MR applications in

healthcare were shown to aim to deal with a variety of

health issues. Applications’ purpose were classiﬁed

as three groups with eight subgroups, for prevention

(including disease prevention, health behaviour train-

ing/health knowledge teaching, and encouraging ex-

ercise), for treatment (including mental treatment, ad-

juvant therapy, physiotherapy and rehabilitation), and

for medical education and training.

As shown in Table 1, there were 15 soft-

ware/applications mentioned in the survey answers.

The game/gamiﬁcation techniques were mainly used

HEALTHINF 2021 - 14th International Conference on Health Informatics

736

Table 1: The general information of the software/applications.

Name/feature Purpose User Occasion

Game/gamiﬁcation

Digital patient meeting

Mental treatment,

Disease prevention

Hospital-based care provider,

Patient

Hospital

Null

Medical education and training,

Health behaviour training/ Health knowledge teaching,

Adjuvant therapy

Hospital-based care provider,

Home-based care provider,

Medical student

Hospital, Home,

School/University/Institute,

Outdoors

Treatment for autistic children Mental treatment, Adjuvant therapy Patient Hospital

Word puzzle Disease prevention Patient, High-risk group of certain diseases Home

HUAWEI health wrist ring

Health behaviour training/ Health knowledge teaching,

Encouraging exercise

Hospital-based care provider,

Home-based care provider

Hospital, Home, Outdoors,

School/University/Institute

Game/gamiﬁcation + AR/VR/MR

”Deep”

Mental treatment,

Health behaviour training/ Health knowledge teaching

High-risk group of certain diseases,

Patient, Home-based care provider,

Hospital-based care provider,

Lecturer and Researcher, Patient

Hospital, Home,

School/University/Institute

”Stress jam”

Mental treatment,

Health behaviour training/ Health knowledge teaching

High-risk group of certain diseases,

Patient

Home,

Hospital

VR technique in simulation CPR Medical education and training Medical student School/University/Institute

VR exercise game

Encouraging exercise,

Health behaviour training/ Health knowledge teaching

High-risk group of certain diseases,

Patient

Home

The model video of operation

Mental, treatment,

Medical education and training,

Health behaviour training/ Health knowledge teaching,

Adjuvant therapy, Disease prevention

Home-based care provider,

Hospital-based care provider,

Lecturer and Researcher, Patient

Hospital, Home,

School/University/Institute

AR/VR/MR

For autism spectrum disorder treatment Mental treatment Patient Hospital

3D body map,

3D human body anatomy

Medical education and training Medical student

Home, Hospital,

School/University/Institute

Training software for X-ray imaging Medical education and training Hospital-based care provider

Hospital,

School/University/Institute

Virtual surgery,

Surgical simulation

Medical education and training Medical student

Hospital,

School/University/Institute

”MindMaze”

(stroke rehabilitation training)

Rehabilitation, Encouraging exercise

Hospital-based care provider,

Home-based care provider, Patient

Home, Hospital

Note: The ”Null” means the information was not mentioned in the participant’s answer.

in mental health treatment, and health behaviour

training/health knowledge teaching, especially com-

bined with the AR/VR/MR technology. Those two

kinds of software/applications purposes involved the

treatment of mental health problems such as autism,

stress (”Stressjam”), and anxiety (”Deep”); as well as

physical training.

Except for being used in the mental health treat-

ment and health behaviour training/health knowledge

teaching, the AR/VR/MR techniques were widely

used in medical education and training, such as in

cardiopulmonary resuscitation (CPR), surgery, and

3D body map (for medical students to study the

knowledge of human anatomy). It is worth noting

that AR/VR/MR in medical education and training

was commonly used without game/gamiﬁcation tech-

niques.

Based on the purpose and features, the primary

target user of the software/applications were hospital-

based care provider and patient. They were generally

used in the hospital and home. Furthermore, the soft-

ware/applications for medical students were mainly

used in a school, university, or institute.

4.2 Use Time and Frequency

Just four participants mentioned the soft-

ware/applications use time, and two refer to the

frequency. There were four years for the HUAWEI

health wrist ring used in the interviewee’s organi-

sation and two years for the Deep and Stressjam,

as well as one year for the CPR simulation. The

HUAWEI health wrist ring used in the interviewee’s

organisation was reported to be used two hours per

week, and for Deep, it was about ﬁve hours. The

other 27 participants claimed they did not know the

use time and frequency of the software/applications

in their organisations.

4.3 Advantages and Disadvantages

From the view of advantages in game/gamiﬁcation

software/applications, some participants believed

they were helpful, such as the HUAWEI health wrist

ring for users’ health, as well as autism spectrum

disorder (ASD) treatment software for ASD patients,

their families, and doctors. Moreover, an intervie-

wee said, ”game is more in line with children’s cogni-

tion and acceptance”. Another person thought game

applications provide stronger motivation than non-

game activities. On the other hand, the interviewees

mentioned the disadvantages of game/gamiﬁcation

techniques as well. One of them pointed out the

cost of suitable equipment and software support be-

ing quite expensive. Two others said problems of

game/gamiﬁcation could be that they might not be ap-

propriate and could be addictive.

A Survey of Possibilities and Challenges with AR/VR/MR and Gamiﬁcation Usage in Healthcare

737

From the view of advantages of AR/VR/MR tech-

niques in healthcare software/applications, the partic-

ipant who mentioned the application for the model

video operation believed it is more precise, more use-

ful for the end of the operation and to get a better re-

sult. Using a CPR simulation application, one inter-

viewee claimed the application provided the ability to

change the setting for the patient/student. Moreover,

the user of the 3D body map said, it is intuitive, clear,

and easier to perceive the spatial position and rela-

tionship of various organs, bone, and blood vessels

than in books. Furthermore, interviewees mentioned

the advantage of simulation surgery was beneﬁcial for

being able to repeat several times. For Deep, safety

was mentioned, for VR exercise game, it was the bet-

ter gaming experience, and for X-ray imaging train-

ing applications, it was also safe and inﬁnitely repeat-

able. The disadvantages of AR/VR/MR techniques

in healthcare software/applications mainly focused on

high cost and equipment. Especially the high cost

of costumers led by over-reliance on high-tech elec-

tronic equipment, as well as problems with ﬂuency

and low quality presented caused by data transmis-

sion delays.

4.4 Obstacles to Mass Adoption

As shown in Fig. 4, the largest obstacle of

game/gamiﬁcation techniques to mass adoption was

the user experience (25), and the following three were

the main challenges: content offering (18), the cost

to consumers (12) and the ﬁnancing and investment

(12) were tied for third. The ﬁrst two obstacles of

the AR/VR/MR techniques to mass adoption were the

same as the game/gamiﬁcation techniques: user expe-

rience (25) and content offering (23). However, the

following were different. For the AR/VR/MR tech-

niques, the cost to consumers (18) was the third ob-

stacle, and the ﬁnancing and investment (17) and the

consumer and business reluctance (17) were tied as

the fourth obstacle.

4.5 Requirements and Concerns

From the 30 answers, there were ten requirements

of game/gamiﬁcation and AR/VR/MR techniques in

healthcare, as shown in Table 2. The keywords can

be separated into two groups, the aim, purpose, and

target users. We can see the requirements of physi-

cal health issues, such as exercise, rehabilitation, and

muscle relaxation. Furthermore, the requirements put

more attention to psychological and mental health

problems both in treatment and training. Moreover,

medical education and training were also mentioned

in the requirements. The target user was not only or-

dinary and patients; some interviewees pointed out,

especially the requirements facing students, elderly,

children, and babies. However, the interviewees also

expressed concerns about the process of technology

realisation. Someone believed it is difﬁcult to com-

mercialise. Another thought that AR/VR/MR tech-

niques might be too high tech for their patient groups

who are the elderly over 65 years old. Moreover, the

clinical effect could be challenging to verify.

5 DISCUSSION

Although several interviewees coming from academia

may cause many software/applications related to

medical education and training, it was undeniable

the numbers of mature software/applications used in

healthcare. Students and researchers, as their primary

target users, may lead to an easier acceptance of new

techniques. The requirements of the medical edu-

cation and training software/applications were more

straightforward and clear to analyse, such as the 3D

body map for studying the human body anatomy, and

X-ray imaging training for learning how to take X-

ray images. Thus, the design and prototype process

of such software was more comfortable to implement

and evaluate.

A safe, controlled, and a repeatable environment;

multi-angle presentation; and simulation of the real

world were the main reasons for the AR/VR/MR tech-

niques used in healthcare. The AR/VR/MR tech-

niques provided a virtual environment similar to the

real world. It is a safe and controllable environ-

ment for expensive and high-risk teaching or train-

ing. Users can consolidate knowledge and skills in

actual operations and avoid the serious consequences

of incorrect operations. However, for serious pur-

poses, the proportion of using games/gamiﬁcation

techniques in this kind of software/applications is rel-

atively small. However, there is still space to combine

game/gamiﬁcation techniques into medical education

and training software/applications.

Training skills in taking care of babies could

look like a ”medical education and training” soft-

ware, but for new parents instead of users with

a medical background. It was mentioned both in

the game/gamiﬁcation and AR/VR/MR technique re-

quirements in healthcare. In an environment that

maintains social distancing and avoids gatherings for

safety reasons, software/applications have become a

better solution instead of face-to-face instruction.

The topic of treating psychological and mental

health problems is an account for a large proportion

HEALTHINF 2021 - 14th International Conference on Health Informatics

738

0 1 2 3 4 5 6 7 8 9

Doctor

Nurse

Assistant nurse

Pharmacise

Den�st

Psychologist

Physiotherapist

Therapist

Anaesthe�st

Researcher and developer

Research student

Others

0 1 2 3 4 5 6 7 8 9

Doctor

Nurse

Assistant nurse

Pharmacise

Den�st

Psychologist

Physiotherapist

Therapist

Anaesthe�st

Researcher and developer

Research student

Others

0 5 10 15 20 25 30

User experience

Content oﬀerings

Consumer and business reluctance to

embrace game/gamiﬁca�on technique

Regula�on and legal risks

Financing and investment

Cost to consumers

Government oversigh

Other

0 5 10 15 20 25 30

User experience

Content oﬀerings

Consumer and business reluctance to…

Regula�on and legal risks

Financing and investment

Cost to consumers

Government oversigh

Other

Game/Gamiﬁca�on AR/VR/MR

0 5 10 15 20 25 30

User experience

Content oﬀerings

Consumer and business reluctance to…

Regula�on and legal risks

Financing and investment

Cost to consumers

Government oversigh

Other

Game/Gamiﬁca�on AR/VR/MR

0 5 10 15 20 25 30

User experience

Content oﬀerings

Consumer and business reluctance

Regula�on and legal risks

Financing and investment

Cost to consumers

Government oversigh

Other

Game/Gamiﬁca�on AR/VR/MR

0 5 10 15 20 25 30

User experience

Content oﬀerings

Consumer and business reluctance

Regula�on and legal risks

Financing and investment

Cost to consumers

Government oversigh

Other

Game/Gamiﬁca�on AR/VR/MR

0 5 10 15 20 25 30

User experience

Content oﬀerings

Consumer and business reluctance

Regula�on and legal risks

Financing and investment

Cost to consumers

Government oversigh

Other

Game/Gamiﬁca�on AR/VR/MR

0 5 10 15 20 25 30

User experience

Content oﬀerings

Consumer and business reluctance

Regula�on and legal risks

Financing and investment

Cost to consumers

Government oversigh

Other

Game/Gamiﬁca�on AR/VR/MR

0 5 10 15 20 25 30

User experience

Content oﬀerings

Consumer and business reluctance

Regula�on and legal risks

Financing and investment

Cost to consumers

Government oversigh

Other

Game/Gamiﬁca�on AR/VR/MR

0 5 10 15 20 25 30

User experience

Content oﬀerings

Consumer/business reluctance

Regula�on and legal risks

Financing and investment

Cost to consumers

Government oversigh

Other

Game/Gamiﬁca�on AR/VR/MR

0 5 10 15 20 25 30

User experience

Content oﬀerings

Consumer/business reluctance

Regula�on and legal risks

Financing and investment

Cost to consumers

Government oversigh

Other

Game/Gamiﬁca�on AR/VR/MR

0 5 10 15 20 25 30

User experience

Content oﬀerings

Consumer/business reluctance

Regula�on and legal risks

Financing and investment

Cost to consumers

Government oversigh

Other

Game/Gamiﬁca�on AR/VR/MR

0 5 10 15 20 25 30

User experience

Content oﬀerings

Consumer/business reluctance

Regula�on and legal risks

Financing and investment

Cost to consumers

Government oversight

Other

Game/Gamiﬁca�on AR/VR/MR

Figure 4: Obstacles to mass adoption of game/gamiﬁcation and AR/VR/MR.

Table 2: Requirement of game/gamiﬁcation and AR/VR/MR technique in healthcare software/applications.

Requirement

Game/

Gamiﬁcation

AR/VR/MR

Prescribe exercise to clients •

Rehabilitation • •

Treatment of psychological and mental problems, such as phobia, PTSD, and dementia. • •

Game for the elderly health • •

Psychological diagnosis or psychotherapy, especially for the elderly and children. • •

Taking care skills training of baby • •

Memory training •

Mental relaxation and distraction • •

Teaching students •

Muscle relaxation after prolonged sitting •

of the overall, both in the results of existing software

and the requirements designed. This theme has great

demand and space for development. Similar to the

medical education and training software/applications,

the wide application of AR/VR/MR techniques in

such software mainly lies in being safe and control-

lable. Virtual objects provide a safe and controllable

treatment environment for doctors. They can inﬂu-

ence the mental state of users through the tasks they

are designed to provide treatment or relief, such as

Deep and Stressjam. Games or gamiﬁcation tech-

niques could bring fun and motivation in the treat-

ment, promoting usage time and effect. As mentioned

by one interviewee, the gamiﬁcation interaction was

more friendly to children’s cognitive level. In gami-

ﬁed tasks, they were easier to understand and accept

in the software/application. This could also apply to

users with cognitive impairments.

As the problem of population ageing on a

global scale becomes increasingly serious, the health

problems of the elderly are receiving more atten-

tion. This is also conﬁrmed from the results of

the requirements in healthcare. Many interviewees

mentioned game/gamiﬁcation and AR/VR/MR soft-

ware/applications requirements for elderly healthcare.

Although the sample size of the total intervie-

wees and each job position could be more extensive,

the above results, show some usage and development

trends of game/gamiﬁcation and AR/VR/MR tech-

niques in healthcare. Similar to the results in the

related work, we can see the software/applications

aimed at location-based exergaming and rehabilita-

tion. It can also be seen that the AR/VR/MR tech-

niques can be effective in some healthcare issues. On

the other hand, the result of obstacles to mass adop-

tion was similar to the related work as well. The

user experience and content should be the most cru-

cial quality indexes to consider. The usefulness of the

software as the solution in healthcare and the accept-

ability of using AR/VR/MR were essential as well.

Including medical experts and target users in the de-

velopment and evaluation may help to solve this issue.

Moreover, the customer cost (hardware and software)

should balance with the user experience and effects.

6 CONCLUSIONS

This paper has presented an online survey, based on

30 participants, exploring the possibilities and chal-

lenges with AR/VR/MR and game/gamiﬁcation usage

in healthcare. The participants believed these tech-

niques were helpful for some health issues due to the

A Survey of Possibilities and Challenges with AR/VR/MR and Gamiﬁcation Usage in Healthcare

739

acceptance and motivation in game/gamiﬁcation; as

well as the safety, repeatability, and potential for an

improved experience in AR/VR/MR. However, the

seen challenges were relating to cost and equipment.

There were some requirements of game/gamiﬁcation

and AR/VR/MR in healthcare software/applications,

such as in mental and psychological treatment, el-

derly health, and skills training in caretaking. The

interviewees also pointed out the worries of the tech-

niques and obstacles to mass adoption. Although the

sample size was limited, a conclusion was still ob-

tained, that game/gamiﬁcation and AR/VR/MR have

been accepted and applied in healthcare, as well as

highlighting further development space of software

and hardware. These insights are useful for both re-

searchers and developers to know the state and devel-

opment trends of game/gamiﬁcation and AR/VR/MR

in healthcare. Future work will extend the survey to

a larger sample size, in several countries, to obtain

more universal results.

REFERENCES

Blusi, M. (2014). E-health and information-and communi-

cation technology (ICT) as support systems for older

family caregivers in rural areas. PhD thesis, Mittuni-

versitetet.

de Lima, R. M., de Medeiros Santos, A., Neto, F. M. M.,

de Sousa Neto, A. F., Le

ao, F. C. P., de Macedo, F. T.,

and de Paula Canuto, A. M. (2016). A 3d serious game

for medical students training in clinical cases. In 2016

ieee international conference on serious games and

applications for health (segah), pages 1–9. IEEE.

Deterding, S., Dixon, D., Khaled, R., and Nacke, L. (2011).

From game design elements to gamefulness: deﬁn-

ing” gamiﬁcation”. In Proceedings of the 15th inter-

national academic MindTrek conference: Envisioning

future media environments, pages 9–15.

Jorge, J., Campos, P., and Lopes, D. S. (2019). Approaches

and challenges to virtual and augmented reality in

health care and rehabilitation. In SIGGRAPH Asia

2020 Courses, pages 1–90.

Li, X., Yi, W., Chi, H.-L., Wang, X., and Chan, A. P.

(2018). A critical review of virtual and augmented

reality (vr/ar) applications in construction safety. Au-

tomation in Construction, 86:150–162.

Lima, T., Barbosa, B., Niquini, C., Ara

ujo, C., and Lana,

R. (2017). Playing against dengue design and devel-

opment of a serious game to help tackling dengue. In

2017 IEEE 5th International Conference on Serious

Games and Applications for Health (SeGAH), pages

1–8. IEEE.

LLP, P. C. and the XR Association (2019). 2019 augmented

and virtual reality survey report.

Ma, M., Jain, L. C., and Anderson, P. (2014). Future trends

of virtual, augmented reality, and games for health.

In Virtual, augmented reality and serious games for

healthcare 1, pages 1–6. Springer.

Okeil, A. (2010). Hybrid design environments: immer-

sive and non-immersive architectural design. Journal

of Information Technology in Construction (ITcon),

15(16):202–216.

Sacks, R. and Pikas, E. (2013). Building information mod-

eling education for construction engineering and man-

agement. i: Industry requirements, state of the art, and

gap analysis. Journal of Construction Engineering

and Management, 139(11):04013016.

Singer, J. and Vinson, N. G. (2002). Ethical issues in em-

pirical studies of software engineering. IEEE Trans-

actions on Software Engineering, 28(12):1171–1180.

Voigt, P. and Von dem Bussche, A. (2017). The eu general

data protection regulation (gdpr). A Practical Guide,

1st Ed., Cham: Springer International Publishing.

Wang, P., Wu, P., Wang, J., Chi, H.-L., and Wang, X.

(2018). A critical review of the use of virtual real-

ity in construction engineering education and training.

International journal of environmental research and

public health, 15(6):1204.

HEALTHINF 2021 - 14th International Conference on Health Informatics

740