Immersive Visualizations in a VR Cave Environment for the Training
and Enhancement of Social Skills for Children with Autism
Skevi Matsentidou and Charalambos Poullis
Immersive and Creative Technologies Lab,
Cyprus University of Technology, Limassol, Cyprus
Keywords: Training, Immersion, Visualization, VR Systems and Toolkits, Autism.
Abstract: Autism is a complex developmental disorder characterized by severe impairment in social, communicative,
cognitive and behavioral functioning. Several studies investigated the use of technology and Virtual Reality
for social skills training for people with autism with promising and encouraging results (D. Strickland,
1997; Parsons S. & Cobb S., 2011). In addition, it has been demonstrated that Virtual Reality technologies
can be used effectively by some people with autism, and that it had helped or could help them in the real
world; (S. Parsons, A. Leonard, P. Mitchell, 2006; S. Parsons, P. Mitchell, 2002). The goal of this research
is to design and develop an immersive visualization application in a VR CAVE environment for educating
children with autism. The main goal of the project is to help children with autism learn and enhance their
social skills and behaviours. Specifically, we will investigate whether a VR CAVE environment can be
used in an effective way by children with mild autism, and whether children can benefit from that and apply
the knowledge in their real life.
1 INTRODUCTION
The goal of the project is to design and develop an
immersive visualization application in a VR CAVE
environment for educating and treating children with
mild autism. The main goal of the project is to help
children with autism enhance and improve their
social skills and behaviors. We address the
following research questions:
- Can virtual reality technologies, and in
particular the aforementioned application, be
effectively involved in enhancing the social skills
and behaviors of children?
- Can the immersive visualization application be
considered as a new and innovative method of
treatment?
Autism is a complex developmental disorder
characterized by severe impairment in social,
communicative, cognitive and behavioral
functioning. It belongs to a group of disorders
known as Autism Spectrum Disorder (ASD) and a
broader category of pervasive developmental
disorders (D. Strickland, 1997).
Autism Spectrum Disorder (ASD) is
characterized by impairments in social interaction,
social communication and imagination, stereotyped
and repetitive behaviors and a resistance to change
in routine. Intellectual disability is present in a large
proportion of individuals (B. Robins, K.
Dautenhahn, R. te Boekhorst, 2005; G. Dawson,
2010; K. Dautenhahn, 2000). Some basic
characteristics of people with autism are reduced
emotional attachment, absence or abnormal speech,
ritualistic behaviors, aggression and self-harm.
Typical is also the lack of eye contact (Matson J.L.,
Matson M.L., Rivet T.T., 2007; C. Lord, 2000),
attention deficit, motor incoordination, symptoms of
anxiety and depression (E. L. Hill and U. Frith,
2003). In addition some other characteristics are the
reduced repertoire of activities and interests, and a
tendency of fixation to stable environments (K.
Dautenhahn, 2000).
Recent research investigated the use of virtual
environments for social skills training, as an
augmentation to existing methods and approaches.
People with ASDs had the ability to use Virtual
Environments (VEs) successfully, and learn simple
social skills using the technology. VR technologies
provide safe, realistic-looking 3-D scenarios that can
be built to depict everyday social scenarios. The
possibility of exploring these scenarios in real-time
makes them an attractive tool for teaching social
230
Matsentidou S. and Poullis C..
Immersive Visualizations in a VR Cave Environment for the Training and Enhancement of Social Skills for Children with Autism.
DOI: 10.5220/0004844602300236
In Proceedings of the 9th International Conference on Computer Vision Theory and Applications (VISAPP-2014), pages 230-236
ISBN: 978-989-758-009-3
Copyright
c
2014 SCITEPRESS (Science and Technology Publications, Lda.)
skills to people with ASDs (S. Parsons, A. Leonard,
P. Mitchell, 2006).
Based on information provided by the Cyprus
Autistic Association (2009), an interesting treatment
method of enhancing social skills are ‘social stories’.
Social stories is an educational strategy developed
by Carol Gray (The Gray Center) which aims to
teach people with autism how to carry out some
social processes to their daily life through story
telling. A social story is presented with the help of
pictures in a specific order and shows the
performance of an action.
A study (K S. Thiemann and H. Goldstein, 2001)
that investigated the effects of social stories, on the
social communication of 5 students with autism and
social deficits, showed increases in targeted social
communication skills when the treatment was
implemented. (K S. Thiemann and H. Goldstein,
2001).
In this work, we present the design and
development of an immersive visualization
application for the treatment of social skills for
children with autism which unifies the advantages of
the ‘social story’ teaching method and state-of-the-
art technologies and in particular a VR CAVE.
The goal is to develop virtual social stories in an
immersive VR CAVE environment which will be
used to educate children about some specific social
skills. The training will focus on real-life situations
which are difficult or impossible to explain to
children with the traditional way - pictorial cuing.
For example, consider an unsafe situation or a
situation where the child must have an experience of
it in order to learn how to act e.g. crossing the road
or avoiding cars.
This research is conducted in
collaboration/cooperation and support of Cyprus
Autistic Association.
The paper is organized as follows: Section 2
presents a brief overview of the state-of-the-art in
the area of teaching with the use of virtual
environments. Section 3 describes our methodology
and in Section 4 the design and development of the
application is presented.
2 RELATED WORK
A plethora of work has been conducted in the area.
Below we provide a brief overview of the state-of-
the-art.
The benefits of using virtual environments as
remedial learning environments for children with
autism are many. The features of virtual
environments which make them suitable as learning
tools for children with autism are:
Controlled and safe learning environments: In
virtual environments input stimuli can be controlled
and the behavior of the child can be monitored.
Environments can be customized to account for
individual differences. Children can be guided
through learning experiences and explore new
behavioral opportunities by themselves. Virtual
learning environments can provide safe
environments, a less dangerous and more forgiving
environment for developing skills associated with
activities of daily living. In addition, mistakes are
less catastrophic compared with the real world (D.
Strickland, 1997; K. Dautenhahn, 2000). Tina R.
Goldsmith and Linda A. LeBlanc (2004) indicate
that some of the most notable benefits of virtual
reality are the incomparable control over the
environment, and that it may offer highly realistic
but safe environment in which to teach skills that are
associated with some level of danger when taught in
the natural environment (T. Goldsmith, L. LeBlanc,
2004).
Generalization: An important issue and problem
of all therapeutic approaches to autism is
generalization. It is difficult to the child to
generalize the learning experiences and applying the
skills to non-classroom situations, even if the child
shows improved performance in the classroom.
Virtual environments have the benefit to change
dynamically, create alternative scenarios or
variations, and increase the complexity of scenario
very easy (K. Dautenhahn, 2000; S. Parsons, P.
Mitchell, 2002). Few modifications across similar
scenes may allow generalization. A child with
autism who learns how to cross a virtual street, in
one scene might generalize to another street scene if
the differences are reduced until the similarities are
recognizable. (D. Strickland, 1997).
A primarily visual/auditory world: Virtual
environments highlight visual and auditory
responses rather than other senses such as touch.
Such responses effectively involved in teaching
abstract concepts to people with autism. Individuals
with autism indicate their thought patters are
primarily visual (D. Strickland, 1997).
Individualized Treatment: Individuals with
autism vary widely in their abilities, strengths and
weaknesses. Each individual may even demonstrate
wide variation in skills and behavior between
different days. Taking this into account, an
individualized approach of treatment and training
based on a careful, personalized assessment is
essential. Virtual environments deemed appropriate
ImmersiveVisualizationsinaVRCaveEnvironmentfortheTrainingandEnhancementofSocialSkillsforChildrenwith
Autism
231
considering the ability to change dynamically and be
customized based on each individual separately (D.
Strickland, 1997; S. Parsons, P. Mitchell, 2002).
Preferred Computer Interactions: The
complexity of social interaction can interfere when
teaching individuals with social disorders. Human
interaction can be so difficult and disruptive that
learning is not possible. Children with autism
characterized by proactive behavior, they prefer a
predictable, structured and in this way ‘safe’
environment. They prefer to be in ‘control’ of the
interaction and they respond well to structure,
explicit, consistent expectations, and challenge
provided by computers. Virtual environments are
stable, familiar, predictable, and allow children to
learn basic social interactions in consistent and
accepting way (D. Strickland, 1997; K. Dautenhahn,
2000; S. Parsons, P. Mitchell, 2002).
Embodied Interaction: Virtual environments
devices (e.g. VR helmets, hand controls), might be
unacceptable for many autistic children, but for
others might be appropriate. Approaches which
support interactions involving the whole body seem
highly promising; set ups where the child can freely
move and is not constrained to sitting at a desk and
is not required to wear special devices. The use of
body and head trackers provides other advantages
and possibilities. The movements and actions of an
individual can be controlled in a virtual
environment, allowing the system to adjust to a
patient’s actions. A large proportion of individuals
with autism never learn to communicate, this allows
interactions in virtual scenes without verbal training
from a teacher or other controllers (D. Strickland,
1997; K. Dautenhahn, 2000).
Immersion: The sense of immersion refers to
the feeling of being part of, or engaged in one virtual
scene. Feeling like you are really inside the virtual
environment (Parsons S. & Cobb S., 2011). Parsons
and Cobb (2011) report that in a research of Mineo
(2009) have been compared responses of 42 children
with autism to three different electronic media
conditions. The thrird condition in which the child
engaged directly in activities within an immersive
VR, was classified as the most immersive media
condition. Children were more engaged with this
technology – they spent more time looking at the
screen. As the author points out, a further
investigation needs to be done, in order to check
whether this aspect of VR can be translated into
effective instruction and learning. Parsons and Cobb
(2011) indicate another study of Wallace (2010),
which explored the responses of adolescents with
autism to an immersive ‘Blue Room’ (animations
projected onto the walls, no headsets needed).
Participants experienced scenes from a street,
playground and a school and asked to rate their
feeling of ‘presence’. Results show positive
experiences and that immersive VR has the
advantage of allowing realistic and accessible scenes
that could form the basis of important social role-
play (S. Parsons, S. Cobb, 2011).
A study of Strickland, Marcus, Mesibov and
Hogan (1996), showed that children with autism
were able and willing to accept and interact within
virtual reality worlds. Also demonstrated that
children respond to the virtual world in a meaningful
way, and tolerated wearing virtual reality equipment.
Max and Burke at 1997 demonstrated that the virtual
environment improved children’s attention and
performance across sessions (T. Goldsmith, L.
LeBlanc, 2004). S. Parsons and P. Mitchell (2002),
conclude that virtual environment is an exciting tool
that can extend the existing teaching practices and
methods for social skills treatment for people with
ASDs. Moreover, it provides a safe and supportive
learning environment that succeeds to transfer
knowledge between virtual and real world (S.
Parsons, P. Mitchell, 2002).
Strickland (1997) investigated the use of Virtual
Worlds as a learning tool for children with autism.
Two autism children (a seven year old girl and a
nine year old boy) took part in the study which
consisted of over forty virtual exposures (each less
than five minutes). The goal of the research was to
help children with autism, cross a road safely. The
first part of exposures was to train the child to
recognize and track a moving car within a street
scene. The second phase was to train the child to
find an object and the color of it in the environment,
walk to it, and stop. Finally, with the learned skills
the child should have the ability to cross a street
alone. They used VR helmets for the immersion in
the 3D environment, and the results proved that
children are able to use them. In addition, the results
have demonstrated that children immersed
themselves in the virtual scene, were able to track
the moving cars and verbally labeled objects and
their colors. Also children tracked moving objects
with eyes, head and body turning and located objects
(signs) and walked towards them. Because of the
small number of participants the results cannot be
generalized. (D. Strickland, 1997).
S. Parsons, A. Leonard, P. Mitchell (2006),
investigated the use of virtual environment for social
skills training with two adolescents boys (14 and 17
years old) with ASDs. There are two types of VEs -
a bus and a café – both of which were presented to
VISAPP2014-InternationalConferenceonComputerVisionTheoryandApplications
232
participants on a laptop. Feedback and instructions
provided through textual, picture and audible
prompts from the program. In both VE scenarios,
the user’s aim is to find an appropriate place to sit in
an empty or busy café/bus and to ask appropriate
questions when needed (e.g. if he can sit next to one
stranger). Participants take a number of sessions and
one session after 3 months, to check whether any
new knowledge from the VE had been maintained
over the summer school break. Results showed that
they had remembered social knowledge gained
during their VE sessions, they have a good
understanding of the purpose of the VEs and were
able to offer examples of how it had helped them
now, and could help them in the future. Also,
participants seemed to enjoy the VE sessions and
showed that they can learn about social ‘errors’ in a
safe way, without any stress (S. Parsons, A.
Leonard, P. Mitchell, 2006).
Strickland, McAllister, Coles, Osborne (2007),
investigated whether children with ASD would use
VR equipment and whether they could learn in a
virtual environment. The VR system consisted from
a head-mounted display, body trackers, and three-
dimensional hand controls. Participants were two
children (a girl and a boy, 7 and 9 years old) and
were placed in a street crossing virtual scene to learn
two basic steps of stopping at a stop sign and
tracking moving cars before crossing a street.
Before exposure, neither child displayed awareness
of street boundaries or demonstrated normal safety
actions. While in the virtual environment both
children immersed themselves in the scenes,
verbally labeled objects and colors of objects,
moved their bodies when tracking a moving car, and
located a stop sign. The parents of the girl indicated
that she did track moving cars in the real world with
her head after VR training (D. Strickland, D.
McAllister, C. Coles, S. Osborne, 2007).
Parsons and Cobb (2011) refer to studies which
suggest that children can learn information from VR
and some can transfer this knowledge to the real
world. Strickland (2007) developed desktop VEs to
teach fire safety skills, like how to recognize the fire
danger and how to respond. Eleven out of the 14
children who took part completed the VE session
without error. An animated character named Buddy
demonstrated proper actions and continually
interacted with the child (D. Strickland, D.
McAllister, C. Coles, S. Osborne, 2007). Self
(2007) developed a fire safety and tornado safety
VR training application. Children were able to use
the program successfully, but the responses of
children varied widely and there was limited
evidence of generalization of understanding to real
fire and tornado situations. Furthermore, Josman
(2008) tested VR as a tool for teaching children with
autism to cross the road safely. The results showed
that children could use the VE successfully and
improve their skills to cross a virtual street during
the study. Some of them were able to transfer this
learned knowledge to a real street. The authors
emphasized that programs need to be carefully
targeted according to the individual needs of
children (S. Parsons, S. Cobb, 2011).
Parsons and Cobb (2011) stated that
nevertheless, the overall scale of the research about
VR for educational purposes generally (and
supporting social skills specifically), is undeniably
limited. Because of the limited scale of research the
results characterized as equivocal. The challenge is
to translate the VR applications into flexible,
workable, useful and realistic tools for everyday
classrooms. The authors refer to an interesting find
that the more realistic a virtual environment, the
more generalization will be achieved. (S. Parsons, S.
Cobb, 2011).
3 METHODOLOGY
3.1 Method
This work is conducted in collaboration/cooperation
and support of Cyprus Autistic Association. The
evaluation is ongoing and preliminary results are
promising. The evaluation proceed as follows: a
number of children with ASDs took a session using
the application in a VR CAVE environment for the
training of specific social skills. The engagement
and the use of the application by children during the
sessions will be measured/ determined which will
provide an insight on the effectiveness of the
application.
The goal of this research is to design and develop
an immersive application in a VR CAVE
environment for educating/treating children with
autism. The training focuced in situations of real
life which are difficult or impossible to explain to
children with the known methods of treatment. For
example, an unsafe situation or a situation where the
child must have an experience of it in order to learn
how to act e.g. crossing the road or avoiding moving
cars.
The treatment focused on situations where the
child will be alone and helpless in an unknown
place. Specifically, we developed a scenario where
the child wonders off in the city alone (“Lost in
ImmersiveVisualizationsinaVRCaveEnvironmentfortheTrainingandEnhancementofSocialSkillsforChildrenwith
Autism
233
city”). He/she should be prepared to stay calm and
act accordingly. The child should learn, under the
guidance of educator, how to cross the road safely.
The training on the scenario splited in various parts
so that the child builds step by step the knowledge
which is necessary in order to cope with the next
part of the treatment. In this way the new
knowledge “digested” better and easier and the
learned skills could lead to new abilities.
The parts of the training at the scenario, “Lost in
city”, executed as follows:
The child trained to recognize, track, and avoid the
moving cars within the virtual street scene
The child trained to recognize and find the
crossing button, walk to it, stop, and press it
The child learned to recognize the lights and to
interprets them
The child trained to recognize the crossing and
walk to it if he can – according to the lights and
cars
The final phase was the most difficult; the child
had to cross the street alone without any help.
Figure 2 (a) shows the user in the VR CAVE
with the scenario (“Lost in city”). The user can
navigate and interact within the virtual environment.
For the evaluation of the application we consider
the degree of involvement and use of the application
by children during the session. Behaviours, fillings,
difficulties, problems, actions or activities of the
children recorded and analysed. Also examined
whether children were able to accept and use the
equipment of the application in the VR CAVE
environment.
In addition, taken into account whether children
were able to successfully complete the session and
whether they succeeded to transfer knowledge from
the virtual to the real world. Furthermore educator
and parents completed questionnaires. Educator was
one of the users during the session. Also, as an
expert with children with autism he was able to
provided informations that we have not noticed or
thought. Parents will be present at sessions, and as
knows their children well, so are the most
appropriate people to mention things that they
observed to their child and they considered as an
important.
During the development process of the
application a first formative evaluation was
conducted with the help of one of the educators of
the Cyprus Autistic Association. This evaluation
provide guiding feedback for the proper completion
of the application. Also we have the opportunity for
a second empirical evaluation that was made
Figure 2: (a) User in the VR CAVE, (b) A scene of the
crossing learning environment.
a second empirical evaluation that was made
towards the end of the development phase with the
participation of twelve children (9-10 years old)
without autism. This evaluation helped us to
evaluate the application as a learning tool for
children, and to understand important features which
must identified a session in order the training be
correct, complete and effective. These features have
taken into account in the training of children with
Autism.
The results from this evaluation are encouraging.
Children learned easily to used and interact with the
system and devices without important difficulties or
problems. Also, completed the sessions with signs
of improvement between the different trials.
Children showed samples of excitement, they laugh
and smile during and after the completion of the
session and gave us to understand that they enjoy the
whole experience.
3.2 Equipment
The equipment that we used is the EON REALITY
iCube VR CAVE environment where the application
run and the 3D virtual world present, the 3D glasses
which the participant wears, and the xbox controller
that participant or educator uses for the interaction
and the navigation in the virtual environment. The
participant be in the VR CAVE and interact with the
virtual environment through the 3D glasses and the
xbox controller. Both 3D glasses and xbox
controller have markers on them so that the position
and direction of them detect. In this way the virtual
environment adjust according to participant’s head
movements. For example, if the participant moves
his head to see beyond a wall then the virtual
environment will be adjusted properly. Also, with
the 3D glasses, the participant will be able to see in
three dimensions the virtual world. The xbox
controller needed for the navigation and interaction
in the 3D virtual environment (e.g. navigate and
select objects in the scene). The VR CAVE consists
of four HD screens, four projectors and cameras.
VISAPP2014-InternationalConferenceonComputerVisionTheoryandApplications
234
Figure 3: VR CAVE, glasses, xbox controller.
The screens are placed in such a way that creates a
box without a roof and front part (See Figure 3).
The participant when use the VR CAVE is within
this and with the help of the glasses see the virtual
world in three dimensions, all that make user to fully
immerse in the virtual environment.
4 DESIGN & DEVELOPMENT
For the design and development of the immersive
application in the VR CAVE environment we used
the EON Studio Professional for the implementation,
Autodesk Maya and 3ds Max for the 3D modelling
and animation production. At the implemented
scenario (‘Lost in City’) a crossing learning
environment has been developed. At this
implementation there is a crossing, with the crossing
button and lights and cars that commute in a street of
a city (Figure 2 (b)). The child has the opportunity
to navigate and interact with the virtual world. The
child can press the button at the crossing, wait until
cars stop and lights change from red to green and
then cross the street.
5 ACKNOWLEDGEMENTS
This work is part of the IPE/NEKYP/0311/02 "VR
CAVE" project (http://www.vrcave.com.cy) and is
financially supported by the Cyprus Research
Promotion Foundation and the European Structural
Funds.
6 CONCLUSIONS
The goal of this research is to design and develop an
immersive visualization application in a VR CAVE
environment for educating children with autism.
The training regards on how to cross the street
safely.
Can the immersive visualization application in
the VR CAVE environment be effectively involved
in enhancing the social skills and behaviors of
children? Furthermore, it will be examined whether
children were able to accept and use the equipment
of the application. And, whether children interact in
a meaningful way with the environment, and
succeeded to transfer knowledge from the virtual to
the real world.
REFERENCES
B. Robins, K. Dautenhahn, R. te Boekhorst, A. Billard,
2005. Robotic assistants in therapy and education of
children with autism: can a small humanoid robot
help encourage social interaction skills? Volume 4,
issue 2.
C. Lord, S. Risi, L. Lambrecht, E. H. Cook, Jr., B. L.
Leventhal, P. C. DiLavore, A. Pickles, M. Rutter,
2000. The Autism Diagnostic Observation Schedule-
Generic: A Standard Measure of Social and
Communication Deficits Associated with the
Spectrum of Autism. Journal of Autism and
Developmental Disorders, Volume 30, No. 3.
Cyprus Autistic Association, 2009. (Online), Available:
http://www.autismsociety.org.cy/index.php?pageid=21
(2009).
D. Strickland, D. McAllister, C. Coles, S. Osborne, 2007.
An Evolution of Virtual Reality Training Designs for
Children With Autism and Fetal Alcohol Spectrum
Disorders. Top Lang Disorders, Volume 27, No. 3, pp.
226-241.
D. Strickland, 1997. Virtual Reality for the Treatment of
Autism. Virtual Reality in Neuro-Psycho-Physiology.
E. L. Hill, U. Frith, 2003. Understanding autism: insights
from mind and brain. Phil. Trans. R. Soc. Lond. B,
358.
G. Dawson, S. Rogers, J. Munson, M. Smith, J. Winter, J.
Greenson, A. Donaldson, J. Varley, 2010.
Randomized, Controlled Trial of an Intervention for
Toddlers with Autism: The Early Start Denver Model.
Pediatrics, Official Journal of the American Academy
of Pediatrics, 125, e17.
J. L. Matson, M. L. Matson, T. T. Rivet, Sep 2007.
Social-skills treatments for children with autism
spectrum disorders: An overview. Behavior
Modification, Volume 31(5), 682-707.
K. Dautenhahn, 2000. Design Issues on Interactive
Environments for Children with Autism.
K. S. Thiemann, H. Goldstein, winter 2001. Social stories,
written text cues, and video feedback: effects on social
communication of children with autism. Journal of
applied behavior analysis, Number 4, 34, 425-446.
S. Parsons, A. Leonard, P. Mitchell, 2006. Virtual
environments for social skills training: comments from
two adolescents with autistic spectrum disorder.
Computers & Education 47, 186-206.
S. Parsons, P. Mitchell, June 2002. The potential of
ImmersiveVisualizationsinaVRCaveEnvironmentfortheTrainingandEnhancementofSocialSkillsforChildrenwith
Autism
235
virtual reality in social skills training for people with
autistic spectrum disorders. Journal of Intellectual
Disability Research, Volume 46, Issue 5, pages 430-
443.
S. Parsons, S. Cobb, 2011. State-of-the-art of Virtual
Reality technologies for children on the autism
spectrum. European Journal of Special Needs
Education, 26:3, 355-366.
The Gray Center, for Social Learning and Understanding.
(Online), Available: http://www.thegraycenter.org/.
Tina R. Goldsmith, Linda A. LeBlanc, 2004. Use of
Technology in Interventions for Children with Autism.
Volume 1, Issue Number 2.
VISAPP2014-InternationalConferenceonComputerVisionTheoryandApplications
236
