Creative Coding for Dance Movement Therapy in Children with Autism

Nicol

as Araya

, Javier Gomez

and Germ

an Montoro

Higher Polytechnic School, Autonomous University of Madrid, Madrid, Spain

Keywords:

Autism, Dance Movement Therapy, Creative Coding, Accessibility, Human Computer Interaction.

Abstract:

People diagnosed with an Autism Spectrum Disorder (ASD) have deﬁcits in social interaction, communica-

tion and cognitive development. Children with ASD may also present motor difﬁculties growing up, which

motivates interventions of Dance Movement Therapy (DMT) that helps them to develop social skills and inte-

grate in society. Current technological advances have integrated into DMT interventions, enriched with virtual

scenarios, projections, sensors and robot partners. These works have positive outcomes in social skills devel-

opment and motor skills reﬁnement, even though, due to conﬁnement for COVID-19, online DMT has yet to

be further explored. We propose a research methodology for the development of a tool that aims to develop

self expression for ASD youth, with the creation of an artistic image based on dance and body movements.

Our initial study case is Movarte, a web based tool that creates graphic pieces based on body movement and

proxemic areas. 15 users evaluated the application, showing positive outcomes in terms of engagement and

novelty, though it was not considered so clear and limited in terms of parameter control. Future research will

provide more insight to adapt an interface for DMT in self expression for people with ASD.

1 INTRODUCTION

Autism Spectrum Disorder (ASD) is a family of dis-

orders at a social, communicative and behavioral level

(Association, 2013). This neurodevelopmental disor-

der is detectable from a very early age and today has

a prevalence of 1 in 100 children diagnosed annually

(Zeidan et al., 2022).

Commonly, this group of people present deﬁcits

in the development of social and motor skills and the

expression of emotions. In addition, impairments in

motor development, movements and speech are in-

cluded. That is why, depending on the severity of their

diagnosis, they may require therapeutic support in the

initial stages or even for life (Lord et al., 2018).

Art therapy serves to give a free expression of the

deepest emotions and expressions, through the cre-

ation of an artistic piece that reﬂects an internal state.

It is an alternative for people who cannot communi-

cate verbally and provides a space for personal ex-

pression.

Dance Movement Therapy combines the creative

and expressive aspects of dance with psychother-

apy, using both verbal and non-verbal means to help

https://orcid.org/0000-0002-7623-7835

https://orcid.org/0000-0002-7496-7965

https://orcid.org/0000-0001-7393-1226

clients achieve a more realistic body image and expe-

rience relief, pleasure, and integration (Weitz, 2018).

Movement therapy can help children with behavioral

disorders express memories stored in their bodies

from pre-verbal periods, and Dance Movement Ther-

apy has been developed through the collaboration of

professionals from the ﬁelds of psychiatry, psychol-

ogy, and dance.

Technologies contribute to the creation of new

tools for the development of new visual and creative

forms. The contribution in art therapy is favored with

new digital possibilities. The conﬁnement caused by

the COVID-19 pandemic has led many psychothera-

pists to move all, or a good part, of their work online,

which has been little documented for remote dance

music therapy (Garcia-Medrano, 2021).

In this sense, Creative Coding can contribute from

the development of code for expression and the vi-

sual arts, which is enriched with other areas of Hu-

man Computer Interaction, such as affective comput-

ing. Likewise, a user-centered methodology can be

proposed for the development of a tool that explores

corporality and expression in people with ASD.

The present work seeks to present ideas and a ﬁrst

approach towards a tool that allows creating a visual

piece in an intervention from dance for children with

autism.

Araya, N., Gomez, J. and Montoro, G.

Creative Coding for Dance Movement Therapy in Children with Autism.

DOI: 10.5220/0012472100003657

Paper published under CC license (CC BY-NC-ND 4.0)

In Proceedings of the 17th International Joint Conference on Biomedical Engineering Systems and Technologies (BIOSTEC 2024) - Volume 2, pages 761-767

ISBN: 978-989-758-688-0; ISSN: 2184-4305

761

2 BACKGROUND

Some works have been studied that apply DMT using

technological means to potentiate the abilities of chil-

dren with autism. These applications have been eval-

uated in small groups of children with ASD in thera-

peutic contexts that have had positive results. Some

examples are:

1. OSMoSIS is a musical and body game that con-

verts movements into sounds. It uses a Kinect to

capture the body movements of the child partici-

pating in the game, in order to measure motor and

social synchrony (Ragone et al., 2020).

2. DanceCraft is a remote-use system that, through

videos of a DMT instructor, allows children to

play to imitate their movements. It uses a Kinect

sensor and has been well received by family

groups that have tried it (Ringland et al., 2019).

3. Choreograﬁsh is a virtual reality (VR) video game

that simulates choreography for the movement of

a school of ﬁsh. The objective is to reduce the so-

cial anxiety of the participants, by allowing them

to control the rhythm and movement of an artistic

creation (Jr. et al., 2018).

4. ExpressiveBall is a prototype of a tangible ball

that has different sensors to stimulate the stimula-

tion of the senses. The objective is to achieve the

expression of children with ASD with low verbal

skills, through six body activities (Wilson et al.,

2020).

In these works, children with ASD are encour-

aged to the arts through games on body interfaces and

movement. However, the objectives of these studies

are to promote certain social skills or autonomy, and

few focus on creative expression. Furthermore, the re-

sult is often imitation of an instructor to improve the

precision of body movements, leaving little room for

self expression and exploration.

The present research is motivated by these last as-

pects to implement a new system that considers the

needs of children with ASD, who can explore corpo-

rality for the execution of an artistic piece. The pro-

cess will have more value than the result and will help

you communicate in a different way.

Creative Coding is the use of programming in cre-

ative applications. In different contexts, user inter-

faces are enriched with embodied alternatives. This

philosophy has a major presence in the Maker move-

ment, where users develop creative technological ar-

tifacts, though it has not yet been used for therapy

or well being scenarios. In accessibility, the idea is

to adapt and develop interfaces for users with special

needs, in this case, people with ASD may ﬁnd an en-

gaging form to develop self expression and creative

endeavours through art creation with dance.

DanceON is an educationally-focused program-

ming system that allows users to create visual anima-

tions in response to body movement data, with the

goal of engaging young women of color in creating

artistic computational artifacts within culturally rele-

vant dance learning experiences (Payne et al., 2021).

Children beneﬁt from creation with an increase

of self-expression and autonomy development levels.

Our goal is to explore how this can be developed

through an embodied interface, noting that users are

keen on visual and are engaged with virtual worlds

and videogames.

Some common tools for creative coding are

Processing, a library for graphical creation. The

JavaScript language is commonly used for web de-

velopment, in this case we ﬁnd p5.js, a library that

adapts Processing with WebGL standards. A comple-

mentary library is ml5.js, that uses Machine Learning

models from Pytorch in JavaScript. TouchDesigner is

a software that helps to create graphical, 2D and 3D

rendering in creation projects.

In summary, DMT can beneﬁt from the creative

currents of programming, to support new tools that

stimulate creative development. Recently, as a re-

sult of the conﬁnement caused by the COVID-19 pan-

demic, psychotherapists who practice DMT have had

to seek remote and online avenues for their work,

demonstrating similar results of empathy and body

exploration (Garcia-Medrano, 2021). Currently, it is

necessary to continue documenting these processes

and propose tools that can adapt to these challenges,

in order to achieve a novel and stimulating effect for

this group of users.

3 METHODOLOGY

A ﬁrst approach is proposed to evaluate a body inter-

face that allows creating a graphic piece, developed

with creative programming tools. This ﬁrst stage re-

quires validation and further we propose a more com-

plete project, designed with experts in the ﬁeld.

Movarte emerges as an interactive art proposal

that aims to establish links between movement and

a dynamic visual image. For the recognition of poses

and body movements, a neural network model was

used, which was associated with a JavaScript library

to produce graphic images within a web browser. Ad-

ditionally, the concept of proxemics was used to es-

tablish imaginary zones that allowed variations to be

made when generating the ﬁnal image. In particular,

HEALTHINF 2024 - 17th International Conference on Health Informatics

762

it is intended to investigate the use of proxemic zones

as a way of generating layers to create an image com-

posed of simple geometric ﬁgures.

The web application has six proxemic zones that

represent a layer of the image that the user can manip-

ulate. To set a zone, it is necessary to approach until

it is indicated to be in the next zone. These are:

Layer 1. Choose the background color by varying

the vertical position of the elbows and the hori-

zontal position of the waist. The RGB values are

chosen, where the right elbow represents the Red

channel, the left elbow represents the Green chan-

nel, and the right waist represents the Blue chan-

nel.

Layer 2. Allows the user to choose the position of

lines that align with the position of the forearms,

measuring from the elbow to the wrist. The colors

are triad of the previous color.

Layer 3. A triangle is formed whose vertices are the

points of each shoulder and the midpoint between

the points of each hip. The color is complemen-

tary to the background.

Layer 4. Allows the user to generate a sequence of

circles of different colors and sizes, which follow

the position of the wrist and elbow for each arm.

Layer 5. Two triangles are formed with each ear, eye

and nose as vertices as a common point. The col-

ors are triad of the color of the third zone.

Layer 6. Circles of different sizes are formed whose

center is the nose. Unlike the third zone, these are

created every 2 seconds.

Having completed these areas, you can download

the ﬁnal image, an example is shown in Figure 1.

Figure 1: Example of image generated with Movarte.

In order to gain proof of concept, we developed a

web-based tool for interactive art creation based on an

embodied interface called Movarte.

The approach is to use proxemic spaces that rep-

resent image layers, on which different graphical ele-

ments are placed as the user moves closer to the web

camera. The main idea is to control basic graphical

ﬁgures according to the body movements and overall

position, so that body movements and dance are en-

hanced to advance into a ﬁnal visual product, that can

be exported into a 2D image.

It was developed using the p5.js library, which

extends Processing in the JavaScript language, and

the React framework to integrate to a web based app

hosted in Heroku. For body recognition, it uses the

PoseNet model integrated with the ml5.js library, a

Machine Learning model that collects 17 body and

points.

The UEQ is a quick and reliable questionnaire to

measure the user experience of interactive products

(Laugwitz et al., 2008) . It consists of 26 questions of

pairs of opposite words in which the user evaluates the

level of identiﬁcation of an adjective about another,

on a seven-point scale. This allows evaluating the

scales of: Attraction, Transparency, Efﬁciency, Con-

trollability, Stimulation and Novelty. Furthermore, it

separates the measurement of quality at a pragmatic

and hedonic level. The Spanish version was used and

the results are processed by the data analysis tool pro-

vided on their website.

The evaluation used the User Experience Ques-

tionnaire (UEQ) and a set of qualitative questions.

The UEQ sets 6 metrics composed of 26 closed ques-

tions in a likert scale. Additionally, we added the fol-

lowing open questions:

Q1. What contexts do you consider suitable for this

application? Options: Dance, Physical exercise,

Performance, Therapy, Artistic creation, Leisure

activities, Kinesiology and Other.

Q2. What do you think about this experience?

Q3. What aspects would you change for future ver-

sions of the application?

Proposals will be reviewed to improve the sys-

tem and move towards integration in educational or

daily use contexts. Changes regarding technological

advances and a new review of the literature will be in-

corporated, as well as proposals from researchers and

experts in the area. New case studies will be deﬁned

to propose workshops and sessions that promote cre-

ativity applied in schools and support centers for chil-

dren with ASD.

It is planned to have an active group of partici-

pants, between 10 and 20 children with ASD. ASD

and their families. During a deﬁned time, creative

exploration sessions will be developed through the

proposed system. The evaluation will correspond to

Creative Coding for Dance Movement Therapy in Children with Autism

763

the complexity of the artistic piece created, as well as

your experience and usability with applications of this

type. All integration stages will include interviews

with experts from centers interested in collaborating.

4 A CASE STUDY

As a proof of concept, a study was carried out on the

usability and acceptance of Movarte as a tool that con-

tributes to the validity of this research. Initially, it is

intended to validate with people who are not experts

in the topics addressed in this project, who allow an

external view and have different levels of technolog-

ical or artistic knowledge. To do this, a group of 15

participants between 19 and 52 years, they were sent

the link to the project page and some instructions on

its use. Users carried out the tests in their homes, with

different web cameras, computers, internet browsers,

spaces and environmental conditions, to avoid condi-

tioning a response under deﬁned parameters. They

were invited to freely explore the tool, to generate at

least one image that they could optionally share it in

the gallery. At the end, they were asked to respond

to the UEQ and the questions posed. The results of

the survey for the pairs of items are in Figure 4. A

high score can be seen for the characteristics of orig-

inal, innovative, good, creative and activator. On the

other hand, it has been considered unpredictable by

a majority of users, and to a lesser extent, confusing,

difﬁcult to learn and overloaded.

Figure 2: Quality factor in UEQ.

In the quality ﬁelds presented in Figure 2, Attrac-

tiveness scores the highest with 2.56, close to Hedo-

nic Quality with 2.54. Pragmatic Quality scores lower

with 1.63

As noted in Figure 3, Dependability scores only

1.25 which is the lowest overall, given that most users

have described the system as more unpredictable, and

some of them have classiﬁed it as not secure, compli-

cated and confusing, as reﬂected on answers of Fig-

ure 4. Efﬁciency scores 1.78, followed by Perspicuity

with 1.87, noting an experience that is complete but

struggles a little to be deﬁned as practical, organized

and understandable. Simulation is 2.33.

The UEQ applies a normalization for the ques-

tions (or items) into -3 to 3 scale, for each questions

ﬁeld group. In the quality ﬁelds presented in Figure

2, Attractiveness scores the highest with 2.56, close

to Hedonic Quality with 2.54 and Pragmatic Quality

scores the lowest with 1.63.

This is then classiﬁed in the UEQ sections noted

in Figure 3. Attractiveness and Perspicuity score 2.56

and 1.87, respectively, representing on higher values

for the engagement and visual aspects. In Figure 4 the

results are presented, noting their relevance with the

quality factors, overall the application was found to be

quite unpredictable and sometime hard to understand,

but showed a great visual impact and was also valued

for entertainment.

Figure 3: Mean of UEQ sections.

As for the qualitative questions, users in general

though that Movarte had value for DMT interven-

tions. The most important aspects of Q1 is that users

found this useful for artistic creation with 14 out of

15 answers, and Therapy with two thirds of the votes,

other relevant topics where Therapy, dance, physical

exercise and leisure activities, as noted in Figure 5.

Users responded to Q2 to have an overall en-

tertaining experience, where they felt present in the

dance and art creation, some even played music while

they were dancing to increase their creative patterns.

They felt their well-being increased and felt more re-

laxed after the experience.

For Q3, users think the idea is good but still ﬁnd

it limited. They propose new graphic ﬁgures, the use

of 3D graphics, Virtual or Augmented Reality, as well

as being able to control the parameters such as colour,

size of images and image format.

HEALTHINF 2024 - 17th International Conference on Health Informatics

764

Figure 4: Answers per item.

5 DISCUSSION

Movarte overall showed positive outcomes with high

levels in terms of novelty, attraction and engagement,

they feel this tool can help to increase interaction for

creative and self expression.

Users assured they need more control on param-

eters such as colour and size, more diversity in the

displayed ﬁgures, explore 3D graphics. Instructions

were not clear for many, people took some time to

understand application.

Users see this tool suitable for anxiety and relief,

they felt that disconnected from their physical envi-

ronments and felt an immersive experience they took

to their home places. Some included music and en-

joyed the exploration that could be made with their

bodies with no additional device needed.

General audience and software experts validate

the use of this embodiment tool for therapeutic and

artistic uses, which gives a direction that DMT can

proﬁt form an intervention that uses Movarte for cre-

ative expression in an artistic piece with dance move-

ments. The layers are found to be an interesting idea,

but is not clear how to use it, people got confused on

Figure 5: Answers for Q1.

where to pass from one layer to the other or to revert

changes, feeling like they lost their previous image

and had to start over again.

As an initial proof of concept, it can be seen that

Movarte has generated good acceptance and has given

space to discuss possible improvements. At a general

level, it proves to be attractive to a non-specialized

general public, achieving high levels of hedonistic

and attractive quality, based on the positive assess-

ment in terms of novelty, attraction and stimulation.

By varying the colors by random levels and accord-

ing to each base color chosen, the images are unique

for each experience, added to the possibility of shar-

ing them in the gallery, invites users to actively par-

ticipate to explore new results. On the other hand,

randomness plays a factor that does not allow con-

trollability, which lowers the levels of predictability

and control over the result, reﬂected in a lower level

of pragmatic quality. You must continue exploring to

ﬁnd a way that allows you to control the creative pro-

cess, but without limiting it so that it loses its novelty

or causes frustration.

Additionally, graphic possibilities and audiovisual

resources must be increased. The p5.js library allows

you to manipulate basic ﬁgures although they are lim-

ited, 3D can be integrated with three.js or Babylon.js

to achieve an immersive vision. Along these lines, the

use of virtual or augmented reality can be explored,

without its use involving high computing use or re-

quiring expensive hardware that would mean losing

Creative Coding for Dance Movement Therapy in Children with Autism

765

the portability of an online system. Some options

to consider for the web are AR.js, A-Frame and Ar-

gon.js. You should study how to adapt existing tech-

nologies to an environment designed for people with

ASD. For motion detection, if web cameras associ-

ated with a body recognition machine learning model

are chosen, it must be veriﬁed that their result is sim-

ilar to depth cameras (RGB-D). Of this group, Kinect

is the most used and validated in different research

projects with children with ASD, however, today it is

discontinued; Although there are alternatives, they in-

volve a higher cost and the need to have the necessary

equipment. The study should then be of the techni-

cal capabilities to capture the precision of movement,

such as level of clarity, angle, resolution and distance

of camera, as well as prevention for obstruction or

erroneous captures, both for one or multiple partici-

pants.

Another point to assess is the integration of math-

ematical models with graphic and creative program-

ming libraries. The input data is usually the posi-

tion on the two-dimensional x and y axes, which are

captured, and the percentage probability of accuracy.

As an alternative to PoseNet, Tensorﬂow today fea-

tures BlazePose and MoveNet, which increases cap-

ture from 17 to 33 body and facial points, or runs

at over 50 frames per second for mobile devices and

desktop, respectively. A model that has had a leading

role in research is OpenPose, which detects differ-

ent body and facial points, its integration with p5.js

or ml5.js. The results of the open questions have

provided positive validation to replicate the use of

Movarte in contexts of artistic creation, therapy and

dance, which is the focus of this work.

The next step is to validate the tool with experts

in the thematic lines of ASD, therapists, artists and

relatives of diagnosed people, to have a global vision

that allows us to propose a speciﬁc use case that will

be evaluated in later stages. Creative coding offers a

simplet language that extends and adapts for people

that do not come from technological backgrounds to

enhance their work with the nature of code.

It would be interesting to propose an interface that

allows p5.js code and graphic resources to be added

easily. Other contexts to explore arise from physi-

cal exercise, leisure activities and performance, which

can follow the same line. Rehabilitation is already

widely explored in literature, the focus today is on

creative expression and training. It is also necessary

to establish evaluative concepts for the complexity

of the image, such as the management of colors and

shapes, as well as expressive degree.

6 CONCLUSIONS

Users with ASD need to refer creative endeavours

with an inclusive approach, like art therapy. Dance

Movement Therapy (DMT) is an evolving ﬁeld that

needs to be promoted with new technological ad-

vances and can be suitable for motor therapy, well

being, anxiety relief.

We believe that Creative Coding as important and

empowerment tool for autism and has yet a big jour-

ney to be validated, though our approach gives hints

for future works.

Future work will be made for testing with ASD

groups, apply questionnaires and validation of experts

for creative assessment, as well as extending the use

of the application to educational environments, work-

shops and leisure use.

Dance Movement Therapy (DMT) provides expe-

riential beneﬁts for people diagnosed with an autism

spectrum disorder (ASD), at the level of well-being

and expressive development. This research is part of

a project that seeks to propose a tool for artistic cre-

ation with creative programming tools.

In a ﬁrst stage, a proof of concept is proposed to

validate a prototype with a group of users, quantita-

tively and qualitatively. The main results denote that

it is a novel and attractive idea, suitable for contexts

of artistic creation and therapy. However, it is not en-

tirely controllable and has aspects to improve in terms

of the diversity of graphic options and usability.

The work will continue with these results towards

its next phase, it has the support of collaborating orga-

nizations and experts who will provide a lot of infor-

mation to consider in future stages. A user-centered

design will continue and subsequent validation with

an environment adapted to end users.

ACKNOWLEDGEMENTS

The authors acknowledge the support from the Project

Indigo! (Ministry of Science and Innovation) with

reference number PID2019-105951RB-I00 / AEI /

10.13039 / 501100011033). This work has been sup-

ported by the FPI-UAM predoctoral scholarship from

the Autonomous University of Madrid.

REFERENCES

Association, A. P. (2013). Diagnostic and Statistical Man-

ual of Mental Disorders. American Psychiatric Asso-

ciation.

HEALTHINF 2024 - 17th International Conference on Health Informatics

766

Garcia-Medrano, S. (2021). Screen–bridges: dance move-

ment therapy in online contexts. Body, Movement and

Dance in Psychotherapy, 16(1):64–72.

Jr., R. A., Handman, E., Bayles, G., Jackman, J., Cheng,

K., Ritchie, S., Newell, T., and Wright, C. (2018).

Choreograﬁsh: Co-designing a choreography-based

therapeutic virtual reality system with youth who have

autism spectrum advantages. pages 381–389. Associ-

ation for Computing Machinery.

Laugwitz, B., Held, T., and Schrepp, M. (2008). Construc-

tion and Evaluation of a User Experience Question-

naire, page 63–76. Springer Berlin Heidelberg.

Lord, C., Elsabbagh, M., Baird, G., and Veenstra-

Vanderweele, J. (2018). Autism spectrum disorder.

The Lancet, 392:508–520.

Payne, W. C., Bergner, Y., West, M. E., Charp, C., Shapiro,

R. B., Szaﬁr, D. A., Taylor, E. V., and DesPortes, K.

(2021). Danceon: Culturally responsive creative com-

puting. In Proceedings of the 2021 CHI Conference

on Human Factors in Computing Systems, CHI ’21,

New York, NY, USA. Association for Computing Ma-

chinery.

Ragone, G., Good, J., and Howland, K. (2020). Osmo-

sis: Interactive sound generation system for children

with autism. Extended Abstracts - Proceedings of the

2020 ACM Interaction Design and Children Confer-

ence, IDC 2020, pages 151–156.

Ringland, K. E., Wolf, C. T., Boyd, L., Brown, J. K.,

Palermo, A., Lakes, K., and Hayes, G. R. (2019).

Dancecraft: A whole-body interactive system for chil-

dren with autism. pages 572–574. Association for

Computing Machinery.

Weitz, N. (2018). The importance of the dance movement

therapy in general and child therapy. In The Euro-

pean Proceedings of Social and Behavioural Sciences,

ERD. Cognitive-Crcs.

Wilson, C., Sitbon, L., Ploderer, B., Opie, J., and Brereton,

M. (2020). Self-expression by design: Co-designing

the expressiball with minimally-verbal children on the

autism spectrum. pages 1–13. Association for Com-

puting Machinery.

Zeidan, J., Fombonne, E., Scorah, J., Ibrahim, A., Durkin,

M. S., Saxena, S., Yusuf, A., Shih, A., and Elsabbagh,

M. (2022). Global prevalence of autism: A systematic

review update. Autism Research, 15:778–790.

Creative Coding for Dance Movement Therapy in Children with Autism

767