Eye-tracking Dataset to Support the Research on Autism Spectrum

Disorder

Federica Cilia

, Romuald Carette

, Mahmoud Elbattah

3,4

, Jean-Luc Guérin

and Gilles Dequen

Laboratoire CRP-CPO, Université de Picardie Jules Verne, Amiens, France

Evolucare Technologies, Villers-Bretonneux, France

Laboratoire MIS, Université de Picardie Jules Verne, Amiens, France

Faculty of Environment and Technology, University of the West of England, Bristol, U.K.

Keywords: Autism Spectrum Disorder, ASD, Eye-tracking, Machine Learning.

Abstract: The availability of data is a key enabler for researchers across different disciplines. However, domains, such

as healthcare, are still fundamentally challenged by the paucity and imbalance of datasets. Health data could

be inaccessible due to a variety of hurdles such as privacy concerns, or lack of sharing incentives. In this

regard, this study aims to publish an eye-tracking dataset developed for the purpose of autism diagnosis. Eye-

tracking methods are used intensively in that context, whereas abnormalities of the eye gaze are largely

recognised as the hallmark of autism. As such, it is believed that the dataset can allow for developing useful

applications or discovering interesting insights. As well, Machine Learning is a potential application for

developing diagnostic models that can help detect autism at an early stage of development.

1 INTRODUCTION

Autism Spectrum Disorder (ASD) is a neuro-

developmental disorder, which is characterized by

various impairments, mainly social communication

and interaction issues, and repetitive behaviour

(American Psychiatric Association, 2013). ASD-

diagnosed individuals usually suffer from troubles in

interaction and communication in multiple forms.

The most remarkable symptom is the poor

development of non-verbal skills such as the lack or

absence of eye contact. With such troubling deficits,

a considerable strain can unfortunately be placed on

the well-being of autistic individuals and their

families as well. From an economic standpoint, it was

estimated that autism costs the UK, for example,

more than heart disease, cancer, and stroke combined

(Buescher et al., 2014).

The detection of autism at an early stage of

development is highly favourable to realise common

benefits for children and their families. Multiple

studies (e.g., Smith et al., 2000; Dawson et al., 2010)

reported improved outcomes of treatment such as

intellectual capacity, communication, adaptive

behaviour, and educational support. However, the

diagnosis of autism has been considered as a

challenging task. The diagnosis process includes a

variety of cognitive tests that typically require hours

of intensive clinical examinations. Furthermore,

standardised tests require a considerable amount of

time and effort to be conducted, and the diversity of

symptoms increase the complexity of identifying an

accurate classification.

In this regard, a large body of the psychological

research endeavoured to develop assistive

instruments based on observational measures or

diagnostic interviews. Examples include Childhood

Autism Rating Scale (Schopler et al., 1980), Autism

Diagnostic Observation Schedule (ADOS) (Lord et

al., 1989), and Autism Diagnostic Interview (ADI-R)

(Lord, Rutter, and Le Couteur, 1994). More recently,

a range of technologies have been embraced for

aiding the process of screening and diagnosis. In

particular, the eye-tracking technology has received

an extensive research interest in the ASD context.

Abnormalities of the eye gaze have been largely

recognised as the hallmark of autism (Guillon et al.,

2014), which makes eye-tracking methods suitable

for a variety of diagnostic tasks.

This study seeks to contribute to the research

community of ASD by providing an eye-tracking

dataset. On the one hand, part of our earlier work

(Carette et al., 2018) has published an image dataset,

which represented the visual patterns of eye-tracking

Cilia, F., Carette, R., Elbattah, M., Guérin, J. and Dequen, G.

Eye-tracking Dataset to Support the Research on Autism Spectrum Disorder.

DOI: 10.5220/0011540900003523

In Proceedings of the 1st Workshop on Scarce Data in Artiﬁcial Intelligence for Healthcare (SDAIH 2022), pages 59-64

ISBN: 978-989-758-629-3

 2023 by SCITEPRESS – Science and Technology Publications, Lda. Under CC license (CC BY-NC-ND 4.0)

scanpaths. On the other hand, the present work aims

to publish the raw eye-tracking output, which should

allow for extended opportunities for studying and

analysing the gaze behaviour of ASD. This study has

been generally initiated in the frame of an

interdisciplinary research of Psychology and AI. The

collaboration has brought together psychologists

from the CRP-CPO lab along with AI researchers

from the MIS lab, based at the University of Picardie

Jules Verne in France.

2 BACKGROUND AND RELATED

WORK

This section aims to provide a review of the literature

as follows. Initially, a brief history is given on the

origins of eye-tracking. Further, we review selective

studies that employed eye-tracking to analyse the

gaze behaviour among ASD-diagnosed individuals.

2.1 Brief History of Eye-tracking

Eye-tracking refers to the process of capturing and

measuring eye movements and the absolute point of

gaze (POG) (Majaranta, and Bulling, 2014). The

POG represents the focal point of the eye gaze in the

visual scene. Modern eye-trackers largely fall into

three categories including screen-based eye-trackers,

eye-tracking glasses, and Virtual Reality (VR)

headsets as well.

However, the use of eye-tracking interestingly has

a long history that dates back to the 19th century. The

early development of eye-tracking is credited to the

French ophthalmologist Louis Javal from the

Sorbonne University. In seminal research that

commenced in 1878, Javal had produced the novel

observations of fixations and saccades based on the

gaze behaviour during the reading process (Javal

1878; Javal 1879). A fixation describes the brief

moments while the eye gaze is paused on a particular

object, which allow the brain to perform the

perception process. The average duration of fixation

was estimated to be around 330ms (Henderson,

2003). While saccades include a constant scanning

with very rapid and short eye movements. Saccades

consist of quick ballistic jumps of 2

or longer, which

continue for about 30–120ms (Jacob, 1995).

Afterwards, Edmund Huey built a primitive eye-

tracking tool for analysing eye movements while

reading (Huey, 1908). More advanced

implementations of eye-tracking instruments were

developed by (Buswell 1922; Buswell 1935).

Photographic films were utilised to record the eye

movements while looking at a collection of paintings.

The eye-tracking records included both of the

direction and duration of movements. Eye trackers

are currently utilised in a plethora of applications. To

name a few, applications of marketing (Boerman, and

Müller, 2022), psychology (Åsberg Johnels et al.,

2022), product design (Jeon, Cho, and Oh, 2021), and

many others.

2.2 Eye-tracking Research in Autism

Making an accurate diagnosis of autism has been

considered as a challenging task, which usually

requires an intensive clinical assessment and

experience. With contemporary advances in

technology, new approaches have come into

prominence to assist the procedures of diagnosis and

assessment. Examples includes a variety of

technologies such as Electroencephalography (EEG)

(Abdulhay et al., 2020), Magnetic Resonance

Imaging (MRI) (Dekhil et al., 2019), and eye-tracking

(Cilia et al., 2021) in particular.

In this context, the literature contains abundant

contributions that endeavoured to apply eye-tracking

methods for analysing and understanding the

characteristics of autism. Various interesting

physiological elements were reported based on

findings output from eye-tracking experiments. For

instance, eye movements in face-to-face interactions

were observed to be different for individuals who had

different levels of the autism severity (Vabalas, and

Freeth, 2016). Specifically, highly autistic individuals

were noticed to experience saccades of shorter

duration and less frequency as well. In another

application of eye-tracking, ASD toddlers could be

identified based on the frequency of saccades and

fixations (Pierce et al., 2011). The results showed that

the ASD-diagnosed group spent significant longer

periods of fixations on dynamic geometric images.

Other studies sought to integrate eye-tracking

with contemporary AI approaches to advance the

diagnostic applications of autism. For example, a

Deep Learning model was implemented to detect

autism using eye-tracking tasks of free-image

viewing (Jiang, and Zhao, 2017). Deep Learning was

utilised to extract features automatically from a

collection of discriminative images. Likewise, CNN-

based architecture was applied for the detection of

eye contact during social interactions (Chong et al.,

2017). Their results reported a precision and recall of

76% and 80%, respectively. Another Deep Learning-

based framework was developed for ASD screening

using photo-taking tasks (Chen, and Zhao, 2019).

SDAIH 2022 - Scarce Data in Artiﬁcial Intelligence for Healthcare

They applied LSTM models for encoding the

temporal information of eye movements.

The present study endeavours to contribute with

an eye-tracking dataset to the research community of

autism. The dataset is based on a set of eye-tracking

experiments. Our earlier work (Carette et al., 2018)

published an image dataset of the eye-tracking output.

The images illustrated the dynamics of eye

movements in visualisations of the eye-tracking

scanpaths. The dataset was used in numerous

publications such as (Ahmed, and Jadhav, 2020),

(Akter et al., 2021), (Gaspar et al., 2022), and

(Elbattah et al., 2022). It is hoped that the present

dataset will also deem useful to develop further

applications for the ASD diagnosis.

3 DATA DESCRIPTION

3.1 Brief History of Eye-tracking

The participants included a set of 59 children

recruited from a few schools in the region of Hauts-

de-France. The age of participants ranged from 3 to

12 years old. It was highly desirable for the

participants to be at an early stage of development. A

parental permission was obtained for every child

before taking part in our experiments. Further, the

parents were acquainted with the study objectives

through orientation sessions.

Initially, the participants were organised based on

a basic binary grouping as: i) Typically developing

(TD), and ii) ASD-diagnosed. In addition, the CARS

score (Schopler et al., 1980) was employed to classify

the severity of autism more precisely. The CARS

method has been widely applied in the Psychology

practice for describing the severity of ASD symptoms

(Jones, and Klin, 2013). The scale includes various

ratings on different behavioural aspects (e.g., verbal

communication, activity level). Table 1 gives a

summary of the characteristics of participants.

3.2 Experimental Protocol

The eye-tracking experiments included a set of photos

and video scenarios, which were particularly

designed to stimulate the eye gaze across different

parts of the screen. The participants were seated at

approximately 60cm distance away from the monitor.

A quiet room at the university campus was used for

running our experiments. In addition, physical

barriers were applied around the screen to avoid

visual distractions.

Table 1: Summary of participants.

Count of Participants (TD, ASD) 59 (30, 29)

Gender (Female, Male) 21 (≈ 36%), 38

(

≈ 64%

)

Age (Mean, Median) 7.88, 8.1 years

CARS Score (Mean, Median) 32.97, 34.50

Figure 1: A screenshot from one of the videos used in the

eye-tracking experiments.

We used a SMI Red-M eye tracker with 60 Hz

sampling rate, which is a screen-based eye-tracker.

The eye-tracker was operated along with a 17-inch

monitor in our experiments. The screen resolution

was 1280x1024.

The content and length of videos varied to allow

for analysing the ocular activity from different

aspects and levels. The content of photos and videos

was generally aimed to include visual items (e.g.,

colourful balloons, cartoons), which could be

attractive to children in particular. The position of

items can change over the experiment timeline. In

addition, other videos included human presenters.

The presenter usually attempted to turn the

participant’s attention to elements, which could be

visible or invisible around the display area. The

French language was the working language in all

videos and conversations, as the mother tongue of

participants. Figure 1 presents a sample screenshot

captured from one of the videos.

We used other stimuli provided by the SMI

Experiment Center Software. The Stimuli included a

variety of types, which are used in the eye-tracking

research. For instance, static and dynamic naturalistic

scenes with and without receptive language, static

face or objects and cartoons stimuli, and other joint

attention stimuli. Eye-tracking experiments usually

took about 5 minutes. The participants were inspected

with respect to the quality of eye contact with the

presenter, and the level of focus on other elements. A

five-point scheme of calibration was applied. A set of

verification procedures followed the calibration

scheme.

Eye-tracking Dataset to Support the Research on Autism Spectrum Disorder

Table 2: Summary of participants.

Timestamp [ms]

Eye Movement

Category

Point-Regard X

[px]

Point-Regard Y

[px]

Pupil Diameter-

Right [mm]

Pupil Diameter-

Left [mm]

8005654.069 Fixation 1033.9115 834.0902 4.3785 4.5431

8005673.953 Fixation 1030.3754 826.0894 4.4050 4.5283

8005693.85 Saccade 1027.337 826.3127 4.4273 4.6036

8005713.7 Saccade 1015.0085 849.2188 4.3514 4.5827

8005733.589 Saccade 613.7673 418.1735 4.3538 4.5399

The eye-tracking device captured three categories

of eye movements including fixations, saccades, and

blinks. Additionally, the POG coordinates were

captured by the eye tracker. Table 2 gives a simplified

view of the eye-tracking records. As an example, the

table lists five eye-tracking records that represent a

sequence of two fixations and three saccades. The

records also give the POG coordinates for the right

and left eye over time. Due to limited space, many

variables had to be excluded from the table below (e.g.

pupil position, pupil diameter, pupil size).

3.3 Dataset

The dataset is distributed over 25 CSV-formatted files.

Each file represents the output of an eye-tracking

experiment. However, a single experiment usually

included multiple participants. The participant ID is

clearly provided at each record at the ‘Participant’

column, which can be used to identify the class of

participant (i.e., TD or ASD).

Furthermore, a set of metadata files is included.

The main metadata file, Participants.csv, is used to

describe the key characteristics of participants (e.g.

gender, age, CARS). Every participant was also

assigned a unique ID. Table 2 presents a couple of

metadata examples. Table 3 provides some examples

in this regard.

Table 3: Summary of participants.

ID Gender Age CARS Class

27 F 5.6 40 ASD

51 F 10.7 NA TD

3.4 Ethical Approval

The study received the ethical approval by the ethics

committee of Rouen University (Reference: 2016-02-

B). The CNIL (Commission nationale de

l'informatique et des libertés) declaration number of

research conformity is 2208663v0.

The study fulfils the principles and terms of the

1964 Helsinki declaration. Before starting the study,

the approval was obtained from the heads of the

regional and district education authorities, as well as

the head and teachers of the participating schools. The

parents of participants had also given their written

informed consent.

3.5 Usage Notes

The dataset is shared publicly on the Figsahre

repository. The files along with metadata can be

downloaded directly from the URL below.

https://doi.org/10.6084/m9.figshare.20113592

4 LIMITATIONS AND

CONCLUSIONS

There are some limitations that should be considered

as follows. First, the relatively small number of

participants is one limitation of the dataset. Second,

the duration of eye-tracking experiments was also

relatively short (≈ 5 min). Longer scenarios could

have allowed for a more exhaustive representation of

the gaze behaviour.

Despite limitations, it is conceived that the dataset

can be effectively utilised in the ASD research in

many ways. We believe that further studies can avail

of the dataset by applying data analysis or Machine

Learning. For instance, predictive models could be

developed to assist the diagnosis process.

ACKNOWLEDGEMENT

We acknowledge the financial support provided by

Fonds European de Development Régional (FEDER)

and Hauts-de-France.

SDAIH 2022 - Scarce Data in Artiﬁcial Intelligence for Healthcare

REFERENCES

Abdulhay, E., Alafeef, M., Alzghoul, L., Al Momani, M.,

Al Abdi, R., Arunkumar, N., ... & de Albuquerque, V.

H. C. (2020). Computer-aided autism diagnosis via

second-order difference plot area applied to EEG

empirical mode decomposition. Neural Computing and

Applications, 32(15), 10947-10956.

Akter, T., Ali, M. H., Khan, M. I., Satu, M. S., & Moni, M.

A. (2021). Machine learning model to predict autism

investigating eye-tracking dataset. In Proceedings of

the 2nd International Conference on Robotics,

Electrical and Signal Processing Techniques (ICREST)

(pp. 383-387). IEEE.

Ahmed, Z. A., & Jadhav, M. E. (2020). Convolutional

Neural Network for Prediction of Autism based on Eye-

tracking Scanpaths. International Journal of

Psychosocial Rehabilitation, 24(05).

American Psychiatric Association. (2013). Diagnostic and

statistical manual of mental disorders (DSM-5®).

American Psychiatric Pub.

Åsberg Johnels, J., Hadjikhani, N., Sundqvist, M., &

Galazka, M. A. (2022). Face Processing in School

Children with Dyslexia: Neuropsychological and Eye-

tracking Findings. Developmental Neuropsychology,

47(2), 78-92.

Boerman, S. C., & Müller, C. M. (2022). Understanding

which cues people use to identify influencer marketing

on Instagram: an eye tracking study and experiment.

International Journal of Advertising, 41(1), 6-29.

Buescher, A. V., Cidav, Z., Knapp, M., & Mandell, D. S.

(2014). Costs of autism spectrum disorders in the

United Kingdom and the United States. JAMA

Pediatrics, 168(8), 721-728.

Buswell, G.T. Fundamental Reading Habits: A Study of

Their Development. American Psychological

Association: Worcester, MA, USA, 1922.

Buswell, G.T. How people look at pictures: a study of the

psychology and perception in art. University of

Chicago Press, Chicago, IL, USA, 1935.

Carette, R.; Elbattah, M.; Dequen, G.; Guérin J.; & Cilia F.

(2018). Visualization of eye-tracking patterns in autism

spectrum disorder: method and dataset. In Proceedings

of the 13th International Conference on Digital

Information Management. Berlin, Germany, 24–26

September 2018.

Cilia, F., Carette, R., Elbattah, M., Dequen, G., Guérin, J.

L., Bosche, J., ... & Le Driant, B. (2021). Computer-

aided screening of autism spectrum disorder: eye-

tracking study using data visualization and deep

learning. JMIR Human Factors, 8(4), e27706.

Chen, S., & Zhao, Q. (2019). Attention-based autism

spectrum disorder screening with privileged modality.

In Proceedings of the IEEE/CVF International

Conference on Computer Vision (pp. 1181-1190).

Chong, E., Chanda, K., Ye, Z., Southerland, A., Ruiz, N.,

Jones, R. M., ... & Rehg, J. M. (2017). Detecting gaze

towards eyes in natural social interactions and its use in

child assessment. In Proceedings of the ACM on

Interactive, Mobile, Wearable and Ubiquitous

Technologies, 1(3), 1-20.

Dawson, G., Rogers, S., Munson, J., Smith, M., Winter, J.,

Greenson, J., ... & Varley, J. (2010). Randomized,

controlled trial of an intervention for toddlers with

autism: the Early Start Denver Model. Pediatrics

125(1), e17-e23.

Dekhil, O., Ali, M., El-Nakieb, Y., Shalaby, A., Soliman,

A., Switala, A., ... & Barnes, G. (2019). A personalized

autism diagnosis CAD system using a fusion of

structural MRI and resting-state functional MRI data.

Frontiers in Psychiatry, 10, 392.

Elbattah, M., Guérin, J., Carette, R., Cilia, F. & Dequen, G.

(2022). Vision-based approach for autism diagnosis

using transfer learning and eye-tracking. In

Proceedings of the 15th International Conference on

Health Informatics (HEALTHINF).

Gaspar, A., Oliva, D., Hinojosa, S., Aranguren, I., &

Zaldivar, D. (2022). An optimized Kernel Extreme

Learning Machine for the classification of the autism

spectrum disorder by using gaze tracking images.

Applied Soft Computing, 120, 108654.

Guillon, Q., Hadjikhani, N., Baduel, S., & Rogé, B. (2014).

Visual social attention in autism spectrum disorder:

Insights from eye tracking studies. Neuroscience &

Biobehavioral Reviews, 42, 279-297.

Lord, C., Rutter, M., Goode, S., Heemsbergen, J., Jordan,

H., Mawhood, L., & Schopler, E. (1989). Autism

diagnostic observation schedule: A standardized

observation of communicative and social behavior.

Journal of Autism and Developmental Disorders, 19(2),

185-212.

Lord, C., Rutter, M., & Le Couteur, A. (1994). Autism

Diagnostic Interview-Revised: a revised version of a

diagnostic interview for caregivers of individuals with

possible pervasive developmental disorders. Journal of

Autism and Developmental Disorders, 24(5), 659-685.

Henderson, J. M. (2003). Human gaze control during real-

world scene perception. Trends in Cognitive Sciences,

7(11), 498-504.

Huey, E.B. The psychology and pedagogy of reading. The

Macmillan Company: New York, NY, USA, 1908.

Jacob, R. (1995). Eye tracking in advanced interface design.

In W. Barfield W, T.A. Furness (eds), Virtual

Environments and Advanced Interface Design, pp.

258–288. New York: Oxford University Press.

Javal, L. (1878). Essai sur la physiologie de la lecture.

Annales d'Ocilistique, 80, 240-274.

Javal, L. (1879). Essai sur la physiologie de la lecture.

Annales d'Ocilistique, 82, 242-253.

Jeon, Y., Cho, M.S. and Oh, J. (2021), "A study of customer

perception of visual information in food stands through

eye-tracking", British Food Journal, Vol. 123 No. 12,

pp. 4436-4450.

Jiang, M., & Zhao, Q. (2017). Learning visual attention to

identify people with autism spectrum disorder. In

Proceedings of the IEEE International Conference on

Computer Vision (ICCV) (pp. 3267-3276).

Eye-tracking Dataset to Support the Research on Autism Spectrum Disorder

Jones, W., & Klin, A. (2013). Attention to eyes is present

but in decline in 2–6-month-old infants later diagnosed

with autism. Nature, 504(7480), 427-431.

Majaranta, P., & Bulling, A. (2014). Eye tracking and eye-

based human–computer interaction. In P. Majaranta, H.

Aoki, M. Donegan, D. W. Hansen, J.P. Hansen, A.

Hyrskykari & K.J. Räihä (Eds.), Gaze Interaction and

Applications of Eye Tracking: Advances in Assistive

Technologies (pp. 39-65). Hershey: IGI-Gloal.

Pierce, K., Conant, D., Hazin, R., Stoner, R., & Desmond,

J. (2011). Preference for geometric patterns early in life

as a risk factor for autism. Archives of General

Psychiatry, 68(1), 101-109.

Smith, T., Groen, A. D., & Wynn, J. W. (2000).

Randomized trial of intensive early intervention for

children with pervasive developmental disorder.

American Journal on Mental Retardation, 105(4), 269-

285.

Schopler, E., Reichler, R. J., DeVellis, R. F., & Daly, K.

(1980). Toward objective classification of childhood

autism: Childhood Autism Rating Scale (CARS).

Journal of Autism and Developmental Disorders, 10(1),

91-103.

Vabalas, A., & Freeth, M. (2016). Brief report: Patterns of

eye movements in face to face conversation are

associated with autistic traits: Evidence from a student

sample. Journal of Autism and Developmental

Disorders, 46(1), 305-314.

SDAIH 2022 - Scarce Data in Artiﬁcial Intelligence for Healthcare