Brain Rehabilitation in Clinical Trials Setup by Eye-Tracking
Bartosz Kunka
, Robert Kosikowski
, Jessica Barlinn
and Karol Kozak
AssisTech Sp. z o.o., R&D Department, ul. Trzy Lipy 3, 80-172 Gdańsk, Poland
Clinic for Nuerology, Medical Faculty, Dresden Univ. of Technology, Fetscherstrasse 74, 01307 Dresden, Germany
{bartosz.kunka, robert.kosikowski}, { jessica.barlinn, karol.kozak }
Keywords: Brain rehabilitation, Neurology, Stroke, Human-computer interaction
Abstract: The number of patients with traumatic brain injury in Germany is about 280,000 per year. Eighty percent of
the patients hospitalized in these cases exhibit minor traumatic brain injury, while approximately 10 percent
are suffering from moderate and another 10 percent from severe traumatic brain injury. The goals of
rehabilitation are to help survivors become as independent as possible and to attain the best possible quality
of life. For the last few years, eye tracking has been used as an assistive tool, especially as a tool for
alternative communication. Within the paper we explore new patent pending approach in brain injury
rehabilitation. However, eye tracking questionnaire need a full implementation into clinical studies and
medical documentation systems. In this paper we present integration of cognitive test into eye tracking
technology based on electronic case report form.
Increasing number of people who undergo brain
damage is one of the most characteristic features of
our contemporary society. Brain injury” is a term
used in terms of traumatic brain injury (TBI), a
cerebral stroke, or changes occurring in the brain
that are consequent from cerebral hypoxia (e.g. due
to perinatal incidents, sudden cardiac arrest (SCA)
or suicidal strangulation). These examples of TBI
lead to serious neurological disorders directly related
to cognitive disturbances and need to be assessed in
an objective way.
Neurological rehabilitation pursues different
goals for recovery that can be divided into several
steps: in the acute phase of TBI restitution of
neurobiological processes should be facilitated by
different therapeutic strategies. If improvement of
functional deficits is not achieved or is not expected
to occur (e.g. because of a large brain lesion), it
should be aimed at compensation strategies, for
instance by the use of assistive tools. Individuals
may need to learn how to communicate and express
their own feelings. If sensomotor, language or
cognitive deficits cannot be compensated, the
patient’s environment should be adapted to his needs
(Turner-Stokes, 2007). As recently recommended
for stroke patients, the patient’s functional and
cognitive status must be accurately assessed within
the first few days after stroke to identify special
needs for further therapeutic and rehabilitative
strategies (Hebert, 2016).
Eye tracking technology might be useful in
several steps of brain rehabilitation process, from
diagnosis to therapeutic implications, especially
when eyeballs movements is the only channel of
communication. Nowadays eye tracking technology
is well-known and it is reasonable to use it in
medical purposes, especially in supporting
neurological examination of patients with serious
communication barriers (Doležal, 2015) (Kunka,
2014). Therefore, within the paper we present a new
approach for objectivized neurological assessment
procedure based on tests included in a case report
form and adopted for eye tracking.
Patients with cognitive disturbances originating
from TBI, should be immediately assessed in
regards to the extensiveness of the damage and the
level of their consciousness. Properly conducted
diagnostics is absolutely crucial, since the whole
rehabilitation management will be later based on it.
Furthermore, the proposed approach based on eye
Kunka B., Kosikowski R., Barlinn J. and Kozak K.
Brain Rehabilitation in Clinical Trials Setup by Eye-Tracking.
DOI: 10.5220/0006227500890094
In Proceedings of the Fifth International Conference on Telecommunications and Remote Sensing (ICTRS 2016), pages 89-94
ISBN: 978-989-758-200-4
2016 by SCITEPRESS Science and Technology Publications, Lda. All rights reserved
tracking technology supporting diagnosis of
neurological patients could be also employed at their
intensive rehabilitation by stimulating particular
structures of the central nervous system (CNS).
Neurorehabilitation concerned with cognitive
functions stimulation is very important. It is a time-
consuming process requiring cyclic, systematic work
and motivation of all involved people. When the
CNS is constantly stimulated by different stimuli
and intellectual efforts, the repair mechanism can be
activated in a damaged brain. The main mechanism
is neuroplasticity which could compensate damaged
brain centers’ functions due to creation of new
neural connections. Repair mechanism of brain
consists of reorganization, adaptation, changeability,
self-repair, learning and remembering of neurons
(Doležal, 2015).
It is worth mentioning that cyclic monitoring and
observation of rehabilitation progress would be also
supported by proposed in this paper neurological
assessment utilizing eye tracking system.
Eye tracking is a technology allowing for
determining user’s gaze direction (it measures
coordinates of eye fixation point). Eye tracking
follows the path of an observer visual attention thus
it enables controlling of mouse cursor in the system,
as well as detection and analysis of user’s regions of
Currently eye tracking interfaces are based on
video processing. They utilize near-infrared
technology along with a high-resolution camera to
track gaze direction.
In the paper we focus on the remote eye trackers,
especially on the C-Eye system described in C-Eye
System section.
Eye tracking technology has many applications
in different areas. The most known are related to
human-computer interaction (HCI), entertainment
(Drewes, 2010), (Jacob, 2003), and research,
including psychology and neuroscience (Kooiker,
2015), (Hahn, 2015), as well as medicine (Kunka,
Neuroscience and psychology employ eye
tracking technology to analyze the scan paths (gaze
patterns) and heat maps to gain deeper insights into
cognitive processes underlying attention, learning,
and memory. Another research shows that eye
tracking gives us insights into i.a. word processing,
particularly how eye movements during reading are
affected by the emotional content of the texts (Urry,
Eye tracking in combination with standard
research methods or other biometric sensors can also
support diagnosis of neurological diseases such as
Autism Spectrum Disorder (ASD), Attention Deficit
Hyperactivity Disorder (ADHD), Schizophrenia,
Parkinson‘s (PD), and Alzheimer‘s disease (AD). It
is worth mentioning that iMotions exists (iMotions,
2016). It is the integrated web biometric research
platform integrating best-in-class biosensors and
synchronizes eye tracking, facial expression
analysis, EEG, GSR, EMG, ECG and surveys.
iMotions platform helps its users conduct state-of-
the-art human behavior research in the areas of
psychology, neuroscience, human factors
engineering, education, health, business and HCI.
The platform is used worldwide by leading
universities such as Harvard, Yale and Stanford.
This section presents possibilities of eye tracking
technology offered to different branches, especially
for specialized medical examinations and objective
assessment of the measured data. It has been proven
that eye tracking can be successively used both in
diagnostic and therapeutic applications.
A case report form (or CRF) (Bellary, 2014) is a
paper or electronic questionnaire specifically used in
clinical trial research. The Case Report Form is the
tool used by the sponsor of the clinical trial to
collect data from each participating site. All data on
each patient participating in a clinical trial are held
and/or documented in the CRF, including adverse
events. Originally all case report forms were made
on paper. But recently there is a changing trend to
perform clinical studies using an electronic case
report form (eCRF). Commonly encountered
challenges in CRF designing are consistency in the
design, collection of precise data and user-
friendliness also from assisting devices. These
challenges can be overcome by proper planning by a
using clinical supporting system. This way of
working has many advantages: faster and efficient,
high security, environmentally friendly. In this paper
we present integration of cognitive test into eye
tracking technology based on electronic case report
In this paper we present a concept of integration
of eCRF with eye tracking technology for clinical
trials in TBI rehabilitation.
Clinical trials involves many steps, one of the most
time-consuming elements of conducting clinical
studies is the entry of clinical data onto case report
forms (CRFs). Traditionally, this is done by clinical
research coordinators (CRCs) at various research
centers who use a pen to write the data on paper-
Fifth International Conference on Telecommunications and Remote Sensing
based CRFs, which are then faxed to clinical
monitors (CRAs) where they are examined for
potential errors that may skew the accuracy of
statistical data required to evaluate a drug’s
performance. The most important for facilitating
study management are electronic data capture by
users (doctors and patient) and clinical trial
management software. This paper describes the
advantages of integration C-Eye and study
As eCRFs are created within the C-Eye user
interface, all the field definitions, data types, control
positions, and validation rules are stored in a single
table that saves CRF definitions for all trials, no
matter how different they are. Data on patient-
specific CRFs is stored in a similar manner all
field values are checked for data type compliance at
the application level and written to a table as records
that can easily be extracted using a single reporting
tool for all trials. Elements of the system for study
management in TBI domain:
2.1 Data Capture Section
A data element in an eCRF represents the smallest
unit of observation captured for a subject in a
clinical investigation. Examples of data elements
include IQ test, color recognition, object recognition,
or other clinical observations made and documented
during a study Data capture interface allow:
1. Electronic Source Data Origination
2. Test Data Capture
3. Data Element Identifiers
4. Modifications and Corrections
5. Use of Electronic Prompts, Flags, and Data
Quality Checks in the eCRF
Many data elements in a clinical investigation
can be obtained at a study visit and can be entered
directly into the C-Eye eCRF form by an authorized
data originator. This direct entry of data can
eliminate errors by not using a paper transcription
step before entry into the eCRF.
The forms with eye tracking system are
providing a possibility to collect specific records:
Heat maps: aggregations of gaze points and
fixations revealing the distribution of visual
Scan paths (or Fixation sequences): sequence
representing the order of subjects' looking and
how much time they spend
Time of interesting: parameter quantifies the
amount of time that subjects have spent on
Areas if Interest (AIOs) being predefined
subregions of displayed content (e.g. subregions
representing the right answer)
Time To First Fixation (TTFF): the time to first
fixation indicates the amount of time it takes a
respondent to look at a specific AOI.
Typically, images (eye motion, eye symptoms,
face images) are not included as data elements in an
eCRF, but rather the clinical interpretation of the
image is included as a predefined data field.
2.2 Data Review
To comply with the requirement to maintain
accurate clinical test, clinical investigator should
review and electronically sign the completed eCRF
for each subject before the data are archived or
submitted to clinical research organization (CRO).
To comply with the requirement to maintain
accurate test histories, data elements might call for
modification or correction during data review. Either
the clinical investigator can enter the revised data
element. Modified and/or corrected data elements
must have data element identifiers that reflect the
date, time, originator, and reason for the change, and
must not obscure previous entries.
If changes are made to the eCRF after the
clinical investigator(s) has already signed, the
changes should be reviewed and electronically
signed by the clinical investigator(s).
2.3 Retention of Records by Clinical
The clinical investigator(s) should retain control of
the records (i.e., completed and signed eCRF or
certified copy of the eCRF).
2.4 C-Eye System as Study Manage-
ment Interface
The C-Eye system is a fully integrated certified
medical device supporting the evaluation of the state
of consciousness of patients suffering from any
central nervous system disorder, enabling
neurorehabilitation for people with neurological
dysfunctions and impaired development. The C-Eye
system also supports alternative communication
thanks to eye tracking technology implemented as a
remote interface.
Brain Rehabilitation in Clinical Trials Setup by Eye-Tracking
The evaluation and neurorehabilitation of a
patient suffering from neurological disorders and
impaired development consists in performing special
tasks based on multimedia contents. The subject
establishes interaction with contents displayed on
the screen using their sight, i.e.: graphics,
photographs and captions. This way, specific centers
of the central nervous system are both evaluated and
stimulated, especially centers responsible for sight,
hearing, speaking and cognitive functions.
As presented in the Figure 1, the system consists
of the processing unit (PU), which is the integrated
into the touch display (TD) and the speakers (S),
which are integrated into the touch display (TD).
Figure 1: C-Eye structure of the system
The system is equipped with two infrared light
sources (IRLS), that enable to indicate the visual
fixation point position through generating infrared
light reflections, that are reflected from the surface
of subject’s cornea and acquired by the infrared
sensitive camera (IrSC). The infrared light sources
(IRLS) are integrated with the infrared sensitive
camera (IrSC) in the way that the infrared light
sources (IrLS) are located symmetrically and
uniaxially on the both sides of the camera (IrSC) and
put together into the longitudinal cover to be formed
into the movable attachment (MA). The movable
attachment (MA), which is connected with the
processing unit (PU) and located in the lower part of
the display (TD), is up and down tiltable in a range
of 45 degrees in relation to the perpendicular
location of the attachment towards the display (TD).
The patient with potential cognitive disordered is
located before the C-Eye system. The C-Eye is
attached to the movable extension arm and adjusted
to the subject through the adjusting movements of
the movable extension arm in this way, that the
subject is located 60 cm before the system. The C-
Eye is parallel to the patient’s interpupillary line, so
that the patient’s eyes are situated in the angle of
view of the camera (IrSC), as it was presented in
Figure 2.
Figure 2: Examination session with the C-Eye system
2.5 C-Eye and Integrated Medical
System for Study Management
There are a set of many tasks included in eCRFs that
can be fully, objectively performed only with
support of eye tracking technology. The C-Eye and
eCRF approach assist the physician to use as system
in clinical practice. C-Eye could be a platform for a
comprehensive patient management and the
integration of study documentation into clinical
practice orientation guide for ideal progression
control of the therapy. The current version of the C-
Eye contains various tasks that correspond to the
tasks located in specific subsections of the eCRF
being used in everyday clinical practice. C-Eye
combines clinical documentation, medical records
register, specific therapy documentation, and
research projects in one platform. It is necessary to
adapt some of them to the structure and template of
content presentation to the C-Eye system.
Adaptation of eCRF tasks being dedicated for eye
tracking-based interaction is associated with content
development and its proper implementation. There is
a scope for interdisciplinary cooperation between
AssisTech engineers and representatives of the
medical world.
Conducting full adequate and objective
neurological assessment of patient after TBI requires
efficiency evaluation of the communication senses
(vision and hearing). Sometimes patients who have
suffered craniocerebral injury experience visual
impairment, and hemispatial neglect, unilateral
Fifth International Conference on Telecommunications and Remote Sensing
(partial) visual inattention (agnosia), and unilateral
"neglect" of space occurs. Due to the preliminary
assessment we may take into account the patient's
difficulties with perception in the half of space
opposite to the damaged brain hemisphere.
In the next step, we may conduct simplified
hearing examination. It is very important, as patients
following craniocerebral trauma are auditory
oversensitive (Landon, 2012). At times, such sounds
cause physical pain, and significantly reduce the
patient's comfort. Therefore, it is very important to
adjust loudness of all sounds produced by the C-Eye
system to suit the patients' needs.
The C-Eye can effectively expand the use of
existing assessment forms for cognitive functions of
patients that have been deprived contact with the
world and increase the efficiency of their evaluation.
Especially, the following sections of Assessment
Forms for Cognitive Rehabilitation should be
mentioned here: attention problems, visual
processing problems, memory problems,
information processing problems, executive
functions problems. Cognitive assessment procedure
adopted to the C-Eye system with considering
objectivized evaluation of the communication senses
was presented in Figure 3.
Figure 3: Adaptation of eCRF test to the C-Eye system
Furthermore, the C-Eye can be an effective tool
for rehabilitation that allows to perform the
following exercises dedicated for patients
communicating by eyeballs movements, as well as
engaged in Cognitive Stimulation Program (Ribeiro,
2011) at home: spatial/writing skills, search and
find, recall of pictures and places, recall of story
material, visual scanning, language exercises,
categorization exercises, reading comprehension,
time sense. It is worth mentioning that the C-Eye
being a medical device is simple to use - feature
especially important in everyday practice. It doesn’t
require any software installation or configuration.
The C-Eye is fully operable couple seconds after
turning it on, and does not require calibration
procedure which, in fact, would disqualify the use of
the system in case of patients in severe state after
TBI. Our approach presented in the paper is
comprehensive. We propose employing specialized
eye tracking system for holistic assessment, as well
as neurorehabilitation (Figure 4).
Figure 4: C-Eye system in neurorehabilitation and cyclic,
constant assessment based on profiled eCRF
Cognitive Test data includes all information in
original records and certified copies of original
records of rehabilitation procedure, observations in a
rehabilitation and diagnostic after TBI.
Access to cognitive test data is critical to the
review and inspections of clinical investigations.
The review of cognitive test data by both the clinic
and sponsor is important to ensure adequate
protection of the rights, welfare, and safety of
human subjects and the quality and integrity of the
clinical investigation data. Cognitive clinical test
should be attributable, legible, contemporaneous,
original, and accurate and must meet the regulatory
requirements for record keeping.
Capturing eye tracking cognitive test data
electronically and transmitting it to the eCRF will
Brain Rehabilitation in Clinical Trials Setup by Eye-Tracking
Eliminate unnecessary duplication of data
Reduce the possibility for transcription errors
Encourage entering source data during a
subject’s visit, where appropriate
Eliminate transcription of source data prior to
entry into an eCRF
Facilitate remote monitoring of data
Promote real-time access for data review
Facilitate the collection of accurate and
complete data
Cognitive Stimulation and rehabilitation allows for
great flexibility so that patients can tailor their
program of rehabilitation and follow individual
schedules. TBI survivors may participate in an
intensive level of therapy several hours per week or
follow a less demanding regimen. Eye tracking
rehabilitation efforts to address the continuum-of-
care needs of TBI patients are being developed. Eye
tracking service providers and researchers will need
to put in place service delivery plans backed by
strong research components, which include control
populations, prospective evaluations, and rigorous
methodology for the assessment of functional vision.
In conclusion, our study yielded relevant
information related to a structured TBI rehabilitation
service and represents an alternative for patients and
families afflicted by TBI, enabling the generation of
clinical protocols in eCRF Format.
We are very grateful to GoGlobal Team from
Fraunhofer Center for International Management
and Knowledge Economy for their insights and
consultancy into the German medical system and
the role cognitive devices in this country.
Turner-Stokes L, Disler PB, Nair A et al.
Multidisciplinary rehabilitation for acquired brain
injury in adults of working age. Cochrane Database
Syst Rev 2007; 2: CD004170
Hebert D, Lindsay MP, McIntyre A, Kirton A, Rumney
PG, Bagg S, et al, Canadian stroke best practice
recommendations: Stroke rehabilitation practice
guidelines, update 2015. Int J Stroke. 2016 Apr 14. pii:
J. Doležal, V. Fabian, Application of eye tracking in
neuroscience, Clinical Neurophysiology, Volume 126,
Issue 3, pp. e44, 2015.
B. Kunka, T. Sanner, A. Czyżewski, A. Kwiatkowska,
Consciousness study of subjects with unresponsive
wakefulness syndrome employing multimodal
interfaces. In: Ślȩzak, D., Tan, A.-H., Peters, J.F.,
Schwabe, L. (eds.) BIH 2014. LNCS, vol. 8609, pp.
5767. Springer, Heidelberg (2014).
H. Drewes, Eye Gaze Tracking for Human Computer
Interaction, LFE Medien-Informatik der Ludwig-
Maximilians-Universität, Monachium 2010.
R. J. K. Jacob, K. S. Karn, Eye tracking in Human-
Computer Interaction and usability research: ready to
deliver the promises (section commentary) in The
Mind's Eyes: Cognitive and Applied Aspects of Eye
Movements”, (J. Hyona, R. Radach, H. Deubel),
Elsevier Science, Oxford 2003.
M. J. G. Kooiker, J. J. M. Pel, J. van der Steen, The
Relationship Between Visual Orienting Responses and
Clinical Characteristics in Children Attending Special
Education for the Visually Impaired. Journal of Child
Neurology, 30(6), 690697, 2015.
N. Hahn, J. Snedeker, H. Rabagliati, Rapid Linguistic
Ambiguity Resolution in Young Children with Autism
Spectrum Disorder: Eye Tracking Evidence for the
Limits of Weak Central Coherence: Rapid lexical
ambiguity resolution in ASD. Autism Research, 8(6),
717726, 2015.
H. L. Urry, Seeing, thinking, and feeling: emotion-
regulating effects of gaze-directed cognitive
reappraisal, Emotion, 10(1), pp. 125-135, 2010.
iMotions platform, available at:, last
access: April 14
, 2016.
S. Bellary, B. Krishnankutty and M. S. Latha: Basics of
case report form designing in clinical research.
Perspect Clin Res. 2014 Oct-Dec; 5(4): 159166.
J. Landon, D. Shepherd, S. Stuart, A. Theadom, S.
Freundlich, Hearing every footstep: Noise sensitivity
in individuals following traumatic brain injury,
Neuropsychological Rehabilitation, 22(3), pp. 391-
407, 2012
F. R. Freire, F. Coelho, J. R. Lacerda, M. F. da Silva, V. T.
Gonçalves, S. Machado, B. Velasques, P. Ribeiro, L.
F. Hindi Basile, A. Maynart, P. Oliveira, Wellingson
S. P., P. A. Medeiros Kanda, R. Anghinah: Cognitive
rehabilitation following traumatic brain injury.
Dement Neuropsychol 2011 March;5(1):17-25
Fifth International Conference on Telecommunications and Remote Sensing