Brain Rehabilitation in Clinical Trials Setup by Eye-Tracking

Bartosz Kunka

, Robert Kosikowski

, Jessica Barlinn

and Karol Kozak

AssisTech Sp. z o.o., R&D Department, ul. Trzy Lipy 3, 80-172 Gdańsk, Poland

Clinic for Nuerology, Medical Faculty, Dresden Univ. of Technology, Fetscherstrasse 74, 01307 Dresden, Germany

{bartosz.kunka, robert.kosikowski}@assistech.eu, { jessica.barlinn, karol.kozak }@uniklinikum-dresden.de

Keywords: Brain rehabilitation, Neurology, Stroke, Human-computer interaction

Abstract: The number of patients with traumatic brain injury in Germany is about 280,000 per year. Eighty percent of

the patients hospitalized in these cases exhibit minor traumatic brain injury, while approximately 10 percent

are suffering from moderate and another 10 percent from severe traumatic brain injury. The goals of

rehabilitation are to help survivors become as independent as possible and to attain the best possible quality

of life. For the last few years, eye tracking has been used as an assistive tool, especially as a tool for

alternative communication. Within the paper we explore new patent pending approach in brain injury

rehabilitation. However, eye tracking questionnaire need a full implementation into clinical studies and

medical documentation systems. In this paper we present integration of cognitive test into eye tracking

technology based on electronic case report form.

1 INTRODUCTION

Increasing number of people who undergo brain

damage is one of the most characteristic features of

our contemporary society. “Brain injury” is a term

used in terms of traumatic brain injury (TBI), a

cerebral stroke, or changes occurring in the brain

that are consequent from cerebral hypoxia (e.g. due

to perinatal incidents, sudden cardiac arrest (SCA)

or suicidal strangulation). These examples of TBI

lead to serious neurological disorders directly related

to cognitive disturbances and need to be assessed in

an objective way.

Neurological rehabilitation pursues different

goals for recovery that can be divided into several

steps: in the acute phase of TBI restitution of

neurobiological processes should be facilitated by

different therapeutic strategies. If improvement of

functional deficits is not achieved or is not expected

to occur (e.g. because of a large brain lesion), it

should be aimed at compensation strategies, for

instance by the use of assistive tools. Individuals

may need to learn how to communicate and express

their own feelings. If sensomotor, language or

cognitive deficits cannot be compensated, the

patient’s environment should be adapted to his needs

(Turner-Stokes, 2007). As recently recommended

for stroke patients, the patient’s functional and

cognitive status must be accurately assessed within

the first few days after stroke to identify special

needs for further therapeutic and rehabilitative

strategies (Hebert, 2016).

Eye tracking technology might be useful in

several steps of brain rehabilitation process, from

diagnosis to therapeutic implications, especially

when eyeballs movements is the only channel of

communication. Nowadays eye tracking technology

is well-known and it is reasonable to use it in

medical purposes, especially in supporting

neurological examination of patients with serious

communication barriers (Doležal, 2015) (Kunka,

2014). Therefore, within the paper we present a new

approach for objectivized neurological assessment

procedure based on tests included in a case report

form and adopted for eye tracking.

Patients with cognitive disturbances originating

from TBI, should be immediately assessed in

regards to the extensiveness of the damage and the

level of their consciousness. Properly conducted

diagnostics is absolutely crucial, since the whole

rehabilitation management will be later based on it.

Furthermore, the proposed approach based on eye

Kunka B., Kosikowski R., Barlinn J. and Kozak K.

Brain Rehabilitation in Clinical Trials Setup by Eye-Tracking.

DOI: 10.5220/0006227500890094

In Proceedings of the Fifth International Conference on Telecommunications and Remote Sensing (ICTRS 2016), pages 89-94

ISBN: 978-989-758-200-4

tracking technology supporting diagnosis of

neurological patients could be also employed at their

intensive rehabilitation by stimulating particular

structures of the central nervous system (CNS).

Neurorehabilitation concerned with cognitive

functions stimulation is very important. It is a time-

consuming process requiring cyclic, systematic work

and motivation of all involved people. When the

CNS is constantly stimulated by different stimuli

and intellectual efforts, the repair mechanism can be

activated in a damaged brain. The main mechanism

is neuroplasticity which could compensate damaged

brain centers’ functions due to creation of new

neural connections. Repair mechanism of brain

consists of reorganization, adaptation, changeability,

self-repair, learning and remembering of neurons

(Doležal, 2015).

It is worth mentioning that cyclic monitoring and

observation of rehabilitation progress would be also

supported by proposed in this paper neurological

assessment utilizing eye tracking system.

Eye tracking is a technology allowing for

determining user’s gaze direction (it measures

coordinates of eye fixation point). Eye tracking

follows the path of an observer visual attention thus

it enables controlling of mouse cursor in the system,

as well as detection and analysis of user’s regions of

interests.

Currently eye tracking interfaces are based on

video processing. They utilize near-infrared

technology along with a high-resolution camera to

track gaze direction.

In the paper we focus on the remote eye trackers,

especially on the C-Eye system described in C-Eye

System section.

Eye tracking technology has many applications

in different areas. The most known are related to

human-computer interaction (HCI), entertainment

(Drewes, 2010), (Jacob, 2003), and research,

including psychology and neuroscience (Kooiker,

2015), (Hahn, 2015), as well as medicine (Kunka,

2014).

Neuroscience and psychology employ eye

tracking technology to analyze the scan paths (gaze

patterns) and heat maps to gain deeper insights into

cognitive processes underlying attention, learning,

and memory. Another research shows that eye

tracking gives us insights into i.a. word processing,

particularly how eye movements during reading are

affected by the emotional content of the texts (Urry,

2010).

Eye tracking in combination with standard

research methods or other biometric sensors can also

support diagnosis of neurological diseases such as

Autism Spectrum Disorder (ASD), Attention Deficit

Hyperactivity Disorder (ADHD), Schizophrenia,

Parkinson‘s (PD), and Alzheimer‘s disease (AD). It

is worth mentioning that iMotions exists (iMotions,

2016). It is the integrated web biometric research

platform integrating best-in-class biosensors and

synchronizes eye tracking, facial expression

analysis, EEG, GSR, EMG, ECG and surveys.

iMotions platform helps its users conduct state-of-

the-art human behavior research in the areas of

psychology, neuroscience, human factors

engineering, education, health, business and HCI.

The platform is used worldwide by leading

universities such as Harvard, Yale and Stanford.

This section presents possibilities of eye tracking

technology offered to different branches, especially

for specialized medical examinations and objective

assessment of the measured data. It has been proven

that eye tracking can be successively used both in

diagnostic and therapeutic applications.

A case report form (or CRF) (Bellary, 2014) is a

paper or electronic questionnaire specifically used in

clinical trial research. The Case Report Form is the

tool used by the sponsor of the clinical trial to

collect data from each participating site. All data on

each patient participating in a clinical trial are held

and/or documented in the CRF, including adverse

events. Originally all case report forms were made

on paper. But recently there is a changing trend to

perform clinical studies using an electronic case

report form (eCRF). Commonly encountered

challenges in CRF designing are consistency in the

design, collection of precise data and user-

friendliness also from assisting devices. These

challenges can be overcome by proper planning by a

using clinical supporting system. This way of

working has many advantages: faster and efficient,

high security, environmentally friendly. In this paper

we present integration of cognitive test into eye

tracking technology based on electronic case report

form.

In this paper we present a concept of integration

of eCRF with eye tracking technology for clinical

trials in TBI rehabilitation.

2 C-EYE SYSTEM

Clinical trials involves many steps, one of the most

time-consuming elements of conducting clinical

studies is the entry of clinical data onto case report

forms (CRFs). Traditionally, this is done by clinical

research coordinators (CRCs) at various research

centers who use a pen to write the data on paper-

Fifth International Conference on Telecommunications and Remote Sensing

based CRFs, which are then faxed to clinical

monitors (CRAs) where they are examined for

potential errors that may skew the accuracy of

statistical data required to evaluate a drug’s

performance. The most important for facilitating

study management are electronic data capture by

users (doctors and patient) and clinical trial

management software. This paper describes the

advantages of integration C-Eye and study

management.

As eCRFs are created within the C-Eye user

interface, all the field definitions, data types, control

positions, and validation rules are stored in a single

table that saves CRF definitions for all trials, no

matter how different they are. Data on patient-

specific CRFs is stored in a similar manner – all

field values are checked for data type compliance at

the application level and written to a table as records

that can easily be extracted using a single reporting

tool for all trials. Elements of the system for study

management in TBI domain:

2.1 Data Capture Section

A data element in an eCRF represents the smallest

unit of observation captured for a subject in a

clinical investigation. Examples of data elements

include IQ test, color recognition, object recognition,

or other clinical observations made and documented

during a study Data capture interface allow:

1. Electronic Source Data Origination

2. Test Data Capture

3. Data Element Identifiers

4. Modifications and Corrections

5. Use of Electronic Prompts, Flags, and Data

Quality Checks in the eCRF

Many data elements in a clinical investigation

can be obtained at a study visit and can be entered

directly into the C-Eye eCRF form by an authorized

data originator. This direct entry of data can

eliminate errors by not using a paper transcription

step before entry into the eCRF.

The forms with eye tracking system are

providing a possibility to collect specific records:

 Heat maps: aggregations of gaze points and

fixations revealing the distribution of visual

attention.

 Scan paths (or Fixation sequences): sequence

representing the order of subjects' looking and

how much time they spend

 Time of interesting: parameter quantifies the

amount of time that subjects have spent on

Areas if Interest (AIOs) being predefined

subregions of displayed content (e.g. subregions

representing the right answer)

 Time To First Fixation (TTFF): the time to first

fixation indicates the amount of time it takes a

respondent to look at a specific AOI.

Typically, images (eye motion, eye symptoms,

face images) are not included as data elements in an

eCRF, but rather the clinical interpretation of the

image is included as a predefined data field.

2.2 Data Review

To comply with the requirement to maintain

accurate clinical test, clinical investigator should

review and electronically sign the completed eCRF

for each subject before the data are archived or

submitted to clinical research organization (CRO).

To comply with the requirement to maintain

accurate test histories, data elements might call for

modification or correction during data review. Either

the clinical investigator can enter the revised data

element. Modified and/or corrected data elements

must have data element identifiers that reflect the

date, time, originator, and reason for the change, and

must not obscure previous entries.

If changes are made to the eCRF after the

clinical investigator(s) has already signed, the

changes should be reviewed and electronically

signed by the clinical investigator(s).

2.3 Retention of Records by Clinical

Investigator

The clinical investigator(s) should retain control of

the records (i.e., completed and signed eCRF or

certified copy of the eCRF).

2.4 C-Eye System as Study Manage-

ment Interface

The C-Eye system is a fully integrated certified

medical device supporting the evaluation of the state

of consciousness of patients suffering from any

central nervous system disorder, enabling

neurorehabilitation for people with neurological

dysfunctions and impaired development. The C-Eye

system also supports alternative communication

thanks to eye tracking technology implemented as a

remote interface.

Brain Rehabilitation in Clinical Trials Setup by Eye-Tracking

The evaluation and neurorehabilitation of a

patient suffering from neurological disorders and

impaired development consists in performing special

tasks based on multimedia contents. The subject

establishes interaction with contents displayed on

the screen using their sight, i.e.: graphics,

photographs and captions. This way, specific centers

of the central nervous system are both evaluated and

stimulated, especially centers responsible for sight,

hearing, speaking and cognitive functions.

As presented in the Figure 1, the system consists

of the processing unit (PU), which is the integrated

into the touch display (TD) and the speakers (S),

which are integrated into the touch display (TD).

Figure 1: C-Eye – structure of the system

The system is equipped with two infrared light

sources (IRLS), that enable to indicate the visual

fixation point position through generating infrared

light reflections, that are reflected from the surface

of subject’s cornea and acquired by the infrared

sensitive camera (IrSC). The infrared light sources

(IRLS) are integrated with the infrared sensitive

camera (IrSC) in the way that the infrared light

sources (IrLS) are located symmetrically and

uniaxially on the both sides of the camera (IrSC) and

put together into the longitudinal cover to be formed

into the movable attachment (MA). The movable

attachment (MA), which is connected with the

processing unit (PU) and located in the lower part of

the display (TD), is up and down tiltable in a range

of 45 degrees in relation to the perpendicular

location of the attachment towards the display (TD).

The patient with potential cognitive disordered is

located before the C-Eye system. The C-Eye is

attached to the movable extension arm and adjusted

to the subject through the adjusting movements of

the movable extension arm in this way, that the

subject is located 60 cm before the system. The C-

Eye is parallel to the patient’s interpupillary line, so

that the patient’s eyes are situated in the angle of

view of the camera (IrSC), as it was presented in

Figure 2.

Figure 2: Examination session with the C-Eye system

2.5 C-Eye and Integrated Medical

System for Study Management

There are a set of many tasks included in eCRFs that

can be fully, objectively performed only with

support of eye tracking technology. The C-Eye and

eCRF approach assist the physician to use as system

in clinical practice. C-Eye could be a platform for a

comprehensive patient management and the

integration of study documentation into clinical

practice – orientation guide for ideal progression

control of the therapy. The current version of the C-

Eye contains various tasks that correspond to the

tasks located in specific subsections of the eCRF

being used in everyday clinical practice. C-Eye

combines clinical documentation, medical records

research projects in one platform. It is necessary to

adapt some of them to the structure and template of

content presentation to the C-Eye system.

Adaptation of eCRF tasks being dedicated for eye

tracking-based interaction is associated with content

development and its proper implementation. There is

a scope for interdisciplinary cooperation between

AssisTech engineers and representatives of the

medical world.

Conducting full adequate and objective

neurological assessment of patient after TBI requires

efficiency evaluation of the communication senses

(vision and hearing). Sometimes patients who have

suffered craniocerebral injury experience visual

impairment, and hemispatial neglect, unilateral

Fifth International Conference on Telecommunications and Remote Sensing

(partial) visual inattention (agnosia), and unilateral

"neglect" of space occurs. Due to the preliminary

assessment we may take into account the patient's

difficulties with perception in the half of space

opposite to the damaged brain hemisphere.

In the next step, we may conduct simplified

hearing examination. It is very important, as patients

following craniocerebral trauma are auditory

oversensitive (Landon, 2012). At times, such sounds

cause physical pain, and significantly reduce the

patient's comfort. Therefore, it is very important to

adjust loudness of all sounds produced by the C-Eye

system to suit the patients' needs.

The C-Eye can effectively expand the use of

existing assessment forms for cognitive functions of

patients that have been deprived contact with the

world and increase the efficiency of their evaluation.

Especially, the following sections of Assessment

Forms for Cognitive Rehabilitation should be

mentioned here: attention problems, visual

processing problems, memory problems,

information processing problems, executive

functions problems. Cognitive assessment procedure

adopted to the C-Eye system with considering

objectivized evaluation of the communication senses

was presented in Figure 3.

Figure 3: Adaptation of eCRF test to the C-Eye system

Furthermore, the C-Eye can be an effective tool

for rehabilitation that allows to perform the

following exercises dedicated for patients

communicating by eyeballs movements, as well as

engaged in Cognitive Stimulation Program (Ribeiro,

2011) at home: spatial/writing skills, search and

find, recall of pictures and places, recall of story

material, visual scanning, language exercises,

categorization exercises, reading comprehension,

time sense. It is worth mentioning that the C-Eye

being a medical device is simple to use - feature

especially important in everyday practice. It doesn’t

require any software installation or configuration.

The C-Eye is fully operable couple seconds after

turning it on, and does not require calibration

procedure which, in fact, would disqualify the use of

the system in case of patients in severe state after

TBI. Our approach presented in the paper is

comprehensive. We propose employing specialized

eye tracking system for holistic assessment, as well

as neurorehabilitation (Figure 4).

Figure 4: C-Eye system in neurorehabilitation and cyclic,

constant assessment based on profiled eCRF

3 RESULTS

Cognitive Test data includes all information in

original records and certified copies of original

records of rehabilitation procedure, observations in a

rehabilitation and diagnostic after TBI.

Access to cognitive test data is critical to the

review and inspections of clinical investigations.

The review of cognitive test data by both the clinic

and sponsor is important to ensure adequate

protection of the rights, welfare, and safety of

human subjects and the quality and integrity of the

clinical investigation data. Cognitive clinical test

should be attributable, legible, contemporaneous,

original, and accurate and must meet the regulatory

requirements for record keeping.

Capturing eye tracking cognitive test data

electronically and transmitting it to the eCRF will

help:

Brain Rehabilitation in Clinical Trials Setup by Eye-Tracking

 Eliminate unnecessary duplication of data

 Reduce the possibility for transcription errors

 Encourage entering source data during a

subject’s visit, where appropriate

 Eliminate transcription of source data prior to

entry into an eCRF

 Facilitate remote monitoring of data

 Promote real-time access for data review

 Facilitate the collection of accurate and

complete data

4 CONLUSION

Cognitive Stimulation and rehabilitation allows for

great flexibility so that patients can tailor their

program of rehabilitation and follow individual

schedules. TBI survivors may participate in an

intensive level of therapy several hours per week or

follow a less demanding regimen. Eye tracking

rehabilitation efforts to address the continuum-of-

care needs of TBI patients are being developed. Eye

tracking service providers and researchers will need

to put in place service delivery plans backed by

strong research components, which include control

populations, prospective evaluations, and rigorous

methodology for the assessment of functional vision.

In conclusion, our study yielded relevant

information related to a structured TBI rehabilitation

service and represents an alternative for patients and

families afflicted by TBI, enabling the generation of

clinical protocols in eCRF Format.

ACKNOWLEDGMENTS

We are very grateful to GoGlobal Team from

Fraunhofer Center for International Management

and Knowledge Economy for their insights and

consultancy into the German medical system and

the role cognitive devices in this country.

REFERENCES

Turner-Stokes L, Disler PB, Nair A et al.

Multidisciplinary rehabilitation for acquired brain

injury in adults of working age. Cochrane Database

Syst Rev 2007; 2: CD004170

Hebert D, Lindsay MP, McIntyre A, Kirton A, Rumney

PG, Bagg S, et al, Canadian stroke best practice

recommendations: Stroke rehabilitation practice

guidelines, update 2015. Int J Stroke. 2016 Apr 14. pii:

1747493016643553.

J. Doležal, V. Fabian, Application of eye tracking in

neuroscience, Clinical Neurophysiology, Volume 126,

Issue 3, pp. e44, 2015.

B. Kunka, T. Sanner, A. Czyżewski, A. Kwiatkowska,

Consciousness study of subjects with unresponsive

wakefulness syndrome employing multimodal

interfaces. In: Ślȩzak, D., Tan, A.-H., Peters, J.F.,

Schwabe, L. (eds.) BIH 2014. LNCS, vol. 8609, pp.

57–67. Springer, Heidelberg (2014).

H. Drewes, Eye Gaze Tracking for Human Computer

Interaction, LFE Medien-Informatik der Ludwig-

Maximilians-Universität, Monachium 2010.

R. J. K. Jacob, K. S. Karn, Eye tracking in Human-

Computer Interaction and usability research: ready to

deliver the promises (section commentary) in “The

Mind's Eyes: Cognitive and Applied Aspects of Eye

Movements”, (J. Hyona, R. Radach, H. Deubel),

Elsevier Science, Oxford 2003.

M. J. G. Kooiker, J. J. M. Pel, J. van der Steen, The

Relationship Between Visual Orienting Responses and

Clinical Characteristics in Children Attending Special

Education for the Visually Impaired. Journal of Child

Neurology, 30(6), 690–697, 2015.

N. Hahn, J. Snedeker, H. Rabagliati, Rapid Linguistic

Ambiguity Resolution in Young Children with Autism

Spectrum Disorder: Eye Tracking Evidence for the

Limits of Weak Central Coherence: Rapid lexical

ambiguity resolution in ASD. Autism Research, 8(6),

717–726, 2015.

H. L. Urry, Seeing, thinking, and feeling: emotion-

regulating effects of gaze-directed cognitive

reappraisal, Emotion, 10(1), pp. 125-135, 2010.

iMotions platform, available at: https://imotions.com/, last

access: April 14

, 2016.

S. Bellary, B. Krishnankutty and M. S. Latha: Basics of

case report form designing in clinical research.

Perspect Clin Res. 2014 Oct-Dec; 5(4): 159–166.

J. Landon, D. Shepherd, S. Stuart, A. Theadom, S.

Freundlich, Hearing every footstep: Noise sensitivity

in individuals following traumatic brain injury,

Neuropsychological Rehabilitation, 22(3), pp. 391-

407, 2012

F. R. Freire, F. Coelho, J. R. Lacerda, M. F. da Silva, V. T.

Gonçalves, S. Machado, B. Velasques, P. Ribeiro, L.

F. Hindi Basile, A. Maynart, P. Oliveira, Wellingson

S. P., P. A. Medeiros Kanda, R. Anghinah: Cognitive

rehabilitation following traumatic brain injury.

Dement Neuropsychol 2011 March;5(1):17-25

Fifth International Conference on Telecommunications and Remote Sensing