Farzin Deravi and Shivanand P. Guness
School of Engineering and Digital Arts, University of Kent, Canterbury, CT2 7NT, U.K.
Keywords: Biometrics, Gaze tracking.
Abstract: Could everybody be looking at the world in a different way? This paper explores the idea that every
individual has a distinctive way of looking at the world and thus it may be possible to identify an individual
by how they look at external stimuli. The paper reports on a project to assess the potential for a new
biometric modality based on gaze. A gaze tracking system was used to collect gaze information of
participants while viewing a series of images for about 5 milliseconds each. The data collected was firstly
analysed to select the best suited features using three different algorithms: the Forward Feature Selection,
the Backwards Feature Selection and the Branch and Bound Feature Selection algorithms. The performance
of the proposed system was then tested with different amounts of data used for classifier training. From the
preliminary experimental results obtained, it can be seen that gaze does have some potential as being used
as a biometric modality. The experiments carried out were only done on a very small sample; more testing is
required to confirm the preliminary findings of this paper.
Biometric systems aim to establish the identity of
individuals using data obtained from their physical
or behavioural characteristics. In recent years there
has been an increasing range of systems developed
using a wide variety of biometric modalities –
including fingerprints, face, voice and gait. In this
work we propose the human gaze as a new modality
that may also be used to establish identity.
Gaze tracking is the process of continuously
measuring the point or direction of gaze of the eyes
of an individual. Up to now we are not aware of any
studies on the use of gaze as a source of biometric
information. In particular, the possibility of using
gaze direction as a means for identifying individuals
will be explored in this work.
Gaze may be considered a type of behavioural
biometrics. Such behavioural modalities are based
on acquired behaviour, style, preference, knowledge,
motor-skills or strategy used by people while
accomplishing different everyday tasks such as
driving an automobile, talking on the phone or using
a computer (Goudelis, Tefas, & Pitas, 2009;
Gutiérrez-García, Ramos-Corchado, & Unger, 2007)
Human Computer Interaction (HCI) is the
interaction between the user and devices such as
Personal Computers, Smart phones etc. HCI can also
be used as a source of biometric information because
the interaction of a user and his computer may be
quite distinctive if not unique. One attraction for
using HCI as a biometric modality is the potential to
develop a non-intrusive authentication mechanism
(Yampolskiy, 2007).
Gaze may therefore be considered in part as a
behavioural biometrics. At the same time it may also
have physiological aspects determined by the tissues
and muscles that determine its capabilities and
limitations. In this respect, it may be likened to text-
dependent automatic speaker recognition where a
user is asked to read a pre-defined text and the sound
generated is analysed and compared with a database
to establish his or her identity. In a similar fashion
the user of gaze biometrics may be shown a
predefined sequence of images and the gaze
information is analysed and compared with a
database of previously stored gaze data to recognize
the individual.
In particular the data generated may be analysed
in a similar fashion to Online Signature Verification
because the gaze data is very similar to online
signature, except for the fact that the gaze data is
collected with reference to a screen where the
stimulus images are presented and the signature data
is obtained on a digitalised pad. A study of HCI-
based biometric modalities such as Keystroke and
Deravi F. and Guness S..
DOI: 10.5220/0003275803350341
In Proceedings of the International Conference on Bio-inspired Systems and Signal Processing (BIOSIGNALS-2011), pages 335-341
ISBN: 978-989-8425-35-5
2011 SCITEPRESS (Science and Technology Publications, Lda.)
Mouse Dynamics may also be helpful in the
development of gaze as a biometric modality.
The rest of the paper is organized as follows: In
Section 2 a review of some related sources of
biometric information is presented. Section 3
describes the design of our proposed system while
Section 4 presents the experimental setup and some
preliminary test results. Finally Section 5 provides
tentative conclusions and suggestions for further
In this section of the dissertation we are looking at
traditional behavioural biometrics such as Online
Signature Verification and also HCI based
biometrics such as Keystroke and Mouse Dynamics.
The behavioural aspect of gaze and eye movement is
also investigated.
2.1 Biometric Modalities
2.1.1 Online Signature Verification
The study done by Lei et al was to investigate the
consistency of the features used for signature
verification. The speed, coordinate points and,
angles between the speed vector and the X-axis are
found to be the most consistent and reliable. The
False Accept Rate (FAR), False Reject Rate (FRR)
and the Equal Error Rate (EER) are calculated for
each feature and this data is used to find the most
consistent features (Lei & Govindaraju, 2005). In
Chapran et al, the 35 most widely used features used
in handwritten signatures. ANOVA (Analysis of
Variance) statistic method was used to find the
variance of the different features. This enables
finding the features that changed when a forged
signature was entered. This also helped to find the
features which are most suitable for the different
types of writing activities such as writing cheques
and signing forms (Chapran, Fairhurst, Guest, &
Ujam, 2008).
2.1.2 Keystroke Biometrics
Keystroke biometric modality involves collecting
data about the typing pattern of the user. There are
two different types of keystroke biometrics. The
keystroke biometrics can be either static or
continuous. In static keystroke biometrics, keystroke
biometric is only used during login time whereas
with continuous keystroke biometrics; the modality
is continuously being monitored. The advantage of
continuous keystroke over static is that an imposter
user can be detected even if an imposter is
substituted for a genuine user after the initial
authentication process. The features that can be used
are the time between keystrokes, the duration of the
keystroke, finger placement and applied pressure on
the keys in the paper proposed by Monrose & Rubin
(Monrose & Rubin, 2000).
2.1.3 Mouse Dynamics
Mouse Dynamics uses information such as the
direction of the movement of the mouse, timing and
monitoring when actions such as clicking are
performed to authenticate a user. There are two ways
in which data is captured for this modality. Data can
be collected by the continuous monitor the activities
of the user. Another method used is by capturing the
mouse interaction in an application such as a game.
In a paper proposed by Revett et al, a survey is
conducted to investigate mouse movement based
biometric authentication systems. Revett et al also
proposed a novel graphical authentication system
called Mouse Lock. The preliminary result showed
that mouse movement or mouse dynamics could be a
viable biometric modality. For testing their
application with 5 user and they showed that the
FAR of the system to be in the range of 2% to 6%
and FRR to be in the range of 0% to 7% (Revett,
Jahankhani, Magalhães, & Santos, 2008).
2.1.4 Eye Behaviour
In a paper by Adolphs the relationship between the
size of the pupils and emotions were investigated.
The study was looking at how the size of the pupil of
individual is changes while looking at sad faces.
Pupil size is well-known to be influenced by
stimulus luminance, but it turns out also to be
influenced by other factors, including salience and
emotional meaning (Adolphs, 2006).
In Harrison et al, the size of the pupil was
investigated to see how it changes while viewing the
expressions of another person. The study showed
that the size of the pupil becomes smaller while
viewing sad facial expressions and there is no visible
change caused by expressions of happiness, neutral
expression or expressions of anger (Harrison,
Singer, Rotshtein, Dolan, & Critchley, 2006).
Wang et al used eye tracking and pupil dilation
to see if a person is telling the truth. It was observed
that the pupils were dilated when deceptive
messages were sent and that the dilation was related
to the magnitude of the deception (Wang, Spezio, &
BIOSIGNALS 2011 - International Conference on Bio-inspired Systems and Signal Processing
Camerer, 2010).
In this study, Castelhano et al (2008) investigated
the influence of the gaze of another person on the
direction of gaze of an observer. In their experiment
the participants were shown a sequence of scene
photographs that told a story. Some of the scenes
contained an actor fixating an object in the scene. It
was observed that the first fixation point of the
participants were the face of the actor, the eye then
moved to focus on the object the actor was focusing
on. Furthermore it was observed that even in the
presence of other object in the scene the participants
would always focus their gaze on the object the actor
is looking at.(Castelhano, Wieth, & Henderson,
Another study conducted by Castelhano (2009),
the influence of task on the movement of the eye is
being investigated. The experiments consisted of 20
participants who were asked to view color
photographs of natural scenes but under two
different instruction sets. The first instruction set
was to do a visual search of the image and the
second task was to memorization of the image. The
results of the experiments show that the fixation
points and the gaze duration of the different
participants were influenced by the task they were
performing. It was also seen that the areas of
fixation were different for both tasks but the
movement amplitude and the duration of the fixation
were not affected (Castelhano, Mack, & Henderson,
2.2 Tracking Performance
An important consideration is the accuracy with
which gaze data can be captured for further analysis.
The result in the paper by Chao-Ning et al shows
their technique can be used to calculate gaze with an
accuracy of 85% to 96% within 2 meters. This was
done by calculating the estimated gaze of the system
and comparing the result to the coordinate of the dot
the user was viewing on the screen. The test was
carried out at various distances (Chao-Ning Chan,
Oe, & Chern-Sheng Lin, 2007).
3.1.1 Overview
The overall system setup is shown in Figure 1. This
consists of a display screen where stimulus images
are presented and a webcam facing the user to
capture their gaze information. Software in the
computer attached to the webcam is then used to
analyse the images captured by the camera to extract
gaze information which is subsequently further
processed to extract gaze features. Once the features
are extracted, a suitable classifier is used to compare
the captured data with previously stored data for this
and other users. In this way it is possible to establish
error rates for the system and explore its feasibility
as a biometric system.
3.1.2 Stimulus
The stimuli used were obtained from an image
quality database (Engelke, Maeder, & Zepernick,
2009; Le Callet & Autrusseau, 2005). Five images
were chosen from the database. Alternate images of
objects or nature and human are displayed to the
user. This is done so as to avoid the user’s gaze to be
influenced by the content of the image and to offer
the user a variety of image types. The images are all
assumed to be of the same quality and thus the gaze
of the user would not be influenced by the quality of
the image.
Figure 1: Stimulus images.
3.1.3 Gaze Data
Table 1 describes the structure of the data to be
stored for the gaze information that is captured. This
data has to contain enough information to enable
further processing for biometric feature extractions.
3.1.4 Gaze Capture
Figure 2 show the flow diagram for the enrolment
process. User enrolment is the process whereby the
Table 1 : Data structure of the data retrieved from the
# Field Name Description
1 Frame Number Unique incremental number
2 Time The time the gaze was
3 Interval The interval time used by the
4 X coordinate of left
The X coordinate of the
centre of the left pupil
5 Y coordinate of left
The Y coordinate of the
centre of the left pupil
6 Tracking status of
left pupil
The tracking status of the left
7 X coordinate of
right pupil
The X coordinate of the
centre of the right pupil
8 Y coordinate of
right pupil
The Y coordinate of the
centre of the right pupil
9 Tracking status of
right pupil
The tracking status of the
right pupil
10 Type of data The type of the data
11 Size of left pupil The size of the left pupil
12 Size of right pupil The size of the right pupil
13 Stimulus The stimulus being presented
to the user
14 X coordinate of
gaze point
The X coordinate of the
estimated gaze point
15 Y coordinate of
gaze point
The Y coordinate of the
estimated gaze point
16 Interoccular
The distance between the left
and right pupil of the user
biometrics of the user is processed and added to the
system. The user is presented with a number of
images. The images are presented to the user in a
full screen and modal mode i.e. the user would not
see any other activity on the screen and the
application would have the focus the during the gaze
capture session so as to prevent the user from getting
distracted from other operations on the screen. The
gaze data of the user on the images are recorded.
Between images the screen is greyed out and the
user is asked to fix the location in the middle of the
screen. This data is used to calibrate the gaze of the
user and also to ensure that the user starts looking at
each of the images from the same location. The
mapping from the location of the pupil to the centre
of the screen is stored as the calibration data and is
used to estimate the gaze of the user when the user is
looking at screen/images.
Figure 2: Flow chart of user enrolment.
3.1.5 Gaze Features
Table 2 is a list of features to be extracted from the
gaze data:
Table 2: List of features from gaze data.
Feature Description
2 Duration
The duration the gaze was at the
current position
4,5 Left Pupil
The location of the left pupil
7,8 Right Pupil
The location of the right pupil
11,12 Size of Pupil
The size of the pupil
14,15 Gaze Point
The calculated gaze point on
the screen
The data column number corresponds to the data
column in Table 1.
4.1 Set Up
For the experiment the equipment are placed in a
table mounted configuration. The camera is placed
in front and in the middle of the screen. The user is
placed at a distance of 30-60cm from the screen. The
procedure of the test is based on (Duchowski, 2007;
Judd, Ehinger, Durand, & Torralba, 2009; Van,
Rajashekar, Bovik, & Cormack, 2009).
4.2 Procedure
4.2.1 Initialisation
The purpose of the initialisation phase is to detect
and initialise the location of the face, eyes and nose.
2. Gaze tracking
3. Extract
4. Generate
1. Display picture
5. Store template6. Get Next pictureScreen
BIOSIGNALS 2011 - International Conference on Bio-inspired Systems and Signal Processing
The purpose of this phase is to verify if the user is
facing the sensor and viewing the scene. Once a face
is detected and the eye and pupil centres are detected
the calibration phase can begin.
4.2.2 Calibration
For the calibration phase the user is presented with 9
dots on the screen. The dots are shown at one
location at a time. The user has to fix their gaze on
the dots when they appear. Each dot is displayed to
the user for a period of 5 seconds (5000
4.2.3 Gaze Capture
Once the calibration is complete, the user is shown a
set of images. The images used were obtained from
a quality image database (Engelke et al., 2009; Le
Callet & Autrusseau, 2005) so as to prevent the
quality of the images used to influence the data
captured. The images are assumed to be of similar
quality. The user is shown the images for a period of
5 seconds (5000 milliseconds). For each set of
images 2 gaze capture sessions are required. This is
because as it is seen in the literature review that the
gaze or eye movement is based on the task being
carried out (Castelhano et al., 2009).
4.2.4 Rest Period
The rest period is used to rest the gaze of the user by
making the user look at the centre of the screen. The
rest period is shown between showing the different
stimulus images for gaze capture. The duration of
the rest period is 3 seconds (3000 milliseconds).
4.3 Results
4.3.1 Performance
For this test the experiment the whole feature list as
described in Table 1 : Data structure of the data
retrieved from the gaze.Table 1.
4.3.2 Feature Analysis
Figure 4, shows the difference in the performance of
the system using features selected using the Forward
Feature Selection (FFS), Backwards Feature
Selection (BFS) and Branch and Bound (B&B)
algorithms. The performance data is compared
against the performance using all the features which
is used as a control. As it can be seen when selecting
2 to 3 features, the performance did not improved
Figure 3: ROC curve with 20% of data as Training data.
Table 3: Results from testing with classifiers.
Error Rate
Error Rate
training data)
Error Rate
0.152 0.064 0.030
Support Vector
Classifier (SVC)
0.077 0.004 0.010
densities based
linear classifier
0.077 0.004 0.010
Minimum Least
Square Linear
0.005 0.004 0.000
using the features selection algorithms. On selecting
4 features, there was a slight improvement in the
performance of the system with the feature selection
algorithms. With 5 to 7 features being selected, it
can be seen that the features obtained from the
feature selection algorithms improved the
performance of the system. When 8 features were
selected, using the FFS algorithm only a slightly
improvement of the performance was noticed and
with the BFS and B&B algorithms the performance
was equal to the control performance using all the
features. The best performance is obtained using the
BFS and the B&B algorithms with 7 features
selected from Table 2. The features selected are:
Figure 4: Error rate before and after feature selection.
Table 4: Best Feature selection algorithms and features
Algorithm 1 2 3 4 5 6 7 8 9
BFS 1 2 3 4 5 6 7
B&B 2 4 7 5 1 6 3
This work has relied on techniques developed in the
field of behavioural biometric; HCI based biometric
modalities and combining results with techniques
from gaze tracking, pupillometry and facial feature
extraction to create a new biometric modality based
on gaze. From the preliminary result obtained, it can
be seen that gaze information may have some
potential for being used as a biometric modality. The
experiments carried out were only done on a very
small sample; more testing is required to confirm the
preliminary findings of this project.
A gaze-based biometric modality would be both
an affordable and nonintrusive way of verifying the
user’s identity. In addition, a gaze-based biometric
modality would also open the way to a number of
new application areas. It would be well suited for
verification of users which interact with a whole
range of devices containing a camera such as smart
phones, personal computers etc. Gaze-based
biometric systems could also be used as a remote
authentication system for web sites or e-commerce
sites. Another potential area where such
technologies can be used is in liveness detection. In
such an approach, liveness detection may be based
on the movement of the pupil using as stimulus
either the variation of lighting condition or using
Future research on this topic should be directed
at increasing overall accuracy of the gaze tracking
system as well as looking into possibility of
developing multimodal biometric system based on
other existing biometric modalities such as iris,
fingerprint or HCI-based biometric modalities such
as keystroke or mouse dynamics.
Adolphs, R. (2006). A landmark study finds that when we
look at sad faces, the size of the pupil we look at
influences the size of our own pupil. Social Cognitive
and Affective Neuroscience, 1(1), 3-4. doi:10.1093/
Castelhano, M. S., Mack, M. L., & Henderson, J. M.
(2009). Viewing task influences eye movement control
during active scene perception. Journal of Vision, 9(3)
Castelhano, M. S., Wieth, M., & Henderson, J. M. (2008).
I see what you see: Eye movements in real-world
scenes are affected by perceived direction of gaze.,
251-262. doi:
Chao-Ning Chan, Oe, S., & Chern-Sheng Lin. (2007).
Active eye-tracking system by using quad PTZ
cameras. Industrial Electronics Society, 2007. IECON
2007. 33rd Annual Conference of the IEEE, 2389-
Chapran, J., Fairhurst, M. C., Guest, R. M., & Ujam, C.
(2008). Task-related population characteristics in
handwriting analysis. Computer Vision, IET, 2(2), 75-
Duchowski, A. T. (2007). Eye tracking methodology:
Theory and practice. Secaucus, NJ, USA: Springer-
Verlag New York, Inc.
Engelke, U., Maeder, A., & Zepernick, H. -. (2009).
Visual attention modelling for subjective image
quality databases. , Rio De Janeiro
Goudelis, G., Tefas, A., & Pitas, I. (2009). Emerging
biometric modalities: A survey. Journal on
Multimodal User Interfaces, , 1-19.
Gutiérrez-García, J. O., Ramos-Corchado, F. F., & Unger,
H. (2007). User authentication via mouse biometrics
and the usage of graphic user interfaces: An
application approach. 2007 International Conference
on Security and Management, SAM'07, Las Vegas,
NV. 76-82.
Harrison, N. A., Singer, T., Rotshtein, P., Dolan, R. J., &
Critchley, H. D. (2006). Pupillary contagion: Central
mechanisms engaged in sadness processing. Social
Cognitive and Affective Neuroscience, 1(1), 5-17.
Judd, T., Ehinger, K., Durand, F., & Torralba, A. (2009).
Learning to predict where humans look. IEEE
International Conference on Computer Vision (ICCV),
Le Callet, P., & Autrusseau, F. (2005). Subjective quality
BIOSIGNALS 2011 - International Conference on Bio-inspired Systems and Signal Processing
assessment IRCCyN/IVC database.
Lei, H., & Govindaraju, V. (2005). A comparative study
on the consistency of features in on-line signature
verification. Pattern Recognition Letters, 26(15),
2483-2489. doi:DOI: 10.1016/j.patrec.2005.05.005
Monrose, F., & Rubin, A. D. (2000). Keystroke dynamics
as a biometric for authentication. Future Generation
Computer Systems, 16(4), 351-359. doi:DOI:
Revett, K., Jahankhani, H., Magalhães, S. T., & Santos, H.
M. D. (2008). A survey of user authentication based
on mouse dynamics. In H. Jahankhani, K. Revett & D.
Palmer-Brown (Eds.), Global E-security (, pp. 210-
219) Springer Berlin Heidelberg.
Van, D. L., Rajashekar, U., Bovik, A. C., & Cormack, L.
K. (2009). Doves: A database of visual eye
movements. Spatial Vision, 22(2), 161-177.
Wang, J. T., Spezio, M., & Camerer, C. F. (2010).
Pinocchio's pupil: Using eyetracking and pupil
dilation to understand truth telling and deception in
sender-receiver games
Yampolskiy, R. V. (2007). Human computer interaction
based intrusion detection. Information Technology,
2007. ITNG '07. Fourth International Conference on,