TOWARD DOMOTIC APPLIANCES CONTROL
THROUGH A SELF-PACED P300-BASED BCI
F. Aloise
1
, F. Schettini
1
, P. Aricò
1
, F. Leotta
1
, S. Salinari
2
, D. Mattia
1
F. Babiloni
1,3
and F. Cincotti
1
1
Neuroelectrical Imaging and BCI Lab, Fondazione Santa Lucia IRCCS, Rome, Italy
2
Deparment of Computer Science and Engineering, University of Rome “Sapienza”, Rome, Italy
3
Department of Human Physiology and Pharmacology, University of Rome “Sapienza”, Rome, Italy
Keywords: Brain Computer Interface, P300, Domotic, EEG, Asynchronous Control.
Abstract: During recent years there has been a growing interest in Brain Computer Interface (BCI) systems as an
alternative means of interaction with the external world for people with severe motor disabilities. The use of
the P300 event-related potentials as control feature allows users to choose between various options (letters
or icons) requiring a very short calibration phase. The aim of this work is to improve performances and
flexibility of P300 based BCIs. An efficient BCI system should be able to understand user's intentions from
the ongoing EEG, abstaining from doing a selection when the user is engaged in a different activity, and
changing its speed of selection depending on current user's attention level. Our self-paced system addresses
all these issues representing an important step beyond the classical synchronous P300 BCI that forces the
user in a continuous control task. Experimentation has been performed on 10 healthy volunteers acting on a
BCI-controlled domestic environment in order to demonstrate the potential usability of BCI systems in
everyday life. Results show that the self-paced BCI increases information transfer rate with respect to the
synchronous one, being very robust, at the same time, in avoiding false negatives when the user is not
engaged in a control task.
1 INTRODUCTION
A BCI system allows to control simple devices,
including communication facilities, without using
muscles and peripheral nerves (Wolpaw et al. 2002);
in particular an EEG based BCI system is able to
understand user's intentions translating his brain
activity into a control signal (Wolpaw and
McFarland, 2004). Progresses made in recent years
have brought these systems to be considered as an
alternative means of communication and control in
situations where environmental conditions can
transiently compromise the mobility of the user, and
not only for people with severe disabilities. The
P300 event-related potential is typically a large and
positive deflection in the EEG activity (ca. 10-
20µV) with a latency ranging between 250 and 400
ms (Polich et al. 1995, Fabiani et al.1987). It is
elicited when the subject recognizes a particular
stimulus (Target stimulus) presented within a train
of frequent stimuli (NoTarget stimuli). An average
of several epochs related to a specific target stimulus
is required to distinguish the P300 potential from the
spontaneous EEG activity. For this reason classic
P300 based BCI systems provide a well-defined
number of stimulation repetition at the end of which
a selection is made; this last issue is the cause of
obvious drawbacks because the user is continuously
engaged in controlling the interface and his
distractions produce wrong classifications.
Moreover, the number of stimuli repetitions needed
to make a selection (and therefore time) depends on
the user's attention level; few sequences are needed
if the user is very concentrated, but fatigue or
distractions may cause a significant decrease in
performance and in this case it is preferable that the
system refrains from making selections avoiding a
wrong classification. This work presents a
methodology for the classification of EEG signals
related to P300 potentials which allows the user to
divert attention from the stimulation interface,
suspending the control, without incurring in a wrong
239
Aloise F., Schettini F., Aricò P., Leotta F., Salinari S., Mattia D., Babiloni F. and Cincotti F..
TOWARD DOMOTIC APPLIANCES CONTROL THROUGH A SELF-PACED P300-BASED BCI.
DOI: 10.5220/0003162002390244
In Proceedings of the International Conference on Bio-inspired Systems and Signal Processing (BIOSIGNALS-2011), pages 239-244
ISBN: 978-989-8425-35-5
Copyright
c
2011 SCITEPRESS (Science and Technology Publications, Lda.)
classification (selection). In order for the BCI
systems to move from research labs to people's
homes it is absolutely necessary that they are able to
recognize user's state without any outside input; in
particular they have to detect user’s control state or,
in other words, to distinguish when the user intends
to operate through a control interface from when he
is engaged in a different task and therefore to avoid
making any selections. This is why in recent years
many research groups have been involved in the
problem of a self-paced BCI, both for the motor
imagery (Mason and Birch, 2000; Millan and
Mourino, 2003; Townsend et al, 2004) and for the
P300-based systems (Zhang et al, 2008).
In this work we show a simple and fast self-
paced system based on an heuristic method; we
allow the system to recognize user's status,
introducing thresholds in the classifier. Moreover the
choice of Target stimulus become dynamic and this
allows system to improve its selection speed
depending on the user’s ability and attention level.
For this reason particular attention will be also put
on system performances in terms of information
transfer rate (ITR) and accuracy. Another important
thing to emphasize is that we conducted our
experiments in an environment completely operated
by the BCI system (Cincotti et al. 2008) and we
based user’s task on real life situations. This is
because we want to demonstrate the feasibility of
using these systems in everyday life.
2 MATERIALS AND METHODS
For the aim of acquisition protocol we investigated
the use of P3Speller application (Farwell and
Donchin, 1988) provided with the BCI2000
framework (Schalk et al. , 2004) to control an home
automation system. We organized 8 different
stimulus classes in a 4 by 4 matrix, each consisting
of a row or a column.
Matrix elements were 16 simple B&W icons (in
order to minimize the variability and possible VEPs
due to a high variety of colors and load information
of each icon) representing the actions that the user
could perform on the environment (e.g. light control,
DVD player, webcam for remote monitoring, mobile
phone and opening the door). Stimulation consisted
in a random intensifications of each stimulus class,
with a duration of 125ms each one. Inter Stimulus
Interval (ISI) was set to 125ms, so 250ms lag
between two stimuli.
We distinguished two different states in which
the user can be: the Control State, during which
he/she was attending to the stimulation because he
intended to exercise control over the surroundings
through the interface; and the NoControl State,
during which the user was engaged in another task
and then he wanted the system refrained to make
decisions. The EEG signal was reorganized in
overlapping epochs representing the 800 ms time
intervals immediately following the onset of each
stimulus; we can distinguish the epochs acquired
during Control trials in Target Epochs and NoTarget
Epochs. Target Epochs relate to the onset of the row
or the column stimulus containing the icon that the
user intends to select, while NoTarget are related to
no relevant stimuli. Then epochs were grouped into
sequences; a sequence denotes a single presentation
of each stimulus class on the control interface; in
this case a sequence consisted of 8 epochs, one for
each stimulus class of the interface. A single
sequence lasted 2 seconds. With the term Trial we
refer to a set of sequences at the end of which a
selection is made. Between 2 Trials we took 4
seconds during which the system presented to the
users the Target icon. All the icons on the interface
were proposed as a Target to the subject with the
same frequency and this because we wanted that
each stimulus was equally likely for subsequent
analysis. Finally, a Run consists of a series of trials
at the end of which data acquisition is stopped.
2.1 Data Acquisition Protocol
10 volunteers participated to data acquisition
protocol (4 female and 6 male) aged between 23 and
38 years; 5 subjects had already experience with the
BCI systems. Scalp EEG data were acquired from
each subject during BCI sessions using the
g.MobiLab device from g.Tec (Austria, 256Hz). The
EEG was recorded from 8 Ag-AgCl electrodes : Fz,
Cz, Pz, Oz, P3, P4, PO7 and PO8. This channel set
represents the union of the classical channels used to
extract the P300 response (Fz, Cz and Pz) with
channels in the posterior regions that have strong
correlation with desired matrix target (Krusienski et
al. 2006).
Each subject completed 2 recording sessions, we
used collected data for subsequent off-line analysis
and during recording session subject had not any
feedback about classification results. The first
recording session included a total of 8 runs, the first
2 compounded of 8 trials each during which the
subject was asked to always exercise control on the
interface; the number of stimulation sequences per
trial was fixed a priori to 10. Over the last 6 runs
Control Trials and NoControl Trials were alternated
BIOSIGNALS 2011 - International Conference on Bio-inspired Systems and Signal Processing
240
for 10 trials in total. During the NoControl Trials the
subject was asked to focus the eyes on a fixation
cross in the center of the stimulation interface. This
is because we wanted that the subject was not
completely immune to the stimulation even when he
was in the NoControl state. It is assumed that a BCI
user can’t move his head or eyes to ignore the
stimulation, so it is essential that the NoControl state
is robust to random stimuli which the user does not
actually paying attention to.
The second session provided 2 runs in Control
State to verify the subject's attitude to control the
interface using the P300 potential, and 6 runs in
which Control trials and NoControl trials were
alternated, however we considered different
NoControl tasks; in fact, the last 6 runs can be
divided into 2 groups of 3 runs each:
Control and View: The subject during the
NoControl Trials was watching a video on the
other half of the screen.
Control and Answer: During the NoControl
trials the subject had to do computation as
quickly as possible looking at the fixation cross
at the center of the interface.
Following this approach the data set contains
NoControl trials representing real situations in
which the user turns his attention elsewhere or
speaks with another person.
2.2 Features Extraction
and Classification
Signal preprocessing is necessary before perform the
classification. In particular, the EEG signal was
divided into overlapped epochs of 800 ms starting
from the onset of each stimulus, and then each epoch
was down-sampled at 85 Hz. Such down-sampling
reduces the data size and at the same time speeds up
the ensuing EEG processing significantly. Then,
EEG epochs were reorganized into a three-
dimensional array: each 2D matrix of the array
represents a single epoch related to a single stimulus,
where rows represent acquisition channels and
columns correspond to samples of each epoch.
Despite this first preprocessing the amount of
data was still significant and a further reduction in
features space was performed using the Stepwise
Linear Discriminant analysis (SWLDA). Stepwise
Linear Discriminant analysis (Draper and Smith,
1981) is an extension of Fisher’s linear discriminant
(FLD, Fisher 1936) that performs feature space
reduction by selecting suitable features to be
included in the discriminant function. Farwell and
Donchin first introduced this method for classifying
P300 features into EEG signal (Farwell and
Donchin, 1988); Krusienski et al. confirmed that this
simple technique is really efficient for online
communication (Krusienski et al., 2008). We ran the
stepwise function on a testing data set including
NoControl trials. We assigned a label equal to zero
to the NoTarget and NoControl epochs while label
was equal to 1 for Target epochs. Using SWLDA,
the final discriminant function was restricted to
contain a maximum of 60 features. Nonzero weights
were assigned to these features, w. Then the scores
values for each epoch were calculated as:
=
+
(1)
Where i denote all features related to single
stimulus j. It is assumed that a P300 is elicited for
one of the four row/column intensifications during
Control trials, and that the P300 response is invariant
to row/column stimuli, the resultant classification is
taken as the maximum of the scored feature vectors
for the respective rows, as well as for the columns:
 =max


 =max


The icon that appears at the intersection of the
predicted row and column in the matrix is the one
chosen.

= 
 
2.3 Threshold Values
Self-paced control is based on the introduction of
some thresholds in the classifier; the classification
was performed as explained before but the system
will refrain from making a selection until a row and
a column scores exceeds the threshold value. The
threshold values were chosen through a procedure
that relies on the use of ROC curves. In particular
we calculated the scores value on the Target,
NoTarget and NoControl epochs. A normal
distribution well fit the scores distributions and to
confirm this we ran a t-test on the 3 different score
distributions for each subject. T-test results show
that the hypothesis of normal distribution is true
with 95% confidence level. Next step was
investigating if the 3 score's distributions can be
considered different; for this reason we ran the
Kolmogorov-Smirnov test on each pair of samples.
Table 1 report the results of this test; the hypothesis
of different distribution was confirmed with the 95%
TOWARD DOMOTIC APPLIANCES CONTROL THROUGH A SELF-PACED P300-BASED BCI
241
confidence level for all subjects except one (Subject
1: No Target vs NoControl). For this reason it is
necessary to take account of NoControl trials for
thresholds extraction.
Table 1: Kolmogorov-Smirnov test on each pair of
sample.
Target vs
NoTarget
Target vs
NoControl
NoTarget vs
NoControl
SUBJ 1 8,75E-192 9,65E-196 0,44287182
SUBJ 2 1,74E-154 1,84E-136 6,99E-04
SUBJ 3 8,52E-256 6,38E-205 2,01E-21
SUBJ 4 2,74E-213 1,79E-182 2,82E-11
SUBJ 5 1,24E-198 3,95E-159 2,48E-10
SUBJ 6 1,81E-255 3,42E-207 9,35E-19
SUBJ 7 1,77E-277 2,77E-241 2,67E-23
SUBJ 8 2,34E-177 4,52E-134 8,90E-22
SUBJ 9 1,04E-147 2,30E-107 2,39E-13
SUBJ 10 6,67E-178 1,14E-126 1,31E-24
The threshold values were chosen according to the
number of stimulation sequences accumulated in the
trial. In fact the scores for the general stimulus i at
the sequence s will be defined as:

=

+
=1,2,
= 1,2,
(2)
Where
is given by (1), Ns = 8 for the domotic
interface and the number of stimulation sequences
was fixed to 10.
Subsequently, for each sequence we looked for the
maximum score of the row stimuli and the
maximum score of the column stimuli, and then to
them was assigned a label equal to 1 if the maximum
scores were relative to a Target stimulus and equal
to 0 if it referred to NoTarget or NoControl stimuli.
In this way we are sure to include the maximum
score values related to NoControl trial in ROC
curves training, so threshold values taking into
account possible artifacts that may occur when the
subject was not engaged in BCI control. Now ROC
curves can be plotted for each sequence using the
corresponding scores. An example is shown in
figure 1 where it is evident that when the number of
sequences accumulated in the trial increases the
ROC curves assume an ideal trend. Finding a
tradeoff between false positive rate and false
negative rate, is necessary in order to choose the
threshold. We have chosen to set the maximum
False Positive Rate (FPR) to 0.05 and the lowest
True Positive Rate (TPR) to 0.5, so the threshold
will be chosen at the intersection of the ROC curve
to the straight line joining points (0.1) and (0.05 and
0.5) .
Figure 1: Area under the ROC curve: thresholds are
identified from a tradeoff between the True Positive Rate
(yellow area) and False Positive Rate(blue area). Figure
shows only the first five stimulation sequences, over the
trend is approximately the same.
3 RESULTS
This section describes the results obtained through
the off-line analysis performed on data acquired
during the 2 recording sessions. Then we compare
our self-paced system with a classic P300 based BCI
in terms of information transfer rate considering
only the Control trials.
3.1 Off-line Analysis
An off-line cross validation was performed on the
data collected during recording sessions. In
particular, we divided the data into a training data
set consisting of 3 runs from the first session and 4
runs from the second session. In this way we
included in the training data set NoControl trials
related to all 3 different NoControl tasks. The
remainder of the data set was used as a test data set.
Specifically, the train data set was used for features
extraction and to select the threshold values. The
figure 2 shows the results obtained in cross
validation.
There are 5 different classification outcomes,
depending on the user’s state.
During Control trials we can distinguish
between:
Correct Classification: the target is
correctly recognized;
Wrong Classification: there’s a target
misclassification;
Missed Classification: the thresholds are
never exceeded, for this reason the system
abstains from take a decision.
During NoControl trials possible classification
outcome can be:
BIOSIGNALS 2011 - International Conference on Bio-inspired Systems and Signal Processing
242
Abstention: the system properly refrain
from taking decisions;
Missed Abstention: the thresholds are
exceeded and the system wrongly makes a
choice.
Figure 2: Results of offline cross validation for Control
and No-Control task.
Figure 2 also shows the averaged number of
sequences needed to make a selection when the user
was in the Control state. From the graph it can be
see how the system was proved to be robust during
NoControl trials in fact Abstentions reached an
average of 98%. On the other hand there is a high
percentage of abstentions during the Control trials
(average 8,4%), this may seem an inconvenience but
it really represents the system's ability to avoid
misclassification, because the percentage of wrong
classification did not exceed an average of less than
3%. We used the first 2 runs of the first 2 sessions,
in which the subject was always in a Control state, to
assess his ability with a classic P300-based BCI. We
found the ‘optimal’ number of stimulation sequences
just doing a cross validation offline: particularly we
used 2 runs to train SWLDA and to extract
significant features and the other 2 to test these
parameters, then we averaged the results of
classification for each possible combination of
training and testing data set. The figure 3 shows the
trend of the percentages of correct classification
based on the number of stimulation sequences
accumulated for each subject. The black line
represents an accuracy of 95%, which corresponds
to a false positive rate of 5% that is the maximum of
false positive allowed in the self-paced system
through ROC curves. We used these results to
estimate the information transfer rate.
Figure 3: Results of offline cross validation for Control
task depending on number of stimulation sequences.
3.2 Information Transfer Rate (ITR)
To assess the efficiency of the two systems in terms
of information transfer rate we used the definition of
bit rate given by Wolpaw et al.(2000) and widely
used in BCI systems, this is based on the definition
of information rate proposed by Shannon for noisy
channels with some simplifying assumptions: the
symbols have all the same a priori occurrence
probability =1/
, the classifier accuracy P is the
same for all target symbols and that the
classification error 1 is equally distributed
amongst all remaining symbols.

=log
+log
+
1
log
1
1
(3)
This express the bit rate or bit/trial for each
selection. The information transfer rate (bits/minute)
is equal to B
Wolpaw
multiplied by speed of selection S
(Selection per minute). In turn the speed selection
for P300-based system depends on the number of
sequences of stimulation used and when we
calculated it we have taken account of the 4 seconds
between a trial and the other which are used to
present the results of the classification. The Table 2
shows the values of ITR for each subject calculated
using the average number of sequences and the
percentage of accuracy obtained by off-line analysis
both for self-paced BCI and for the synchronous
one. In particular, for the synchronous system we
imposed the number of stimulation sequences that
allowed subject to achieve 95% of accuracy, if this
did not happen we have used the minimum number
of sequences that produced the highest accuracy.
TOWARD DOMOTIC APPLIANCES CONTROL THROUGH A SELF-PACED P300-BASED BCI
243
Table 2: ITR in both modalities.
SYNC SELF
SUBJ 1 12,39 4,24
SUBJ 2 5,08 11,37
SUBJ 3 16,81 16,36
SUBJ 4 12,01 11,87
SUBJ 5 11,03 18,00
SUBJ 6 16,81 15,00
SUBJ 7 14,01 16,06
SUBJ 8 5,98 15,38
SUBJ 9 4,67 10,17
SUBJ 10 5,63 11,83
Mean 10,44 13,03
4 CONCLUSIONS
The introduction of a threshold based classification
system in the P300-based BCIs allows the user to
divert his attention from control interface at any time
and without the use of external inputs, and it also
brings positive effects on the bit rate that is
incremented when the user is in the best control
conditions. A further advantage consists in
increasing the accuracy of the system by preventing
errors through abstentions; in this way the BCI
system acquires more dynamicity and flexibility by
reducing its gap with traditional input interfaces.
Future applications could consider the use of
dynamic thresholds fitting the user's current action
or the environment state: the system would be able
to automatically identify the user's most likely action
and facilitate its selection by reducing the threshold
values for that item. This work represents a step
towards the use of the BCI systems as an aid for
functional communication and environmental
control for people with severe motor disabilities or
as alternative means when the usual channels of
communication and interaction are temporarily
compromised.
ACKNOWLEDGEMENTS
This work is partly supported by the EU grant FP7-
224332 “SM4ALL” project, and FP7-224631
“TOBI” project. This paper only reflects the authors’
views and funding agencies are not liable for any use
that may be made of the information contained
herein.
REFERENCES
Cincotti, F. et al., 2008. Non-invasive brain-computer
interface system: towards its application as assistive
technology. Brain Research Bulletin, 75(6), 796-803.
Draper, N. R. & Smith, H., 1998. Applied Regression
Analysis Third., Wiley-Interscience.
Fabiani, M. et al., 1987. Definition, identification and
reliability of measurement of the P300 component of
the event-related brain potential. , 2, 1-78.
Farwell, L. A. & Donchin, E., 1988. Talking off the top of
your head: toward a mental prosthesis utilizing event-
related brain potentials. Electroencephalography and
Clinical Neurophysiology, 70(6), 510-523.
Fisher, R. A., 1936. The Use of Multiple Measurements in
Taxonomic Problems. [[Annals of Eugenics]], 7, 179–
188.
Krusienski, D. J. et al., 2008. Toward enhanced P300
speller performance. Journal of Neuroscience
Methods, 167(1), 15-21.
Krusienski, D. J. et al., 2006. A comparison of
classification techniques for the P300 Speller. Journal
of Neural Engineering, 3(4), 299-305.
Mason, S. G. & Birch, G. E., 2000. A brain-controlled
switch for asynchronous control applications. IEEE
Transactions on Bio-Medical Engineering, 47(10),
1297-1307.
Millán, J. D. R. & Mouriño, J., 2003. Asynchronous BCI
and local neural classifiers: an overview of the
Adaptive Brain Interface project. IEEE Transactions
on Neural Systems and Rehabilitation Engineering: A
Publication of the IEEE Engineering in Medicine and
Biology Society, 11(2), 159-161.
Polich, J. & Kok, A., 1995. Cognitive and biological
determinants of P300: an integrative review.
Biological Psychology, 41(2), 103-146.
Schalk, G. et al., 2004. BCI2000: a general-purpose brain-
computer interface (BCI) system. IEEE Transactions
on Bio-Medical Engineering, 51(6), 1034-1043.
Townsend, G., Graimann, B. & Pfurtscheller, G., 2004.
Continuous EEG classification during motor imagery--
simulation of an asynchronous BCI. IEEE
Transactions on Neural Systems and Rehabilitation
Engineering: A Publication of the IEEE Engineering
in Medicine and Biology Society, 12(2), 258-265.
Wolpaw, J. R. et al., 2002. Brain-computer interfaces for
communication and control. Clinical
Neurophysiology: Official Journal of the International
Federation of Clinical Neurophysiology, 113(6), 767-
91.
Wolpaw, J. R. & McFarland, D. J., 2004. Control of a
two-dimensional movement signal by a noninvasive
brain-computer interface in humans. Proceedings of
the National Academy of Sciences of the United States
of America, 101(51), 17849-17854.
Zhang, H., Guan, C. & Wang, C., 2008. Asynchronous
P300-based brain-computer interfaces: a
computational approach with statistical models. IEEE
Transactions on Bio-Medical Engineering, 55(6),
1754-1763.
BIOSIGNALS 2011 - International Conference on Bio-inspired Systems and Signal Processing
244