A REAL-TIME FRACTAL-BASED BRAIN STATE

RECOGNITION FROM EEG AND ITS APPLICATIONS

Olga Sourina, Qiang Wang, Yisi Liu and Minh Khoa Nguyen

Nanyang Technological University, Nanyang Avenue, Singapore, Singapore

Keywords: BCI, Neurofeedback, Emotion recognition, Fractal dimension, EEG, Real-time applications.

Abstract: EEG-based immersion is a new direction in research and development on human computer interfaces. It has

attracted recently more attention from the research community and industry as wireless EEG reading

devices became easily available on the market. EEG-based technology has been applied in anaesthesiology,

psychology, serious games or even in marketing. As EEG signal is considered to have a fractal nature, we

proposed and developed a novel spatio-temporal fractal based approach to the brain state quantification. The

real-time algorithms of emotion recognition and concentration level recognition were implemented and

integrated in human-computer interfaces of EEG-enable applications. In this paper, EEG-based “serious”

games for concentration training and emotion-enable applications including emotion-based music therapy

on the Web were proposed and implemented.

1 INTRODUCTION

Immersive human interaction with computer systems

should use all human senses such as visual, audio,

tactile, odour, taste, etc to make virtual or even non-

virtual experience more real. In the case of human

computer systems, human receives information from

the system using eyes, ears, skin, nose, etc and the

information is processed by the corresponding lobes

of the brain. Then, the user enters information into

the computer system using the implemented human-

computer interface system scenarios. To make the

human-computer interfaces more seamlessness the

information could be entered involuntary as well. It

could be entered into the computer by cameras,

sensors based tracking systems, and by biofeedback

sensors. Then, the information could be processed

by the corresponding algorithms depending on the

system application. In this paper, we study a novel

dimension of human-computer interfaces that is

based on real-time EEG recordings and its

recognition. Electroencephalogram (EEG) is a non-

invasive technique recording the electrical potential

over the scalp which is produced by the activities of

brain cortex, and reflects the state of the brain

(Nunez and Srinivasan, 2006). EEG technique gives

us an easy and portable way to monitor brain

activities by using suitable signal processing and

classification methods and algorithms.

We proposed new algorithms of brain state

recognition including emotion recognition and

concentration level recognition, and innovative

integrated methods and tools for implementation of

the EEG-based user immersion and interaction.

Algorithms of the “inner” brain state quantification

including emotion recognition would advance

research on human computer interaction bringing up

the proposed novel objective quantification methods

and algorithms as new research tools in medical

applications, entertainment, and even novel digital

art methodology applications, and allowing us an

integration of the brain state quantification

algorithms in the human computer interfaces. It

would lead to the implementation of the applications

such as EEG-based serious games including

neurofeedback games, emotion-enable personalized

search on the Web, experimental art animation,

personalized avatars seamlessness communicating

with virtual objects, other avatars, or even social

robots working with elderly people, etc.

In this paper, we describe a novel spatio-

temporal fractal-based approach to brain state

recognition particularly to recognition of the

concentration levels and emotion recognition. The

method is general, and it could be used as a basis for

the development of other brain states recognition

Sourina O., Wang Q., Liu Y. and Nguyen M..

A REAL-TIME FRACTAL-BASED BRAIN STATE RECOGNITION FROM EEG AND ITS APPLICATIONS.

DOI: 10.5220/0003335300820090

In Proceedings of the International Conference on Bio-inspired Systems and Signal Processing (BIOSIGNALS-2011), pages 82-90

ISBN: 978-989-8425-35-5

 2011 SCITEPRESS (Science and Technology Publications, Lda.)

algorithms as well. Our approach consists from two

parts: interactive tools allowing dynamic analysis of

EEG signals amplitudes and other parameters

distribution over the brain lobes, and fractal

dimension algorithms with sliding window that can

estimate complexity of the signal by calculating

fractal dimensions values changing over time. The

algorithms of concentration level recognition and

emotion recognition use just one fractal feature per

channel that allows us to implement real-time EEG-

enable applications. We implemented real-time

applications such as blobby 3D mapping,

concentration games, emotion-enable search on the

Web, etc. that are described in this paper.

In Section 2.1, we describe Brain Computer

Interfaces (BCI) and neurofeedback techniques. In

Section 2.2, emotion recognition algorithms are

reviewed. Then, we describe the proposed spatio-

temporal fractal-based approach to the brain state

recognition. In Section 3.1, 3D Mapping of EEG

signal with blobby model is presented. In Section

3.2, fractal dimension algorithms applied for

extraction of fractal dimension features of EEG

signal with sliding window are elaborated. An

implementation of the overall algorithm is given in

Section 4. Real-time EEG-enable applications

including EEG-based serious games and emotion-

based music search are described in Section 5.

2 RELATED WORK

2.1 BCI and Neurofeedback

Brain Computer Interfaces (BCIs) are the systems

that use brain signals to create a new channel of

interaction of humans with computers or other

devices. Mostly the systems are used for disabled or

elderly persons. Recently, efforts have been made on

the development of EEG-based real-time

applications in multimedia communication,

rehabilitation games, interaction in virtual

environments, etc. Traditionally, neurofeedback is a

technique that allows the user voluntary change

his/her brain state based on the visual or audio

feedback from the system corresponding to the

recognized from the user EEG state of the brain. The

user by doing some exercises recommended by the

doctor or just by playing the serious game with

neurofeedback learns how to improve his/her brain

plasticity. Neurofeedback could recover some

psychological disorders or just help to improve some

skills of concentration, meditation, etc. Some

research demonstrates that both the EEG and Event

Related Potential (ERP) distortion can reflect

psychological disorders such as Attention Deficit

Hyperactivity Disorder (ADHD) (Lubar et al., 1995,

Fuchs et al., 2003), Autistic Spectrum Disorders

(ASD) (Coben et al., 2010, Kouijzer et al., 2010),

Substance Use Disorders (SUD) including alcoholics

and drug abuse (Saxby and Peniston, 1995,

Sokhadze et al., 2008), etc. Neurofeedback can be

used for treating these disorders besides medical

treatments. Many neurofeedback games were

assessed, and it was proved that they have a healing

effect on patients with ADHD while the patient has

abnormal θ/β ratio of EEG. Besides the ratio, the

distortion in Slow Cortical Potential (SCP) was also

notified in (Gevensleben et al., 2009). Both the

frequency band neurofeedback training and the SCP

neurofeedback training could achieve a good healing

effect for ADHD (Gevensleben et al., 2009). Two

EEG signal processing methods are prevalent in BCI

systems: power spectrum analysis for different

frequency bands and event related potential analysis.

As the different frequency band reflects different

brain functions (Demos, 2005), frequency training is

a well-known technique applied in clinic

applications together with the Quantitative EEG

(QEEG) protocol. In QEEG protocol, the power over

different bands is assessed from the patients EEG

signals, and compared to the reference QEEG

database. Pathology and the corresponding recovery

protocol can be generated with the statistical model.

The ERP analysis including SCP and P300 analysis

is a technique to analyze the event synchronized

EEG potential. SCP has shown its usability in

ADHD treatment in (Gevensleben et al., 2009), and

P300 component training could be used for drug

abuse rehabilitation (Sokhadze et al., 2008).

Although an efficiency of EEG linear features

application were proved in clinical treatments, the

nonlinear methods, e.g. entropy analysis and fractal

dimension analysis, became popular in EEG

processing due to the nonlinearity of the EEG

signals. The hypothesis is that a non-linear fractal

dimension approach allows quantify brain states

corresponding to the concentration levels, pain

levels, etc. In work (Wang et al., 2010b, Wang et al.,

2010a), two well-known algorithms such as Box-

counting (Block et al., 1990) and Higuchi (Higuchi,

1988) were applied in concentration level

recognition in neurofeedback games, and the

efficiency was approved.

A REAL-TIME FRACTAL-BASED BRAIN STATE RECOGNITION FROM EEG AND ITS APPLICATIONS

2.2 Emotion Recognition Algorithms

Emotion recognition from EEG could reveal the

“inner” feeling of the user, and then, it could be used

in a therapy or to create an emotion-enable avatar of

the user or other real-time applications. Emotion

recognition algorithms consist from two parts:

feature extraction and classification. For real-time

applications, an objective is to develop fast

algorithms recognizing more emotions with fewer

electrodes used. Currently, mostly off-line

recognition algorithms were proposed which are

shown in the Table 1. EEG-based emotion

recognition algorithms could be divided into two

groups: a subject-dependent and a subject-

independent one. In the Table 1, the algorithms are

compared by feature extraction and classification

algorithms used, by emotion types recognized and

by the algorithms accuracy. The algorithms are also

differed by the number of the electrodes used in the

emotion recognition. In the Table 1, in works (Ishino

and Hagiwara, 2003, Zhang and Lee, 2009,

Takahashi, 2004, Petrantonakis and Hadjileontiadis,

2010) 3 or 2 electrodes were used. All other works

employed more than 32 electrodes to collect EEG

data.

In (Liu et al., 2010), we proposed real-time

algorithm only using 3 channels in total. Fractal

dimension algorithms were applied to compute

fractal based features, and a real time EEG-based

emotion recognition algorithm was implemented

with predefined thresholds based on the training

session analysis. In our work, by recognizing arousal

and valence level with an accuracy of 84.9% and

90% respectively, the satisfied, pleasant, happy,

frustrated, sad, fear, and neutral emotions were

differentiated. Since the discrete emotions can be

mapped to the 2D emotion model, and fractal

dimension values can be mapped to 2D emotion

model as well, more emotions that are defined in 2D

model could be distinguished.

3 SPATIO-TEMPORAL

APPROACH

The spatio-temporal approach combines two

methods: a spatio-temporal analysis and fractal

based analysis. The spatio-temporal analysis

includes real-time 3D mapping of EEG signal

amplitude or other parameters, for example, fractal

dimension values, with blobby model defined by

implicit functions and applying set-theoretic

Table 1: Off-line emotion recognition algorithms.

Author

Feature

and Classification

Emotion

Result

Subject-dependent emotion recognition works

(Ishino and

Hagiwara,

2003)

Feature

FFT;

Wavelet transform;

Variance, mean

Classification

Neural Network

Joy,

sad,

angry,

relaxed

Joy:

54.5%

Anger:

67.7%

Sorrow:

59%

Relaxati-on:

62.9%

(Zhang and

Lee, 2009)

Feature

PCA

Classification

Linear Kernel SVM;

RBF Kernel SVM

Negative

and

positive

73.00%

(Chanel et

al., 2006)

Feature

6 frequency bands from

different locations

Classification

Naïve Bayes;

Fisher Discriminant

Analysis

3 degree of

arousal

58%

(Chanel et

al., 2009)

Feature

Short Time Fourier

Transform; Mutual

Information

Classification

Discriminant Analysis;

SVM; Relevance Vector

Machine

Positive/

arousal,

neutral/

calm,

negative/

arousal

63%

(Lin et al.,

2009)

Feature:

ASM 12

Classification

SVM

Joy,

anger,

sadness,

pleasure

90.72%

Subject-independent emotion recognition works

(Khalili and

Moradi,

2009)

Feature

Statistical feature

combined with

Correlation dimension

Classification

Quadratic Discriminant

Analysis

Calm,

positive

aroused,

negative

aroused

76.66%

(Takahashi,

2004)

Feature

statistical features

Classification

SVM, Neural Networks

Joy, anger,

sadness,

fear and

realizati-

41.7% for

five

emotions,

66.7% for

three

emotions

(Petrantona

kis and

Hadjileontia

dis, 2010)

Feature:

Statistical features,

wavelet based features,

higher order crossings.

Classification:

SVM, QDA, KNN,

Mahalanobis Distance

Happy,

surprised,

angry,

fear,

disgust,

sad

62.3% for

single

channel

case,

83.33% for

combined

channel

case

BIOSIGNALS 2011 - International Conference on Bio-inspired Systems and Signal Processing

operations over the moving shapes to isolate

activities common for the signal during the time

interval, as well as those that are unique one. The

proposed fractal based method allows us to estimate

the signal complexity changing over time and then,

recognize the brain state.

3.1 3D Mapping of EEG

We proposed a novel method of EEG analysis based

on 3D mapping of EEG data. We employed a

concept of a dynamic 3D volumetric “blobby” shape

to visualize the EEG signal changes over time. The

blobby-like objects were firstly introduced in (Blinn,

1982, Wyvill et al., 1986). A time-dependent

“blobby” object is defined using implicit functions

that allow us to propose and implement set-theoretic

operations over the time changing shapes for further

analysis.

This object is defined using so-called FRep

representation proposed in (Pasko et al., 1995) and

extended to spatio-temporal model in (Kulish et al.,

2006a, Sourina et al., 2009) where it was described

by the following formula:

()

()()()

222

,,, 0

rb t

ii ii

fxyzt ae g

rxx yy zz

−

=−≥

=−+−+−

∑

(1)

where a is a scale factor, b is an exponent scale

factor changing over time, g is a threshold value, and

m is the number of electrodes used.

At any given point (x, y, z, t), function f can take

negative, positive or zero values. The point is

considered on the surface of the object if the

function value is zero, inside the object if the

function value is positive, and outside the object

otherwise. The shape changes through time due to

the variable values of the exponent factor b

according to the signal. Its size and appearance

visually reflect the brain activity. For a better visual

impression, the blobby shape is superimposed on a

3D head model. Besides just a visual comparison,

we proposed to apply set-theoretic (“Boolean”)

operations to the moving shapes to isolate activities

common for both of them per time point, as well as

those that are unique for either one. Furthermore, the

group set-theoretic operations applied to the

individual time frames of the moving shape allow us

to isolate idle parts of the brain as well as to estimate

an average level of the brain activity. The proposed

operations could be applied over one or/and over

two datasets. On one data set, we can do intersection

of all shapes to show constant activity on the time

interval, and union of all shapes to show the overall

maximum activity. On two data sets, we could apply

an intersection to show common activity, union to

show overall maximum activity and subtraction to

show activities which are characteristic to one set.

3.2 Fractal-based Approach

Fractal dimension (FD) is a measurement of

complexity and irregularity of the object based on an

entropy analysis. Entropy is a measure of the

disorder in physical systems, or an amount of

information that may be gained by observations of

the disordered systems. A common practice to

distinguish among possible classes of time series is

to determine their so-called correlation dimension.

The correlation dimension, however, belongs to an

infinite family of fractal dimensions (Hentschel and

Procaccia, 1983). Hence, there is a hope that the use

of the whole family of fractal dimensions may be

advantageous in comparison to using only some of

these dimensions. The concept of generalized

entropy of a probability distribution was introduced

by Alfred Renyi (Renyi, 1955). Based on the

moments of order

of the probability

p , Renyi

obtained the following expression for entropy

log

−

∑

(2)

where q is not necessarily an integer and log denotes

log

. Note that for

1q →

, Eq. (2) yields the well-

known entropy of a discrete probability distribution

(Shannon, 1998):

log

Spp

=−

∑

(3)

There are various methods to calculate fractal

dimensions. In works (Kulish et al., 2006b, Kulish et

al., 2006a), the generalized Renyi approach based on

Renyi entropy and calculation of the whole spectra

of fractal dimensions to quantify brain states were

studied. In our real-time applications, we apply only

Hausdorff dimension when

0q =

in (2). We

implemented two well-known Higuchi (Higuchi,

1988) and Box-counting (Block et al., 1990)

algorithms calculating fractal dimension. Both of

them were evaluated using mono-fractal Brownian

and Weierstrass functions where theoretical FD

values are known (Wang et al., 2010a). Higuchi

algorithm gave a better accuracy as FD values were

closer to the theoretical FD ones.

A REAL-TIME FRACTAL-BASED BRAIN STATE RECOGNITION FROM EEG AND ITS APPLICATIONS

The algorithms were used in the proposed

algorithm of FD feature extraction with sliding

window for concentration level recognition and

emotion level recognition.

The Box-counting and Higuchi algorithms are

described as follows.

3.2.1 Box-counting Method

Fractal dimension

D is defined in Box counting

method (Block et al., 1990) as

(

)

lim

→

−

⎛⎞

⎜⎟

⎝⎠



(4)

Equivalently,

(

)

()

lim

→

=−

(5)

where

()N

is the number of boxes of length

which cover the whole data set.

Our implementation of the above box counting

formula is based on (Phothisonothai and Nakagawa,

2007). A time series data are covered by a grid of

boxes of length

. ()N

is the number of boxes

that the curve intersects with the grid. Different

values of

result in different number of boxes and

hence different values of

()N

. Therefore, the curve

ln ( )N

is plotted versus ln( )

and

D is

calculated as the slope of that curve multiplied by

1−

3.2.2 Higuchi Method

The following implementation is based on work

(Higuchi, 1988).

Suppose we want to calculate fractal dimension

for a time series

() ( ) ( )

1 , 2 ,...,

xxn

Step 1: Choose one value of

Step 2: Construct the sub-series

from the

time series as following

() ( )

, ,..., .

mxmk xm k

⎛⎞

−

⎡⎤

⎜⎟

⎢⎥

⎣⎦

⎝⎠

(6)

where

1, 2,...,mk=

and [ ] denotes Gaussian

notation which rounds a number in the brackets to its

largest integer which is equal to or smaller than

itself,

m the initial time and

the time interval. For

example, when

3k =

and

100n =

we have 3 sub-

series as follows:

(

)

(

)() ( )

: 1 , 4 , 7 ,..., 100Xx x x x

(

)

(

)

(

)

(

)

: 2 , 5 , 8 ,..., 98Xx x x x

(

)

(

)

(

)

(

)

: 3 , 6 , 9 ,..., 99Xx x x x

Then, every length of each sub-series

calculated. Length

()

Lk of

is equal to

()

()((1).) 1

xm ik xm i k N

−

⎧

⎫

⎛⎞

+− +− −

⎪

⎜⎟

⎪

⎝⎠

⎨

⎬

−

⎡⎤

⎪

⎢⎥

⎪

⎣⎦

⎩⎭

∑

(7)

Step 3: Calculate the average length

()Lk of all

()

Lk.

Step 4: Repeat step 1 to 3 for several values of

Step5: Slope of the curve of

ln( ( ))Lk versus

ln( )k is approximated. Fractal dimension value is

the slope multiplied by -1.

4 IMPLEMENTATION

The proposed real-time system diagram is shown in

Figure 1. The user receives stimuli from the

computer system such as visual, audio, etc. Then,

the mental process of the user thinking is recognized

from his/her EEG that is acquired by the EEG

device. An overall recognition algorithm used in the

real-time applications consists from the following

steps: data sampling and pre-processing including

data filtering, feature extraction, and subject-

dependent machine learning algorithm. Then the

command to the feedback system is formed based on

the recognition results.

Figure 1: Diagram for non-invasive BCI system. ^

BIOSIGNALS 2011 - International Conference on Bio-inspired Systems and Signal Processing

4.1 Pre-processing

The collected data are filtered by a 2-42 Hz

bandpass filter since major waves of EEG lie in this

band (Sanei and Chambers, 2007).

4.2 Features Extraction

The next step after the data pre-processing is feature

extraction. We apply a sliding window and calculate

one FD value per sample per channel. Number of

channels used in the recognition algorithm defines a

size of the feature vector as follows:

{ , ,... }

FD FD FD=

(8)

where m is number of channels

In the concentration level recognition algorithm,

we have one feature as only one channel is used

(Wang et al., 2010a). In the emotion recognition

algorithm, there are 3 features in the vector as 3

channels are used (Liu et al., 2010). Thus, in our

approach we use one FD value per channel.

4.3 Classification Algorithms

Currently, we implemented a simple real-time

subject-dependent classification algorithm based on

threshold FD values that are calculated during a

short training session. Note that off-line processing

with SVM classifier of the EEG labelled with

emotions and concentration levels gave us similar

accuracy as the real time implementation algorithm

used thresholds.

5 APPLICATIONS

EEG data are collected by Emotiv device with 14

electrodes located at AF3, F7, F3, FC5, T7, P7, O1,

O2, P8, T8, FC6, F4, F8, AF4 standardized by the

American Electroencephalographic Society (1991).

The sampling rate is 128Hz. To be able to use any

EEG device a program reading raw EEG signals is

needed to be implemented. Currently, our

applications could also work with Pet 2 and Mindset

24. All electrodes can be active in the system. The

steps of an overall algorithm of the real-time

application are as follows. First, raw data are read

from the EEG device, filtered with band pass filter

2-42 Hz, and entered to the corresponding brain state

recognition algorithm. Then, the results of the

recognition are fed to the developed game, web site,

or any other real-time software.

5.1 3D Mapping of EEG

We proposed a spatio-temporal approach to EEG

analysis. 3D blobby-based EEG mapping was

implemented for offline processing. To monitor and

analyze the subject/user brain state in real time, a

system “VisBrain” was implemented. Signal

amplitude values are visualized with blobby model,

color or “pins”. In Figure 2, spatio-temporal

visualization of EEG signals is shown with the 3D

blobby mapping. The blobby model allows assessing

a spatio-temporal pattern of the subject/user EEGs

corresponding to different brain states.

Figure 2: Real-time system “VisBrain” for spatio-temporal

analysis.

5.2 EEG-based Serious Games

The EEG-based serious game design includes two

parts: signal processing algorithms and a 2D/3D or

virtual reality game part. Raw EEG signals collected

by the device from the user brain are filtered and

analyzed by signal processing algorithms, and the

resulting values are interpreted in the game as an

additional game control using just the “brain

power”. A therapeutic effect of such games consists

from combination of a distraction effect of the game

and an effect from the learning by the user/patient

how to control the game by changing voluntary

his/her brain state, for example, learning how to

improve the user’s concentration level. We

developed two concentration games named “Brain

Chi” and “Dancing Robot”, and one game for stress

management named “Pipe”. They are simple single-

player games implemented with the game engines

SDL, Panda3D, and Adobe Flash CS4

correspondingly. The recognized relaxation/

concentration/ stress level values from EEG could be

interpreted in the games as any visual/audio effects

or even as a behaviour change of the game

characters.

In the “Brain Chi” game, the relaxation/

concentration level of the user is associated with

radius of a “growing/shrinking” ball. It allows the

“little boy” character to fight enemies by “growing”

A REAL-TIME FRACTAL-BASED BRAIN STATE RECOGNITION FROM EEG AND ITS APPLICATIONS

the ball. In the “Dancing Robot” game, the

relaxation/concentration level is associated with the

“robot” character behaviour. When the concentration

level of the user increases, the robot character starts

to move faster. If the user is fully relaxed, the robot

stops dancing. In our implementation, the

concentration and relaxation levels could be easily

associated either with concentration training or

relaxation training depending on the therapeutic

purpose of the game. In Figure 3, a change of the

quantified level of the user concentration level is

interpreted as a “faster/slower” movement of the

“robot”.

Figure 3: EEG-enable concentration “Dancing Robot”

game.

The “Pipe” game is implemented more as a

traditional neurofeedback game. In Figure 4, the

“blue” bar located under the “green” bar on the

screen shows the level of the user stress. In the

“Pipe” game, water flows faster when the player’s

stress level increases, and hence it makes the game

playing more difficult.

We also did preliminary study how to use the

EEG-enable serious games for pain management and

have got some promising results.

Figure 4: EEG-enable stress management “Pipe” game.

5.3 Emotion-based Digital Experience

We proposed and implemented a real-time fractal-

based emotion recognition algorithm where the

calculated fractal dimension values were mapped to

2D Valence-Arousal emotion model. It is possible to

recognize in real time any discrete emotions that

could be defined with the 2-dimensional emotion

model. Satisfied, pleasant, happy, frustrated, sad,

fear, and neutral emotions were recognized. In our

algorithm, only 3 channels are used. We

implemented two emotion-enable real-time

applications. First, we implemented an application

with the EEG-enable avatar (Liu et al., 2010). The

music stimuli were used for emotion induction as it

was proposed in (Sourina et al., 2008). We used an

avatar available with free version of Haptek

development

package for our application (Haptek).

(a)

(b)

Figure 5: Emotion-enabled applications: a) “Pleasant”

emotion is recognized and visualized on the user 3D

avatar; b) “Sad” emotion is recognized and sent to the

music therapy Web site.

Haptek Activex control provides functions and

commands to change facial expressions of 3D

avatars. We defined six emotions by changing the

parameters controlling the facial muscles of the

Haptek emotion avatar. Those emotions are: fear,

frustrated, sad, happy, pleasant and satisfied. In the

application, emotions of the user are recognized

from EEG and visualized in real-time on the user’s

avatar with Haptek system. In Figure 5(a), the user

was listening to music pieces for emotion induction,

and the algorithm recognized “pleasant” emotion

that was visualized on the user’s avatar. The avatar

BIOSIGNALS 2011 - International Conference on Bio-inspired Systems and Signal Processing

emotions are changed according to emotions that the

user is feeling during the music listening. Second,

we implemented an EEG-enable music therapy web-

site. The user/patient’s emotion is recognized from

EEG, and the corresponding music piece is

downloaded according to the emotion recognized

from the EEG of the user. In Figure 5(b), emotions

are induced by music stimuli played through the

earphone, and then the “inner” user’s emotion is

recognized from the EEG signal in real-time. Then,

a pleasant song is played to the user upon

recognizing the user is being sad to improve his/her

mood or the corresponding music is played to calm

down the user if he/she is too excited (happy or

angry) or too nervous feeling “fear” emotion. A

simple music therapy algorithm was proposed

allowing automatically down load the corresponding

music piece based on the recognized emotion to

change the user mood. The choice of the appropriate

music from the list could be given the user as well.

6 CONCLUSIONS

In this paper, we proposed and described a novel

spatio-temporal fractal based approach to study

different brain states such as concentration levels,

human emotions, and in future other brain states

such as “central pain” feeling, attention levels, etc.

Using just one fractal dimension feature per channel

and the simple machine learning algorithm allows us

to implement real-time brain recognition

applications with acceptable accuracy that could be

improved by subject-based training. We expect

further use of the proposed approach in different

real-time applications. It could be also used in

studies such as validation of the hypotheses: emotion

induction could change pain level in the patients; the

positive emotions could improve human

performance, etc. We also work on the improvement

on the real-time filtering of artefacts of different

origin. The work described in the paper is a part of

the project EmoDEx presented in (IDM-Project,

2008).

ACKNOWLEDGEMENTS

This project is supported by the grant NRF2008

IDM-IDM004-020 “Emotion-based personalized

digital media experience in Co-Spaces” of National

Research Fund of Singapore.

REFERENCES

1991. American electroencephalographic society

guidelines for standard electrode position

nomenclature. Journal of Clinical Neurophysiology, 8,

200-202.

Blinn, J. F. 1982. Generalization Of Algebraic Surface

Drawing. Computer Graphics (ACM), 16, 273.

Block, A., Von Bloh, W. & Schellnhuber, H. J. 1990.

Efficient box-counting determination of generalized

fractal dimensions. Physical Review A, 42, 1869-1874.

Chanel, G., Kierkels, J. J. M., Soleymani, M. & Pun, T.

2009. Short-term emotion assessment in a recall

paradigm. International Journal of Human Computer

Studies, 67, 607-627.

Chanel, G., Kronegg, J., Grandjean, D. & Pun, T. 2006.

Emotion assessment: Arousal evaluation using EEG's

and peripheral physiological signals

Coben, r., Linden, M. & Myers, T. E. 2010.

Neurofeedback for autistic spectrum disorder: A

review of the literature. Applied Psychophysiology

Biofeedback, 35, 83-105.

Demos, J. N. 2005. Getting Started with Neurofeedback,

New York, WW Norton & Company.

Fuchs, T., Birbaumer, N., Lutzenberger, W., Gruzelier, J.

H. & Kaiser, J. 2003. Neurofeedback treatment for

attention-deficit/hyperactivity disorder in children: A

comparison with methylphenidate. Applied

Psychophysiology Biofeedback, 28, 1-12.

Gevensleben, H., Holl, B., Albrecht, B., Schlamp, D.,

Kratz, O., Studer, P., Wangler, S., Rothenberger, A.,

Moll, G. H. & Heinrich, H. 2009. Distinct EEG effects

related to neurofeedback training in children with

ADHD: A randomized controlled trial. International

Journal of Psychophysiology, 74, 149-157.

HAPTEK. Available: http://www.haptek.com [Accessed].

hentschel, H. G. E. & Procaccia, I. 1983. The infinite

number of generalized dimensions of fractals and

strange attractors. Physica D: Nonlinear Phenomena,

8, 435-444.

Higuchi, T. 1988. Approach to an irregular time series on

the basis of the fractal theory. Physica D: Nonlinear

Phenomena, 31, 277-283.

IDM-PROJECT. 2008. Emotion-based personalized

digital media experience in Co-Spaces [Online].

Available: http://www3.ntu.edu.sg/home/eosourina/

CHCILab/projects.html [Accessed].

Ishino, K. & Hagiwara, M. Year. A feeling estimation

system using a simple electroencephalograph. In

2003. 4204-4209.

Khalili, Z. & Moradi, M. H. Year. Emotion recognition

system using brain and peripheral signals: Using

correlation dimension to improve the results of EEG.

In: Proceedings of the International Joint Conference

on Neural Networks, 2009. 1571-1575.

Kouijzer, M. E. J., Van Schie, H. T., De Moor, J. M. H.,

Gerrits, B. J. L. & Buitelaar, J. K. 2010.

Neurofeedback treatment in autism. Preliminary

findings in behavioral, cognitive, and

A REAL-TIME FRACTAL-BASED BRAIN STATE RECOGNITION FROM EEG AND ITS APPLICATIONS

neurophysiological functioning. Research in Autism

Spectrum Disorders, 4, 386-399.

Kulish, V., Sourin, A. & Sourina, O. 2006a. Analysis and

visualization of human electroencephalograms seen as

fractal time series. Journal of Mechanics in Medicine

and Biology, World Scientific, 26(2), 175-188.

Kulish, V., Sourin, A. & Sourina, O. 2006b. Human

electroencephalograms seen as fractal time series:

Mathematical analysis and visualization. Computers in

Biology and Medicine, 36, 291-302.

Lin, Y. P., Wang, C. H., Wu, T. L., Jeng, S. K. & Chen, J.

H. Year. EEG-based emotion recognition in music

listening: A comparison of schemes for multiclass

support vector machine. In: ICASSP, IEEE

International Conference on Acoustics, Speech and

Signal Processing - Proceedings, 2009 Taipei. 489-

492.

Liu, Y., Sourina, O. & Nguyen, M. K. Year. Real-time

EEG-based Human Emotion Recognition and

Visualization In: Proc. 2010 Int. Conf. on

Cyberworlds, 20-22 Oct, 2010 2010 Singapore. 262-

269.

Lubar, J. F., Swartwood, M. O., Swartwood, J. N. & O'

Donnell, P. H. 1995. Evaluation of the effectiveness of

EEG neurofeedback training for ADHD in a clinical

setting as measured by changes in T.O.V.A. scores,

behavioral ratings, and WISC-R performance.

Biofeedback and Self-Regulation, 20, 83-99.

Nunez, P. L. & Srinivasan, R. 2006. Electric Fields of the

Brain, Oxford University Press.

Pasko, A., Adzhiev, V., Sourin, A. & Savchenko, V. 1995.

Function representation in geometric modeling:

concepts, implementation and applications. The Visual

Computer, 11, 429-446.

Petrantonakis, P. C. & Hadjileontiadis, L. J. 2010.

Emotion recognition from EEG using higher order

crossings. IEEE Transactions on Information

Technology in Biomedicine, 14, 186-197.

Phothisonothai, M. & Nakagawa, M. 2007. Fractal-based

EEG data analysis of body parts movement imagery

tasks. Journal of Physiological Sciences, 57, 217-226.

Renyi, A. 1955. On a new axiomatic theory of probability.

Acta Mathematica Academiae Scientiarum

Hungaricae, 6, 285-335.

Sanei, S. & Chambers, J. 2007. EEG signal processing,

Chichester, England ; Hoboken, NJ, John Wiley &

Sons.

Saxby, E. & Peniston, E. G. 1995. Alpha-theta brainwave

neurofeedback training: An effective treatment for

male and female alcoholics with depressive symptoms.

Journal of Clinical Psychology, 51,

685-693.

Shannon, C. E. 1998. The mathematical theory of

communication University of Illinois Press.

Sokhadze, T. M., Cannon, R. L. & Trudeau, D. L. 2008.

EEG biofeedback as a treatment for substance use

disorders: Review, rating of efficacy, and

recommendations for further research. Applied

Psychophysiology Biofeedback, 33, 1-28.

Sourina, O., Kulish, V. V. & Sourin, A. Year. Novel Tools

for Quantification of Brain Responses to Music

Stimuli. In: Proc of 13th International Conference on

Biomedical Engineering ICBME 2008, 3-6 December

2008. 411-414.

Sourina, O., Sourin, A. & Kulish, V. Year. EEG data

driven animation and its application. In: Proc of

International Conference Mirage 2009, 4-6 May 2009

Rocquencourt. 380-388.

Takahashi, K. Year. Remarks on emotion recognition from

multi-modal bio-potential signals. In, 2004. 1138-

1143.

Wang, Q., Sourina, O. & Nguyen, M. K. Year. EEG-based

"Serious" Games Design for Medical Applications. In:

In Proc. 2010 Int. Conf. on Cyberworlds, 2010a

Singapore. 270-276.

Wang, Q., Sourina, O. & Nguyen, M. K. Year. Fractal

Dimension Based Algorithm for Neurofeedback

Games. In: Proc. CGI 2010. SP25, 2010b Singapore.

Wyvill, G., Mcpheeters, C. & Wyvill, B. 1986. Data

structure for soft objects. The Visual Computer, 2,

227-234.

Zhang, Q. & Lee, M. 2009. Analysis of positive and

negative emotions in natural scene using brain activity

and GIST. Neurocomputing, 72, 1302-1306.

BIOSIGNALS 2011 - International Conference on Bio-inspired Systems and Signal Processing