AUTOMATIC VIDEO DETECTION OF NOCTURNAL

EPILEPTIC MOVEMENT BASED ON MOTION TRACKS

Kris Cuppens

1,2

, Bert Bonroy

, Anouk Van de Vel

, Berten Ceulemans

, Lieven Lagae

Tinne Tuytelaars

, Sabine Van Huffel

2,6

and Bart Vanrumste

1,2

Mobilab, K. H. Kempen, Geel, Belgium

KULeuven, ESAT, SISTA, BioMed, Leuven, Belgium

UZAntwerpen, Child Neurology, Edegem, Belgium

UZLeuven, Pediatrics, Leuven, Belgium

KULeuven, ESAT, PSI, Leuven, Belgium

KU Leuven-IBBT Future Health Department, Leuven, Belgium

Keywords: Epilepsy, Seizure detection, Image motion analysis, Video monitoring, Optical flow, Mean shift clustering.

Abstract: Epileptic seizure detection in a home situation is often not feasible due to the complicated attachment of the

EEG-electrodes on the scalp. We propose to detect nocturnal seizures with a motor component in patients

by means of a single video camera. To this end we use a combination of optical flow and mean shift

clustering to register moving body parts. After extraction of seven features, related to amplitude, duration

and direction of the motion, we carry out a first validation with a linear support vector machine classifier.

This resulted in a sensitivity of 80.60% and a positive predictive value of 62.07%.

1 INTRODUCTION

Epilepsy is one of the most common neurological

disorders in the world, that affects approximately

one percent of the world’s population. In 80% of the

patients the seizures can be controlled by medication

or surgery. In the other group (20%), patients need

to be monitored on a regular basis to follow up their

disease, especially during the night when there is no

social control by the patient’s environment.

However knowing the number of seizures during the

night would provide the neurologist with an

objective measure to alter the medication and

increase the quality of life of the patient. A second

advantage of a nocturnal monitoring system is that

an alarm can be given if a heavy seizure occurs

which needs care afterwards.

Currently the most widely used method for

monitoring epilepsy is video/EEG-monitoring. But

the attachment of EEG-electrodes, which measure

the electrical activity of the brain, is complicated,

labor intensive and mostly restricted to a clinical

setting. For monitoring patients in a home setting, a

more practical setup is required.

Monitoring a patient with a video recording

system can offer a solution, as it monitors the patient

without making physical contact and it is easy to

install in the bedroom.

Human motion analysis has already been studied

extensively in other applications such as gesture

recognition or surveillance (Hu et al, 2004); (Turaga

et al., 2008). The nocturnal detection of seizures is

in some points more challenging. We cannot use the

skin color information, which is often used in

gesture analysis, due to the gray scale images

because of night vision. Also, detecting the patient’s

extremities is most of the time not feasible because

the patients are covered with a blanket. And finally,

there are different types of motor manifestations in

seizures. Therefore we focus on one type of seizure

only, namely the myoclonic seizure. Even then,

there is a considerable variation in manifestation

within a dataset of one patient and the datasets

between patients.

In Cuppens et al. (Cuppens et al., 2010) body

movement is detected in nocturnal recordings from

epileptic patients as a data reduction step. The

optical flow based method reaches a sensitivity of

100% and a positive predictive value between 82%

and 100% when testing it on three test sets.

342

Cuppens K., Bonroy B., Van de Vel A., Ceulemans B., Lagae L., Tuytelaars T., Van Huffel S. and Vanrumste B..

AUTOMATIC VIDEO DETECTION OF NOCTURNAL EPILEPTIC MOVEMENT BASED ON MOTION TRACKS.

DOI: 10.5220/0003742903420345

In Proceedings of the International Conference on Bio-inspired Systems and Signal Processing (BIOSIGNALS-2012), pages 342-345

ISBN: 978-989-8425-89-8

 2012 SCITEPRESS (Science and Technology Publications, Lda.)

Karayiannis (Karayiannis et al., 2006) already

studied the detection of seizures in neonates, with a

focus on focal clonic and myoclonic seizures. The

patients were monitored in the neonatal intensive

care unit where all the body parts were clearly

visible for the camera. The motion of the body parts

was quantified by temporal motion-strength signals

based on optical flow computation, and by motion

trajectories. The classification based on neural

networks resulted in a sensitivity and specificity

above 90%.

The difference in our study is that we focus on a

different patient group. Furthermore, the setting is as

natural as possible, i.e. the patients can be covered

with a blanket and we do not make use of markers,

just as in a normal home situation. And finally, we

also make use of new features that we extract from

the motion tracks.

2 METHOD

2.1 Data Acquisition

The data was acquired at the Pulderbos rehabilitation

centre for children and youth. In this centre,

epileptic patients up to the age of 18 years are

monitored. Video data is acquired during the night

with a near-infrared camera. The frame rate is 25

frames per second and the resolution is 320 by 240

pixels. The datasets are labelled by an expert based

on the video and EEG recordings.

Figure 1: Motion vector field in one frame from epileptic

patient, the colors represent the direction of motion

vectors (a) the frame after thresholding (b).

2.2 Motion Detection

The extraction of motion from the video sequences

is carried out by the Horn-Schunck optical flow

calculation (Horn and Schunck, 1981). The motion

vector field is thresholded to remove low amplitude

noisy motion vectors and to get a first segmentation,

as shown in figure 1.

Figure 2: Result after clustering the motion vectors. The

red circles indicate the modes of the clusters. The colors

indicate here the direction of the motion.

2.3 Spatial Clustering

In a next step the motion vectors are clustered in

homogenous blobs with similar motion directions.

This is realized by a slightly modified version of the

mean-shift clustering algorithm (Cheng, 1995);

(Fukunaga and Hostetler, 1975), which was

proposed in (Min et al., 2008) for the classification

of movements in ballet sequences.

The mean shift algorithm is an iterative

procedure that starts in a pixel in the segmente d

area. In the next iteration the point moves to a new

location according to eqn. (1).

(1)

where p is the original location, q are the pixels in

image I, K is the chosen kernel, F is the amplitude of

the motion vector and w is a weight function based

on vector similarity.

The point moves towards the location with the

local maximal vector amplitude (mode). All the

points in the segmented frame that iterate to the

same local maximum belong to the same cluster.

As a kernel we use a parabolic function with a

fixed bandwidth. The weight function w gives a

measure for the similarity of the vector directions,

with a value of zero if the angle between the two

vectors is 180° and one if the angle is 0°. This way,

vectors with a different direction are not clustered

together.

Figure 2 shows the result of clustering the frame

from figure 1. The two parts in the frame with a

different movement direction are well split up.

)()()(

)(









qwqFqpK

qqwqFqpK

AUTOMATIC VIDEO DETECTION OF NOCTURNAL EPILEPTIC MOVEMENT BASED ON MOTION TRACKS

343

2.4 Tracking

After the motion vectors in the frames are clustered,

they are tracked over time based on the location of

the cluster modes, as explained in (Crocker and

Grier, 1996), assuming that the location of the object

would not change too much from one frame to

another (proximity) and the maximum velocity

defines an upper bound on the object velocity and

limits the possible correspondences to a circular

neighborhood around the object (Yilmaz et al.,

2006). If the tracked body part stops moving for

more than half a second, the movement is split up in

two tracks, otherwise they are considered as one.

Figure 3: Motion tracks (green and red) with the

corresponding motion vectors in blue, the YZ-plane

represent the locations in the image frame, the X-direction

corresponds to the time (a) motion tracks represented with

the amplitude and sign (Y-axis) and time (X-axis) (b).

2.5 Feature Extraction

From every cluster in every frame the average

motion vector is calculated. Every motion track

results in a series of two dimensional motion vectors

from which we derive different features. This is

visualized in figure 3.

Because the myoclonic shocks are characterized

by a short and intense movement of the limbs,

mostly in arms or shoulders, in one direction

followed by a movement in the opposite direction,

e.g. the relaxation after the contraction of the arms

muscles, we incorporate features that quantify the

change in motion direction.

Therefore we use two features covering the

change in direction, namely slow and fast phase

changes. Other features we use are the duration of

the movement, the maximal, median and average

vector amplitude and the number of vectors in the

track. The number of vectors is sometimes smaller

than the length because the body part can stop

moving for up to half a second and still be in the

same track.

2.6 Classification

The classification of the seizures is carried out by a

support vector machine (SVM) classifier. We use a

linear kernel on all seven features for classification.

This classification was done on one training and test

set, to get our first preliminary results. Notice that

the classification was performed on the tracks, not

on the seizures as a whole.

3 RESULTS

Table 1: Feature values of tracks.

#vectors

Mean

amplitude

Median

amplitude

Max

amplitude

Quick

changes

Slow

changes

Length

(frames)

Normal movement

#1 3 0.67 0.71 0.73 0 0 3

#2 48 0.72 0.75 0.95 0 2 49

#3 15 0.43 0.47 0.52 2 0 33

#4 6 0.71 0.71 0.84 0 0 7

#5 6 0.41 0.39 0.48 0 0 6

Epileptic movement

#1 6 0.26 0.28 0.37 1 0 8

#2 5 0.09 0.10 0.11 1 0 7

#3 33 0.20 0.13 0.59 0 1 44

#4 4 0.38 0.42 0.51 2 0 7

#5 12 0.11 0.12 0.22 0 0 24

Some typical values for the track features can be

found in table 1. This table shows the values for 5

epileptic and 5 non-epileptic tracks from motions.

For the validation of our approach we use a

training set containing 100 motion tracks from

normal nocturnal movement from an epileptic

patient, and 113 tracks from 10 myoclonic shocks.

All the data is from one patient. During the time that

BIOSIGNALS 2012 - International Conference on Bio-inspired Systems and Signal Processing

344

a seizure occurs, all the tracks are labeled as

epileptic. The test set consists also of 100 normal

motion tracks and 67 epileptic motion tracks from 5

myoclonic seizures. For the time being, the

classification is based on the single tracks, so not on

the whole seizures, thus we have 113 positive

samples in the training set and 67 in the test set, each

of them being labeled as part of a myoclonic shock.

After training the linear SVM model once, we

tested it on our test set which resulted in 54 true

positives, 67 true negatives, 33 false positives and

13 false negatives. This corresponds to a sensitivity

of 80.60%, a Positive Predictive Value of 62.07%

and a specificity of 67.00%.

4 DISCUSSION

The classification is carried out on individual motion

tracks. The motion tracks extracted during an

epileptic shock are possibly not all from the epileptic

movement itself, but also from e.g. movements of

the bed because of the seizure. So the classification

can be improved on this point. Moreover, the

features from different tracks originating from one

seizure can be combined, to further improve the

detection.

The training and testing is now performed on a

small dataset. To have more solid validation, a larger

dataset should be used. These results are preliminary

but give an indication that the detection of specific

types of seizures by the proposed algorithm is

possible.

The obtained results in this paper are less optimal

than in (Karayiannis et al., 2006), namely a

sensitivity of 80.60% and a specificity of 67.00%

compared to a sensitivity and specificity above 90%

in (Karayiannis et al., 2006). But the circumstances

in our setup were more difficult as the patients’ body

parts are most of the time not clearly visible.

Removing the blankets is not an option as it would

reduce the sleeping quality of the patients too much.

But notice that there is still some room for

improvement in our algorithm.

5 CONCLUSIONS

The detection of seizures based on motion tracks

extracted from the optical flow calculation and the

mean shift clustering algorithm is possible. In the

first test on 15 myoclonic shocks a sensitivity of

80.60% and a positive predictive value of 62.07% is

reached. Further research is required to confirm

these first results and to test the algorithm on other

seizures.

ACKNOWLEDGEMENTS

Research supported by Research Council KUL:

GOA-MANET, IWT: TBM070713-Accelero,

Belgian Federal Science Policy Office IUAP P6/04

(DYSCO, ‘Dynamical systems, control and

optimization, 2007–2011); EU: Neuromath

(COSTBM0601). Kris Cuppens is funded by a Ph.D

grant of the Institute for the Promotion of Innovation

through Science and Technology in Flanders (IWT-

Vlaanderen)

REFERENCES

Karayiannis N. B., Xiong Y., Tao G., Frost J. D. Jr., Wise

M. S., Hrachovy R. A. and Mizrahi E. M., Automated

detection of videotaped neonatal seizures of epileptic

origin. Epilepsia, vol. 47, pp. 966-980, 2006.

Min J. H., Kasturi R. and Camps O., Extraction and

temporal segmentation of multiple motion trajectories

in human motion. Image and Vision Computing, vol.

26, pp. 1621-1635, 2008.

Crocker J. C. and Grier D. G., Methods of digital video

microscopy for colloidal studies. Journal of Colloid

and Interface Science, vol. 179, pp. 298-310, 1996.

Yilmaz A., Javed O. and Shah M., Object tracking: A

survey. Acm Computing Surveys, vol. 38, 2006.

Horn B. K. P. and Schunck B. G., Determining Optical-

Flow. Artificial Intelligence, vol. 17, pp. 185-203,

1981.

Cuppens K., Lagae L., Ceulemans B., Van Huffel S. and

Vanrumste B., Automatic video detection of body

movement during sleep based on optical flow in

pediatric patients with epilepsy. Medical & biological

engineering & computing, vol. 48, N° 9, pp. 923-931,

2010.

Cheng Y. Z., Mean Shift, Mode Seeking, And Clustering.

IEEE Transactions on Pattern Analysis and Machine

Intelligence, vol. 17, pp. 790-799, 1995.

Fukunaga K. and Hostetler L. D., Estimation Of Gradient

Of A Density-Function, With Applications In Pattern-

Recognotion. IEEE Transactions on Information

Theory, vol. 21, pp. 32-40, 1975.

Hu W. M., Tan T. N., Wang L., Maybank S., A survey on

visual surveillance of object motion and behaviors.

IEEE Transactions on Systems Man and Cybernetics

Part C-Applications and Reviews, vol. 34, pp. 334-

352, 2004.

Turaga P., Chellappa R., Subrahmanian V. S., Udrea O.,

Machine Recognition of Human Activities: A Survey.

IEEE Transactions on Circuits and Systems for Video

Technology, vol. 18, pp. 1473-1488, 2008.

AUTOMATIC VIDEO DETECTION OF NOCTURNAL EPILEPTIC MOVEMENT BASED ON MOTION TRACKS

345