Multi-class Motor Imagery EEG Classification using Convolution

Neural Network

Amira Echtioui

1,2 a

, Wassim Zouch

, Mohamed Ghorbel

, Chokri Mhiri

4,5 d

and Habib Hamam

ATMS Lab, Advanced Technologies for Medicine and Signals, ENIS, Sfax University, Sfax, Tunisia

Faculty of Engineering, Moncton University, NB, E1A3E9, Canada

King Abdulaziz University (KAU), Jeddah, Saudi Arabia

Department of Neurology, Habib Bourguiba University Hospital, Sfax, Tunisia

Neuroscience Laboratory “LR-12-SP-19”, Faculty of Medicine, Sfax University, Sfax, Tunisia

Keywords: Electroencephalogram (EEG), BCI, CNN, LSTM.

Abstract: Electroencephalogram (EEG) signals based on Motor Imagery (MI) are a widely used form of input in Brain

Computer Interface (BCI). Although there are several ways to classify data, a question remains as to which

method to use in EEG signals based on motor imagery. This article presents an attempt to reach the best

classification method based on deep learning methods by comparing two models: Convolutional Neural

Network (CNN) and Long Short-Term Memory (LSTM), on the same basic data set. The BCI Competition

IV dataset 2a was used as the base dataset to test the two classification methods. Experimental results show

that the proposed CNN model outperforms the LSTM model, with an accuracy value of 74%, and other state-

of-the-art methods.

1 INTRODUCTION

BCI is a method of communication between a system

and a user that does not depend on the muscles or

normal nerve pathways that exit the brain. The

process begins with the acquisition of the user's brain

activities and is followed by the processing of EEG

signals to detect the user's intentions. This signal is

then sent to an external device, such as a wheelchair,

which is then controlled according to the detected

signal. Hence, the identification of motor imagery

movement intentions is of great importance in the

field of Artificial Intelligence Rehabilitation

Medicine. As per the literature, in-depth learning

methods have gained enormous success in signal,

image, video, speech, and other areas, which perform

better than hand-craft methods.

In reference (Tabar et al., 2017), the authors

present a hybrid deep learning model that combines

https://orcid.org/0000-0003-2041-1301

https://orcid.org/0000-0003-1047-1968

https://orcid.org/0000-0003-0821-0398

https://orcid.org/0000-0001-7591-6994

https://orcid.org/0000-0002-5320-1012

the CNN model and Stacked AutoEncoder (SAE).

They demonstrate a powerful capability for the

classification of IM tasks. Other authors of reference

(Ma et al., 2018) developed an LSTM by proposing a

temporal and spatial recurrent neural network that

outperforms other methods with a gain of 8.25% in

classification accuracy.

In reference (Shen et al., 2017), the authors

combine RNN with CNN in order to improve the

feature representation and classification capabilities

of MI-EEG. In reference (Shen et al., 2017), an end-

to-end LD approach using CNNs and LSTMs in the

long and short term is proposed for the classification

of raw EEG data without applying any pre-

processing. A model with CNN input for a separate

spatial and temporal filtering producing good results

despite the architecture is very simple (Schirrmeister

et al., 2017). Our objective is to propose a new Deep

Learning-based method to enhance the performance

Echtioui, A., Zouch, W., Ghorbel, M., Mhiri, C. and Hamam, H.

Multi-class Motor Imagery EEG Classiﬁcation using Convolution Neural Network.

DOI: 10.5220/0010425905910595

In Proceedings of the 13th International Conference on Agents and Artiﬁcial Intelligence (ICAART 2021) - Volume 1, pages 591-595

ISBN: 978-989-758-484-8

591

of the motor imagery classification. The main

research contributions to this work may be

summarized as follows:



Application of a simple pre-processing of the

data: removal of EOG channels and extraction

of three second period epochs from the data set

in 288 events for the 4 classes;



Selection of EEG channels;



We propose two new methods of classifying

MIs based on deep learning: modified CNN

and modified RNN-LSTM;



The proposed method based on CNN allows for

the classification of MI with an accuracy of

74%.



The comparative results show that the proposed

modified CNN-based method could offer the

best performance in achieving the highest

accuracy value compared to recent state of the

art approaches.

The remainder of this paper is organized as follows.

Our proposed method is presented in Section 2.

Results and the discussion are detailed in Section 3.

Section 4 concludes the paper.

2 METHODOLOGY

In this section, we provide a description of the

database used in this work. We also present a detailed

description of the methodology proposed to classify

MIs based on deep learning methods.

2.1 Data Set Description

In this work, we use multi-class motor imagery EEG

data from the BCI Competition IV 2a dataset. This

dataset includes nine subjects who performed motor

imagery tasks in four classes: left hand, right hand,

feet and tongue. The EEG data for each subject

consists of 288 trials of the MI task in four classes and

each class of tasks was tested 72 times. EEG signals

were recorded with 22 EEG channels and 3 EOG

channels, sampled at 250 Hz and a bandwidth filtered

between 0.5 and 100 Hz. The paradigm of the single

MI task is shown in Figure 1.

2.2 Proposed Work

We propose two deep learning-based methods for the

classification of motor imagery. Figure 2 displays the

block

diagram of our proposed technique. Our

Figure 1: The paradigm time sequence.

proposal begins with a pre-processing step where we

removed the EOG channels and extracted the epochs

of 3 s time period from the dataset into 288 events for

all 4 classes. A channel selection step follows. We

tested our methodology on 22 electrodes, after which

we selected 12 electrodes: Fz, FC3, FC1, FCz, FC2,

FC4, C5, C3, C1, Cz, C2, and C4. Finally, we tested

two classifiers: modified CNN and modified RNN-

LSTM.

Figure 2: Block diagram of our proposed technique.

2.3 Proposed Modified CNN

Figure 3 shows the proposed modified CNN

architecture used to classify the motor imagery tasks.

We fine-tuned this model by 250, 500, 750 and 1000

epochs. The batch size is set to 64; the ADAM

optimizer is used to optimize the loss function; and

the learning rate is 0.001. The activation functions

used in this model are ReLu, Elu, Selu, and Tanh.

2.4 Proposed Modified RNN-LSTM

LSTMs are a modified version of RNNs, allowing for

easier storage of past data in memory. The problem

of the disappearance gradient of RNNs is foreign to

LSTM. It is well suited to classify, process and

predict time series based on time lags of unknown

duration. It drives the model using backpropagation.

For more details of RNN and LSTM cells, the reader

is referred to reference (Zhang et al., 2019).

Fixation cross

Break

Cue

Motor imagery

(

)

6 7 8

SDMIS 2021 - Special Session on Super Distributed and Multi-agent Intelligent Systems

592

Figure 3: Block diagram of the proposed modified CNN method.

Figure 4 shows the proposed modified RNN-

LSTM. We tested this model by 250, 500, 750 and

1000 epochs. The batch size is set to 100, the ADAM

optimizer is used to optimize the loss function, and

the learning rate is 0.001. The activation functions

used in this model are ReLu, Elu, Selu, and Tanh.

Figure 4: Block diagram of the proposed modified

RNN-LSTM method.

3 RESULTS AND DISCUSSION

The performance criterion we used to evaluate the

performance of the two models is the accuracy value.

It is defined by the rate of correctly classified motor

imagery tasks. It is defined by the following equation:

𝐴𝑐𝑐𝑢𝑟𝑎𝑐𝑦  𝑇𝑃  𝑇𝑁/𝑇𝑃  𝑇𝑁  𝐹𝑃  𝐹𝑁

(1)

where:

• TP (True Positive) is the number of motor

imagery tasks that are correctly detected;

• TN (True Negative) is the number of non-motor

imagery tasks that are correctly classified as non-

motor imagery tasks;

• FP (False Positive) is the number of motor

imagery tasks that are incorrectly determined by

a classifier; and

• FN (False Negative) is the actual number of

motor imagery tasks that are incorrectly assigned

to other classes.

In tables 1 and 2 and Figure 5, the classification

accuracy of each network is calculated for the 22 EEG

channels. It can be seen that the modified RNN-

LSTM did not produce a superior performance,

scoring a 70% accuracy with the Elu activation

function (1000 epochs) compared to the modified

CNN classifier. The latter produced the greatest

accuracy value of our work reaching 74% with the

Elu activation function after 750 epochs.

At the same time, the results obtained for the

modified CNN and the modified RNN-LSTM-based

methods with the selection of 12 channels

demonstrated both the same classification precision

value, that is 56% after 1000 epochs.

This value is achieved by the modified CNN with

the ReLu activation function and by the modified

RNN-LSTM with Selu. After comparing these

methods, it is found that the modified CNN is the most

optimal architecture, of which the classification

accuracy far exceeds the accuracy rates of the

modified RNN-LSTM.

Multi-class Motor Imagery EEG Classiﬁcation using Convolution Neural Network

593

Figure 5: Classification result with: (a) the Relu function, (b) the Elu function, (c) the Selu function, and (d) the Tanh function.

Table 1: Accuracy values obtained by the modified CNN.

Epochs

number

Modified CNN (22 electrodes) Modified CNN (12 electrodes)

Relu Elu Selu Tanh Relu Elu Selu Tanh

250 50% 52% 50% 52% 50% 42% 26% 52%

500 60% 66% 60% 58% 40% 40% 48% 36%

750 64% 74% 70% 60% 54% 48% 40% 38%

1000 62% 58% 62% 62% 56% 48% 50% 54%

Table 2: Accuracy values obtained by the modified RNN-LSTM.

Epochs

number

Modified RNN -LSTM (22 electrodes) Modified RNN -LSTM (12 electrodes)

Relu Elu Selu Tanh Relu Elu Selu Tanh

250 46% 50% 52% 54% 50% 42% 46% 50%

500 40% 52% 60% 50% 54% 54% 44% 36%

750 46% 60% 60% 62% 44% 52% 46% 50%

1000 56% 70% 54% 64% 52% 54% 56% 54%

Table 3: Summary of the research on classification of motor

imagery tasks.

Reference Method Accuracy

(

Fadel et al., 2020

)

DCNN-LSTM 70.64%

(Zhuozheng et al.,

2019)

EEGnet 67.76%

Proposed model Modified CNN

model

74.00%

We consider in Table 3 a summary of the results

on the classification of motor imagery tasks of EEG

signals and a comparison with our proposed method.

The summary proves that the proposed modified

CNN model outperforms the other models in terms of

accuracy.

In reference (Fadel et al., 2020), the authors

proposed a classification method in which the EEG

signals are transformed into images using deep

learning. They used a Physionet dataset, which

includes 109 subjects, and MI EEG signals for three

frequency bands were transformed into three-channel

SDMIS 2021 - Special Session on Super Distributed and Multi-agent Intelligent Systems

594

images using Azimuthal equidistant projection and

the Clough- Tocher algorithm for interpolation. These

2D images represent the input data of the DCNN

which is used to extract frequency and spatial

characteristics. A LSTM is applied to extract

temporal features and classify the results into 5

different classes (4 MIs tasks and a pause). They

obtained an average accuracy of 70.64%.

In reference (Zhuozheng et al., 2019), the authors

adopted a shallow EEGnet network, and used one-

dimensional convolution for EEG classification in the

time domain. They extract only the one-dimensional

characteristics of the EEG signals. They obtained an

accuracy value of 67.76%.

Our proposed method provides the highest

accuracy value despite its simplicity and the minimal

pre-processing of the data. Moreover, we notice that

reducing the number of electrodes does not always

give better results.

4 CONCLUSIONS

We applied two methods based on deep learning for

the classification of MI tasks using EEG signals. We

demonstrated that the modified CNN is more efficient

than the modified RNN-LSTM model in terms of

classification. We compared our results with recent

results obtained using other classification methods

and showed that our proposed method gives a higher

accuracy than other methods. We believe that our

approach can be used to increase the efficiency of

BCI based on motor imagery.

ACKNOWLEDGMENT

This work was carried out under the MOBIDOC

scheme, funded by the European Union (EU) through

the EMORI program and managed by the National

Agency for the Promotion of Scientific Research

(ANPR).

REFERENCES

Fadel, W., Kollod, C., Wahdow, M., Ibrahim, Y., & Ulbert,

I., 2020. Multi-Class Classification of Motor Imagery

EEG Signals Using Image-Based Deep Recurrent

Convolutional Neural Network. 8th International

Winter Conference on Brain-Computer Interface (BCI).

Ma, X., Qiu, S., Du, C., et al., 2018. Improving EEG-Based

Motor Imagery Classification via Spatial and Temporal

Recurrent Neural Networks. 40th Annual International

Conference of the IEEE Engineering in Medicine and

Biology Society (EMBC). IEEE.

Schirrmeister, R. T., Springenberg, J. T., Fiederer, L. D. J.,

Glasstetter, M., Eggensperger, K., Tangermann, M.,

Hutter, F., Burgard, W., and Ball, T., 2017. Deep

learning with convolutional neural networks for EEG

decoding and visualization, Human Brain Mapping,

vol. 38, no. 11, pp. 5391–5420.

Shen, Y., Lu, H., and Jia, J., 2017. Classification of motor

imagery EEG signals with deep learning models, in

International Conference on Intelligent Science and

Big Data Engineering (Lanzhou: Springer), pp. 181–

190.

Shen, Y. Lu, H. and Jia, J., 2017. Classification of motor

imagery EEG signals with deep learning models, in

Intelligence Science and Big Data Engineering: 7

International Conference, IScIDE 2017. Springer

International Publishing, pp. 181–190.

Tabar, Y R., Halici U., 2017. A novel deep learning

approach for classification of EEG motor imagery

signals. Journal of Neural Engineering.

Zhang, G., Davoodnia, V., Sepas-Moghaddam, A., Zhang,

Y., & Etemad, A., 2019. Classification of Hand

Movements from EEG using a Deep Attention-based

LSTM Network. IEEE Sensors Journal.

Zhuozheng, W., Zhuo, M., Xiuwen, D., Yingjie, D., Wei,

L. 2019. Research on the Key Technologies of Motor

Imagery EEG Signal Based on Deep Learning, Journal

of Autonomous Intelligence.

Multi-class Motor Imagery EEG Classiﬁcation using Convolution Neural Network

595