Advances in EEG‑Based Emotion Recognition: Methods and

Challenges

Jiayu Yang

School of Engineering, Rutgers University, New Jersey, U.S.A.

Keywords: Emotion Recognition, EEG Signal Processing, Deep Learning.

Abstract: EEG-Based emotion recognition is a key area in affective computing and brain-computer interfaces (BCI),

offering real-time insights into human emotional states. Unlike facial expressions or speech, EEG provides

direct neural activity data, making it a robust tool for emotion decoding. However, several challenges hinder

its effectiveness, including low signal-to-noise ratio (SNR), individual variability, and dataset inconsistencies.

These issues affect model generalizability and classification accuracy, limiting real-world applications. This

review is about the preprocessed EEG, machine as well as deep models, as well as cross-dataset generalization

challenges. Comparative evaluation with traditional models such as SVM as well as the PCA is given with

the implementation of the deep models such as the CNNs, LSTMs, as well as the implementation of the

Transformer. Cross-subject variance reduction as well as standardization of databases is necessary for the

advancement of emotional decoding with the use of the EEG. Future research should be targeted toward light

models of AI, as well as the implementation of multiple modes as well as the domain adaptation.

1 INTRODUCTION

Human emotion perception as well as interpretation

is a significant area in BCI, as well as affective

computing. Of the numerous forms of the kinds of

body’s physiological signals, electroencephalography

is a highly promising method toward the recognition

of emotions due to the close correspondence with the

processes at the level of the neurons. Compared with

expressions or voice, electroencephalography

measures inherent emotional states with less

likelihood of being masked, thus being a credible

measure for use in affective computing. EEG signals

have the preference due to their high temporal

resolution as well as the presence of portable as well

as low-cost devices. EEG has been forwarded as a

more effective method toward the recognition of

emotions compared with other forms of the body's

physiological signals due to the close correspondence

with the activities at the level of the neurons as well

as the resistance toward being masked through

voluntary expressions (Rahman et al., 2021). These

attributes coupled with the portability of

electroencephalography as well as the provision of

real-time details have made it a preferable candidate

for use in affective computing systems.

Despite its potential, EEG-based emotion

recognition faces several challenges. Firstly, the EEG

signal is characterized by a low signal-to-noise ratio

(SNR), making it susceptible to various artifacts from

muscle movements and environmental interference.

Improper preprocessing can significantly impact

classification accuracy in EEG-based emotion

recognition (Liu et al., 2011). Secondly, individual

differences remain a significant issue, as emotional

responses differ across individuals due to personal

experiences, cultural influences, and

neurophysiological variations. The need to identify

more fundamental and universal emotion patterns has

been emphasized, requiring the use of convolutional

layers or attention mechanisms, as well as a deeper

understanding of human emotions (Abibullaev et al.,

2023). Additionally, the limited availability of

standardized EEG emotion datasets hinders the

development and validation of generalized models.

Existing datasets have inconsistencies in stimulus

types and labeling methodologies (Wang & Wang,

2021). Finally, existing machine learning and deep

learning models struggle with generalization across

different datasets, limiting their real-world

applicability. Challenges in cross-dataset adaptation

and transfer learning in EEG-based emotion

recognition have been extensively discussed (Jafari et

Yang, J.

Advances in EEG-Based Emotion Recognition: Methods and Challenges.

DOI: 10.5220/0014386500004933

Paper published under CC license (CC BY-NC-ND 4.0)

In Proceedings of the 1st International Conference on Biomedical Engineering and Food Science (BEFS 2025), pages 46-52

ISBN: 978-989-758-789-4

al., 2023).

This review is purposed as a broad overview

of the varied signal processing techniques utilized in

emotional recognition using EEG, the signals

processed in the frequency as well as the time

domains. A brief introduction is given to the emotions

theories before proceeding. It also compares the

performance of machine learning (ML) as well as

deep learning (DL) techniques with their limits.

Further, the current EEG emotional databases as well

as the cross-dataset learning challenges are explored.

Finally, the current applications of emotional

recognition using EEG in the fields of monitoring of

mental health, adaptive learning systems as well as

experience-based immersive situations using virtual

experience is explored.

2 EMOTION THEORIES

Emotion is the essential ingredient of cognition as

well as behavior in humans, influencing decision-

making, perception, as well as social interaction.

Emotion is characterized in the multiple theories of

emotion in psychology as well as neuroscience.

One of the more popular models is the Discrete

Emotion Model, which acknowledges the presence of

the core emotions of happiness, sadness, anger, fear,

surprise, and disgust. These have been characterized

as being present in all societies and being

accompanied by certain facial expressions as well as

certain bodily reactions. This model, originally

advanced in 1992 by Ekman, is also being explored

with regard to cognitive as well as bodily significance

(Lench et al., 2011). Nevertheless, despite the

categorical framework of this model being quite

unique, the model cannot adequately capture the

diversity as well as the complexity of emotional

experience.

An alternative is the Dimensional Emotion Model,

the Valence-Arousal model specifically, originally

proposed by Russell in 1980 and later extended

(Harmon-Jones et al., 2017). It locates emotions in a

dimensional plane: valence, from positive through to

negative emotions, and arousal, the intensity of the

emotional state. For example, joy is at the point of

high valence and high arousal, but sadness is at low

valence and low arousal. It is widely applied in the

area of EEG-based emotional recognition because it

is a flexible as well as scalable method of representing

emotional states.

In addition to these models, the Component

Process Theory emphasizes that emotions are not

discrete categories but rather dynamic processes that

are determined by cognitive appraisals, physiological

reactions, and situational contexts (Scherer, 2001).

This theory is well compatible with EEG-based

emotion decoding since EEG records the real-time

dynamics of brain activity related to emotional

reactions.

In EEG-based emotional recognition, the V-A

model is utilized in the tagging of emotional states in

multiple databases like SEED, DREAMER, and

DEAP, thereby cementing its use in computational

models. EEG's ability to capture discrete patterns of

the brain for different valences as well as arousal

levels makes the method highly suited for the

examination of affective states. Understanding these

models is required for the creation of EEG-based

emotional recognition models because these models

control the process of feature extraction,

classification method, as well as model evaluation

procedures.

3 EEG SIGNAL

CHARACTERISTICS &

PREPROCESSING

TECHNIQUES

3.1 EEG Signals and Emotional

Correlation

Electroencephalography (EEG) is widely utilized for

the identification of emotions due to the precision

with which the activity of the brain can be measured.

Brain oscillations as represented through the signals

of EEG correspond with distinct frequency bands,

each with specific cognitive as well as emotional

processes. Frequency bands also serve as vital signals

for the decoding of emotional states (Wang & Wang,

2021).

Delta (0.5–4 Hz) is largely associated with

unconsciousness and deep sleep but is also linked

with emotional regulation and stress. Similarly, Theta

(4–8 Hz) is associated with emotional arousal and

processing of the memory, with increased theta

activity observed with the processing of emotional

stimuli. Alpha (8–13 Hz), on the other hand, is linked

with relaxation as well as inhibitory control, with

alpha asymmetry in the front highly correlated with

the state of emotions. Beta (13–30 Hz) is involved in

cognitive processing as well as elevated emotional

states with increased activity observed with tasks of

emotional intensity. Subsequently, Gamma (>30 Hz)

Advances in EEG-Based Emotion Recognition: Methods and Challenges

is involved with higher-level cognition, emotional

perception, as well as integrative processing.

Emotional processing is not uniform in the

distribution across the brain but is localized in

specific regions. For instance, the prefrontal cortex of

the frontal lobe is responsible for the regulation of

emotional responses as well as the process of

decision-making (Val-Calvo et al., 2020).

Meanwhile, the temporal lobe, the amygdala, as well

as the hippocampus, is involved with emotional

recognition as well as the process of encoding the

memory. Subsequently, the parietal lobe processes

sensory as well as emotional input, with the process

being integral in emotional perception as well as

regulation.

3.2 Signal Processing Techniques

EEG signals are very prone to noise from different

sources such as muscle activity, eye movement, and

environmental interference. Preprocessing is

necessary to enhance signal quality and increase

classification accuracy. Typical preprocessing

methods are filtering, artifact removal, and signal

transformation.

3.2.1 Filtering Techniques

Independent Component Analysis (ICA) separates

EEG sources into statistically independent

components, thereby aiding in the removal of artifacts

(Dadebayev et al., 2022). Similarly, Principal

Component Analysis (PCA) reduces dimensionality

and retains the most informative EEG components,

which is particularly useful for classification tasks

(Wang & Wang, 2021). In addition, Wavelet

Transform (WT) decomposes EEG signals into

different frequency bands, effectively enhancing

signal denoising and further improving signal

processing (Dadebayev et al., 2022).

3.2.2 Time-Domain vs. Frequency-Domain

Approaches

Time-domain features, such as entropy, variance, and

statistical properties, are commonly used to describe

EEG amplitude fluctuations related to emotions

(Dadebayev et al., 2022). Furthermore, frequency-

domain features, including Fast Fourier Transform

(FFT) and Discrete Wavelet Transform (DWT),

decompose EEG signals into different frequency

bands, providing deeper insights into affective states

(Luo et al., 2020). However, a primary challenge in

EEG emotion recognition is inter-subject variability,

where differences between individuals lead to

inconsistent EEG patterns. Cross-subject adaptation

methods, such as domain adaptation and transfer

learning, aim to mitigate these inconsistencies

(Dadebayev et al., 2022).

3.2.3 Experiment: Comparison of EEG

Preprocessing Methods

The objective of this study is to evaluate the

effectiveness of Bandpass Filtering, Independent

Component Analysis (ICA), Principal Component

Analysis (PCA), and Wavelet Transform in

improving EEG signal quality. To achieve this, the

DEAP, SEED, and DREAMER datasets will be

utilized. The methodology involves comparing

signal-to-noise ratio (SNR) variations before and

after preprocessing, using EEG visualization

techniques to assess the effects of filtering, and

training SVM and CNN classifiers to evaluate

emotion classification accuracy. The results will

include a comparison of SNR values (numerical

data), a table summarizing classification accuracy,

and figures illustrating EEG signals before and after

preprocessing. The expected conclusion is that ICA

will be highly effective for artifact removal, while

Wavelet Transform will provide superior denoising

capabilities.

Traditional feature extraction methods, such as

FFT and PCA, remain widely used. However, deep

learning-based methods, particularly CNNs and

Transformers, are demonstrating promising results in

automatically extracting relevant EEG features for

emotion recognition.

4 MACHINE LEARNING VS.

DEEP LEARNING FOR EEG

EMOTION DECODING

4.1 Traditional Machine Learning

Approaches

Traditional machine learning (ML) approaches have

been widely used for the recognition of emotions

from EEG since they possess the advantage of being

low computational cost as well as being interpretable.

Most traditional ML approaches leverage hand-

engineered characteristics from the EEG signals such

as PSD, Hjorth parameters, as well as other statistical

metrics.

BEFS 2025 - International Conference on Biomedical Engineering and Food Science

Support Vector Machines (SVMs), k-Nearest

Neighbors (KNN), Linear Discriminant Analysis

(LDA), and Random Forest (RF) classifiers have

been extensively applied in the field of EEG emotion

decoding. SVMs have been seen as a strong classifier

for the recognition of emotions from EEG, as also

other models such as Decision Trees and Random

Forests (Dadebayev et al., 2022). Decision-level

fusion-based random forest classifiers, when applied,

also support the enhancement of the recognition of

emotions from EEG under noisy conditions (Wang et

al., 2022). However, the greatest limitation of these

models is their inability to capture the temporal

dependencies in the EEG signals effectively.

Though limited, traditional ML models are still a

suitable choice for applications where interpretability

is essential. Feature selection is critical to enhance

EEG-based emotion classification accuracy,

especially to counteract limited feature availability

and excessive signal noise (Luo et al., 2020). As EEG

datasets increase in complexity and size, however,

traditional ML methods are confronted with

mounting difficulties in generalization and

scalability.

4.2 Deep Learning Models

Deep learning (DL) revolutionized the use of EEG-

based emotional recognition through the automatic

acquisition of hierarchical representations from the

EEG signals. Compared with traditional ML models

that rely upon hand-coded features, DL models learn

the spatial and temporal dependencies from the raw

EEG signals.

4.2.1 CNN for Spatial Feature Extraction

Convolutional Neural Networks have been

extensively utilized in the modeling of EEG signals

as patterned spatial data, considering the placements

of the electrodes as image-like topographic

representations. CNNs have been utilized for the

extraction of spatial features with notable accuracy

improvements in the classification of emotions

compared with the use of traditional ML methods

(Jafari et al., 2023). The use of CNNs in the analysis

of EEG signals has been widely explored, specifically

their processing of the spatial features (Dadebayev et

al., 2022).

4.2.2 RNN/LSTM for Sequential EEG

Modeling

Though CNNs can effectively capture spatial

characteristics, RNNs and LSTM networks are more

suited for representing temporal dependencies in the

case of EEG. LSTM networks have been effectively

utilized in the pipelines of EEG emotion recognition,

demonstrating their capabilities for representing

long-range dependencies in the sequences of EEG as

well as improving the accuracy of classification

(Hassouneh et al., 2020).

4.2.3 Transformer-Based EEG Emotion

Decoding

Recent advances in the area of deep learning have

seen the use of Transformer-based models applied in

the recognition of emotions from EEG. In contrast

with RNNs, the use of self-attention mechanisms

ensures long-term dependencies without vanishing

gradients. Transformer models have been found to

have enhanced classification accuracy (Abibullaev et

al., 2023). Some of the benefits of self-attention have

been found in the analysis of EEG, specifically the

capture of long-term dependencies more effectively

compared with RNNs and LSTMs (Dadebayev et al.,

2022)).

4.2.4 Experiment: Traditional vs. Deep

Learning Feature Extraction

The objective of this study is to compare the

performance of hand-crafted features, such as Power

Spectral Density (PSD) and Hjorth parameters,

against deep learning-based features derived from

models like CNN and Transformer in emotion

classification. To achieve this, the DEAP and SEED

datasets will be utilized. The traditional approach

involves extracting features like PSD and Hjorth

parameters, while the deep learning approach focuses

on training CNN and LSTM models for automated

feature extraction. For classification, both SVM and

CNN will be trained and their performance evaluated.

The results will include a comparison of classification

accuracy between traditional and deep learning

methods (presented in a table) and a visualization of

CNN-extracted features using t-SNE. The expected

conclusion is that CNN-extracted features will

outperform hand-crafted features, with Transformers

potentially further enhancing classification

performance.

Advances in EEG-Based Emotion Recognition: Methods and Challenges

4.3 Hybrid & Advanced Deep Learning

Models

To leverage the strengths of different architectures,

hybrid models have been proposed to enhance EEG-

based emotion recognition. One such approach

involves CNN-LSTM fusion models, which combine

the spatial feature extraction capabilities of CNNs

with the sequential modeling capabilities of LSTMs.

This hybrid architecture has been shown to

significantly improve EEG emotion classification

accuracy (Wang et al., 2022). Another emerging

approach is the use of Spiking Neural Networks

(SNNs), which mimic biological neural mechanisms

for energy-efficient processing. SNNs have

demonstrated superior performance compared to

traditional FFT-DWT processing in EEG-based

emotion recognition by preserving more

neurophysiological information (Luo et al., 2020).

5 CHALLENGES AND FUTURE

RESEARCH DIRECTIONS

5.1 Open Challenges

Despite significant research in the area of EEG-based

emotional recognition, there have been numerous

essential challenges. A significant problem is the low

signal-to-noise ratio of the EEG signals, which makes

the extraction of meaningful emotional

characteristics problematic. Noise from eye

movements, eye blinks, as well as other electronic

interferences, also compromise the quality of the

signals, with the resultant classification accuracy

being low (Zeng et al., 2024).

Another persistent challenge is inter-subject

variability, where differences in brain activity across

individuals result in inconsistent model performance.

Inter-subject variability poses a significant challenge

in EEG-based emotion recognition, as individual

differences in EEG signals affect model

generalizability, making subject-independent models

perform worse than subject-dependent models

(Dadebayev et al., 2022).

Existing EEG emotional databases, SEED and

DEAP, have discrepancies in the method of recording

signals, recording conditions, as well as the

population under investigation, introducing variance

in research results as well as cross-dataset

generalization difficulties (Yang et al., 2024). A

unified experiment set with standardized protocols is

necessary for improved model comparability.

Constraints of processing in real-time also hinder the

use of emotion decoding models in actual systems.

Most of the present recognition models have been

created for offline processing due to the

computational overhead of feature extraction and

classification, which makes their implementation in

real-time impossible (Liu et al., 2011).

5.2 Promising Future Directions

To address these challenges, several promising

research directions have emerged. Hybrid deep

learning models, such as CNN-LSTM and Spiking

Neural Networks (SNNs), offer potential solutions by

combining spatial and temporal feature extraction,

improving generalization and efficiency (Luo et al.,

2020).

Another promising method is multimodal fusion,

wherein the EEG signals can be coupled with other

body signals such as the galvanic skin response

(GSR) and facial expressions. Research has proved

the use of multiple modalities can be more robust with

enhanced classification accuracy in the recognition of

emotions (Yang et al., 2024).

The development of light AI models for

implementation in real-time BCI is also receiving

attention. Low power-efficient models are being

proposed and being low power-optimized for low

power devices, enabling the implementation of real-

time EEG emotional recognition in wearable devices.

To mitigate dataset bias and improve model

generalization, researchers are exploring techniques

for better cross-dataset learning. Strategies such as

optimizing multimodal datasets and improving

emotion classification models contribute to more

robust and generalizable approaches in EEG-based

emotion recognition (Yang et al., 2024). Future EEG

emotion decoding research must focus on mitigating

cross-subject variability and dataset biases.

Multimodal fusion and domain adaptation hold

promise for improving accuracy, while lightweight

AI models will be essential for real-time BCI

applications.

6 CONCLUSION

EEG-based emotion recognition has become an

essential part of affective computing and brain-

computer interfaces. In the last decade, there has been

considerable advancement in feature extraction,

machine learning models, and deep learning methods

BEFS 2025 - International Conference on Biomedical Engineering and Food Science

to allow more precise emotion classification from

EEG signals.

Traditional machine learning models like SVM

and Random Forest have been the basis for

classification using EEG but have been limited in

their capabilities in extracting intricate temporal

patterns, which have given birth to the use of deep

models. LSTMs, Transformer models, as well as

CNNs, have been found to perform much better in

extracting significant features as well as classification

accuracy. But these models use large databases as

well as require large computational power, which

makes them non-practicable in the context of real-

time.

Despite advancements, there continue to be

significant challenges with the use of EEG-based

emotional decoding due to the low signal-to-noise

ratio, the large inter-subject variance, as well as the

non-availability of a standard EEG dataset. Cross-

dataset adaptation techniques, multimodal fusion, as

well as the implementation of hybrid deep networks,

can be utilized in countering these challenges.

Future research would be directed toward real-time

BCI implementation, wherein the light models of AI,

being highly efficient, can be utilized with portable

and wearable EEG devices. Also, incorporating

multimodal paradigms through the combination of

EEG with facial expressions, voice, and body signals

like GSR can be used to boost the accuracy of

recognizing emotions. Domain adaptation and

transfer learning will play a crucial role in creating

models that generalize well across different EEG

datasets and recording conditions.

In summary, though EEG-based emotion

recognition is progressing, making it practical to

deploy in real-world scenarios is still a persisting

challenge. Standardized datasets, refined deep

learning models, and real-time inference

optimizations are the key to moving the field forward

and enabling EEG-based emotion decoding as a

practical approach for affective computing and BCI

applications.

REFERENCES

Abibullaev, B., Keutayeva, A., & Zollanvari, A. 2023.

Deep learning in EEG-based BCIs: A comprehensive

review of Transformer models, advantages, challenges,

and applications. IEEE Access 11:127271–127301.

Dadebayev, D., Goh, W. W., & Tan, E. X. 2022. EEG-

based emotion recognition: Review of commercial

EEG devices and machine learning techniques. Journal

of King Saud University - Computer and Information

Sciences 34(7):4385–4401.

Harmon-Jones, E., Harmon-Jones, C., & Summerell, E.

2017. On the importance of both dimensional and

discrete models of emotion. Behavioral Sciences

7(4):Article 66.

Hassouneh, A., Mutawa, A. M., & Murugappan, M. 2020.

Development of a real-time emotion recognition system

using facial expressions and EEG based on machine

learning and deep neural network methods. Informatics

in Medicine Unlocked 20:100372.

Jafari, M., Shoeibi, A., Khodatars, M., Bagherzadeh, S.,

Shalbaf, A., López García, D., Gorriz, J. M., &

Acharya, U. R. 2023. Emotion recognition in EEG

signals using deep learning methods: A review.

Computers in Biology and Medicine 165:107450.

Lench, H. C., Flores, S. A., & Bench, S. W. 2011. Discrete

emotions predict changes in cognition, judgment,

experience, behavior, and physiology: A meta-analysis

of experimental emotion elicitations. Psychological

Bulletin 137(5):834–855.

Liu, Y., Sourina, O., & Nguyen, M. K. 2011. Real-time

EEG-based emotion recognition and its applications. In

M. L. Gavrilova, C. J. K. Tan, A. Sourin, & O. Sourina

(Eds.), Transactions on Computational Science XII:

Special Issue on Cyberworlds. Lecture Notes in

Computer Science, vol 6670. Springer, Berlin,

Heidelberg, 256–277.

Luo, Y., Fu, Q., Xie, J., Qin, Y., Wu, G., Liu, J., Jiang, F.,

Cao, Y., & Ding, X. 2020. EEG-based emotion

classification using spiking neural networks. IEEE

Access 8:46007–46016.

Rahman, Md. M., Sarkar, A. K., Hossain, Md. A., Hossain,

Md. S., Islam, Md. R., Hossain, Md. B., Quinn, J. M.

W., & Moni, M. A. 2021. Recognition of human

emotions using EEG signals: A review. Computers in

Biology and Medicine 136:104696.

Scherer, K. R. 2001. Emotions, psychological structure of.

In N. J. Smelser & P. B. Baltes (Eds.), International

Encyclopedia of the Social & Behavioral Sciences.

Pergamon, 4472–4477.

Val-Calvo, M., Álvarez-Sánchez, J. R., Ferrández-Vicente,

J. M., Díaz-Morcillo, A., & Fernández-Jover, E. 2020.

Real-time multi-modal estimation of dynamically

evoked emotions using EEG, heart rate and galvanic

skin response. International Journal of Neural Systems

30(4):2050013.

Wang, J., & Wang, M. 2021. Review of the emotional

feature extraction and classification using EEG signals.

Cognitive Robotics 1:29–40.

Wang, Q., Wang, M., Yang, Y., & Zhang, X. 2022. Multi-

modal emotion recognition using EEG and speech

signals. Computers in Biology and Medicine

149:105907.

Yang, P., Liu, N., Liu, X., Shu, Y., Ji, W., Ren, Z., Sheng,

J., Yu, M., Yi, R., Zhang, D., & Liu, Y.-J. 2024. A

multimodal dataset for mixed emotion recognition.

Scientific Data 11(1):847.

Advances in EEG-Based Emotion Recognition: Methods and Challenges

Zeng, Y., Zhang, J.-W., & Yang, J. 2024. Multimodal

emotion recognition in the metaverse era: New needs

and transformation in mental health work. World

Journal of Clinical Cases 12(34):6674–6678.

BEFS 2025 - International Conference on Biomedical Engineering and Food Science