EEG Emotion Classification Using an Enhanced Transformer-CNN-BiLSTM Architecture with Dual Attention Mechanisms

Abstract: Electroencephalography (EEG)-based emotion recognition plays a critical role in affective computing and emerging decision-support systems, yet remains challenging due to high-dimensional, noisy, and subject-dependent signals. This study investigates whether hybrid deep learning architectures that integrate convolutional, recurrent, and attention-based components can improve emotion classification performance and robustness in EEG data. We propose an enhanced hybrid model that combines convolutional feature extraction, bidirectional temporal modeling, and self-attention mechanisms with regularization strategies to mitigate overfitting. Experiments conducted on a publicly available EEG dataset spanning three emotional states (neutral, positive, and negative) demonstrate that the proposed approach achieves state-of-the-art classification performance, significantly outperforming classical machine learning and neural baselines. Statistical tests confirm the robustness of these performance gains under cross-validation. Feature-level analyses further reveal that covariance-based EEG features contribute most strongly to emotion discrimination, highlighting the importance of inter-channel relationships in affective modeling. These findings suggest that carefully designed hybrid architectures can effectively balance predictive accuracy, robustness, and interpretability in EEG-based emotion recognition, with implications for applied affective computing and human-centered intelligent systems.

• The paper presents a novel hybrid Transformer-CNN-BiLSTM model that integrates dual multi-head attention to capture both spatial and temporal EEG features.
• It employs advanced regularization and dual pooling strategies to achieve 99.19% test accuracy with a minimal 0.56% overfitting gap.
• Feature importance analysis reveals that covariance-based EEG attributes are key drivers of emotion prediction, emphasizing spatial-temporal dynamics.

Enhanced Transformer-CNN-BiLSTM with Dual Attention for EEG Emotion Classification

Introduction

The paper "EEG Emotion Classification Using an Enhanced Transformer-CNN-BiLSTM Architecture with Dual Attention Mechanisms" (2602.06411) addresses EEG-based emotion recognition, a central problem in affective computing and human-centric intelligent systems. The inherent complexity of EEG signals — characterized by high dimensionality, noise, and subject-dependence — has historically limited the accuracy and robustness of emotion classification. Classical approaches have relied on handcrafted features and traditional classifiers, but recent advances in deep learning architectures have significantly improved the modeling of spatial and temporal signal structures.

This work systematically investigates the impact of hybrid deep architectures integrating convolutional (CNN), recurrent (BiLSTM), and transformer-based attention modules, augmented with advanced regularization strategies. The primary goal is to achieve state-of-the-art classification performance, robust generalization, and interpretable modeling for clinical and applied settings.

Architecture Design and Methodology

The Enhanced Transformer-CNN-BiLSTM architecture is devised as a seven-stage pipeline encompassing spatial feature extraction, bidirectional temporal modeling, dual multi-head self-attention, dual pooling, deep classification, and robust regularization.

The architecture begins with 1D convolutional layers with residual connections, capturing localized spatial patterns across EEG channels. Subsequent Bidirectional LSTM layers model sequential dependencies, accounting for both past and future context. Dual multi-head self-attention mechanisms (16 and 8 heads) are applied after BiLSTM to attend to both local feature dynamics and global sequence relevance. The dual pooling strategy (global average and max pooling) enables retention of both holistic trends and salient activations prior to dense classification. Regularization components include dropout, layer normalization, label smoothing, and weight decay, applied to enhance generalization and prevent overfitting.

Figure 1: Enhanced Transformer-CNN-BiLSTM architecture detailing CNN, BiLSTM, dual multi-head attention, pooling, and dense classifier components.

A comparative baseline suite is implemented — Random Forest, SVM, MLP — to quantify improvements attributable to hybrid modeling and attention.

Figure 2: Overview of feature integration and prediction flow in the Enhanced Transformer-CNN-BiLSTM. CNNs extract spatial features, BiLSTM models temporal sequences, dual attention layers weight feature relevance, leading to dense emotion prediction.

EEG preprocessing involves z-score normalization of 988-dimensional feature vectors for 2,529 samples, stratified train/test splits, and Gaussian noise/data augmentation during training. Optimization employs AdamW with cosine annealing and early stopping.

Empirical Performance and Statistical Validation

The proposed model demonstrates substantial improvements over classical and deep learning baselines. Random Forest achieves 96.5% accuracy, SVM 96.8%, and MLP 97.2%. In contrast, the Enhanced Transformer-CNN-BiLSTM reaches 99.19% test accuracy with a mere 0.56% overfitting gap. Per-class precision, recall, and F1-score all exceed 98.89% for Neutral, Positive, and Negative states. These improvements are confirmed by the Friedman test ($\chi^2 = 12.45$, $p < 0.01$) and Wilcoxon signed-rank post-hoc analysis, which attests to statistical superiority over all baselines.

Figure 3: Random Forest confusion matrix, baseline performance reference (96.5% accuracy).

Figure 4: Model performance comparison, statistical significance, and per-class accuracy. Enhanced model displays dominance in all evaluation metrics.

Training and validation accuracy curves evidence stable convergence and minimal overfitting (0.56%), attributable to the regularization strategies.

Figure 5: Training and validation accuracy/loss curves evidencing robust generalization and minimal overfitting.

Test set confusion matrix reveals strong diagonal dominance, indicating negligible misclassification across emotion categories.

Figure 6: Enhanced model confusion matrix on test data (99.19% accuracy), demonstrating balanced performance and minimal off-diagonal errors.

Feature Importance and Interpretability Analysis

To address interpretability and feature contribution, five complementary feature importance metrics were integrated: Random Forest importance, Extra Trees, mutual information, ANOVA F-statistic, and Pearson correlation. Consensus ranking identifies covariance-based EEG features and eigenvalue-derived statistics as most discriminative, followed by frequency-domain features. Category-wise ablation reveals that removing covariance features causes a 15.3% drop in accuracy, indicating that functional connectivity drives emotion classification.

Figure 7: Top 15 EEG features by consensus importance across ensemble, statistical, and correlation-based methods.

SHAP analysis further corroborates the dominance of temporal connectivity and covariance features, providing both directionality and marginal impact on prediction.

Figure 8: SHAP feature contribution analysis, directional impact versus Random Forest importance.

Pearson correlation analysis highlights strong inter-feature and feature-label dependencies, informing dimensionality reduction and biomarker selection.

Figure 9: Feature correlation heatmap, revealing clusters and strong linear associations with emotion labels.

Robustness and Overfitting Mitigation

The enhanced architecture achieves a low overfitting gap (0.56%) via label smoothing, dropout, weight decay, layer normalization, and rigorous data augmentation. Training dynamics demonstrate consistent learning and generalization across epochs.

Figure 10: Overfitting gap analysis, showing stable training and validation accuracy convergence.

Implications and Future Directions

Practical Impact

The model's reliability with near-perfect accuracy positions it for applications in real-time affect monitoring, mental health diagnostics, and adaptive BCI systems. With a compact parameter count and robust regularization, the architecture is deployable on edge and wearable devices, enhancing accessibility outside clinical labs.

Theoretical Contributions

Results validate the hypothesis that dual attention combined with spatial-temporal modeling is uniquely suited for physiological signals, transcending NLP origins of transformers. The ablation and correlation analyses reinforce the theoretical claim that emotion is an emergent property of distributed neural activity — spatial relationships and temporal dependencies are more predictive than isolated spectral summaries.

Generalization and Limitations

Current findings are limited to a single dataset with discrete three-class emotion representation. Generalization to other data sources and finer-grained affective states requires further validation. Interpretability methods provide correlational, not causal, insights. Real-time deployment necessitates artifact handling and subject adaptation.

Speculative Research Directions

Integration with graph neural networks, multimodal fusion (e.g., facial, physiological signals), uncertainty quantification, model compression, and bias mitigation represent critical avenues for advancing EEG-based affective computing. Longitudinal and cross-population studies are warranted for clinical translation.

Conclusion

The Enhanced Transformer-CNN-BiLSTM architecture with dual attention not only surpasses existing baselines in EEG-based emotion classification but also offers clinical-grade robustness and feature-level interpretability. Covariance features emerge as key emotion biomarkers. The hybrid design facilitates the balance of accuracy, generalization, and interpretability, laying groundwork for scalable, practical, and transparent affective computing. Further cross-dataset validation, multimodal integration, and ethical governance will shape future developments in AI-driven emotion monitoring.