Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
126 tokens/sec
GPT-4o
47 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Joint Contrastive Learning with Feature Alignment for Cross-Corpus EEG-based Emotion Recognition (2404.09559v1)

Published 15 Apr 2024 in cs.HC and cs.AI

Abstract: The integration of human emotions into multimedia applications shows great potential for enriching user experiences and enhancing engagement across various digital platforms. Unlike traditional methods such as questionnaires, facial expressions, and voice analysis, brain signals offer a more direct and objective understanding of emotional states. However, in the field of electroencephalography (EEG)-based emotion recognition, previous studies have primarily concentrated on training and testing EEG models within a single dataset, overlooking the variability across different datasets. This oversight leads to significant performance degradation when applying EEG models to cross-corpus scenarios. In this study, we propose a novel Joint Contrastive learning framework with Feature Alignment (JCFA) to address cross-corpus EEG-based emotion recognition. The JCFA model operates in two main stages. In the pre-training stage, a joint domain contrastive learning strategy is introduced to characterize generalizable time-frequency representations of EEG signals, without the use of labeled data. It extracts robust time-based and frequency-based embeddings for each EEG sample, and then aligns them within a shared latent time-frequency space. In the fine-tuning stage, JCFA is refined in conjunction with downstream tasks, where the structural connections among brain electrodes are considered. The model capability could be further enhanced for the application in emotion detection and interpretation. Extensive experimental results on two well-recognized emotional datasets show that the proposed JCFA model achieves state-of-the-art (SOTA) performance, outperforming the second-best method by an average accuracy increase of 4.09% in cross-corpus EEG-based emotion recognition tasks.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (46)
  1. Soraia M Alarcao and Manuel J Fonseca. 2017. Emotions recognition using EEG signals: A survey. IEEE transactions on affective computing 10, 3 (2017), 374–393.
  2. Mashail Alsolamy and Anas Fattouh. 2016. Emotion estimation from EEG signals during listening to Quran using PSD features. In 2016 7th International Conference on computer science and information technology (CSIT). IEEE, 1–5.
  3. Cognitive behavioral therapy for anxiety and related disorders: A meta-analysis of randomized placebo-controlled trials. Depression and anxiety 35, 6 (2018), 502–514.
  4. A simple framework for contrastive learning of visual representations. In International conference on machine learning. PMLR, 1597–1607.
  5. Emotion recognition in human-computer interaction. IEEE Signal processing magazine 18, 1 (2001), 32–80.
  6. Convolutional neural networks on graphs with fast localized spectral filtering. Advances in neural information processing systems 29 (2016).
  7. Differential entropy feature for EEG-based emotion classification. In 2013 6th international IEEE/EMBS conference on neural engineering (NER). IEEE, 81–84.
  8. Patrick Flandrin. 1998. Time-frequency/time-scale analysis. Academic press.
  9. Nickolaos Fragopanagos and John G Taylor. 2005. Emotion recognition in human–computer interaction. Neural Networks 18, 4 (2005), 389–405.
  10. Domain-adversarial training of neural networks. Journal of machine learning research 17, 59 (2016), 1–35.
  11. Lester I Goldfischer. 1965. Autocorrelation function and power spectral density of laser-produced speckle patterns. Josa 55, 3 (1965), 247–253.
  12. ASTDF-Net: Attention-Based Spatial-Temporal Dual-Stream Fusion Network for EEG-Based Emotion Recognition. In Proceedings of the 31st ACM International Conference on Multimedia. 883–892.
  13. Deep mul timodal learning for emotion recognition in spoken language. In 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 5079–5083.
  14. PGCN: Pyramidal graph convolutional network for EEG emotion recognition. arXiv preprint arXiv:2302.02520 (2023).
  15. Clocs: Contrastive learning of cardiac signals across space, time, and patients. In International Conference on Machine Learning. PMLR, 5606–5615.
  16. Domain adaptation techniques for EEG-based emotion recognition: a comparative study on two public datasets. IEEE Transactions on Cognitive and Developmental Systems 11, 1 (2018), 85–94.
  17. Domain adaptation for EEG emotion recognition based on latent representation similarity. IEEE Transactions on Cognitive and Developmental Systems 12, 2 (2019), 344–353.
  18. A multi-domain adaptive graph convolutional network for EEG-based emotion recognition. In Proceedings of the 29th ACM International Conference on Multimedia. 5565–5573.
  19. EEG based emotion recognition: A tutorial and review. Comput. Surveys 55, 4 (2022), 1–57.
  20. A novel transferability attention neural network model for EEG emotion recognition. Neurocomputing 447 (2021), 92–101.
  21. A Novel Neural Network Model based on Cerebral Hemispheric Asymmetry for EEG Emotion Recognition. In IJCAI. 1561–1567.
  22. Contrastive self-supervised representation learning for sensing signals from the time-frequency perspective. In 2021 International Conference on Computer Communications and Networks (ICCCN). IEEE, 1–10.
  23. Real-time movie-induced discrete emotion recognition from EEG signals. IEEE Transactions on Affective Computing 9, 4 (2017), 550–562.
  24. Contrastive representation learning for electroencephalogram classification. In Machine Learning for Health. PMLR, 238–253.
  25. Survey on emotional body gesture recognition. IEEE transactions on affective computing 12, 2 (2018), 505–523.
  26. Henri J Nussbaumer and Henri J Nussbaumer. 1982. The fast Fourier transform. Springer.
  27. Antonia Papandreou-Suppappola. 2018. Applications in time-frequency signal processing. CRC press.
  28. Soheil Rayatdoost and Mohammad Soleymani. 2018. Cross-corpus EEG-based emotion recognition. In 2018 IEEE 28th international workshop on machine learning for signal processing (MLSP). IEEE, 1–6.
  29. Contrastive learning of subject-invariant EEG representations for cross-subject emotion recognition. IEEE Transactions on Affective Computing (2022).
  30. Variational instance-adaptive graph for EEG emotion recognition. IEEE Transactions on Affective Computing 14, 1 (2021), 343–356.
  31. Graph-embedded convolutional neural network for image-based EEG emotion recognition. IEEE Transactions on Emerging Topics in Computing 10, 3 (2021), 1399–1413.
  32. MPED: A multi-modal physiological emotion database for discrete emotion recognition. IEEE Access 7 (2019), 12177–12191.
  33. EEG emotion recognition using dynamical graph convolutional neural networks. IEEE Transactions on Affective Computing 11, 3 (2018), 532–541.
  34. Johan AK Suykens and Joos Vandewalle. 1999. Least squares support vector machine classifiers. Neural processing letters 9 (1999), 293–300.
  35. Exploring Contrastive Learning in Human Activity Recognition for Healthcare. arXiv preprint arXiv:2011.11542 (2020).
  36. Laurens Van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-SNE. Journal of machine learning research 9, 11 (2008).
  37. Mixing up contrastive learning: Self-supervised representation learning for time series. Pattern Recognition Letters 155 (2022), 54–61.
  38. Ts2vec: Towards universal representation of time series. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 36. 8980–8987.
  39. Facial expression recognition via learning deep sparse autoencoders. Neurocomputing 273 (2018), 643–649.
  40. mixup: Beyond empirical risk minimization. arXiv preprint arXiv:1710.09412 (2017).
  41. Emotionmeter: A multimodal framework for recognizing human emotions. IEEE transactions on cybernetics 49, 3 (2018), 1110–1122.
  42. Wei-Long Zheng and Bao-Liang Lu. 2015. Investigating critical frequency bands and channels for EEG-based emotion recognition with deep neural networks. IEEE Transactions on autonomous mental development 7, 3 (2015), 162–175.
  43. EEG-based emotion recognition using regularized graph neural networks. IEEE Transactions on Affective Computing 13, 3 (2020), 1290–1301.
  44. PR-PL: A novel prototypical representation based pairwise learning framework for emotion recognition using EEG signals. IEEE Transactions on Affective Computing (2023).
  45. EEG-based Emotion Style Transfer Network for Cross-dataset Emotion Recognition. arXiv preprint arXiv:2308.05767 (2023).
  46. Emotion self-regulation training in major depressive disorder using simultaneous real-time fMRI and EEG neurofeedback. NeuroImage: Clinical 27 (2020), 102331.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets