Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
158 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Toward cross-subject and cross-session generalization in EEG-based emotion recognition: Systematic review, taxonomy, and methods (2212.08744v3)

Published 16 Dec 2022 in cs.LG

Abstract: A systematic review on machine-learning strategies for improving generalizability (cross-subjects and cross-sessions) electroencephalography (EEG) based in emotion classification was realized. In this context, the non-stationarity of EEG signals is a critical issue and can lead to the Dataset Shift problem. Several architectures and methods have been proposed to address this issue, mainly based on transfer learning methods. 418 papers were retrieved from the Scopus, IEEE Xplore and PubMed databases through a search query focusing on modern machine learning techniques for generalization in EEG-based emotion assessment. Among these papers, 75 were found eligible based on their relevance to the problem. Studies lacking a specific cross-subject and cross-session validation strategy and making use of other biosignals as support were excluded. On the basis of the selected papers' analysis, a taxonomy of the studies employing Machine Learning (ML) methods was proposed, together with a brief discussion on the different ML approaches involved. The studies with the best results in terms of average classification accuracy were identified, supporting that transfer learning methods seem to perform better than other approaches. A discussion is proposed on the impact of (i) the emotion theoretical models and (ii) psychological screening of the experimental sample on the classifier performances.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (143)
  1. American clinical neurophysiology society guideline 2: guidelines for standard electrode position nomenclature. The Neurodiagnostic Journal, 56(4):245–252.
  2. Domain-adversarial neural networks. arXiv preprint arXiv:1412.4446.
  3. Emotions recognition using eeg signals: A survey. IEEE Transactions on Affective Computing, 10(3):374–393.
  4. A survey on eeg-based solutions for emotion recognition with a low number of channels. IEEE Access, pages 1–1.
  5. Combining inter-subject modeling with a subject-based data transformation to improve affect recognition from eeg signals. Sensors, 19(13):2999.
  6. Two-level domain adaptation neural network for eeg-based emotion recognition. Frontiers in Human Neuroscience, 14.
  7. Barbaresco, F. (2008). Innovative tools for radar signal processing based on cartan’s geometry of spd matrices & information geometry. In 2008 IEEE Radar Conference, pages 1–6. IEEE.
  8. Learning representations from eeg with deep recurrent-convolutional neural networks. arXiv preprint arXiv:1511.06448.
  9. Using neurophysiological signals that reflect cognitive or affective state: six recommendations to avoid common pitfalls. Frontiers in neuroscience, 9:136.
  10. Multiple transferable recursive feature elimination technique for emotion recognition based on eeg signals. Symmetry, 11(5):683.
  11. Affective computing and sentiment analysis. In A practical guide to sentiment analysis, pages 1–10. Springer.
  12. Application of a novel measure of eeg non-stationarity as ‘shannon-entropy of the peak frequency shifting’for detecting residual abnormalities in concussed individuals. Clinical Neurophysiology, 122(7):1314–1321.
  13. Multi-source and multi-representation adaptation for cross-domain electroencephalography emotion recognition. Frontiers in Psychology, 12:809459–809459.
  14. Casson, A. J. (2019). Wearable eeg and beyond. Biomedical engineering letters, 9(1):53–71.
  15. A fast, efficient domain adaptation technique for cross-domain electroencephalography (eeg)-based emotion recognition. Sensors, 17(5):1014.
  16. Unsupervised domain adaptation techniques based on auto-encoder for non-stationary eeg-based emotion recognition. Computers in biology and medicine, 79:205–214.
  17. Multi-subject subspace alignment for non-stationary eeg-based emotion recognition. Technology and Health Care, 26(S1):327–335.
  18. Ms-mda: Multisource marginal distribution adaptation for cross-subject and cross-session eeg emotion recognition. Frontiers in Neuroscience, 15.
  19. Meernet: Multi-source eeg-based emotion recognition network for generalization across subjects and sessions. In 2021 43rd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), pages 6094–6097. IEEE.
  20. Personal-zscore: Eliminating individual difference for eeg-based cross-subject emotion recognition. IEEE Transactions on Affective Computing.
  21. Learning with progressive transductive support vector machine. Pattern Recognition Letters, 24(12):1845–1855.
  22. Investigating the use of pretrained convolutional neural network on cross-subject and cross-dataset eeg emotion recognition. Sensors, 20(7):2034.
  23. The state and trait nature of frontal eeg asymmetry in emotion.
  24. An automatic pre-processing pipeline for eeg analysis (app) based on robust statistics. Clinical Neurophysiology, 129(7):1427–1437.
  25. Boosting for transfer learning. In Proceedings of the 24th International Conference on Machine Learning, ICML ’07, page 193–200, New York, NY, USA. Association for Computing Machinery.
  26. What does clean eeg look like? In 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society, pages 3963–3966. IEEE.
  27. The geneva affective picture database (gaped): a new 730-picture database focusing on valence and normative significance. Behavior research methods, 43(2):468.
  28. Davidson, R. J. (1984). Hemispheric asymmetry and emotion. Approaches to emotion, 2:39–57.
  29. Brain lateralization of emotional processing: historical roots and a future incorporating “dominance”. Behavioral and cognitive neuroscience reviews, 4(1):3–20.
  30. Eeg emotion enhancement using task-specific domain adversarial neural network. In 2021 International Joint Conference on Neural Networks (IJCNN), pages 1–8. IEEE.
  31. An efficient lstm network for emotion recognition from multichannel eeg signals. IEEE Transactions on Affective Computing.
  32. Ekman, P. (1999). Basic emotions. Handbook of cognition and emotion, 98(45-60):16.
  33. A wavelet-based approach to emotion classification using eda signals. Expert Systems with Applications, 112:77–86.
  34. Cross-subject eeg-based emotion recognition through neural networks with stratified normalization. Frontiers in neuroscience, 15:11.
  35. Unsupervised visual domain adaptation using subspace alignment. In Proceedings of the IEEE international conference on computer vision, pages 2960–2967.
  36. Fisher, R. A. (1992). Statistical methods for research workers. In Breakthroughs in statistics, pages 66–70. Springer.
  37. Emotion recognition with domain adaptation based on few-shot eeg learning. In 2021 IEEE 10th Global Conference on Consumer Electronics (GCCE), pages 1–2. IEEE.
  38. Unsupervised domain adaptation by backpropagation. In International conference on machine learning, pages 1180–1189. PMLR.
  39. Domain-adversarial training of neural networks. The journal of machine learning research, 17(1):2096–2030.
  40. Time domain feature extraction and classification of eeg data for brain computer interface. In 2012 9th International Conference on Fuzzy Systems and Knowledge Discovery, pages 1136–1139. IEEE.
  41. A robust data-driven approach identifies four personality types across four large data sets. Nature human behaviour, 2(10):735–742.
  42. Scatter component analysis: A unified framework for domain adaptation and domain generalization. IEEE transactions on pattern analysis and machine intelligence, 39(7):1414–1430.
  43. Verification, analytical validation, and clinical validation (v3): the foundation of determining fit-for-purpose for biometric monitoring technologies (biomets). npj digital Medicine, 3(1):1–15.
  44. Geodesic flow kernel for unsupervised domain adaptation. In 2012 IEEE conference on computer vision and pattern recognition, pages 2066–2073. IEEE.
  45. Eeg-based emotion detection using unsupervised transfer learning. In 2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pages 694–697. IEEE.
  46. A kernel method for the two-sample-problem. Advances in neural information processing systems, 19.
  47. Multi-source domain transfer discriminative dictionary learning modeling for electroencephalogram-based emotion recognition. IEEE Transactions on Computational Social Systems.
  48. Cross-subject emotion recognition using flexible analytic wavelet transform from eeg signals. IEEE Sensors Journal, 19(6):2266–2274.
  49. Learning subject-generalized topographical eeg embeddings using deep variational autoencoders and domain-adversarial regularization. Sensors, 21(5):1792.
  50. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778.
  51. An adversarial discriminative temporal convolutional network for eeg-based cross-domain emotion recognition. Computers in biology and medicine, 141:105048.
  52. Cross-day eeg-based emotion recognition using transfer component analysis. Electronics, 11(4):651.
  53. A machine learning emotion detection platform to support affective well being. In 2018 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), pages 2694–2700. IEEE.
  54. Detecting epilepsy in eeg signals using time, frequency and time-frequency domain features. In Computer science and engineering—theory and applications, pages 167–182. Springer.
  55. Manifold feature fusion with dynamical feature selection for cross-subject emotion recognition. Brain Sciences, 11(11):1392.
  56. Densely connected convolutional networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 4700–4708.
  57. Learning cnn features from de features for eeg-based emotion recognition. Pattern Analysis and Applications, 23(3):1323–1335.
  58. Subject-independent eeg-based emotion recognition using adversarial learning. In 2020 8th International Winter Conference on Brain-Computer Interface (BCI), pages 1–4. IEEE.
  59. Im, C.-H. (2018). Computational eeg analysis. Springer Singapore, Singapore, 10:978–981.
  60. A new segmentation method of electroencephalograms by use of akaike’s information criterion. Cognitive brain research, 3(1):33–40.
  61. Feature extraction and selection for emotion recognition from eeg. IEEE Transactions on Affective computing, 5(3):327–339.
  62. Jiang, J. (2008). A literature survey on domain adaptation of statistical classifiers. URL: http://sifaka. cs. uiuc. edu/jiang4/domainadaptation/survey, 3(1-12):3.
  63. Cross-subject emotion recognition with a decision tree classifier based on sequential backward selection. In 2019 11th International Conference on Intelligent Human-Machine Systems and Cybernetics (IHMSC), volume 1, pages 309–313. IEEE.
  64. Removal of artifacts from eeg signals: a review. Sensors, 19(5):987.
  65. Eeg-based emotion recognition using domain adaptation network. In 2017 international conference on orange technologies (ICOT), pages 222–225. IEEE.
  66. Joachims, T. et al. (1999). Transductive inference for text classification using support vector machines. In Icml, volume 99, pages 200–209.
  67. Dreamer: A database for emotion recognition through eeg and ecg signals from wireless low-cost off-the-shelf devices. IEEE journal of biomedical and health informatics, 22(1):98–107.
  68. Face morphing attack detection in the presence of post-processed image sources using neighborhood component analysis and decision tree classifier. In International Conference on Information and Communication Technology and Applications, pages 340–354. Springer.
  69. Kitchenham, B. (2004). Procedures for performing systematic reviews. Keele, UK, Keele University, 33(2004):1–26.
  70. Siamese neural networks for one-shot image recognition. In ICML deep learning workshop, volume 2, page 0. Lille.
  71. Deap: A database for emotion analysis; using physiological signals. IEEE transactions on affective computing, 3(1):18–31.
  72. Eeg correlates of different emotional states elicited during watching music videos. In International Conference on Affective Computing and Intelligent Interaction, pages 457–466. Springer.
  73. Cross-subject and cross-device wearable eeg emotion recognition using frontal eeg under virtual reality scenes. In 2021 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), pages 3630–3637. IEEE.
  74. Introducing the open affective standardized image set (oasis). Behavior research methods, 49(2):457–470.
  75. Domain adaptation techniques for eeg-based emotion recognition: a comparative study on two public datasets. IEEE Transactions on Cognitive and Developmental Systems, 11(1):85–94.
  76. Lang, P. J. (2005). International affective picture system (iaps): Affective ratings of pictures and instruction manual. Technical report.
  77. A statistically robust eeg re-referencing procedure to mitigate reference effect. Journal of neuroscience methods, 235:101–116.
  78. Cross-subject emotion recognition using deep adaptation networks. In International conference on neural information processing, pages 403–413. Springer.
  79. Foit: Fast online instance transfer for improved eeg emotion recognition. In 2020 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), pages 2618–2625. IEEE.
  80. Cross-subject eeg emotion recognition with self-organized graph neural network. Frontiers in Neuroscience, page 689.
  81. Domain adaptation for eeg emotion recognition based on latent representation similarity. IEEE Transactions on Cognitive and Developmental Systems, 12(2):344–353.
  82. Multisource transfer learning for cross-subject eeg emotion recognition. IEEE transactions on cybernetics, 50(7):3281–3293.
  83. Eeg responses to emotional videos can quantitatively predict big-five personality traits. Neurocomputing, 415:368–381.
  84. Can emotion be transferred?–a review on transfer learning for eeg-based emotion recognition. IEEE Transactions on Cognitive and Developmental Systems.
  85. Exploring eeg features in cross-subject emotion recognition. Frontiers in neuroscience, 12:162.
  86. Variational autoencoder based latent factor decoding of multichannel eeg for emotion recognition. In 2019 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), pages 684–687. IEEE.
  87. Latent factor decoding of multi-channel eeg for emotion recognition through autoencoder-like neural networks. Frontiers in neuroscience, 14:87.
  88. A novel bi-hemispheric discrepancy model for eeg emotion recognition. IEEE Transactions on Cognitive and Developmental Systems, 13(2):354–367.
  89. Adaptive batch normalization for practical domain adaptation. Pattern Recognition, 80:109–117.
  90. A novel neural network model based on cerebral hemispheric asymmetry for eeg emotion recognition. In IJCAI, pages 1561–1567.
  91. From regional to global brain: A novel hierarchical spatial-temporal neural network model for eeg emotion recognition. IEEE Transactions on Affective Computing.
  92. A bi-hemisphere domain adversarial neural network model for eeg emotion recognition. IEEE Transactions on Affective Computing.
  93. Reducing the calibration effort of eeg emotion recognition using domain adaptation with soft labels. In 2021 43rd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), pages 5962–5965. IEEE.
  94. The prisma statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. Journal of clinical epidemiology, 62(10):e1–e34.
  95. Lin, Y.-P. (2019). Constructing a personalized cross-day eeg-based emotion-classification model using transfer learning. IEEE journal of biomedical and health informatics, 24(5):1255–1264.
  96. Improving cross-day eeg-based emotion classification using robust principal component analysis. Frontiers in computational neuroscience, 11:64.
  97. Eeg-based emotion recognition in music listening. IEEE Transactions on Biomedical Engineering, 57(7):1798–1806.
  98. Domain adaptation for cross-subject emotion recognition by subject clustering. In 2021 10th International IEEE/EMBS Conference on Neural Engineering (NER), pages 904–908. IEEE.
  99. Subject-independent emotion recognition of eeg signals based on dynamic empirical convolutional neural network. IEEE/ACM Transactions on Computational Biology and Bioinformatics.
  100. Learning transferable features with deep adaptation networks. In International conference on machine learning, pages 97–105. PMLR.
  101. Transfer sparse coding for robust image representation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 407–414.
  102. Dry eeg electrodes. Sensors, 14(7):12847–12870.
  103. A review of classification algorithms for eeg-based brain–computer interfaces: a 10 year update. Journal of neural engineering, 15(3):031005.
  104. Progressive low-rank subspace alignment based on semi-supervised joint domain adaption for personalized emotion recognition. Neurocomputing, 456:312–326.
  105. Wasserstein-distance-based multi-source adversarial domain adaptation for emotion recognition and vigilance estimation. In 2021 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), pages 1424–1428. IEEE.
  106. Wgan domain adaptation for eeg-based emotion recognition. In International Conference on Neural Information Processing, pages 275–286. Springer.
  107. Reducing the subject variability of eeg signals with adversarial domain generalization. In International Conference on Neural Information Processing, pages 30–42. Springer.
  108. Supervised dictionary learning. Advances in neural information processing systems, 21.
  109. Domain generalization via invariant feature representation. In International Conference on Machine Learning, pages 10–18. PMLR.
  110. The fractal dimension of eeg as a physical measure of conscious human brain activities. Bulletin of Mathematical Biology, 50(5):559–565.
  111. Attentive adversarial network for large-scale sleep staging. In Machine Learning for Healthcare Conference, pages 457–478. PMLR.
  112. A domain adaptation sparse representation classifier for cross-domain electroencephalogram-based emotion classification. Frontiers in Psychology, page 3015.
  113. Cross-subject eeg emotion recognition using domain adaptive few-shot learning networks. In 2021 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), pages 1468–1472. IEEE.
  114. A review on the attention mechanism of deep learning. Neurocomputing, 452:48–62.
  115. A novel eeg feature extraction method using hjorth parameter. International Journal of Electronics and Electrical Engineering, 2(2):106–110.
  116. Analysis and design of echo state networks. Neural computation, 19(1):111–138.
  117. Domain adaptation via transfer component analysis. IEEE transactions on neural networks, 22(2):199–210.
  118. A survey on transfer learning. IEEE Transactions on knowledge and data engineering, 22(10):1345–1359.
  119. Subject independent emotion recognition from eeg using vmd and deep learning. Journal of King Saud University-Computer and Information Sciences.
  120. Feature extraction of eeg for emotion recognition using hjorth features and higher order crossings. In 2016 Conference on Advances in Signal Processing (CASP), pages 429–434. IEEE.
  121. Self-weighted semi-supervised classification for joint eeg-based emotion recognition and affective activation patterns mining. IEEE Transactions on Instrumentation and Measurement, 70:1–11.
  122. A survey on feature extraction methods for eeg based emotion recognition. In International Conference on Innovation in Modern Science and Technology, pages 31–45. Springer.
  123. The neurophysiological bases of emotion: An fmri study of the affective circumplex using emotion-denoting words. Human brain mapping, 30(3):883–895.
  124. Learning densenet features from eeg based spectrograms for subject independent emotion recognition. Biomedical Signal Processing and Control, 74:103485.
  125. Dataset shift in machine learning. Mit Press.
  126. Cross-corpus eeg-based emotion recognition. In 2018 IEEE 28th International Workshop on Machine Learning for Signal Processing (MLSP), pages 1–6. IEEE.
  127. Saganowski, S. (2022). Bringing emotion recognition out of the lab into real life: Recent advances in sensors and machine learning. Electronics, 11(3):496.
  128. Emotion recognition using wearables: A systematic literature review-work-in-progress. In 2020 IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops), pages 1–6. IEEE.
  129. Maximum classifier discrepancy for unsupervised domain adaptation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 3723–3732.
  130. Kernel principal component analysis. In International conference on artificial neural networks, pages 583–588. Springer.
  131. Towards adaptive classification for bci. Journal of neural engineering, 3(1):R13.
  132. Multimodal emotion recognition in response to videos. IEEE transactions on affective computing, 3(2):211–223.
  133. Eeg emotion recognition using dynamical graph convolutional neural networks. IEEE Transactions on Affective Computing, 11(3):532–541.
  134. Bioelectrical signal processing in cardiac and neurological applications, volume 8. Academic press.
  135. Ascertain: Emotion and personality recognition using commercial sensors. IEEE Transactions on Affective Computing, 9(2):147–160.
  136. Eeg-based emotion recognition: A state-of-the-art review of current trends and opportunities. Computational intelligence and neuroscience, 2020.
  137. Inception-v4, inception-resnet and the impact of residual connections on learning. In Thirty-first AAAI conference on artificial intelligence.
  138. Multi-source co-adaptation for eeg-based emotion recognition by mining correlation information. Frontiers in Neuroscience, 15:401.
  139. TenHouten, W. D. (2017). From primary emotions to the spectrum of affect: An evolutionary neurosociology of the emotions. In Neuroscience and social science, pages 141–167. Springer.
  140. Personality first in emotion: a deep neural network based on electroencephalogram channel attention for cross-subject emotion recognition. Royal Society open science, 8(8):201976.
  141. Eeg-based bci emotion recognition: A survey. Sensors, 20(18):5083.
  142. Adversarial discriminative domain adaptation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 7167–7176.
  143. Deep domain confusion: Maximizing for domain invariance. arXiv preprint arXiv:1412.3474.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now