Weight Freezing: A Regularization Approach for Fully Connected Layers with an Application in EEG Classification (2306.05775v2)
Abstract: In the realm of EEG decoding, enhancing the performance of artificial neural networks (ANNs) carries significant potential. This study introduces a novel approach, termed "weight freezing", that is anchored on the principles of ANN regularization and neuroscience prior knowledge. The concept of weight freezing revolves around the idea of reducing certain neurons' influence on the decision-making process for a specific EEG task by freezing specific weights in the fully connected layer during the backpropagation process. This is actualized through the use of a mask matrix and a threshold to determine the proportion of weights to be frozen during backpropagation. Moreover, by setting the masked weights to zero, weight freezing can not only realize sparse connections in networks with a fully connected layer as the classifier but also function as an efficacious regularization method for fully connected layers. Through experiments involving three distinct ANN architectures and three widely recognized EEG datasets, we validate the potency of weight freezing. Our method significantly surpasses previous peak performances in classification accuracy across all examined datasets. Supplementary control experiments offer insights into performance differences pre and post weight freezing implementation and scrutinize the influence of the threshold in the weight freezing process. Our study underscores the superior efficacy of weight freezing compared to traditional fully connected networks for EEG feature classification tasks. With its proven effectiveness, this innovative approach holds substantial promise for contributing to future strides in EEG decoding research.
- Deep convolutional neural network for the automated detection and diagnosis of seizure using eeg signals. Computers in biology and medicine 100, 270–278.
- Multiscale convolutional transformer for eeg classification of mental imagery in different modalities. IEEE Transactions on Neural Systems and Rehabilitation Engineering .
- Filter bank common spatial pattern (fbcsp) in brain-computer interface, in: 2008 IEEE international joint conference on neural networks (IEEE world congress on computational intelligence), IEEE. pp. 2390–2397.
- Filter bank common spatial pattern (fbcsp) algorithm using online adaptive and semi-supervised learning, in: The 2011 International Joint Conference on Neural Networks, IEEE. pp. 392–396.
- Eeg-convtransformer for single-trial eeg-based visual stimulus classification. Pattern Recognition 129, 108757.
- A systematic review on overfitting control in shallow and deep neural networks. Artificial Intelligence Review , 1–48.
- Interpretable and lightweight convolutional neural network for eeg decoding: Application to movement execution and imagination. Neural Networks 129, 55–74.
- Improved regularization of convolutional neural networks with cutout. arXiv preprint arXiv:1708.04552 .
- Emg contamination of eeg: spectral and topographical characteristics. Clinical neurophysiology 114, 1580–1593.
- Generative adversarial networks. Communications of the ACM 63, 139–144.
- Transfer learning for brain–computer interfaces: A euclidean space data alignment approach. IEEE Transactions on Biomedical Engineering 67, 399–410.
- Nonstationary nature of the brain activity as revealed by eeg/meg: methodological, practical and conceptual challenges. Signal processing 85, 2190–2212.
- Eegnet: a compact convolutional neural network for eeg-based brain–computer interfaces. Journal of neural engineering 15, 056013.
- Fixing weight decay regularization in adam URL: https://openreview.net/forum?id=rk6qdGgCZ.
- Regularizing common spatial patterns to improve bci designs: unified theory and new algorithms. IEEE Transactions on biomedical Engineering 58, 355–362.
- Motor imagery. Journal of Physiology-paris 99, 386–395.
- Mbga-net: A multi-branch graph adaptive network for individualized motor imagery eeg classification. Computer Methods and Programs in Biomedicine , 107641.
- Priming cross-session motor imagery classification with a universal deep domain adaptation framework. arXiv preprint arXiv:2202.09559 .
- Lmda-net:a lightweight multi-dimensional attention network for general eeg-based brain-computer interfaces and interpretability. NeuroImage 276, 120209. URL: https://www.sciencedirect.com/science/article/pii/S1053811923003609, doi:https://doi.org/10.1016/j.neuroimage.2023.120209.
- Towards the utilization of eeg as a brain imaging tool. Neuroimage 61, 371–385.
- Assessment of cognitive engagement in stroke patients from single-trial eeg during motor rehabilitation. IEEE Transactions on Neural Systems and Rehabilitation Engineering 23, 351–362.
- Mu rhythm (de) synchronization and eeg single-trial classification of different motor imagery tasks. NeuroImage 31, 153–159.
- Motor imagery and direct brain-computer communication. Proceedings of the IEEE 89, 1123–1134.
- Avoiding overfitting: A survey on regularization methods for convolutional neural networks. ACM Computing Surveys (CSUR) 54, 1–25.
- Deep learning with convolutional neural networks for eeg decoding and visualization. Human brain mapping 38, 5391–5420.
- Meeting brain–computer interface user performance expectations using a deep neural network decoding framework. Nature medicine 24, 1669–1676.
- Eeg conformer: Convolutional transformer for eeg decoding and visualization. IEEE Transactions on Neural Systems and Rehabilitation Engineering 31, 710–719. doi:10.1109/TNSRE.2022.3230250.
- Dropout: a simple way to prevent neural networks from overfitting. The journal of machine learning research 15, 1929–1958.
- Physiological origins and functional correlates of eeg rhythmic activities: implications for self-regulation. Biofeedback and Self-regulation 21, 3–33.
- Attention is all you need. Advances in neural information processing systems 30.
- Dropout training as adaptive regularization. Advances in neural information processing systems 26.
- Brain-computer interfaces (bcis) for communication and control, in: Proceedings of the 9th international ACM SIGACCESS conference on Computers and accessibility, pp. 1–2.
- Towards dropout training for convolutional neural networks. Neural Networks 71, 1–10.
- Mi-cat: A transformer-based domain adaptation network for motor imagery classification. Neural Networks .
- Deep representation-based domain adaptation for nonstationary eeg classification. IEEE Transactions on Neural Networks and Learning Systems 32, 535–545.
Sponsored by Paperpile, the PDF & BibTeX manager trusted by top AI labs.
Get 30 days freePaper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.