EEGDiR: Electroencephalogram denoising network for temporal information storage and global modeling through Retentive Network (2404.15289v2)
Abstract: Electroencephalogram (EEG) signals play a pivotal role in clinical medicine, brain research, and neurological disease studies. However, susceptibility to various physiological and environmental artifacts introduces noise in recorded EEG data, impeding accurate analysis of underlying brain activity. Denoising techniques are crucial to mitigate this challenge. Recent advancements in deep learningbased approaches exhibit substantial potential for enhancing the signal-to-noise ratio of EEG data compared to traditional methods. In the realm of large-scale LLMs, the Retentive Network (Retnet) infrastructure, prevalent for some models, demonstrates robust feature extraction and global modeling capabilities. Recognizing the temporal similarities between EEG signals and natural language, we introduce the Retnet from natural language processing to EEG denoising. This integration presents a novel approach to EEG denoising, opening avenues for a profound understanding of brain activities and accurate diagnosis of neurological diseases. Nonetheless, direct application of Retnet to EEG denoising is unfeasible due to the one-dimensional nature of EEG signals, while natural language processing deals with two-dimensional data. To facilitate Retnet application to EEG denoising, we propose the signal embedding method, transforming one-dimensional EEG signals into two dimensions for use as network inputs. Experimental results validate the substantial improvement in denoising effectiveness achieved by the proposed method.
- Understanding of a convolutional neural network, in: 2017 international conference on engineering and technology (ICET), Ieee. pp. 1–6.
- Layer normalization. arXiv preprint arXiv:1607.06450 .
- Feed-forward neural networks. Ieee Potentials 13, 27–31.
- Removal of eog artefacts by combining wavelet neural network and independent component analysis. Biomedical Signal Processing and Control 15, 67–79.
- A new method for off-line removal of ocular artifact. Electroencephalography and clinical neurophysiology 55, 468–484.
- Deep residual learning for image recognition, in: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770–778.
- Removal of ocular artifacts from electro-encephalogram by adaptive filtering. Medical and biological engineering and computing 42, 407–412.
- Batch normalization: Accelerating deep network training by reducing internal covariate shift, in: International conference on machine learning, pmlr. pp. 448–456.
- Self-gated rectified linear unit for performance improvement of deep neural networks. ICT Express 9, 320–325.
- Removal of artifacts from eeg signals: a review. Sensors 19, 987.
- Reg-ica: a hybrid methodology combining blind source separation and regression techniques for the rejection of ocular artifacts. Biomedical Signal Processing and Control 6, 291–300.
- Datasets: A community library for natural language processing. arXiv preprint arXiv:2109.02846 .
- Validation of ica-based myogenic artifact correction for scalp and source-localized eeg. Neuroimage 49, 2416–2432.
- Long short-term memory. Neural computation 9, 1735–1780.
- Artifact suppression from eeg signals using data adaptive time domain filtering. Neurocomputing 97, 297–308.
- Eegdnet: Fusing non-local and local self-similarity for eeg signal denoising with transformer. Computers in Biology and Medicine 151, 106248.
- Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 .
- A novel end-to-end 1d-rescnn model to remove artifact from eeg signals. Neurocomputing 404, 108–121.
- Retentive network: A successor to transformer for large language models. arXiv preprint arXiv:2307.08621 .
- Sequence to sequence learning with neural networks. Advances in neural information processing systems 27.
- Removal artifacts from eeg signal using independent component analysis and principal component analysis, in: 2014 2nd International Conference on Technology, Informatics, Management, Engineering & Environment, IEEE. pp. 296–302.
- Attention is all you need. Advances in neural information processing systems 30.
- Eeg multiresolution analysis using wavelet transform, in: 2001 Conference Proceedings of the 23rd Annual International Conference of the IEEE Engineering in Medicine and Biology Society, IEEE. pp. 1854–1856.
- Group normalization, in: Proceedings of the European conference on computer vision (ECCV), pp. 3–19.
- Automatic ocular artifacts removal in eeg using deep learning. Biomedical Signal Processing and Control 43, 148–158.
- Recurrent neural network regularization. arXiv preprint arXiv:1409.2329 .
- Eegdenoisenet: A benchmark dataset for end-to-end deep learning solutions of eeg denoising. arXiv preprint arXiv:2009.11662 .
- Understanding adamw through proximal methods and scale-freeness. arXiv preprint arXiv:2202.00089 .