Handwriting Imagery EEG Classification based on Convolutional Neural Networks

Abstract: Handwriting imagery has emerged as a promising paradigm for brain-computer interfaces (BCIs) aimed at translating brain activity into text output. Compared with invasively recorded electroencephalography (EEG), non-invasive recording offers a more practical and feasible approach to capturing brain signals for BCI. This study explores the limit of decoding non-invasive EEG associated with handwriting imagery into English letters using deep neural networks. To this end, five participants were instructed to imagine writing the 26 English letters with their EEG being recorded from the scalp. A measurement of EEG similarity across letters was conducted to investigate letter-specific patterns in the dataset. Subsequently, four convolutional neural network (CNN) models were trained for EEG classification. Descriptively, the EEG data clearly exhibited letter-specific patterns serving as a proof-of-concept for EEG-to-text translation. Under the chance level of accuracy at 3.85%, the CNN classifiers trained on each participant reached the highest limit of around 20%. This study marks the first attempt to decode non-invasive EEG associated with handwriting imagery. Although the achieved accuracy is not sufficient for a usable brain-to-text BCI, the model's performance is noteworthy in revealing the potential for translating non-invasively recorded brain signals into text outputs and establishing a baseline for future research.