EEG-Based Imagined Speech Decoding Using a Hybrid CNN-SNN Architecture

Abstract: Imagined speech decoding using EEG signals has emerged as a promising frontier in brain-computer interface (BCI) research, particularly to restore communication for individuals with severe speech impairments. However, decoding imagined speech remains a complex task due to the non-stationary, low-amplitude, and highly variable nature of EEG signals. Existing methods often rely on classical machine learning or deep learning models that fail to exploit spike-based temporal dynamics or event-driven firing mechanisms of biological neurons, which are naturally modeled by spiking neural networks (SNNs). In this study, we propose a hybrid decoding pipeline that extracts temporal representations using convolutional neural networks (CNNs) followed by biologically inspired temporal classification via SNNs. To our knowledge, this is the first study to integrate SNNs into EEG-based imagined speech decoding. Experimental results show that the proposed CNN-SNN architecture achieves an accuracy of 80.13% on the 2020 BCI Competition III benchmark, surpassing existing methods reported in the literature (up to 70.19%) under comparable evaluation settings. These findings demonstrate the effectiveness of spike-based temporal decoding for imagined speech, highlighting the promise of biologically grounded pipelines for next generation neuromorphic BCI applications.

• The paper introduces a novel hybrid CNN-SNN pipeline that leverages temporal convolution and spiking neural networks to boost EEG-based imagined speech decoding.
• It achieves over 80% accuracy on a 64-channel EEG dataset from BCI Competition III, outpacing prior models by 3–10% improvements.
• The architecture offers low latency and energy-efficient event-driven processing, highlighting its potential for real-time, patient-centered brain-computer interfaces.

Hybrid CNN-SNN Architecture for EEG-Based Imagined Speech Decoding

Introduction and Motivation

EEG-based imagined speech decoding is a critical topic at the intersection of neural engineering and AI, with strong implications for non-invasive brain-computer interfaces (BCIs) targeting communication restoration in patients with severe speech and motor disabilities. Electrophysiological signals recorded during covert (imagined) speech tasks are notably noisy, non-stationary, and subject to high inter- and intra-trial variability, presenting unique challenges for robust neural decoding. While classical ML and DL models have demonstrated some efficacy, they are fundamentally limited in their ability to model the brain's temporally precise, event-driven neuronal dynamics, which are more faithfully represented by SNNs.

The paper "EEG-Based Imagined Speech Decoding Using a Hybrid CNN-SNN Architecture" (2607.03844) proposes a novel hybrid pipeline that leverages temporal convolutional feature extraction (CNN) combined with biologically inspired temporal classification (SNN), marking the first concerted effort to evaluate SNNs in the context of non-invasive EEG-based imagined speech decoding.

Related Work Overview

Prior work on EEG-based imagined speech decoding has evolved from classical ML—primarily SVMs, random forests, and KNNs on handcrafted features—to modern DL methods, notably CNNs, LSTMs, and hybrid architectures. Handcrafted approaches often reach a ceiling due to the poor scalability and limited robustness in capturing high-dimensional, low-SNR neural data. DL models, especially those capitalizing on deep temporal or spatial feature hierarchies, have pushed state-of-the-art (SotA) accuracy to the range of 70–77% for standard multi-class tasks [Alharbi2024Hybrid, Li2023ConvMixer].

However, these models generally operate on continuous-valued activations, offering little alignment with the biological substrate. Emerging neuromorphic/energy-efficient approaches such as SNNs, which model neural spiking and event timing, promise fundamentally lower power consumption and improved deployability on real-time, embedded BCI systems. Despite their advantages, SNNs had not previously been explored for this application domain.

Model Architecture and Pipeline

The proposed system adopts a two-stage pipeline:

1. CNN-Based Temporal Feature Extraction: The raw, artifact-reduced, and normalized 64-channel EEG time-series are input to a 1D CNN stack, extracting deep temporal representations with multi-scale convolutions, batch normalization, and dropout regularization. The structure is tuned to optimize temporal receptive fields while minimizing parameter count and overfitting.
2. SNN-Based Classification: The high-level feature embeddings are quantized and temporally encoded as spike trains provided to a temporal SNN composed of two fully connected LIF layers. The SNN, operating in discrete time steps, integrates input and generates output spike trains, with class prediction derived from the mean output spike rate over the time window.

   Figure 1: Pipeline for EEG-based imagined speech decoding, integrating CNN-based feature extraction with SNN-based temporal classification.

   

Figure 2: Model architecture—(a) CNN-based feature extraction section; (b) LIF SNN-based imagined speech classification module.

The integration of SNNs allows for temporal pattern integration at a granularity unachievable by conventional ANN or RNN classifiers, with the added benefit of computational and power efficiency due to sparse and event-driven processing.

Experimental Setup and Results

The model is evaluated using the 2020 BCI Competition III dataset, consisting of 64-channel EEG from 15 healthy subjects performing five imagined word/phrase tasks. After standard preprocessing (notch filtering, spatial CAR, channel-wise z-scoring), the dataset is divided into 85% training and 15% test trials.

Training is performed via AdamW, using surrogate-gradient backpropagation for the SNN section to address non-differentiability. The architecture achieves 80.13% average accuracy and 80.14% F1-score across subjects in a strict subject-dependent supervised classification setting, representing a strong improvement over previous SotA architectures applied to the same dataset.

For reference, competing approaches on identical data include:

• Spectro-spatio-temporal CNNs (70.19% accuracy)
• Self-attention transfer models (69.00%)
• Mixed CNN-DL hybrids (up to 77.8% accuracy)

The CNN-SNN model thus delivers a clear numerical improvement margin of approximately 3–10% over all competing subject-dependent models.

Inference speed is also competitive: Mean per-trial prediction time is 98 ms (on GPU), enabling near-real-time application given the 2s EEG trial duration and suggesting a model real-time factor close to 0.05.

Discussion and Implications

This work establishes significant findings:

• Superior Classification Accuracy: The hybrid CNN-SNN systematically outperforms all prior published models on the BCI Competition III imagined speech benchmark (>80% vs. ≤77.8%), demonstrating the utility of spike-based temporal modeling, especially for weak, variable neural signals.
• First SNN Application in the Task: Integrating LIF SNN layers for classification, together with surrogate-gradient training, represents a novel demonstration that temporally precise, event-driven neuromorphic computation can offer practical gains in non-invasive EEG decoding.
• Power and Latency Considerations: The event-driven nature of SNNs greatly reduces computational load and energy consumption, supporting deployment in portable, wearable BCIs and enabling real-time interactive systems.

(Figure 1) and (Figure 2) visually clarify the proposed processing pipeline and the division between CNN feature extraction and SNN classification.

Practically, this approach positions SNN-based models as a highly competitive option for on-device, patient-centered speech BCIs, particularly in situations where low latency and power budget are paramount. Theoretically, the results encourage further exploration of SNNs as general-purpose temporal pattern decoders in noisy neural domains.

Remaining limitations include the focus on subject-dependent settings and the need for further validation under cross-subject transfer and real-time acquisition conditions. The degree to which the spike-based regime can generalize to more complex imagined speech tasks or larger, more heterogeneous populations remains to be established.

Future Perspectives

Key avenues for future work include:

• Extension to cross-subject transfer learning, to assess generalization and reduce calibration needs;
• Application of the pipeline to larger/multi-lingual or natural-language datasets for more complex BCIs;
• Hardware realization on dedicated neuromorphic chips to fully exploit the SNN energy-latency tradeoff;
• Integration with closed-loop neurofeedback to enhance imagined speech discrimination via real-time adaptation.

These directions will further clarify the scalability of neuromorphic models for complex human-centric neuroprosthetic communication problems.

Conclusion

The hybrid CNN-SNN architecture for EEG-based imagined speech decoding demonstrates a significant advance in both accuracy and computational efficiency. By coupling deep temporal feature learning with event-driven, biologically inspired classification, the pipeline achieves notable superiority over state-of-the-art methods and establishes the foundation for energy-efficient, real-time BCI deployment. This work validates the relevance of SNNs in non-invasive neural decoding and opens further research into neuromorphic AI for brain-signal translation and communication restoration applications.