EEGNet Fusion: Multi-Stream EEG Analysis

• EEGNet Fusion is a multi-stream neural architecture that fuses convolutional, recurrent, and attention features for advanced EEG decoding.
• It employs varied fusion mechanisms, including multi-scale temporal kernels and attention modules, to enhance robustness against session and subject variability.
• Empirical studies show improved performance in motor imagery and emotion recognition tasks while maintaining computational efficiency with minimal parameter budgets.

EEGNet Fusion is a class of neural network architectures designed for decoding electroencephalography (EEG) signals, especially for tasks such as motor imagery (MI) classification and emotion recognition. It extends the foundational EEGNet by integrating multiple feature extraction streams—typically at different temporal or spatiotemporal scales—and fusing their outputs to improve model robustness, particularly in the context of inter-session or inter-subject variability. Recent developments in EEGNet Fusion also incorporate mechanisms such as attention modules and recurrent neural networks, resulting in a spectrum of architectures unified by the principle of fusing features from multiple, distinct processing paths.

1. Architectures and Fusion Mechanisms

EEGNet Fusion architectures share a multi-stream design, where each stream processes the input EEG with a distinct set of parameters—often different temporal kernel sizes, spatial filters, or attention modules. The classic form, as described in comparative studies, employs two parallel branches, each mirroring the EEGNet temporal–spatial–separable-convolutional pipeline, but with different temporal kernel lengths—typically one long and one short to capture coarse and fine temporal information. Feature vectors from these streams are concatenated prior to classification (Köllőd et al., 2023).

More sophisticated variants, such as EEG-MFTNet, introduce a multi-branch (six-way) temporal convolution block covering kernel sizes $k_i \in \{5,9,13,29,61,125\}$ for fine-to-coarse representation, in parallel with a Transformer encoder that provides long-range temporal context. The outputs are fused using adaptive, trainable channel weights to enable dynamic reweighting of different temporal contexts (Andrikopoulos et al., 7 Apr 2026).

In EEG-CSANet, fusion is achieved across four branches with kernel sizes $K_i \in \{64,32,16,8\}$, each with its own spatial convolutions and attention modules. A centralized sparse-attention framework further enables selectivity in inter-branch feature integration. Outputs are concatenated and post-processed via temporal convolutional networks (Cai et al., 21 Dec 2025).

Hybrid designs (EEGFuseNet) integrate convolutional feature extraction with bidirectional GRUs (BiGRU), yielding fused representations that aggregate spatial features (from CNNs) and temporal dynamics (from RNNs), optionally enhanced by adversarial (GAN) losses. Fusion in these models is frequently realized via concatenation of the time-gated hidden states (Liang et al., 2021).

2. Mathematical Formulation of Fusion

Fusion mechanisms in EEGNet Fusion architectures generally operate at the feature level, post-convolution and/or recurrent processing. In canonical two-branch EEGNet Fusion, if $f_1, f_2$ denote the flattened outputs of each branch, the fused representation is:

$f_{\text{fused}} = \operatorname{Concat}(f_1, f_2)$

which is then passed to a fully connected classifier (Köllőd et al., 2023).

Multi-scale fusion as in EEG-MFTNet applies learned, scalar reweighting across $m$ branches:

$$Y_{\text{concat}}(t) = [\,\alpha_1\,Y_1(t)\;||\;\dots\;||\;\alpha_m\,Y_m(t)\,]$$

The Transformer stream output $F_{\rm trans}$ is combined with the concatenated convolutional features $F_{\rm conv}$ as:

$$F_{\rm fused}(t) = [\,\beta\,F_{\rm conv}(t)\;||\;\gamma\,F_{\rm trans}(t)\,]$$

where $\alpha_i, \beta, \gamma$ are learnable parameters controlling the fusion (Andrikopoulos et al., 7 Apr 2026).

Attention-driven fusion, as in EEG-CSANet, combines outputs with main-branch multi-head self-attention and auxiliary-branch centralized sparse cross-attention. Feature maps $K_i \in \{64,32,16,8\}$0 from branch $K_i \in \{64,32,16,8\}$1 are:

$K_i \in \{64,32,16,8\}$2

and overall fusion is:

$K_i \in \{64,32,16,8\}$3

enabling multiscale, attention-weighted integration (Cai et al., 21 Dec 2025).

In hybrid fusion (EEGFuseNet), the output $K_i \in \{64,32,16,8\}$4 is the flattening of bidirectional GRU states over all time positions:

$K_i \in \{64,32,16,8\}$5

with no explicit learned fusion weights beyond the recurrent dynamics (Liang et al., 2021).

3. Training Protocols and Data Preprocessing

EEGNet Fusion models have been systematically evaluated on large, artifact-cleaned datasets as well as smaller curated sets. Standard preprocessing comprises bandpass filtering (1–45 Hz or 0.5–40 Hz), artifact removal (e.g., FASTER, ICA-based component rejection), and windowing of the continuous signal into epochs.

For supervised MI classification, five-fold cross-validation is standard, with separate within-subject and transfer learning (cross-subject) protocols. Transfer learning is assessed via a two-stage procedure: pre-training on a pool of source subjects, then fine-tuning on the target subject, using held-out validation splits at each stage (Köllőd et al., 2023).

Optimization utilizes Adam or AdamW (learning rates from $K_i \in \{64,32,16,8\}$6 to $K_i \in \{64,32,16,8\}$7), categorical cross-entropy loss, early stopping by validation accuracy or loss, modest batch sizes (16–128), and extensive regularization (batch normalization, dropout). For unsupervised settings, adversarial (GAN) loss and $K_i \in \{64,32,16,8\}$8 reconstruction loss are combined, with alternate optimization of generator and discriminator (Liang et al., 2021).

4. Empirical Performance and Comparative Results

The effectiveness of EEGNet Fusion has been quantified across multiple datasets and model variants. In motor imagery decoding:

• Baseline EEGNet achieves cross-session accuracy of $K_i \in \{64,32,16,8\}$9; EEG-GENet and AA-EEGNet yield $f_1, f_2$0 and $f_1, f_2$1 respectively; EEG-MFTNet (fusion with Transformer) reaches $f_1, f_2$2. Model parameter counts remain small (16 K) and inference latency is low ($f_1, f_2$3 ms) (Andrikopoulos et al., 7 Apr 2026).
• EEG-CSANet, with multiscale attention fusion, reports state-of-the-art multi-dataset performance: $f_1, f_2$4 (BCIC-IV-2A), $f_1, f_2$5 (BCIC-IV-2B), $f_1, f_2$6 (HGD), $f_1, f_2$7 (SEED), $f_1, f_2$8 (SEED-VIG) (Cai et al., 21 Dec 2025).
• Comparative studies found EEGNet Fusion (two-branch variant) sometimes outperformed the original EEGNet in within-subject MI accuracy on large datasets (Physionet, Giga), but was often surpassed by deeper or more specialized models such as MI-EEGNet and Deep ConvNet, especially under transfer learning (Köllőd et al., 2023).

In unsupervised emotion recognition, EEGFuseNet demonstrated marked improvements over classical features, with cross-subject accuracies of $f_1, f_2$9 (SEED 2-class) and $f_{\text{fused}} = \operatorname{Concat}(f_1, f_2)$0 (SEED 3-class), and consistent $f_{\text{fused}} = \operatorname{Concat}(f_1, f_2)$1–$f_{\text{fused}} = \operatorname{Concat}(f_1, f_2)$2 F1-score gains versus the best unsupervised baselines (Liang et al., 2021).

Ablation studies systematically confirm the value of each fusion pathway (e.g., multi-scale convolutions, attention modules, Transformer/brute-force recurrence), with full models universally outperforming ablated counterparts on the tested metrics (Andrikopoulos et al., 7 Apr 2026, Cai et al., 21 Dec 2025).

5. Interpretability, Feature Analysis, and Modality-Specific Insights

Feature visualization with UMAP and power spectra reveals that parallel, multi-scale branches in EEGNet Fusion highlight different frequency bands: large temporal kernels focus on low-frequency components ($f_{\text{fused}} = \operatorname{Concat}(f_1, f_2)$3, $f_{\text{fused}} = \operatorname{Concat}(f_1, f_2)$4, $f_{\text{fused}} = \operatorname{Concat}(f_1, f_2)$5); small kernels capture higher-frequency content. All-branch fusion sharpens class separability and reduces inter-subject/class variance (Cai et al., 21 Dec 2025).

EEGFuseNet's learned representations encode both spatial (channel-wise) and temporal dynamics, with the BiGRU gates controlling local and non-local aggregation. The 64-dimensional feature embeddings significantly outperform bandpower, entropy, or handcrafted features in clustering-based emotion decoding, suggesting superior generality and robustness (Liang et al., 2021).

Attention map inspection in EEG-CSANet reports resonance with physiologically plausible motor patterns, as attention heads track relevant temporal segments and channel groupings.

6. Limitations, Comparative Position, and Future Directions

While EEGNet Fusion variants deliver measurable enhancement over baseline EEGNet, especially in robust MI decoding and generalization across sessions or subjects, they do not always surpass all alternatives. Deeper ConvNets and highly specialized models may offer marginally higher performance on some datasets (Köllőd et al., 2023).

Computational efficiency is a critical strength: most EEGNet Fusion models maintain parameter budgets well below $f_{\text{fused}} = \operatorname{Concat}(f_1, f_2)$6 K, crucial for real-time BCI. Nevertheless, further gains are anticipated via more advanced fusion (learned, attention-augmented, recurrent), domain adaptation for cross-subject transfer, and online or continual learning paradigms (Andrikopoulos et al., 7 Apr 2026).

Identified limitations include modest improvements over single-branch or unrelated architectures in some settings, a still-present gap to fully supervised methods in unsupervised regimes, and a lack of interpretability for some fusion weights. Areas for future research involve explicit attention-based fusion, streaming adaptation, and extensive benchmarking across additional tasks and populations.

7. Summary Table: Core EEGNet Fusion Models

Model	Fusion Modality	Key Components	Example Performance
EEGNet Fusion	Two-branch temporal concat	EEGNet-style dual kernels, flatten-concat	$f_{\text{fused}} = \operatorname{Concat}(f_1, f_2)$7 (BCI IV 2a) (Köllőd et al., 2023)
EEG-MFTNet	Multiscale + Transformer fusion	6× temporal branches + Transformer, adaptive fusion	$f_{\text{fused}} = \operatorname{Concat}(f_1, f_2)$8 (SHU) (Andrikopoulos et al., 7 Apr 2026)
EEG-CSANet	Multibranch + attention fusion	4× branch, centralized sparse attn, TCN	$f_{\text{fused}} = \operatorname{Concat}(f_1, f_2)$9 (BCIC-IV-2A) (Cai et al., 21 Dec 2025)
EEGFuseNet	Hybrid CNN+RNN fusion	CNN-GAN encoder, BiGRU, unsupervised clustering	$m$0 (SEED 2-class) (Liang et al., 2021)

These architectures typify the EEGNet Fusion paradigm: leveraging ensembles of diverse, task-relevant feature extractors and dynamically integrating their outputs, achieving gains in MI decoding accuracy, robustness, and computational viability across diverse EEG analysis domains.