CoSupFormer: Robust EEG Classification

• CoSupFormer is an end-to-end deep learning framework for EEG classification using dual-path convolutional encoding and hybrid contrastive loss.
• It integrates multi-scale feature extraction with a global transformer attention mechanism to model spatio-temporal dependencies accurately.
• Its channel gating and hybrid loss design significantly enhance noise resilience and generalization across diverse human and animal EEG datasets.

CoSupFormer is an end-to-end deep learning framework designed for electroencephalography (EEG) signal classification, with explicit architectural innovations addressing the challenges of multi-scale feature extraction, channel variability, and noise resilience. The model leverages a dual-path convolutional encoder for frequency decomposition, a global transformer-based attention mechanism for modeling spatio-temporal dependencies, a channel-wise gating network for noise suppression, and an integrated hybrid loss function comprising both supervised and contrastive objectives. Empirical validation demonstrates state-of-the-art robustness and generalization across human and animal EEG datasets, with significant improvements under both clean and heavily noised conditions (Darankoum et al., 24 Sep 2025).

1. Architectural Design and Data Flow

CoSupFormer processes raw multichannel EEG data (dimension $C \times L$, where $C$ is the channel count and $L$ is the number of time samples) with a multi-stage encoding and classification pipeline:

1. Multi-Scale Frequency Encoder:

For each channel $x_{(c)} \in \mathbb{R}^{1\times L}$, two parallel 1D convolutional branches are applied: - Branch 1 ($f_1$): Small kernel sizes (3–5) capture high-frequency oscillations. - Branch 2 ($f_2$): Larger kernels (7–15) with dilation factors (2–4) extract low-frequency rhythms with large receptive fields and minimal parameter overhead. The outputs, $f_1(x_{(c)})$ and $f_2(x_{(c)})$, are concatenated along the temporal axis, forming $z_{(c)} \in \mathbb{R}^{E \times L}$ for each channel. Dropout is applied per channel, and all $z_{(c)}$ are stacked to yield $C$0. Each token in $C$1 corresponds to a localized temporal patch embedding, representing both high- and low-frequency dynamics.

1. Global Gated Attention Encoder: The stacked patch sequence $C$2 is layer-normalized to $C$3. Bias-free linear projections produce queries $C$4, keys $C$5, and values $C$6 via $C$7, $C$8, and $C$9 respectively. Rotary positional embeddings (RoPE) introduce temporal order. Attention is computed as $L$0, masked to avoid trivial self-attention, and softmax is applied row-wise. The attended features are updated with layer normalization and a two-layer feedforward network (FFN) with residual connections. The final output is $L$1.
2. Channel-Gating Network: A parallel gating multi-layer perceptron operates on $L$2, generating gate values $L$3. The composite output is $L$4. This gating mechanism modulates the importance of each patch, suppressing representations corresponding to noisy or uninformative channel–time regions.

Temporal average pooling aggregates token features, which are passed to a linear classifier for output prediction.

2. Loss Functions and Training Objectives

CoSupFormer employs a hybrid loss, combining cross-entropy (CE) and supervised contrastive (NT-Xent) objectives: $L$5 where $L$6 balances classification and contrastive learning.

• Cross-Entropy Loss:

After pooling, the embedding $L$7 is fed through a linear classifier with softmax. The loss is

$L$8

for batch size $L$9.

• Supervised Contrastive Loss (NT-Xent):

For each batch, sample embeddings are flattened and normalized. Cosine similarities $x_{(c)} \in \mathbb{R}^{1\times L}$0 are computed. Pairs sharing the same label $x_{(c)} \in \mathbb{R}^{1\times L}$1 form the positive set $x_{(c)} \in \mathbb{R}^{1\times L}$2. The loss for each pair is:

$x_{(c)} \in \mathbb{R}^{1\times L}$3

The contrastive loss averages over $x_{(c)} \in \mathbb{R}^{1\times L}$4:

$x_{(c)} \in \mathbb{R}^{1\times L}$5

This pulls same-class embeddings closer and pushes different-class embeddings apart, enhancing representation structure and generalization.

3. Training Scheme and Implementation

• Optimizer: Adam, without explicit weight decay.
• Learning Rates: 1e-4 (TDBrain, ADFTD), 1e-3 (MACO).
• Batch Sizes: 32 (TDBrain), 128 (ADFTD), 20 (MACO).
• Epochs: 100 for all settings.
• Data Augmentation: Besides dropout in the encoder, no extra augmentations are used. Competitor models may employ random channel drop or temporal jitter.
• Data Splits: All splits are subject-independent:
  • ADFTD: 60% train, 20% validation, 20% test by subject.
  • TDBrain: distinct validation/test sets each from eight subjects.
  • MACO: 70%/10%/20% split by individual animal.

A plausible implication is that the absence of augmentation strategies, combined with the explicit gating and contrastive regularization, shifts the burden of robustness directly onto model architecture and objective design.

4. Performance Evaluation and Comparative Results

4.1 Classification on Clean Data

TDBrain (Parkinson’s vs. Control):

• CoSupFormer: 95.8%±0.4 accuracy, AUROC = 99.45%, AUPRC = 99.50%
• Best baseline (MedFormer): 89.6%±0.8 accuracy, AUROC = 96.4%

ADFTD (Alzheimer’s, FTD, Healthy):

• CoSupFormer: 54.9%±1.8 accuracy, AUROC = 70.4%, AUPRC = 53.7%
• Best baseline (MedFormer): 53.3%±1.5 accuracy, AUROC = 70.9%, AUPRC = 51.2%

4.2 Robustness Under Noise and Challenging Modalities

TDBrain with One-Third Channels Noised:

• CoSupFormer: 95.0%±1.9 accuracy, AUROC = 99.31%, AUPRC = 99.37%
• CrossFormer: 68.8%±1.3 accuracy; MedFormer: 49.8%±1.4

ADFTD+Noise:

• CoSupFormer: 51.2%±1.6 accuracy, AUROC = 66.7%, AUPRC = 50.9%
• Next best: CrossFormer 49.9% accuracy, AUROC = 63.7%

MACO (3-class mouse pharmaco-EEG):

• CoSupFormer: 74.9%±4.5 accuracy, AUROC = 87.7%, AUPRC = 79.5%
• MedFormer/CrossFormer: 32–39% accuracy, AUROC ≈ 50–50

4.3 Ablation Study

• Without Contrastive Loss:

1–2 point accuracy drop under noise (e.g., TDBrain+Noise AUPRC from 99.3→97.8%).

• Without Gating:

Severe degradation under noise (TDBrain+Noise accuracy 94.2→89.5%).

• Without Global Attention:

Moderate drop in accuracy on clean data; occasionally very small gains under extreme noise or minimal-feature settings.

This suggests that all three architectural elements—the dual-path encoder, global attention, and gating—are synergistically required for optimal performance and robustness, particularly in the presence of channel or temporal artifacts.

5. Interpretability and Representational Dynamics

Principal Component Analysis (PCA) of pre-softmax embeddings demonstrates that the supervised contrastive objective induces tighter, more separable class clusters relative to strictly supervised variants. Channel gating weights $x_{(c)} \in \mathbb{R}^{1\times L}$6 can be visualized to highlight which channels and time segments the model down-weights, typically corresponding to physiological artifacts or flat-line noise. Attention maps $x_{(c)} \in \mathbb{R}^{1\times L}$7 elucidate both intra-channel temporal structure and cross-channel couplings at varying time lags, which align with known patterns of cross-regional EEG synchronization in neurological disorders.

A plausible implication is that CoSupFormer’s explicit gating and contrastive clustering facilitate not only accuracy, but also interpretability with respect to neural and artifact sources in EEG.

6. Applications and Broader Impact

CoSupFormer has been validated across domains including:

• Human CNS disorder diagnosis (e.g., Parkinson’s, Alzheimer’s, FTD vs. healthy)
• Mouse pharmaco-EEG classification for drug-response phenotyping

The architecture enables autonomous detection and exclusion of poor-quality signals, enhancing statistical power in both clinical and preclinical paradigms. Its cross-species applicability and reproducible gains under noise highlight its utility for translational biomarker development as well as robust automated EEG analytics.

In summary, CoSupFormer combines multi-scale temporal encoding, long-range global attention, explicit channel–patch gating, and hybrid contrastive–supervised learning to achieve superior EEG classification accuracy and resilience across varied data regimes (Darankoum et al., 24 Sep 2025).