---
title: 'CoSupFormer: Robust EEG Classification'
url: https://www.emergentmind.com/topics/cosupformer
type: topic
---

# CoSupFormer: Robust EEG Classification

CoSupFormer is an end-to-end deep learning framework designed for electroencephalography (EEG) signal classification, with explicit architectural innovations addressing the challenges of multi-scale feature extraction, channel variability, and noise resilience. The model leverages a dual-path convolutional encoder for frequency decomposition, a global transformer-based attention mechanism for modeling spatio-temporal dependencies, a channel-wise gating network for noise suppression, and an integrated hybrid loss function comprising both supervised and contrastive objectives. Empirical validation demonstrates state-of-the-art robustness and generalization across human and animal EEG datasets, with significant improvements under both clean and heavily noised conditions [2509.20489].

## 1. Architectural Design and Data Flow

CoSupFormer processes raw multichannel EEG data (dimension $C \times L$, where $C$ is the channel count and $L$ is the number of time samples) with a multi-stage encoding and classification pipeline:

1. **Multi-Scale Frequency Encoder:**  
   For each channel $x_{(c)} \in \mathbb{R}^{1\times L}$, two parallel 1D convolutional branches are applied:
   - *Branch 1 ($f_1$):* Small kernel sizes (3–5) capture high-frequency oscillations.
   - *Branch 2 ($f_2$):* Larger kernels (7–15) with dilation factors (2–4) extract low-frequency rhythms with large receptive fields and minimal parameter overhead.
   The outputs, $f_1(x_{(c)})$ and $f_2(x_{(c)})$, are concatenated along the temporal axis, forming $z_{(c)} \in \mathbb{R}^{E \times L}$ for each channel. Dropout is applied per channel, and all $z_{(c)}$ are stacked to yield $Z \in \mathbb{R}^{(C\cdot L)\times E}$. Each token in $Z$ corresponds to a localized temporal patch embedding, representing both high- and low-frequency dynamics.

2. **Global Gated Attention Encoder:**  
   The stacked patch sequence $Z$ is layer-normalized to $\widehat{Z}$. Bias-free linear projections produce queries $Q$, keys $K$, and values $V$ via $\widehat{Z}W_Q$, $\widehat{Z}W_K$, and $\widehat{Z}W_V$ respectively. Rotary positional embeddings (RoPE) introduce temporal order. Attention is computed as $A = (Q K^T)/\sqrt{d}$, masked to avoid trivial self-attention, and softmax is applied row-wise. The attended features are updated with layer normalization and a two-layer feedforward network (FFN) with residual connections. The final output is $O_{\text{att}} \in \mathbb{R}^{N\times E}$.

3. **Channel-Gating Network:**  
   A parallel gating multi-layer perceptron operates on $\widehat{Z}$, generating gate values $g = \sigma(\widehat{Z} W_g + b_g) \in \mathbb{R}^{N\times E}$. The composite output is $O = O_{\text{att}} \odot g$. This gating mechanism modulates the importance of each patch, suppressing representations corresponding to noisy or uninformative channel–time regions.

Temporal average pooling aggregates token features, which are passed to a linear classifier for output prediction.

## 2. Loss Functions and Training Objectives

CoSupFormer employs a hybrid loss, combining cross-entropy (CE) and supervised contrastive (NT-Xent) objectives:
\[
L_{\mathrm{total}} = L_{\mathrm{CE}} + \lambda L_{\mathrm{NT\text{-}Xent}}
\]
where $\lambda \in [0,1]$ balances classification and contrastive learning.

- **Cross-Entropy Loss:**  
  After pooling, the embedding $\overline{z}\in\mathbb{R}^E$ is fed through a linear classifier with softmax. The loss is  
  \[
  L_{\mathrm{CE}} = -\frac{1}{B} \sum_{i=1}^B \log p(y_i|\overline{z}_i)
  \]
  for batch size $B$.

- **Supervised Contrastive Loss (NT-Xent):**  
  For each batch, sample embeddings are flattened and normalized. Cosine similarities $s_{ij}$ are computed. Pairs sharing the same label $(y_i = y_j)$ form the positive set $P$. The loss for each pair is:
  \[
  \ell_{ij} = -\log \frac{\exp(s_{ij}/\tau)}{ \sum_{k \neq i} \exp(s_{ik}/\tau) }
  \]
  The contrastive loss averages over $P$:
  \[
  L_{\mathrm{NT\text{-}Xent}} = \frac{1}{|P|}\sum_{(i,j)\in P} \ell_{ij}
  \]
  This pulls same-class embeddings closer and pushes different-class embeddings apart, enhancing representation structure and generalization.

## 3. Training Scheme and Implementation

- **Optimizer:** Adam, without explicit weight decay.
- **Learning Rates:** 1e-4 (TDBrain, ADFTD), 1e-3 (MACO).
- **Batch Sizes:** 32 (TDBrain), 128 (ADFTD), 20 (MACO).
- **Epochs:** 100 for all settings.
- **Data Augmentation:** Besides dropout in the encoder, no extra augmentations are used. Competitor models may employ random channel drop or temporal jitter.
- **Data Splits:** All splits are subject-independent:
  - *ADFTD*: 60% train, 20% validation, 20% test by subject.
  - *TDBrain*: distinct validation/test sets each from eight subjects.
  - *MACO*: 70%/10%/20% split by individual animal.

A plausible implication is that the absence of augmentation strategies, combined with the explicit gating and contrastive regularization, shifts the burden of robustness directly onto model architecture and objective design.

## 4. Performance Evaluation and Comparative Results

### 4.1 Classification on Clean Data

**TDBrain (Parkinson’s vs. Control):**
- CoSupFormer: 95.8%±0.4 accuracy, AUROC = 99.45%, AUPRC = 99.50%
- Best baseline (MedFormer): 89.6%±0.8 accuracy, AUROC = 96.4%

**ADFTD (Alzheimer’s, FTD, Healthy):**
- CoSupFormer: 54.9%±1.8 accuracy, AUROC = 70.4%, AUPRC = 53.7%
- Best baseline (MedFormer): 53.3%±1.5 accuracy, AUROC = 70.9%, AUPRC = 51.2%

### 4.2 Robustness Under Noise and Challenging Modalities

**TDBrain with One-Third Channels Noised:**
- CoSupFormer: 95.0%±1.9 accuracy, AUROC = 99.31%, AUPRC = 99.37%
- CrossFormer: 68.8%±1.3 accuracy; MedFormer: 49.8%±1.4

**ADFTD+Noise:**
- CoSupFormer: 51.2%±1.6 accuracy, AUROC = 66.7%, AUPRC = 50.9%
- Next best: CrossFormer 49.9% accuracy, AUROC = 63.7%

**MACO (3-class mouse pharmaco-EEG):**
- CoSupFormer: 74.9%±4.5 accuracy, AUROC = 87.7%, AUPRC = 79.5%
- MedFormer/CrossFormer: 32–39% accuracy, AUROC ≈ 50–50

### 4.3 Ablation Study

- **Without Contrastive Loss:**  
  1–2 point accuracy drop under noise (e.g., TDBrain+Noise AUPRC from 99.3→97.8%).
- **Without Gating:**  
  Severe degradation under noise (TDBrain+Noise accuracy 94.2→89.5%).
- **Without Global Attention:**  
  Moderate drop in accuracy on clean data; occasionally very small gains under extreme noise or minimal-feature settings.

This suggests that all three architectural elements—the dual-path encoder, global attention, and gating—are synergistically required for optimal performance and robustness, particularly in the presence of channel or temporal artifacts.

## 5. Interpretability and Representational Dynamics

Principal Component Analysis (PCA) of pre-softmax embeddings demonstrates that the supervised contrastive objective induces tighter, more separable class clusters relative to strictly supervised variants. Channel gating weights $g$ can be visualized to highlight which channels and time segments the model down-weights, typically corresponding to physiological artifacts or flat-line noise. Attention maps $a_{ij}$ elucidate both intra-channel temporal structure and cross-channel couplings at varying time lags, which align with known patterns of cross-regional EEG synchronization in neurological disorders.

A plausible implication is that CoSupFormer’s explicit gating and contrastive clustering facilitate not only accuracy, but also interpretability with respect to neural and artifact sources in EEG.

## 6. Applications and Broader Impact

CoSupFormer has been validated across domains including:
- Human CNS disorder diagnosis (e.g., Parkinson’s, Alzheimer’s, FTD vs. healthy)
- Mouse pharmaco-EEG classification for drug-response phenotyping

The architecture enables autonomous detection and exclusion of poor-quality signals, enhancing statistical power in both clinical and preclinical paradigms. Its cross-species applicability and reproducible gains under noise highlight its utility for translational biomarker development as well as robust automated EEG analytics.

In summary, CoSupFormer combines multi-scale temporal encoding, long-range global attention, explicit channel–patch gating, and hybrid contrastive–supervised learning to achieve superior EEG classification accuracy and resilience across varied data regimes [2509.20489].

Source: https://www.emergentmind.com/topics/cosupformer