DARE-EEG: A Foundation Model for Mining Dual-Aligned Representation of EEG

Abstract: Foundation models pre-trained through masked reconstruction on large-scale EEG data have emerged as a promising paradigm for learning generalizable neural representations across diverse brain-computer interface applications. However, a critical yet overlooked challenge is that EEG encoders must learn representations invariant to incomplete observations-when different masked views of the same signal have minimal overlap, existing methods fail to constrain them to a consistent latent subspace, leading to degraded transferability. To address this, we propose DARE-EEG, a self-supervised foundation model that explicitly enforces the mask-invariance property through dual-aligned representation learning during pre-training. Specifically, we introduce mask alignment that constrains representations from multiple masked views of the same EEG sample via contrastive learning, complementing anchor alignment that aligns masked representations to momentum-updated complete features for semantic stability. Additionally, we propose conv-linear-probing, a parameter-efficient strategy that adapts pre-trained representations to heterogeneous electrode configurations and sampling rates through decoupled spectro-spatial projections. Extensive experiments across diverse EEG benchmarks demonstrate that DARE-EEG consistently achieves state-of-the-art in accuracy performance while maintaining relatively low parameter complexity and superior cross-dataset portability compared to existing methods. Furthermore, DARE-EEG contributes to effectively discovering and utilizing the rich potential representations in EEG.

• The paper introduces DARE-EEG, an EEG foundation model that combines masked reconstruction, anchor alignment, and cross-view contrastive mask alignment to enforce mask-invariant representations.
• The model achieves 0.8156 balanced accuracy and 0.8904 AUROC on TUAB, while improving TUEV performance by more than 3% over EEGPT-large and leading five additional benchmarks.
• The paper proposes conv-linear probing for adapting frozen models to different electrode layouts and sampling rates, reducing training costs while preserving strong performance across heterogeneous EEG tasks.

Motivation and problem statement

DARE-EEG addresses a specific deficiency in masked-reconstruction pre-training for EEG foundation models. Existing approaches—BENDR, BIOT, LaBraM, EEGMamba, and EEGPT—train encoders to reconstruct randomly masked portions of the input, but none explicitly constrains the representations produced from different masked views of the same sample to be consistent. The authors formalize this requirement as the Mask-Invariance Property (MIP): representations derived from distinct mask-views of one EEG recording should occupy a consistent latent subspace while remaining discriminative across neural states. They argue that EEG is especially vulnerable to violating MIP because of its low signal-to-noise ratio and sensitivity to incomplete observations; at high masking ratios, where overlap between views is minimal, an encoder can degenerate into encoding the mask pattern itself rather than the underlying neural state—a failure mode they characterize as "waveform memorization."

The paper supports this claim with a theoretical analysis (Appendix). Proposition 1 shows that masked contrastive learning directly minimizes the expected squared distance between representations under two independent masks, making MIP enforcement explicit. Proposition 3 provides a counterexample demonstrating that anchor alignment (AA) alone cannot guarantee MIP: a representation of the form $$z_\theta(X,m) = \mathrm{norm}(\bar{z}_\Delta(X) + \alpha r(m))$$, with $r(m)$ a mask-dependent vector approximately orthogonal to the full-view target, keeps AA loss low while allowing inter-view distance to grow as $\mathcal{O}(\alpha)$. This is the paper's central argument: aligning each masked view to the complete signal does not imply that different masked views resemble each other.

Method

DARE-EEG combines three self-supervised objectives during pre-training:

• Masked reconstruction at the patch level. Temporal patches (64 samples, $N=16$ per segment) are embedded with channel position encodings following the 10–20 system. A notable architectural choice is Global Learnable Tokens (GLTs), embedded along the channel dimension rather than the temporal dimension, which serve as global semantic aggregators and are the sole outputs passed to the predictor. The reconstructor receives encoder outputs for visible positions and predictor outputs only for fully masked positions ("Replace" operation), decoupling representation learning from reconstruction fidelity.
• Anchor alignment via a momentum-updated target encoder ($\tau = 0.99$), with MSE consistency loss between target-encoder features on the complete signal and student-encoder GLT features on masked views.
• Mask alignment, the key addition: two random masks $\mathcal{M}_1$ and $\mathcal{M}_2$ are applied to each sample, with their overlap ratio constrained to $(0.2, 0.8)$ to avoid degenerate configurations. Representations from the two views form positive pairs within a mini-batch, optimized with symmetric InfoNCE using a trainable temperature initialized at 0.1. The total loss is $$\mathcal{L}_{RC} + \mathcal{L}_{AA} + 0.1\,\mathcal{L}_{MA}$$.

For downstream adaptation, the paper proposes conv-linear-probing (CLP): a lightweight spectro-spatial projection inserted before the frozen encoder, consisting of a $1\times1$ convolution mapping arbitrary electrode configurations onto the unified 58-channel pre-training space, followed by a large-kernel temporal convolution for sampling-rate/frequency adaptation. Only this module plus a linear probe head are trained by default, keeping trainable parameters in the tens-of-thousands range for most tasks.

Pre-training used over 264,000 samples (>917 hours) spanning five paradigms—emotion recognition (SEED), motor imagery/execution (PhysioMI), multi-paradigm cognitive tasks (M3CV), SSVEP (TSU), and MATB workload (pBCIW)—all resampled to 256 Hz with overlapping channels retained. The Deep configuration has roughly 103M parameters.

Empirical results

On TUAB abnormality detection, DARE-EEG-Deep reaches 0.8156 balanced accuracy and 0.8904 AUROC, exceeding EEGPT-large (0.7983 / 0.8718) by more than 1.6% and 1.7% respectively. On TUEV event classification, DARE-EEG-Base attains 0.6561 balanced accuracy versus 0.6232 for EEGPT-large, a gain of over 3%. Notably, DARE-EEG-Base uses an encoder of roughly 6M parameters against ~20M for competing pre-trained models, which the authors present as evidence that MIP enforcement—not capacity alone—drives the gains.

Across five additional benchmarks evaluated with CLP, DARE-EEG achieves the highest balanced accuracy on every dataset, with gains of ~2.5% on BCIC-2B (0.7471 vs. EEGPT's 0.7212), ~3% on SEEDIV (0.4357), and ~4% on MMWM (0.6490). It also outperforms classical supervised baselines (EEGNet, DeepCNN, Conformer) throughout, whereas several pre-trained competitors do not—for instance, DeepCNN surpasses BENDR and BIOT on BCIC-2A. On SEEDIV, BENDR degrades due to fixed electrode configurations, which CLP mitigates.

Ablations confirm both alignment terms matter: removing AA costs about 2.6% balanced accuracy on MMWM; removing MA costs about 1.5% on BCIC-2A. Loss analysis shows removing either term inflates the corresponding losses dramatically (AA loss up to twelve-fold). Scaling analysis fitted with saturating functions indicates diminishing returns beyond Base/Deep—an extrapolated gain of 1% would require at least 100M additional parameters. Pre-trained weights reduce convergence epochs from 48.2 to 30.6 and improve BCIC-2B balanced accuracy by ~4.9% relative to training from scratch. In subject-dependent small-sample evaluation on MMWM, DARE-EEG with pre-training reaches 0.7640 balanced accuracy, about 2.5% above EEGPT. t-SNE visualizations show clearer class clustering when MA is enabled.

Limitations and open questions

Several caveats bear directly on the reported results. First, the theoretical treatment of MIP is informal: Propositions 1–3 establish that MA makes mask invariance an explicit objective and that AA alone permits degenerate solutions, but no generalization or identifiability guarantees are provided, and discriminability is acknowledged to require contrastive negatives rather than following from mask invariance itself. Second, the empirical evidence for the MIP-violation claim in prior models is indirect—LaBraM and EEGPT are not shown to actually encode mask patterns; the argument rests on ablation deltas and loss behavior within DARE-EEG. Third, the mask-overlap constraint $r(m)$0 is a heuristic whose sensitivity is not analyzed. Fourth, the scaling-law extrapolation relies on a saturating functional form fitted over only five configurations, so conclusions about parameter efficiency beyond Deep are approximate. Fifth, downstream evaluation on TUAB/TUEV uses fine-tuning (with 20% of the training set held out as test, following prior work), so those results are not strictly comparable to the CLP-only protocol used elsewhere. Finally, whether dual alignment confers benefits at substantially larger data scales or higher masking ratios than tested remains open.

Conclusion

DARE-EEG identifies mask invariance as an overlooked requirement in masked-reconstruction pre-training for EEG and enforces it through a dual-alignment objective combining anchor alignment with cross-view contrastive mask alignment, complemented by a parameter-efficient conv-linear-probing strategy for heterogeneous downstream datasets. The method delivers state-of-the-art balanced accuracy across seven benchmarks—including clinical abnormality detection, motor imagery, emotion recognition, sleep staging, and cognitive workload—with modest parameter overhead, and its ablations attribute the gains specifically to the alignment constraints. The principal open question is whether the observed advantages persist and how the mask-invariance framework behaves under larger-scale pre-training and more aggressive masking regimes.