SingLEM: Single-Channel EEG Foundation Model

• SingLEM is a self-supervised EEG model that learns per-channel representations, enabling hardware agnostic operation across various montage configurations.
• It employs a hybrid CNN and hierarchical transformer architecture to capture both local spectral features and long-range temporal dependencies.
• Empirical results demonstrate that SingLEM outperforms multi-channel models in motor imagery and cognitive tasks using extensive public EEG datasets.

Searching arXiv for SingLEM and related referenced work to ground the article in the relevant papers. Searching arXiv for the SingLEM paper. Using the available arXiv metadata in the prompt for grounding, since no external search interface is exposed in this session. SingLEM, the Single-Channel Large EEG Model, is a self-supervised foundation model for electroencephalography designed to learn robust, general-purpose representations from single-channel EEG rather than from a fixed multi-channel montage. Its central premise is that single-channel pretraining makes the representation layer inherently hardware agnostic, while downstream aggregation by late fusion preserves flexibility across heterogeneous datasets, missing-channel settings, and practical low-channel deployments. The model combines convolutional processing for local structure with a hierarchical transformer for short- and long-range temporal dependencies, and it is pretrained on 71 public datasets comprising over 9,200 subjects and 357,000 single-channel hours of EEG. When used as a frozen feature extractor on six downstream motor imagery and cognitive tasks, it is reported to consistently outperform leading multi-channel foundation models and handcrafted baselines (Sukhbaatar et al., 22 Sep 2025).

1. Motivation and problem setting

SingLEM is motivated by three limitations attributed to prior EEG deep learning systems. First, current models are described as task-specific and often dependent on large labeled datasets, which restricts adaptability across tasks and cohorts. Second, emerging EEG foundation models are characterized as being tightly coupled to fixed, high-density multi-channel montages, which limits their use across heterogeneous datasets and in settings with missing channels or low-density hardware. Third, the need for retraining or fine-tuning and the use of high-dimensional feature spaces can impose a deployment burden in clinical, consumer, or wearable settings (Sukhbaatar et al., 22 Sep 2025).

Within this framing, SingLEM treats the single channel as the atomic unit of EEG representation learning. Rather than learning a representation that is entangled with a specific electrode layout, it learns per-channel representations and defers cross-channel combination to downstream late fusion. This design is presented as simultaneously addressing montage rigidity, hardware variation, and missing-channel operation. A plausible implication is that the model shifts the invariance problem from the representation learner to the aggregation stage, which is easier to adapt across datasets than a fixed multi-channel encoder.

The model is explicitly positioned against foundation models such as BENDR, BIOT, LaBraM, and CBraMod, which are described as being tied to fixed montages and/or susceptible to channel mismatch issues in evaluation. SingLEM’s single-channel formulation is intended to avoid these constraints while retaining broad transferability (Sukhbaatar et al., 22 Sep 2025).

2. Pretraining corpus and data unification

The pretraining regime is unusually broad in dataset coverage. SingLEM is pretrained on 71 public EEG datasets spanning motor imagery, cognitive tasks, sleep, epilepsy, emotion recognition, and more, with a total scale of over 9,200 subjects and 357,000 single-channel hours, described as equivalent to over 10,200 hours of multi-channel recordings (Sukhbaatar et al., 22 Sep 2025).

The datasets are unified through minimal preprocessing. The summary specifies bandpass + notch filter, resampling to 128 Hz, outlier artifact rejection, and scaling. This is significant because the model is not presented as relying on heavy task-specific signal engineering; instead, it is trained on a broad and heterogeneous corpus under a relatively standardized preprocessing pipeline. This suggests an attempt to maximize coverage of inter-subject, inter-session, and inter-paradigm variability while preserving a common sampling and filtering regime.

Because the representation is learned from individual channels, the corpus can be interpreted as a large collection of temporally structured one-dimensional signals rather than as a set of montage-specific tensors. That choice is central to the claim of hardware agnosticism: the training signal depends on channel-local temporal structure, not on a predefined global electrode arrangement.

3. Architectural organization

SingLEM uses a hybrid encoder architecture consisting of convolutional modules followed by a hierarchical transformer (Sukhbaatar et al., 22 Sep 2025). The input is a continuous EEG channel segmented into overlapping temporal tokens:

$\mathbf{T}_i = \mathbf{X}[is : is+\ell-1]$

with $\ell = 128$ samples, overlap $u = 32$, stride $s = 96$, for $\mathbf{X} \in \mathbb{R}^{1 \times S}$.

An end-of-channel (EOC) token is appended to avoid channel-sequence mixing. The encoder then proceeds in three stages.

The temporal encoder is a three-layer 1D CNN. The first layer uses a small kernel, exemplified as $K=3$, to capture high-frequency structure such as gamma-band content. The second layer uses a large kernel, exemplified as $K=61$, to capture slower components such as delta oscillations. The third layer performs channel-wise projection. Each layer is followed by LayerNorm and ELU activation. This module preserves temporal length while producing per-token spectral features.

The feature embedding module linearly projects the CNN output into an embedding space of dimension $d$, exemplified as $128$. A sliding context window of size $w$, exemplified as $\ell = 128$0 tokens, is then processed by a lightweight transformer of, for example, 4 layers, with multi-head self-attention and feed-forward blocks. An MLP bottleneck compresses the resulting representation to a compact $\ell = 128$1, exemplified as $\ell = 128$2. In the reported ablation results, this module is materially important for shorter motor imagery trials.

The global transformer encoder adds positional embeddings $\ell = 128$3 and applies a deeper transformer, described as, for example, 12 layers and 8 heads, to model long-range dependencies across the entire token sequence. The final output is projected by a linear head into a compact latent space, with $\ell = 128$4. The overall architecture therefore separates local spectral extraction, short-range contextualization, and long-range temporal modeling, rather than attempting to learn all three scales in a single block (Sukhbaatar et al., 22 Sep 2025).

4. Self-supervised objective and optimization

Pretraining follows an asymmetric masked autoencoder (MAE) paradigm. A random 50% of tokens are masked by setting them to the zero vector, and the model is trained to reconstruct both masked and unmasked tokens using a lightweight linear decoder. The asymmetry is conceptually important: the encoder, rather than the decoder, is tasked with capturing the salient signal structure, encouraging compact and informative latent representations (Sukhbaatar et al., 22 Sep 2025).

The loss has three components. The masked-token reconstruction term is

$\ell = 128$5

and the unmasked-token reconstruction term is

$\ell = 128$6

A third term preserves frequency content in the 13–50 Hz band:

$\ell = 128$7

The total objective is

$\ell = 128$8

This objective makes the pretraining signal explicitly temporal and spectral. Reconstruction of masked and unmasked tokens encourages information retention across the sequence, while the frequency-preservation term biases the model toward preserving beta/gamma structure regarded as critical in EEG.

Optimization uses AdamW with a cosine learning rate schedule, trained for 16 epochs on four A100 GPUs with batch size 1024 (Sukhbaatar et al., 22 Sep 2025).

5. Downstream use and empirical performance

In downstream evaluation, SingLEM is used as a fixed feature extractor. For each trial, the per-channel representations are concatenated through late fusion and then classified with a lightweight classifier, without further DNN fine-tuning. Evaluation is conducted on six datasets covering motor imagery and cognitive tasks, under strict leave-one-subject-out cross-validation (Sukhbaatar et al., 22 Sep 2025).

The reported accuracies are as follows:

Dataset/task	SingLEM accuracy	Best competitor in summary
Dreyer-MI-2C	75.3%	71.1%
WBCIC-MI-2C	79.6%	77.6%
WBCIC-MI-3C	68.3%	59.9%
N-back-2C	82.3%	78.1%
DSR-2C	84.7%	79.6%
WG-2C	69.9%	69.4%

Across these six tasks, SingLEM is reported to consistently outperform all multi-channel foundation models and handcrafted baselines in accuracy, macro-F1, and Cohen’s kappa, often by several percentage points (Sukhbaatar et al., 22 Sep 2025). The strongest margins in the summary appear on WBCIC-MI-3C and DSR-2C, while WG-2C shows a narrower gain.

The ablation results are also informative. Including or excluding task-specific data in pretraining yields a negligible difference, which is presented as evidence of domain-agnostic generalization. By contrast, omitting the feature embedding module produces a notable ~3% drop in motor imagery tasks with short trials, while the effect on longer cognitive tasks is minor to negligible. This pattern indicates that the hierarchical intermediate context module is particularly important when the downstream decision depends on compact temporal segments rather than extended trial structure.

6. Interpretability, neurophysiological analysis, and practical scope

A distinctive consequence of the single-channel formulation is that the model supports fine-grained neurophysiological analysis at the electrode level. The summary states that SingLEM enables mapping of per-channel discriminative power, with patterns that mirror neurophysiological expectations: the sensorimotor cortex is emphasized for motor imagery, and the prefrontal cortex for working memory and attention (Sukhbaatar et al., 22 Sep 2025).

This same analysis can also expose paradigm confounds. The summary notes that unexpectedly high performance in occipital regions can indicate dependence on visual cues rather than on the intended cognitive construct. That capability is methodologically significant because it turns the feature extractor into a diagnostic instrument for paradigm validation, not only a classifier backbone.

The practical applications named for SingLEM include brain-computer interfaces, sleep, epilepsy, or cognitive/emotion monitoring, wearable neurotechnology, neurophysiology research, and automated clinical screening, anomaly detection, or diagnosis regardless of the acquisition hardware (Sukhbaatar et al., 22 Sep 2025). The model is described as suitable for low-density, wearable, single-channel, or non-standard montages, and its frozen-feature usage avoids the resource cost of DNN fine-tuning. The summary further states that its compact representations provide a better tradeoff between capacity and efficiency than ultrahigh-dimensional models such as CBraMod.

A recurring assumption in EEG foundation modeling is that strong transfer requires fixed, high-density, multi-channel input. SingLEM directly contests that assumption by reporting that a single-channel approach can achieve state-of-the-art generalization while enabling fine-grained neurophysiological analysis and enhancing interpretability (Sukhbaatar et al., 22 Sep 2025). The source code and pretrained models are made available at the project repository listed in the paper.