Brain Tokenizer for EEG Signal Analysis

• Brain Tokenizer is a discrete representation approach that converts continuous, noisy single-channel EEG signals into tokens capturing distinct time-frequency motifs.
• It employs a dual-branch pipeline combining STFT-based frequency embedding with temporal convolutions to mitigate frequency collapse and enhance interpretability.
• Benchmark results demonstrate improved performance and parameter efficiency on EEG tasks, supporting masked pretraining and token-based retrieval.

Brain Tokenizer most directly refers to the TFM-Tokenizer introduced in "Single-Channel EEG Tokenization Through Time-Frequency Modeling," a framework that transforms continuous, noisy electroencephalogram (EEG) signals into a sequence of discrete tokens for downstream EEG analysis (Pradeepkumar et al., 22 Feb 2025). In this formulation, a finite vocabulary $\mathcal V$ is learned so that each token encodes a characteristic time-frequency motif within a short window, providing a discrete alternative to continuous embeddings and enabling masked-token pretraining, retrieval, compression, and interpretation. In a broader sense, subsequent work applies closely related tokenization ideas to fMRI and 3D brain MRI, but the term "Brain Tokenizer" is most specifically associated with the single-channel EEG setting of TFM-Tokenizer (Pradeepkumar et al., 22 Feb 2025).

1. Conceptual basis and problem setting

EEG signals are continuous, high-dimensional, and dominated by noisy, low-frequency components, while clinically relevant events often appear as brief, high-frequency bursts or class-specific oscillatory motifs (Pradeepkumar et al., 22 Feb 2025). Conventional approaches typically rely on continuous embeddings and inter-channel dependencies; the cited work identifies two drawbacks in that regime: frequency collapse, in which models gravitate toward dominant low-frequency bands and fail to preserve diverse spectral content, and poor interpretability and scalability, because continuous embeddings lack a discrete symbolic structure.

The Brain Tokenizer addresses this by learning a finite vocabulary of discrete brain tokens. For a single-channel recording $x\in\mathbb R^T$, the tokenizer is formulated as

$$f_{\rm tokenizer}: \mathbb R^T \longrightarrow \bigl\{v_i\bigr\}_{i=1}^N,\quad v_i\in\mathcal V,\;|\mathcal V|=K,$$

where each $v_i$ summarizes both spectral shape and waveform within the $i$-th segment (Pradeepkumar et al., 22 Feb 2025). The central hypothesis is that critical time-frequency features can be effectively captured from a single channel, rather than requiring explicit inter-channel modeling.

This design reframes EEG representation learning around discrete symbols rather than dense continuous states. The paper explicitly links that shift to substantial noise filtering, data compression, and a bridge to NLP-style masked-token pretraining and retrieval (Pradeepkumar et al., 22 Feb 2025). A common misconception is that EEG tokenization must depend primarily on multi-channel structure; the single-channel formulation challenges that assumption by treating intra-channel time-frequency motifs as the primitive units of representation.

2. Signal processing pipeline and tokenizer architecture

The TFM-Tokenizer converts a single-channel time series into jointly time-frequency patches, encodes each patch into a latent embedding, and vector-quantizes that embedding against a learnable codebook (Pradeepkumar et al., 22 Feb 2025). The pipeline begins with a short-time Fourier transform (STFT) spectrogram

$$\mathbf S(\omega,\tau) = \Bigl|\sum_{l=0}^{L-1}x(\tau H +l)\,w(l)\,\mathrm e^{-j2\pi\omega l/L}\Bigr|,$$

with window length $L$, hop $H$, and $F=L/2$ frequency bins. In parallel, the raw signal is sliced into $N$ time patches $x\in\mathbb R^T$0 of length $x\in\mathbb R^T$1 and stride $x\in\mathbb R^T$2 (Pradeepkumar et al., 22 Feb 2025).

Each spectrogram column $x\in\mathbb R^T$3 is divided into $x\in\mathbb R^T$4 contiguous frequency patches $x\in\mathbb R^T$5, where $x\in\mathbb R^T$6. A frequency patch encoder projects each patch according to

$x\in\mathbb R^T$7

A light-weight transformer along the frequency axis mixes these $x\in\mathbb R^T$8 patches, after which a gated patch-wise aggregation computes a single frequency embedding per window: $x\in\mathbb R^T$9 This branch is explicitly frequency-aware and is intended to mitigate the low-frequency bias described as frequency collapse (Pradeepkumar et al., 22 Feb 2025).

In parallel, each raw EEG patch $$f_{\rm tokenizer}: \mathbb R^T \longrightarrow \bigl\{v_i\bigr\}_{i=1}^N,\quad v_i\in\mathcal V,\;|\mathcal V|=K,$$0 is passed through two small 1D convolution layers with kernel size $$f_{\rm tokenizer}: \mathbb R^T \longrightarrow \bigl\{v_i\bigr\}_{i=1}^N,\quad v_i\in\mathcal V,\;|\mathcal V|=K,$$1, strides $$f_{\rm tokenizer}: \mathbb R^T \longrightarrow \bigl\{v_i\bigr\}_{i=1}^N,\quad v_i\in\mathcal V,\;|\mathcal V|=K,$$2, GeLU activations, and group-norm, producing a temporal embedding $$f_{\rm tokenizer}: \mathbb R^T \longrightarrow \bigl\{v_i\bigr\}_{i=1}^N,\quad v_i\in\mathcal V,\;|\mathcal V|=K,$$3 (Pradeepkumar et al., 22 Feb 2025). The temporal and frequency embeddings are concatenated, projected back to $$f_{\rm tokenizer}: \mathbb R^T \longrightarrow \bigl\{v_i\bigr\}_{i=1}^N,\quad v_i\in\mathcal V,\;|\mathcal V|=K,$$4, and processed by a 2-layer temporal transformer with dimension $$f_{\rm tokenizer}: \mathbb R^T \longrightarrow \bigl\{v_i\bigr\}_{i=1}^N,\quad v_i\in\mathcal V,\;|\mathcal V|=K,$$5 and 8 heads to capture longer-range context.

Vector quantization is then performed against a learnable codebook $$f_{\rm tokenizer}: \mathbb R^T \longrightarrow \bigl\{v_i\bigr\}_{i=1}^N,\quad v_i\in\mathcal V,\;|\mathcal V|=K,$$6 with $$f_{\rm tokenizer}: \mathbb R^T \longrightarrow \bigl\{v_i\bigr\}_{i=1}^N,\quad v_i\in\mathcal V,\;|\mathcal V|=K,$$7 entries of dimension $$f_{\rm tokenizer}: \mathbb R^T \longrightarrow \bigl\{v_i\bigr\}_{i=1}^N,\quad v_i\in\mathcal V,\;|\mathcal V|=K,$$8 (Pradeepkumar et al., 22 Feb 2025). No positional encodings are added, since EEG motifs are described as location-invariant. The resulting codebook entries constitute the discrete brain tokens.

3. Token learning objective and transformer integration

TFM-Tokenizer is trained in a masked reconstruction fashion akin to VQ-VAE (Pradeepkumar et al., 22 Feb 2025). During training, both entire frequency bands, with group size $$f_{\rm tokenizer}: \mathbb R^T \longrightarrow \bigl\{v_i\bigr\}_{i=1}^N,\quad v_i\in\mathcal V,\;|\mathcal V|=K,$$9, and time windows are masked, using a time-window mask ratio of $v_i$0. The masked spectrogram $v_i$1 and the full signal $v_i$2 are encoded and quantized, then passed through an 8-layer transformer decoder and a linear head to reconstruct the original spectrogram.

The total tokenizer loss is

$v_i$3

where $v_i$4 stops gradients, $v_i$5 is the chosen codebook vector for patch $v_i$6, and $v_i$7 are typically $v_i$8 (Pradeepkumar et al., 22 Feb 2025). An exponential moving average update is used to stabilize codebook learning.

After tokenizer training, the same discrete vocabulary is used within a TFM-Encoder (Pradeepkumar et al., 22 Feb 2025). Each EEG channel is discretized into $v_i$9 tokens, those indices are looked up in the shared 64-dimensional codebook, and the embeddings are flattened across channels. Learned channel embeddings, optional position embeddings, and a prepended $i$0 token are then added before a lightweight linear-attention transformer with 4 layers, 8 heads, and hidden dimension 64 processes the sequence.

The encoder is pretrained with masked-token prediction. Approximately $i$1 of tokens across channels are randomly masked, and the model predicts the correct codebook index via

$i$2

The resulting representation is then fine-tuned for classification (Pradeepkumar et al., 22 Feb 2025). This workflow places the Brain Tokenizer at the interface between signal processing, vector quantization, and transformer pretraining.

4. Benchmark performance and parameter efficiency

The Brain Tokenizer and TFM-Encoder were evaluated on four EEG benchmarks: TUEV for 6-way EEG event classification, TUAB for binary abnormality detection, CHB-MIT for binary seizure detection, and IIIC Seizure for 6-way seizure type classification (Pradeepkumar et al., 22 Feb 2025). Across all tasks, TFM-Tokenizer+TFM-Encoder outperformed the continuous-embedding baselines BIOT and LaBraM while using approximately 1.9 M parameters, compared with 3.2–5.8 M parameters for the baselines.

Benchmark	Reported metrics	Reported result
TUEV	BA, Cohen’s $i$3	BA rose from 0.468 to 0.494; $i$4 from 0.489 to 0.534
TUAB	BA, AUC-PR, AUROC	BA from 0.772 to 0.815; AUC-PR from 0.850 to 0.895; AUROC from 0.853 to 0.890
CHB-MIT	AUC-PR, AUROC	AUC-PR from 0.313 to 0.338; AUROC from 0.846 to 0.884
IIIC	$i$5, weighted-F1	$i$6 from 0.455 to 0.499; weighted F1 from 0.539 to 0.585

On TUEV, the reported gain corresponds to a 5% increase in balanced accuracy and a 9% increase in Cohen’s $i$7 over baselines (Pradeepkumar et al., 22 Feb 2025). On TUAB, balanced accuracy increased by 5%; on CHB-MIT, AUC-PR improved by 8% and AUROC by 4.5%; on IIIC, $i$8 rose by 9.5% and weighted F1 by 8.5%. The paper characterizes these results as outperforming state-of-the-art methods across the four datasets.

A plausible implication is that the discrete vocabulary is not merely a compression device but a regularizing representation that improves both transfer and efficiency. That interpretation is consistent with the reported gains relative to larger continuous-embedding models, although the paper’s concrete claim is confined to the benchmark outcomes and parameter counts (Pradeepkumar et al., 22 Feb 2025).

5. Interpretability, token semantics, and retrieval

Interpretability is a central claim of the Brain Tokenizer because each codebook entry corresponds to a concrete time-frequency motif in a 1 s window (Pradeepkumar et al., 22 Feb 2025). The paper reports class-specific token usage on TUEV: the PLED class consistently activates token 4035, which captures a dip-rise waveform, while GPED samples share tokens 5096 and 3751 that encode periodic sharp spikes. These examples are presented as evidence that the learned codebook maps clinically meaningful waveform structure into discrete symbols.

Several quantitative analyses reinforce that interpretation. Class-Token Uniqueness, defined as the fraction of tokens exclusively used by one class, increases from 0.034% for LaBraM’s neural tokenizer to 2.14% for TFM-Tokenizer on TUEV (Pradeepkumar et al., 22 Feb 2025). In token-based retrieval using Jaccard similarity, precision@1 rises from much less than 20% to approximately 60%. Spectral entropy of the token sequence, used to measure the variety of transition frequencies, is approximately 0.26 for the frequency-path encoder and approximately 0.14 for the temporal-only path, which the paper interprets as evidence of richer spectral modeling in the frequency-aware branch.

These analyses directly address another recurring concern about discrete tokenization: that quantization may erase physiologically important variation. In the reported results, quantization instead appears to make class-distinctive structure more legible. This suggests that the learned vocabulary functions as a symbolic interface between raw EEG and clinically relevant motifs, rather than as a lossy discretization that obscures them.

6. Related tokenization frameworks in brain signal and imaging models

Brain tokenization has subsequently been extended beyond the single-channel EEG setting of TFM-Tokenizer. In fMRI, "Brain-Semantoks: Learning Semantic Tokens of Brain Dynamics with a Self-Distilled Foundation Model" introduces a semantic tokenizer that converts parcellated fMRI time series $i$9 into tokens representing functional networks over short temporal patches (Gijsen et al., 12 Dec 2025). Its design uses predefined functional networks, multi-scale temporal encoding, and a student–teacher self-distillation objective with a coding-rate regularizer. On external downstream evaluation with a frozen teacher encoder and linear probing, Brain-Semantoks wins 8/9 tasks, and its scaling analysis reports power-law increases with more unlabeled data and no sign of early saturation for out-of-distribution tasks (Gijsen et al., 12 Dec 2025).

In EEG foundation modeling, "CodeBrain: Bridging Decoupled Tokenizer and Multi-Scale Architecture for EEG Foundation Model" introduces a TFDual-Tokenizer that independently tokenizes temporal and frequency components (Ma et al., 10 Jun 2025). The model maintains two separate codebooks,

$$\mathbf S(\omega,\tau) = \Bigl|\sum_{l=0}^{L-1}x(\tau H +l)\,w(l)\,\mathrm e^{-j2\pi\omega l/L}\Bigr|,$$0

and each patch therefore carries a pair of discrete codes, expanding the joint vocabulary from $$\mathbf S(\omega,\tau) = \Bigl|\sum_{l=0}^{L-1}x(\tau H +l)\,w(l)\,\mathrm e^{-j2\pi\omega l/L}\Bigr|,$$1 to $$\mathbf S(\omega,\tau) = \Bigl|\sum_{l=0}^{L-1}x(\tau H +l)\,w(l)\,\mathrm e^{-j2\pi\omega l/L}\Bigr|,$$2 (Ma et al., 10 Jun 2025). The resulting tokens supervise EEGSSM through masked prediction, while cross-domain token co-occurrence is used for interpretability. The paper reports comprehensive experiments on 10 public EEG datasets with linear probing (Ma et al., 10 Jun 2025).

In structural imaging, "BrainG3N: A Dual-Purpose Tokenizer for Controllable 3D Brain MRI Generation" applies tokenization to fully volumetric 3D brain MRI (Puyvelde et al., 17 Jun 2026). Its tokenizer is explicitly decoupled: a frozen 3D MAE encoder produces clinically informative embeddings from $$\mathbf S(\omega,\tau) = \Bigl|\sum_{l=0}^{L-1}x(\tau H +l)\,w(l)\,\mathrm e^{-j2\pi\omega l/L}\Bigr|,$$3 volumes using non-overlapping $$\mathbf S(\omega,\tau) = \Bigl|\sum_{l=0}^{L-1}x(\tau H +l)\,w(l)\,\mathrm e^{-j2\pi\omega l/L}\Bigr|,$$4 patches, and a dedicated 3D CNN decoder reconstructs voxels from a linear projection of those embeddings (Puyvelde et al., 17 Jun 2026). On a 23-task linear-probing benchmark, the encoder matches or outperforms BrainIAC, BrainSegFounder, and MedicalNet on 21 of 23 tasks, while the same latent space supports conditional generation and patient-specific longitudinal forecasting (Puyvelde et al., 17 Jun 2026).

Taken together, these systems indicate that brain tokenization is not restricted to one modality or one objective. This suggests an emerging family of architectures in which discrete or semantically aggregated tokens serve as the interface between noisy biological measurements and foundation-model objectives, with modality-specific choices determining whether the primary emphasis is interpretability, transfer, controllable generation, or all three.

7. Limitations and open directions

For TFM-Tokenizer, the principal limitation is the fixed windowing scheme (Pradeepkumar et al., 22 Feb 2025). If a biologically meaningful motif spans two adjacent windows, the motif may be split into distinct tokens, fragmenting the representation. The current tokenizer also does not adapt window length to signal content. These are architectural constraints rather than training instabilities, and they delimit the kinds of temporal structure that can be captured by a fixed discrete vocabulary.

The future directions identified in the paper are specific. They include adaptive and hierarchical windowing, such as variable-length segments; joint multi-channel vector quantization to recover spatial patterns; incorporation of wavelet-based decompositions for better time-frequency localization; and fusion of discrete EEG tokens with clinical metadata or multi-modal brain recordings (Pradeepkumar et al., 22 Feb 2025). Each of these extensions preserves the central premise of the Brain Tokenizer while relaxing a different bottleneck in the current design.

Within the broader literature, related limitations appear in modality-specific form. Brain-Semantoks emphasizes stabilization against low signal-to-noise time series and early-training collapse, while BrainG3N notes acquisition confounders and longitudinal slope attenuation (Gijsen et al., 12 Dec 2025); (Puyvelde et al., 17 Jun 2026). A plausible implication is that future brain tokenizers will increasingly separate representation learning from reconstruction, introduce adaptive segmentation, and combine token-based abstractions with stronger controls for site, modality, and temporal heterogeneity.