NeuroRVQ: Scalable EEG Tokenization Model

• NeuroRVQ is a scalable EEG tokenization framework that uses hierarchical Residual Vector Quantization to preserve multi-scale temporal and spectral signal features.
• It integrates a multi-branch temporal encoder with shared transformer modules to generate robust discrete tokens for efficient reconstruction and downstream classification.
• The model achieves superior reconstruction fidelity across EEG bands and competitive classification performance with a compact parameter count.

NeuroRVQ is a scalable Large Brainwave Model (LBM) centered on a codebook-based tokenizer for electroencephalography (EEG), devised to preserve multi-scale temporal and spectral structure during generative masked modeling (Barmpas et al., 15 Oct 2025). Its defining components are a multi-scale feature extractor, hierarchical residual vector quantization (RVQ) codebooks, and an EEG signal phase- and amplitude-aware loss function. In the reported formulation, these elements support efficient EEG compression, accurate reconstruction across all frequency bands, and transfer to multiple downstream EEG classification settings. Closely related later work, particularly BrainRVQ, extends the same general residual-quantization paradigm toward dual-domain tokenization and hierarchical autoregressive pre-training for EEG foundation models (Cui et al., 18 Feb 2026).

1. Conceptual position within EEG foundation modeling

NeuroRVQ was introduced against the background that EEG captures neural activity across multiple temporal and spectral scales, while existing EEG foundation models trained to predict masked signal-tokens were reported to be limited by their tokenization modules (Barmpas et al., 15 Oct 2025). The specific deficiency identified is that existing neural tokenizers fail to preserve high-frequency dynamics, which in turn limits high-fidelity reconstruction. NeuroRVQ addresses that bottleneck by treating tokenization not as a secondary compression stage but as the central representational interface between raw EEG and generative masked modeling.

The architecture is organized into three stages: a Multi-Scale Temporal Encoder, a Hierarchical Residual Vector Quantization Tokenizer, and a Tokenizer Decoder with Fourier-domain reconstruction (Barmpas et al., 15 Oct 2025). Per EEG sample $X \in \mathbb{R}^{C\times T}$, the signal is divided into non-overlapping patches of length $w$, for example $w=200$ samples for $1\,\mathrm{s}$ at $200\,\mathrm{Hz}$, and then transformed into branch-specific latent tokens that are subsequently discretized by stacked RVQ codebooks. This design explicitly targets the full frequency neural spectrum rather than a single latent abstraction level.

A useful way to characterize NeuroRVQ is as a token-first EEG foundation-model pipeline. That characterization remains faithful to the reported workflow: tokenizer pre-training is performed end-to-end for reconstruction, the tokenizer is then frozen, and foundation-model pre-training is carried out on discrete tokens via masked-patch prediction (Barmpas et al., 15 Oct 2025). This suggests that NeuroRVQ’s principal technical contribution lies in the quality of the learned discrete token space as much as in the masked-modeling objective itself.

2. Multi-scale temporal encoding and hierarchical RVQ

The encoder is explicitly multi-branch. NeuroRVQ uses $S=4$ temporal branches in an inception-style module, where each branch applies two stacked sequences of $1$-D convolution, GroupNorm with $4$ groups, GELU, and $1$-D pooling (Barmpas et al., 15 Oct 2025). The four branches use filters $8\to 8$ with kernel stacks $w$0, $w$1, $w$2, and $w$3; the corresponding padding and pooling settings are chosen so that branch $w$4 is tuned to $w$5 structure and branch $w$6 to $w$7. After these operations, each branch yields a scale-specific feature map $w$8.

These features are embedded with learnable temporal embeddings $w$9 and spatial embeddings $w=200$0, then processed by a shared transformer encoder with $w=200$1 layers, $w=200$2 heads, and hidden dimension $w=200$3 to produce per-branch tokens $w=200$4 (Barmpas et al., 15 Oct 2025). The transformer is therefore shared across scales, whereas the subsequent quantization is branch-specific.

Hierarchical RVQ is the core discretization mechanism. For each branch $w=200$5, NeuroRVQ applies $w=200$6 residual-quantization steps with codebooks $w=200$7, each $w=200$8 with $w=200$9 and $1\,\mathrm{s}$0 (Barmpas et al., 15 Oct 2025). The reported encoding recursion is

$1\,\mathrm{s}$1

and decoding reconstructs the branch representation by

$1\,\mathrm{s}$2

This hierarchy implements progressively finer residual correction rather than a single hard assignment. In that sense, NeuroRVQ differs from flat tokenizers that map each patch to one codebook index. The hierarchy is intended to preserve fine structure that would otherwise be lost, particularly in higher-frequency EEG components (Barmpas et al., 15 Oct 2025).

3. Fourier-domain decoder and phase-/amplitude-aware objective

The tokenizer decoder operates in the Fourier domain. After summing the branchwise $1\,\mathrm{s}$3-tokens to reconstruct $1\,\mathrm{s}$4, NeuroRVQ applies a shared transformer decoder with $1\,\mathrm{s}$5 layers and three prediction heads for log-amplitude $1\,\mathrm{s}$6, $1\,\mathrm{s}$7, and $1\,\mathrm{s}$8, followed by an inverse Fourier transform to obtain $1\,\mathrm{s}$9 (Barmpas et al., 15 Oct 2025). This is not a purely time-domain decoder: spectral amplitude and phase are modeled directly, then projected back to waveform space.

The loss function combines amplitude, phase, temporal, and quantization terms. The reported amplitude loss is

$200\,\mathrm{Hz}$0

The phase loss uses a unit-circle representation,

$200\,\mathrm{Hz}$1

with $200\,\mathrm{Hz}$2 (Barmpas et al., 15 Oct 2025). After inverse Fourier transformation, the temporal loss is

$200\,\mathrm{Hz}$3

The total tokenizer loss is

$200\,\mathrm{Hz}$4

with $200\,\mathrm{Hz}$5 in practice (Barmpas et al., 15 Oct 2025).

This objective is technically consequential for two reasons. First, the log-amplitude head emphasizes high frequencies by compressing spectral dynamic range. Second, the phase formulation constrains predictions on the unit circle rather than treating phase as an unconstrained scalar. The reported discussion identifies this hybrid Fourier-domain loss as one of the main reasons NeuroRVQ can reconstruct $200\,\mathrm{Hz}$6–$200\,\mathrm{Hz}$7 structure with low error (Barmpas et al., 15 Oct 2025).

4. Training protocol and foundation-model usage

Tokenizer pre-training is performed end-to-end with the reconstruction objective $200\,\mathrm{Hz}$8. The reported setup uses $200\,\mathrm{Hz}$9 public datasets plus $S=4$0 proprietary dataset comprising approximately $S=4$1 hours of motor EEG, all resampled at $S=4$2, with batch size $S=4$3, AdamW with $S=4$4 and weight decay $S=4$5, a cosine learning-rate schedule from $S=4$6 to $S=4$7, warmup for $S=4$8 epochs, and $S=4$9 training epochs on $1$0 Tesla V100 GPUs (Barmpas et al., 15 Oct 2025).

Foundation-model pre-training then uses the discrete tokens from the frozen tokenizer. The reported objective is masked-patch prediction with symmetric spatial inverse masking, with approximately $1$1 of patches masked; training uses batch size $1$2, AdamW with weight decay $1$3, cosine learning rate $1$4, warmup for $1$5 epochs, and total duration $1$6 epochs (Barmpas et al., 15 Oct 2025). Fine-tuning is evaluated on motor ($1$7 classes), memory ($1$8 classes), sleep ($1$9 classes), and eyes-open/closed ($4$0 classes), using $4$1-fold subject-independent cross-validation, batch size $4$2, AdamW with weight decay $4$3, learning rate $4$4, a linear learning-rate schedule with warmup $4$5 epochs and decay factor $4$6, and $4$7 epochs with early stopping on sleep.

This training sequence places NeuroRVQ in a specific class of EEG systems: the tokenizer is optimized first for faithful signal reconstruction, then reused as a fixed discretizer for representation learning. A plausible implication is that tokenizer quality directly conditions the ceiling of the downstream foundation model, which is consistent with the paper’s emphasis on the tokenization bottleneck (Barmpas et al., 15 Oct 2025).

5. Reconstruction fidelity and downstream performance

The reported reconstruction results are unusually strong across EEG bands. After $4$8 epochs, in-distribution validation MSE is reported as $4$9 on Raw, $1$0 on Delta, $1$1 on Theta, $1$2 on Alpha, $1$3 on Beta, and $1$4 on Gamma, compared with LaBraM values of $1$5, $1$6, $1$7, $1$8, $1$9, and $8\to 8$0, respectively (Barmpas et al., 15 Oct 2025). Out-of-distribution results are likewise favorable: on Memory, Raw MSE is $8\to 8$1 and Gamma MSE is $8\to 8$2, compared with $8\to 8$3 and $8\to 8$4 for “LaBraM (ours)”; on Motor, Raw MSE is $8\to 8$5 and Gamma MSE is $8\to 8$6, compared with $8\to 8$7 and $8\to 8$8 (Barmpas et al., 15 Oct 2025).

These numbers directly address a common misconception about codebook tokenizers for EEG, namely that they mainly preserve low-frequency rhythms while degrading fast dynamics. NeuroRVQ was proposed precisely because existing tokenizers were said to fail at preserving high-frequency dynamics, and the reported gamma-band MSE values are presented as evidence that the redesigned tokenizer materially changes that regime (Barmpas et al., 15 Oct 2025).

Downstream balanced-accuracy results show the same pattern. NeuroRVQ reports $8\to 8$9 on Motor, $w$00 on Memory, $w$01 on Sleep, and $w$02 on Eyes, with mean $w$03 and a $w$04M backbone (Barmpas et al., 15 Oct 2025). The comparison table lists NeuroGPT at mean $w$05 with $w$06M parameters, CBraMod at $w$07 with $w$08M, LaBraM at $w$09 with $w$10M, and EEGPT at $w$11 with $w$12M. The paper summarizes this as the highest mean accuracy, at approximately $w$13 above NeuroGPT, with a compact backbone.

6. BrainRVQ as a dual-domain elaboration of the NeuroRVQ paradigm

BrainRVQ is a later high-fidelity EEG foundation model that replaces standard masked modeling with a Dual-Domain Residual Vector Quantization (DD-RVQ) tokenizer and a hierarchical autoregressive pre-training objective (Cui et al., 18 Feb 2026). Its design can be read as a NeuroRVQ-style extension in which the token space is explicitly factorized into temporal waveform structure and spectral content. At sample rate $w$14, it uses patch length $w$15 samples ($w$16), splits $w$17 into non-overlapping patches $w$18, and encodes each patch with a shared encoder $w$19. The temporal branch uses $w$20 directly, while the spectral branch computes the DFT, keeps amplitude $w$21, discards phase $w$22 in the encoder, and uses phase as a reconstruction target.

Each branch has an independent RVQ cascade. For layer $w$23, codebooks $w$24 are learned with EMA updates. Quantization proceeds by residual subtraction,

$w$25

and the quantized vector is

$w$26

The temporal decoder reconstructs waveform, the frequency decoder predicts amplitude and phase, and the total tokenizer loss is

$w$27

with $w$28 and EMA decay $w$29 (Cui et al., 18 Feb 2026).

BrainRVQ’s pre-training objective is explicitly hierarchical:

$w$30

implemented with a shared transformer encoder, layer-specific heads $w$31, and teacher forcing during training. The layer-wise cross-entropy terms are weighted by $w$32, normalized so that $w$33 (Cui et al., 18 Feb 2026). Masking is not uniform; importance-guided curriculum masking scores patches using neural band ratio with power in $w$34–$w$35, artifact penalty from power outside $w$36–$w$37, Hjorth parameters, and an irregularity measure, then interpolates between random and score-based masking with $w$38, $w$39, and mask ratio $w$40.

The hierarchical code structure is empirically supported in the reported layerwise prediction accuracies: approximately $w$41 for the first layer, $w$42 for layer $w$43, and $w$44 for layer $w$45, indicating that coarser codes are more predictable than deeper residual codes (Cui et al., 18 Feb 2026). On downstream tasks, BrainRVQ is reported to outperform supervised baselines such as EEGNet and ST-Transformer as well as SSL or foundation models including BENDR, BIOT, LaBraM, and CBraMod across $w$46 datasets. Reported highlights include Mental Workload AUROC $w$47 versus $w$48, CHB-MIT Seizure AUROC $w$49 versus $w$50, TUEV Event Classification Kappa $w$51 versus $w$52, and BCICIV-2a Motor Imagery Kappa $w$53 versus $w$54 (Cui et al., 18 Feb 2026). Ablations show that removing either domain branch, collapsing RVQ to $w$55 layer, replacing autoregression with independent prediction, or using uniform masking degrades performance by $w$56–$w$57 relative.

7. Scaling behavior, limitations, and research directions

The reported NeuroRVQ discussion emphasizes three strengths: faithful reconstruction of all EEG bands from $w$58 to $w$59 with orders-of-magnitude lower MSE, a compact tokenizer and foundation model at roughly $w$60M parameters, and a hybrid Fourier-domain loss that enforces valid phase and emphasizes high frequencies through log-amplitude (Barmpas et al., 15 Oct 2025). At the same time, the paper identifies several open questions: the masking strategy could be further optimized, for example with causal or attention-guided masks; the temporal scales are fixed at four branches, whereas adaptive scales may better model subject-specific rhythms; and the method is EEG-only, so extension to ECG, EMG, or fNIRS would require co-tokenization strategies.

BrainRVQ adds complementary scaling observations. In the reported best-practice regime, performance scales smoothly with more pre-training hours from $w$61 to $w$62 and with larger model depths from $w$63 to $w$64 layers; multi-layer RVQ with $w$65 is described as critical for tasks requiring fine detail, with $w$66 layers as a sweet spot; and mask-ratio sensitivity follows an inverted-U pattern, with $w$67–$w$68 masks working best (Cui et al., 18 Feb 2026). These claims do not alter the original NeuroRVQ formulation, but they sharpen the broader design space around hierarchical EEG tokenization.

Taken together, the NeuroRVQ line of work frames EEG tokenization as a spectro-temporal discretization problem rather than a generic latent compression problem. The central idea is stable across variants: preserve neural structure at multiple scales, discretize it hierarchically, and train downstream generative models on those discrete codes. A plausible implication is that future progress will depend less on scaling masked modeling alone than on improving the structure of the token alphabet itself, whether through adaptive temporal scales, richer masking curricula, or multimodal codebooks spanning EEG and other biosignals (Barmpas et al., 15 Oct 2025).