--- title: 'NeuroRVQ: Scalable EEG Tokenization Model' url: https://www.emergentmind.com/topics/neurorvq type: topic --- # NeuroRVQ: Scalable EEG Tokenization Model NeuroRVQ is a scalable Large Brainwave Model (LBM) centered on a codebook-based tokenizer for electroencephalography (EEG), devised to preserve multi-scale temporal and spectral structure during generative masked modeling [2510.13068]. Its defining components are a multi-scale feature extractor, hierarchical residual vector quantization (RVQ) codebooks, and an EEG signal phase- and amplitude-aware loss function. In the reported formulation, these elements support efficient EEG compression, accurate reconstruction across all frequency bands, and transfer to multiple downstream EEG classification settings. Closely related later work, particularly BrainRVQ, extends the same general residual-quantization paradigm toward dual-domain tokenization and hierarchical autoregressive pre-training for EEG foundation models [2602.16951]. ## 1. Conceptual position within EEG foundation modeling NeuroRVQ was introduced against the background that EEG captures neural activity across multiple temporal and spectral scales, while existing EEG foundation models trained to predict masked signal-tokens were reported to be limited by their tokenization modules [2510.13068]. The specific deficiency identified is that existing neural tokenizers fail to preserve high-frequency dynamics, which in turn limits high-fidelity reconstruction. NeuroRVQ addresses that bottleneck by treating tokenization not as a secondary compression stage but as the central representational interface between raw EEG and generative masked modeling. The architecture is organized into three stages: a Multi-Scale Temporal Encoder, a Hierarchical Residual Vector Quantization Tokenizer, and a Tokenizer Decoder with Fourier-domain reconstruction [2510.13068]. Per EEG sample \(X \in \mathbb{R}^{C\times T}\), the signal is divided into non-overlapping patches of length \(w\), for example \(w=200\) samples for \(1\,\mathrm{s}\) at \(200\,\mathrm{Hz}\), and then transformed into branch-specific latent tokens that are subsequently discretized by stacked RVQ codebooks. This design explicitly targets the full frequency neural spectrum rather than a single latent abstraction level. A useful way to characterize NeuroRVQ is as a token-first EEG foundation-model pipeline. That characterization remains faithful to the reported workflow: tokenizer pre-training is performed end-to-end for reconstruction, the tokenizer is then frozen, and foundation-model pre-training is carried out on discrete tokens via masked-patch prediction [2510.13068]. This suggests that NeuroRVQ’s principal technical contribution lies in the quality of the learned discrete token space as much as in the masked-modeling objective itself. ## 2. Multi-scale temporal encoding and hierarchical RVQ The encoder is explicitly multi-branch. NeuroRVQ uses \(S=4\) temporal branches in an inception-style module, where each branch applies two stacked sequences of \(1\)-D convolution, GroupNorm with \(4\) groups, GELU, and \(1\)-D pooling [2510.13068]. The four branches use filters \(8\to 8\) with kernel stacks \((21\to 9)\), \((15\to 7)\), \((9\to 5)\), and \((5\to 3)\); the corresponding padding and pooling settings are chosen so that branch \(1\) is tuned to \(\delta/\theta\) structure and branch \(4\) to \(\beta/\gamma\). After these operations, each branch yields a scale-specific feature map \(F_i\). These features are embedded with learnable temporal embeddings \(TE \in \mathbb{R}^{P\times D}\) and spatial embeddings \(SE \in \mathbb{R}^{C\times D}\), then processed by a shared transformer encoder with \(12\) layers, \(10\) heads, and hidden dimension \(H=200\) to produce per-branch tokens \(p_1,\ldots,p_S \in \mathbb{R}^D\) [2510.13068]. The transformer is therefore shared across scales, whereas the subsequent quantization is branch-specific. Hierarchical RVQ is the core discretization mechanism. For each branch \(i\), NeuroRVQ applies \(N=8\) residual-quantization steps with codebooks \(V_{i,1},\ldots,V_{i,N}\), each \(V_{i,j}\in\mathbb{R}^{K\times D}\) with \(K=8\,192\) and \(D=128\) [2510.13068]. The reported encoding recursion is $$ p_i^{(1)} \leftarrow p_i,\qquad z_{i,j}=\arg\min_{v\in V_{i,j}} \|p_i^{(j)}-v\|_2,\qquad p_i^{(j+1)} \leftarrow p_i^{(j)}-z_{i,j}, $$ and decoding reconstructs the branch representation by $$ \hat p_i=\sum_{j=1}^{N} z_{i,j}. $$ This hierarchy implements progressively finer residual correction rather than a single hard assignment. In that sense, NeuroRVQ differs from flat tokenizers that map each patch to one codebook index. The hierarchy is intended to preserve fine structure that would otherwise be lost, particularly in higher-frequency EEG components [2510.13068]. ## 3. Fourier-domain decoder and phase-/amplitude-aware objective The tokenizer decoder operates in the Fourier domain. After summing the branchwise \(z\)-tokens to reconstruct \(\hat y_i\in\mathbb{R}^D\), NeuroRVQ applies a shared transformer decoder with \(3\) layers and three prediction heads for log-amplitude \(\log(1+\hat A)\), \(\sin\hat\phi\), and \(\cos\hat\phi\), followed by an inverse Fourier transform to obtain \(\hat X\in\mathbb{R}^w\) [2510.13068]. This is not a purely time-domain decoder: spectral amplitude and phase are modeled directly, then projected back to waveform space. The loss function combines amplitude, phase, temporal, and quantization terms. The reported amplitude loss is $$ L_{\mathrm{amp}}=\sum_i \|\log(1+\hat A_i)-\log(1+A_i)\|_2^2. $$ The phase loss uses a unit-circle representation, $$ L_{\mathrm{phase}} = 1-\sum_i \cos(\phi_i-\hat\phi_i) + \lambda_{\mathrm{circle}} \sum_i \left( \|(\cos\hat\phi_i,\sin\hat\phi_i)\|_2-1 \right)^2, $$ with \(\lambda_{\mathrm{circle}}=0.4\) [2510.13068]. After inverse Fourier transformation, the temporal loss is $$ L_{\mathrm{temp}}=\sum_i \|\hat X_i-X_i\|_2^2. $$ The total tokenizer loss is $$ L_T=\alpha L_{\mathrm{amp}}+\beta L_{\mathrm{phase}}+\gamma L_{\mathrm{temp}}+\delta L_Q, $$ with \(\alpha=\beta=\gamma=\delta=1\) in practice [2510.13068]. This objective is technically consequential for two reasons. First, the log-amplitude head emphasizes high frequencies by compressing spectral dynamic range. Second, the phase formulation constrains predictions on the unit circle rather than treating phase as an unconstrained scalar. The reported discussion identifies this hybrid Fourier-domain loss as one of the main reasons NeuroRVQ can reconstruct \(\delta\)–\(\gamma\) structure with low error [2510.13068]. ## 4. Training protocol and foundation-model usage Tokenizer pre-training is performed end-to-end with the reconstruction objective \(L_T\). The reported setup uses \(13\) public datasets plus \(1\) proprietary dataset comprising approximately \(235\) hours of motor EEG, all resampled at \(200\,\mathrm{Hz}\), with batch size \(256\), AdamW with \(\beta=(0.9,0.999)\) and weight decay \(1\mathrm{e}{-4}\), a cosine learning-rate schedule from \(5\mathrm{e}{-5}\) to \(1\mathrm{e}{-5}\), warmup for \(10\) epochs, and \(100\) training epochs on \(4\times\) Tesla V100 GPUs [2510.13068]. Foundation-model pre-training then uses the discrete tokens from the frozen tokenizer. The reported objective is masked-patch prediction with symmetric spatial inverse masking, with approximately \(15\%\) of patches masked; training uses batch size \(64\), AdamW with weight decay \(0.05\), cosine learning rate \(5\mathrm{e}{-4}\to 1\mathrm{e}{-5}\), warmup for \(5\) epochs, and total duration \(50\) epochs [2510.13068]. Fine-tuning is evaluated on motor (\(4\) classes), memory (\(2\) classes), sleep (\(6\) classes), and eyes-open/closed (\(2\) classes), using \(10\)-fold subject-independent cross-validation, batch size \(64\), AdamW with weight decay \(0.01\), learning rate \(5\mathrm{e}{-4}\), a linear learning-rate schedule with warmup \(4\) epochs and decay factor \(0.975\), and \(20\) epochs with early stopping on sleep. This training sequence places NeuroRVQ in a specific class of EEG systems: the tokenizer is optimized first for faithful signal reconstruction, then reused as a fixed discretizer for representation learning. A plausible implication is that tokenizer quality directly conditions the ceiling of the downstream foundation model, which is consistent with the paper’s emphasis on the tokenization bottleneck [2510.13068]. ## 5. Reconstruction fidelity and downstream performance The reported reconstruction results are unusually strong across EEG bands. After \(100\) epochs, in-distribution validation MSE is reported as \(0.016\) on Raw, \(0.006\) on Delta, \(0.002\) on Theta, \(0.002\) on Alpha, \(0.005\) on Beta, and \(0.002\) on Gamma, compared with LaBraM values of \(1.071\), \(1.561\), \(0.184\), \(0.099\), \(0.122\), and \(0.020\), respectively [2510.13068]. Out-of-distribution results are likewise favorable: on Memory, Raw MSE is \(0.084\) and Gamma MSE is \(0.010\), compared with \(1.880\) and \(0.031\) for “LaBraM (ours)”; on Motor, Raw MSE is \(0.090\) and Gamma MSE is \(0.009\), compared with \(1.893\) and \(0.049\) [2510.13068]. These numbers directly address a common misconception about codebook tokenizers for EEG, namely that they mainly preserve low-frequency rhythms while degrading fast dynamics. NeuroRVQ was proposed precisely because existing tokenizers were said to fail at preserving high-frequency dynamics, and the reported gamma-band MSE values are presented as evidence that the redesigned tokenizer materially changes that regime [2510.13068]. Downstream balanced-accuracy results show the same pattern. NeuroRVQ reports \(0.700 \pm .073\) on Motor, \(0.574 \pm .027\) on Memory, \(0.728 \pm .028\) on Sleep, and \(0.869 \pm .026\) on Eyes, with mean \(0.717 \pm .038\) and a \(5.9\)M backbone [2510.13068]. The comparison table lists NeuroGPT at mean \(0.695 \pm .045\) with \(79.5\)M parameters, CBraMod at \(0.666 \pm .056\) with \(4.9\)M, LaBraM at \(0.652 \pm .047\) with \(5.8\)M, and EEGPT at \(0.566 \pm .033\) with \(25.7\)M. The paper summarizes this as the highest mean accuracy, at approximately \(3.2\%\) above NeuroGPT, with a compact backbone. ## 6. BrainRVQ as a dual-domain elaboration of the NeuroRVQ paradigm BrainRVQ is a later high-fidelity EEG foundation model that replaces standard masked modeling with a Dual-Domain Residual Vector Quantization (DD-RVQ) tokenizer and a hierarchical autoregressive pre-training objective [2602.16951]. Its design can be read as a NeuroRVQ-style extension in which the token space is explicitly factorized into temporal waveform structure and spectral content. At sample rate \(f_s=200\,\mathrm{Hz}\), it uses patch length \(P=200\) samples (\(1\,\mathrm{s}\)), splits \(X\in\mathbb{R}^{C\times T}\) into non-overlapping patches \(x_{c,a}\in\mathbb{R}^P\), and encodes each patch with a shared encoder \(f_{\mathrm{enc}}\). The temporal branch uses \(h_{c,a}\) directly, while the spectral branch computes the DFT, keeps amplitude \(A[k]=|X_{\mathrm{freq}}[k]|\), discards phase \(\phi[k]=\arg X_{\mathrm{freq}}[k]\) in the encoder, and uses phase as a reconstruction target. Each branch has an independent RVQ cascade. For layer \(l=1,\ldots,L\), codebooks \(V^{(l)}=\{v_k^{(l)}\in\mathbb{R}^p\mid k=1,\ldots,K\}\) are learned with EMA updates. Quantization proceeds by residual subtraction, $$ z^{(l)}=\arg\min_k \|r^{(l-1)}-v_k^{(l)}\|^2,\qquad r^{(l)}=r^{(l-1)}-v_{z^{(l)}}^{(l)}, $$ and the quantized vector is $$ e_q=\sum_{l=1}^{L} v_{z^{(l)}}^{(l)}. $$ The temporal decoder reconstructs waveform, the frequency decoder predicts amplitude and phase, and the total tokenizer loss is $$ L_{\mathrm{RVQ}}=L_{\mathrm{time}}+L_{\mathrm{freq}}+\beta L_{\mathrm{commit}}, $$ with \(\beta \approx 1.0\) and EMA decay \(\gamma=0.99\) [2602.16951]. BrainRVQ’s pre-training objective is explicitly hierarchical: $$ P(z_i\mid X_{\neg M})=\prod_{l=1}^{L} P(z_i^{(l)}\mid z_i^{(