BrainStack: EEG Silent Speech Decoding

• BrainStack is a modular EEG framework that decodes silent speech using anatomically partitioned local experts and a global transformer integrator.
• It employs adaptive routing and global-to-local distillation to effectively coordinate specialized regional processing and improve classification accuracy.
• The design integrates CNN-based region-specific encoders with a transformer that captures long-range dependencies, outperforming baselines on the SilentSpeech-EEG benchmark.

BrainStack denotes, most specifically, a functionally guided neuro-mixture-of-experts framework for EEG-based language decoding that models the brain as a set of anatomically partitioned specialist modules coordinated by a global integrator. In its primary technical formulation, BrainStack was introduced for silent-speech or covert-articulation word decoding from EEG, combining regional experts, a transformer-based whole-brain expert, adaptive routing, and global-to-local distillation (Zhao et al., 29 Jan 2026). In adjacent work, the same name also functions as a conceptual label for stacked, modular, brain-inspired systems in which local specialization, global coordination, memory stratification, or swarm-level composition are treated as separate but interacting layers (Liu, 2020, Weigang et al., 30 Aug 2025, Bering, 26 Apr 2026).

1. Problem domain and formal task

BrainStack addresses EEG-based word-level language decoding in a silent speech setting. A subject sees a word, prepares it, and then silently repeats it without overt vocalization. The input is an EEG trial

$\mathbf{X}\in\mathbb{R}^{C\times T},$

where $C$ is the number of channels and $T$ the number of time samples, and the target is a class label

$y\in \{1,\dots,K\}.$

In the SilentSpeech-EEG benchmark introduced with the model, $K=24$ (Zhao et al., 29 Jan 2026).

The task is difficult because EEG is low-SNR, non-stationary, and highly variable across subjects. The cortical activity relevant to language and covert articulation is distributed across multiple regions rather than localized to a single source, and the measured scalp potentials are indirect mixtures shaped by nonlinear interactions across both regions and time. The paper’s central motivation is therefore architectural rather than purely parametric: most prior decoders are effectively monolithic, whereas the underlying signal source is functionally modular (Zhao et al., 29 Jan 2026).

The formal decomposition begins by grouping channels into functionally defined regions. If $\mathbf{X}_i$ denotes the submatrix for region $i$, the local representation is

$$\mathcal{I}_{\text{local}^i} = \phi_i(\mathbf{X}_i),$$

and the task-level integrated representation is written as

$$\mathcal{I}_y = \sum_{i=1}^{N} \omega_i^y \cdot \mathcal{I}_{\text{local}^i} = \sum_{i=1}^{N} \omega_i^y \cdot \phi_i(\mathbf{X}_i).$$

Final decoding is expressed as

$$\hat{y} = \arg\max_k \Bigg[ g\left( \sum_{i=1}^{N}\omega_i^y\cdot \phi_i(\mathbf{X}_i) \right) \Bigg]_k.$$

These equations are conceptual; the implemented system extends them with a dedicated global expert and adaptive expert routing (Zhao et al., 29 Jan 2026).

2. Neuro-MoE architecture

BrainStack instantiates this idea as a heterogeneous expert stack. For each EEG example, it constructs one global view over the full montage and seven regional views corresponding to Prefrontal, Frontal, Central, Left-Temporal, Right-Temporal, Parietal, and Occipital channel subsets. The main configuration therefore contains eight experts in total: seven local experts and one global expert (Zhao et al., 29 Jan 2026).

Each regional branch is a specialized CNet encoder inspired by EEGNet. It processes a region-specific input

$C$0

using temporal convolutions, depthwise spatial convolutions, separable convolutions, batch normalization, and dropout, yielding a learned embedding

$C$1

The purpose of these branches is locality: each expert sees only a subset of electrodes and can specialize in region-specific spatiotemporal motifs rather than compete with unrelated channels from the entire head (Zhao et al., 29 Jan 2026).

The global branch is CTNet, which takes the full trial

$C$2

and models long-range whole-brain dependencies. Its front end performs temporal convolution, spatial convolution, batch normalization, ELU activation, average pooling, and a final $C$3 projection to produce a token sequence

$C$4

These tokens are then passed through stacked Transformer layers with multi-head self-attention, feedforward blocks, and residual Add-Norm connections, producing a global representation

$C$5

The intended division of labor is explicit: local CNN experts capture localized neural dynamics, while the transformer-based global expert captures cross-regional dependencies beyond the receptive field of the regional branches (Zhao et al., 29 Jan 2026).

Fusion is handled by a learnable routing gate. If $C$6 denotes the output of expert $C$7, the routing coefficients satisfy

$C$8

and the routed representation is

$C$9

This routed representation is then classified by a fully connected layer and softmax. The routing is soft rather than sparse, sample-dependent rather than static, and operates at the expert level rather than at the token level (Zhao et al., 29 Jan 2026).

3. Distillation, optimization, and training dynamics

A second defining feature of BrainStack is hierarchical training with global-to-local distillation. Because the global expert observes the full montage and learns whole-brain contextual structure, it acts as a teacher for the local experts. The distillation target is not embeddings or attention maps but logits. The distillation loss is

$T$0

with temperature $T$1 (Zhao et al., 29 Jan 2026).

The total objective is

$T$2

Classification uses cross-entropy, distillation uses KL divergence, and no separate routing entropy, load-balancing, or sparsity regularizer is reported. This is notable because the model’s modular behavior is induced primarily through architectural specialization, soft routing, and teacher-style regularization rather than through explicit MoE load-balancing terms (Zhao et al., 29 Jan 2026).

Training is progressive rather than static. The schedule begins with a global warm-up phase and defines training progress as

$T$3

The fusion loss weight is annealed as

$T$4

while auxiliary weights are modulated by both training progress and the current fused loss. In the reported implementation, this becomes a 5-epoch warm-up, fusion loss weight increasing linearly from 0.2 to 1.0, global supervision weight decaying from 0.8 to 0, and regional and distillation weights modulated by training progress and fused loss. Training runs for up to 100 epochs with early stopping patience 5, using PyTorch on one NVIDIA A40 GPU with mixed precision; optimization uses SGD with learning rate $T$5, momentum $T$6, and weight decay $T$7 (Zhao et al., 29 Jan 2026).

This training strategy matters empirically. The paper reports that removing the warm-up schedule causes a substantial drop in performance, indicating that stabilization of the global pathway before full multi-objective coupling is not incidental but central to the method (Zhao et al., 29 Jan 2026).

4. SilentSpeech-EEG benchmark and empirical performance

BrainStack was introduced together with SilentSpeech-EEG (SS-EEG), presented as the largest dataset of its kind for word-level silent speech EEG decoding. The collection includes 12 recorded participants, of whom 2 were excluded due to severe contamination, leaving 10 usable participants; it contains over 120 hours of EEG recordings, 24 commonly used English words, 16 sessions per subject, approximately 6,000 trials per subject, and 60,000 total trials in the comparison table. Acquisition uses a 128-channel Neuroscan system at 1000 Hz, with 128 EEG and 8 EXG channels. Each session contains 375 silent speech trials and lasts about 40 minutes; each trial has a four-stage protocol consisting of Rest, Read, Cue, and Silent speech / imagination (Zhao et al., 29 Jan 2026).

Preprocessing consists of band-pass filtering to 1–45 Hz, segmentation into clean trial epochs, per-channel z-score normalization, and removal of trials with excessive noise or artifacts. The main evaluation uses a session-wise within-subject split with 14 sessions for training, 1 for validation, and 1 for test, all from non-overlapping temporal blocks (Zhao et al., 29 Jan 2026).

The main SS-EEG average accuracies are as follows:

Model	Average accuracy
EEGNet	28.78%
TCNet	29.50%
EEGConformer	23.89%
STTransformer	28.28%
LaBraM	18.29%
BrainStack_Homo	32.78%
BrainStack_RoI5	37.19%
BrainStack	41.87%

The strongest baseline is TCNet at 29.50%, so BrainStack improves over it by $T$8. It achieves the highest accuracy on 8 out of 10 subjects, with subject-wise accuracies ranging from 88.05% on S01 to 10.13% on S05. The model is also parameter-efficient: BrainStack uses 1.06M parameters, whereas LaBraM uses 5.80M yet reaches only 18.29%. The coarser 5-region partition underperforms the 7-region design, and the homogeneous variant underperforms the heterogeneous local–global design despite having 5.93M parameters, supporting the claim that finer-grained functional decomposition and architectural heterogeneity matter more than simply duplicating a heavy encoder (Zhao et al., 29 Jan 2026).

The ablation study isolates the main design decisions. Local-only CNet reaches 30.86%, global-only CTNet 35.50%, token-level fusion 29.41%, removal of knowledge distillation 35.69%, and removal of warm-up 33.97%, compared with 41.87% for the full model. The gain from distillation is $T$9, and the gain from warm-up is $y\in \{1,\dots,K\}.$0. The routing analysis further reports meaningful regional specialization: the Occipital expert has the strongest positive correlation with subject-level accuracy ($y\in \{1,\dots,K\}.$1), the Global expert shows a moderate negative correlation ($y\in \{1,\dots,K\}.$2), and the Temporal experts have weak overall correlation ($y\in \{1,\dots,K\}.$3) but consistently high routing weights. The paper interprets this as evidence that the router learns functionally relevant regional preferences rather than acting as a generic ensemble combiner (Zhao et al., 29 Jan 2026).

5. BrainStack as a broader modular design pattern

Beyond the 2026 Neuro-MoE model, BrainStack also names a broader architectural idea: intelligence as a stack of specialized modules plus a coordinating bottleneck. In BHN: A Brain-like Heterogeneous Network, this appears as many cortex-like units $y\in \{1,\dots,K\}.$4 producing distributed local representations $y\in \{1,\dots,K\}.$5 and one attention-network producing a global representation $y\in \{1,\dots,K\}.$6. The training objective is written in minimax form,

$y\in \{1,\dots,K\}.$7

and the paper explicitly stresses specialization through distributed objectives, self-supervision, and gradient isolation between both units and networks. The result is not a single homogeneous stack trained end-to-end with one global loss, but a modular arrangement in which local specialization and global integration are intentionally separated (Liu, 2020).

A related but higher-level use appears in “LLM-Assisted Iterative Evolution with Swarm Intelligence Toward SuperBrain.” There the architecture is explicitly layered: Subclass Brain layer, forward/backward evolution layer, Swarm Alignment / Swarm Intelligence layer, Meta-LLM distillation layer, Superclass Brain / SuperBrain layer, and Registry / infrastructure layer. The atomic unit is a persistent human–LLM dyad

$y\in \{1,\dots,K\}.$8

with an exportable cognitive signature

$y\in \{1,\dots,K\}.$9

The paper is strongest empirically on forward evolution for UAV scheduling, but the layered decomposition itself is a direct BrainStack-style formulation: local specialized units, shared memory infrastructure, coordination layer, and global synthesis layer (Weigang et al., 30 Aug 2025).

ZenBrain extends the same motif into autonomous-agent memory. It proposes seven memory layers—working, short-term, episodic, semantic, procedural, core, and cross-context—plus replay, forgetting, reconsolidation, priority modulation, and stability protection. Its reported gains are architectural rather than cosmetic: multi-layer routing beats a flat single-layer baseline by 20.7% F1 on LoCoMo and 19.5% on MemoryArena, Simulation-Selection sleep yields 37% stability improvement with 47.4% storage reduction, and TripleCopyMemory retains $K=24$0 at 30 days (Bering, 26 Apr 2026). Taken together, these papers suggest that BrainStack is not restricted to one EEG decoder; it is also a reusable research pattern in which modular locality, global integration, and lifecycle-aware coordination are made explicit rather than left implicit.

6. Data resources, infrastructure, and terminological boundaries

For BrainStack-oriented research, data standardization and infrastructure are recurrent concerns. MindBigData 2022 is relevant here because it provides a benchmark family of mostly raw, stimulus-locked EEG datasets across multiple devices and channel counts, with public availability and a prepared December 2022 Hugging Face release that includes basic Python loading code and a standardized 80% train / 20% test split. Its strengths are cross-device heterogeneity and raw signal access; its major limitation is equally explicit: the corpus is effectively single-subject, which constrains it to within-subject benchmarking and cross-device or cross-session transfer rather than subject-general claims (Vivancos et al., 2022).

At the systems layer, BRAIN: A Blockchain-based Platform for Reliable Inference and Training of Large-Scale Models is not a BrainStack model but is relevant as a trust and coordination substrate for any stack intended to support reliable large-model inference or update. Its core mechanism is a two-phase transaction design with commit-and-reveal committees and VRF-based selection, and its reported throughput is 454.4293 times greater than a naive single-phase implementation in tasks-per-second (Park et al., 2023). This is a different use of “brain-like” terminology: the paper is about reliable inference and training infrastructure rather than modular cognition.

Terminological confusion is increased by nearby names. STACK: State-Aware Reasoning Compression with Knowledge Guidance is not BrainStack; the paper explicitly states that the string “BrainStack” is not mentioned anywhere in it. STACK instead addresses state-aware chain-of-thought compression for large reasoning models, reporting a 59.9% average response length reduction with a 4.8-point accuracy improvement in the reported setting (Sui et al., 10 Apr 2026). The distinction is substantive: BrainStack, in its primary usage, is an anatomically guided Neuro-MoE for EEG language decoding; STACK is a reasoning-compression framework.

The current BrainStack literature therefore has clear boundaries and equally clear limitations. In the EEG formulation, average accuracy remains 41.87% on a 24-word classification problem, subject variability is large, and the regional decomposition is anatomically informed but fixed rather than learned. In the broader modular literature, several BrainStack-like systems remain partly conceptual: SuperBrain formalizes swarm and registry layers more fully than it demonstrates them, MindBigData provides benchmark scale but not population diversity, and the most ambitious memory-stack systems derive much of their evidence from long-horizon agent benchmarks rather than direct neuroscience tasks (Zhao et al., 29 Jan 2026, Weigang et al., 30 Aug 2025, Vivancos et al., 2022). What is stable across these variants is the underlying thesis: robust intelligent systems may benefit from being organized as interacting layers of specialized modules, shared integration mechanisms, and explicit memory or coordination substrates rather than as a single homogeneous model.