NeuroAdapt-Bench: Neural Adaptation Benchmark

• NeuroAdapt-Bench is a benchmark paradigm that standardizes the evaluation of neural model adaptation under realistic distribution shifts.
• It addresses diverse challenges such as source-free adaptation, budgeted evaluation, and shifts across modalities and domains like EEG, connectomics, and DBS.
• The benchmark offers domain-specific instantiations with clear protocols and performance metrics to guide practical deployment and future innovation in neuroengineering.

NeuroAdapt-Bench is a benchmark label associated with standardized evaluation of adaptation under distribution shift in neuroscience and neuroengineering machine learning. In the current literature, the term appears both as the explicit name of a benchmark for test-time adaptation in EEG foundation models and as a benchmark designation, mapping, or blueprint for closely related settings, including connectomic domain adaptation, adaptive deep brain stimulation, fast EEG calibration, channel adaptation across heterogeneous montages, and hardware-aware test-time adaptation on edge devices (Lee et al., 18 Apr 2026, Sawmya et al., 8 Mar 2025, Kuzmina et al., 26 Apr 2025, Sharma et al., 2024, Kokate et al., 25 Apr 2026, Bhardwaj et al., 2022). Across these uses, the recurring problem is deployment under shift: pretrained systems encounter new subjects, sites, tasks, devices, species, modalities, or corruptions, often without source-data access, with strict label budgets, or under latency, energy, and safety constraints.

1. Naming, correspondence, and scope

Only one cited work explicitly introduces NeuroAdapt-Bench as a named benchmark: the EEG foundation model study on test-time adaptation under realistic clinical shifts (Lee et al., 18 Apr 2026). Other works position NeuroAdapt-Bench differently. The adaptive DBS paper states that, in its overview, NeuroAdapt-Bench “maps directly” onto the DBS-Gym environment (Kuzmina et al., 26 Apr 2025). The connectomics paper states that NeuroADDA “provides a concrete recipe” whose datasets, source–target protocols, label-budget regimes, metrics, and ablations “can be directly adopted to design NeuroAdapt-Bench” (Sawmya et al., 8 Mar 2025). The edge-device TTA, fast-adaptability BCI, and EEG channel-adaptation sources similarly present protocols or benchmark-ready specifications that are explicitly framed as practical blueprints for NeuroAdapt-Bench rather than as a single official suite (Bhardwaj et al., 2022, Sharma et al., 2024, Kokate et al., 25 Apr 2026).

Reference	Domain	Role of NeuroAdapt-Bench
(Lee et al., 18 Apr 2026)	EEG foundation models	Official benchmark name for TTA under realistic shifts
(Sawmya et al., 8 Mar 2025)	Connectomics	Benchmark blueprint derived from NeuroADDA
(Kuzmina et al., 26 Apr 2025)	Adaptive DBS	Direct mapping to the DBS-Gym environment
(Sharma et al., 2024)	EEG BCI calibration	Benchmark-ready fast-adaptability specification
(Kokate et al., 25 Apr 2026)	EEG channel adaptation	Controlled benchmark treated as NeuroAdapt-Bench
(Bhardwaj et al., 2022)	Edge-device TTA	Hardware-aware benchmark template

This suggests that NeuroAdapt-Bench is best understood not as one immutable dataset-and-metric package, but as a benchmark paradigm for neurodomain adaptation, with domain-specific instantiations anchored by shared evaluation principles.

2. Benchmark problem setting

A defining property across NeuroAdapt-Bench formulations is adaptation under realistic shift. In EEG TTA, the benchmark spans in-distribution subject variability on TUEV and TUAB, out-of-distribution cross-dataset and cross-task shifts on SleepEDF-78 and CHB-MIT, and an extreme modality shift from scalp EEG to Ear-EEG (Lee et al., 18 Apr 2026). In connectomics, the shift is cross-dataset transfer across six electron microscopy datasets spanning different organisms and even microscopy modalities, with C. elegans-6 being the only TEM dataset and the others SEM (Sawmya et al., 8 Mar 2025). In adaptive DBS, non-stationarity is part of the environment itself through neural drift, electrode drift, encapsulation, localized beta loci, and directional multi-contact stimulation (Kuzmina et al., 26 Apr 2025). In the fast-calibration BCI setting, adaptation is cross-subject and cross-task under a maximum of ten fine-tuning steps (Sharma et al., 2024). In the edge-device study, shift is operationalized through CIFAR-10-C corruption streams processed on embedded hardware with no labels at test time (Bhardwaj et al., 2022).

A second recurrent property is source restriction. The EEG TTA benchmark is strictly source-free during adaptation: pretrained backbones are combined with a shared classifier head, and test-time adaptation uses only unlabeled target data (Lee et al., 18 Apr 2026). NeuroADDA is source-free after initialization: once the minimum-MMD source model is selected, no source images are accessed during target adaptation, and only target images with newly acquired labels enter the loop (Sawmya et al., 8 Mar 2025). The edge-device study likewise focuses on on-device unsupervised adaptation without labeled test data and under conditions where cloud connectivity may be unavailable (Bhardwaj et al., 2022). By contrast, the BCI fast-adaptability setting is not label-free: calibration uses a small support set with 10 shots per class and evaluates performance on a disjoint query set after at most 10 updates (Sharma et al., 2024). NeuroAdapt-Bench, therefore, does not denote a single supervision regime; it denotes controlled adaptation regimes whose constraints are explicitly specified.

A third shared feature is budgeted evaluation. NeuroADDA fixes annotation budgets $A \in \{1,2,4,8,16,32,64\}$ images and a training budget $B=10{,}000$ gradient steps (Sawmya et al., 8 Mar 2025). The EEG TTA benchmark evaluates adaptation batch sizes of 64, 128, and 256 (Lee et al., 18 Apr 2026). The BCI calibration specification fixes $K \le 10$ gradient steps, 10 support samples per class, and 11 query samples per class (Sharma et al., 2024). The edge-device study varies batches of 50, 100, and 200 and reports latency, energy, and memory feasibility on Ultra96-v2, Raspberry Pi 4, and Jetson Xavier NX (Bhardwaj et al., 2022). The aDBS environment defines episode structures and complexity levels Env0, Env1, and Env2, with drift schedules and reward-based evaluation over long-horizon control (Kuzmina et al., 26 Apr 2025).

3. Principal benchmark families

The EEG foundation-model instantiation is the most direct named form of NeuroAdapt-Bench. It evaluates CBraMod, REVE-Base, REVE-Large, and TFM-Tokenizer under a common downstream classifier head, with TTA methods No-TTA, Tent, SHOT, and T3A. The datasets are TUEV, TUAB, CHB-MIT, SleepEDF-78, and EarEEG, all split patient-disjointly, with model selection on held-out validation only and test labels withheld during adaptation. Binary tasks are scored with balanced accuracy, ROC-AUC, and PR-AUC; multiclass tasks with balanced accuracy, Cohen’s $\kappa$, and weighted F1; results are reported as mean $\pm$ standard deviation across five seeds (Lee et al., 18 Apr 2026).

The connectomics formulation derives NeuroAdapt-Bench from NeuroADDA. Its task is 2D membrane segmentation with a 2D U-Net, followed by seeded watershed to produce instance segmentations evaluated with Variation of Information. The dataset suite comprises SNEMI3D, MICrONS, H01, C. elegans-1, C. elegans-6, and FlyWire, all using 1024×1024 tiles, with non-overlapping subsets from densely labeled regions where applicable. The benchmark logic is multi-source transfer: for each target, squared MMD is computed between the target and each candidate source using activations from the final down-projection layer of each source U-Net with spatial max-pooling to $\phi(x)$, the minimum-MMD source is chosen, and active or passive target adaptation proceeds under strict annotation and training budgets (Sawmya et al., 8 Mar 2025).

The adaptive DBS formulation is an RL-ready simulation benchmark. It is implemented as a Gymnasium environment built on JAX and Diffrax, with a spatially embedded Kuramoto oscillator population as a proxy for basal ganglia circuit rhythms. The environment incorporates 15 physiological attributes spanning bandwidth, spatial, and temporal domains, including low-beta and high-beta structure, beta bursting, 3D spatial embedding, beta locus localization, partial observability, directional stimulation, multi-contact electrodes, neural drift, electrode drift, encapsulation, movement-related modulation, and signal instabilities and noise. Controllers observe a sliding LFP window and act through stimulation amplitudes and directions, with evaluation centered on beta-band suppression and energy expenditure (Kuzmina et al., 26 Apr 2025).

The fast-adaptability BCI specification focuses on quick calibration with EEGNet-style CNNs on PhysioNet EEG Motor Movement/Imagery. The benchmark fixes train, validation, and test subjects to 1–87, 88–98, and 99–109, respectively; uses 64×321 inputs derived from the first 2 s of each epoch; and compares CNNs with batch normalization, CNNs with layer normalization, and FO-MAML. Calibration consists of no more than 10 fine-tuning steps on a tiny labeled support set, repeated 100 times per subject-task configuration, with step-wise metrics on a disjoint query set (Sharma et al., 2024).

The channel-adaptation formulation addresses heterogeneous electrode montages in EEG foundation models. It compares four external channel adaptation methods—Conv1d projection, spherical spline interpolation, source-space decomposition, and Riemannian re-centering—across five pretrained models, five downstream tasks, and two training regimes, probe and supervised fine-tuning. The models range from rigid-montage architectures such as BENDR and Neuro-GPT to flexible architectures such as EEGPT, LUNA, and CBraMod; the tasks span motor imagery, clinical event detection, emotion recognition, and depression screening (Kokate et al., 25 Apr 2026).

The hardware-aware edge-device formulation contributes a distinct NeuroAdapt-Bench template centered on test-time unsupervised adaptation costs. Using CIFAR-10-C corruption streams, it evaluates BN-Norm and BN-Opt on robust ResNet-18, Wide-ResNet-40-2, and ResNeXt29_32x4d, measuring prediction error, per-batch latency, per-batch energy, and memory feasibility on FPGA, Raspberry Pi, and Jetson Xavier NX platforms. Although not neurodata-specific, the protocol is explicitly presented as a practical benchmark core for NeuroAdapt-Bench because it ties adaptation algorithms to deployment-time systems constraints (Bhardwaj et al., 2022).

4. Metrics and formalism

The connectomics benchmark uses squared Maximum Mean Discrepancy for source selection and Variation of Information for evaluation. With source and target embedding distributions $P$ and $Q$, MMD is defined in standard RKHS form as

$$\operatorname{MMD}^2(P,Q)=\mathbb{E}_{x,x'\sim P}[k(x,x')] + \mathbb{E}_{y,y'\sim Q}[k(y,y')] - 2\,\mathbb{E}_{x\sim P,\,y\sim Q}[k(x,y)].$$

Lower MMD implies greater transferability. Instance-level performance is then measured by

$$\operatorname{VI}(X,Y)=H(X\mid Y)+H(Y\mid X)=H(X)+H(Y)-2I(X;Y),$$

computed between watershedded instance segmentations and ground-truth neuron labels (Sawmya et al., 8 Mar 2025).

The EEG TTA benchmark formalizes adaptation through entropy-based and prototype-based objectives. Tent minimizes predictive entropy on an unlabeled target batch,

$B=10{,}000$0

while SHOT uses

$B=10{,}000$1

T3A is optimization-free: it maintains per-class support sets $B=10{,}000$2 and class prototypes

$B=10{,}000$3

with prediction defined by

$B=10{,}000$4

These formulations matter because the benchmark’s main empirical contrast is precisely between gradient-based and optimization-free TTA (Lee et al., 18 Apr 2026).

The adaptive DBS environment defines beta-band power as

$B=10{,}000$5

with observations derived from distance-weighted local field potentials. Its primary reward is bicriteria,

$B=10{,}000$6

thereby coupling symptom suppression to stimulation energy. This design makes NeuroAdapt-Bench in the aDBS setting a control benchmark rather than a pure prediction benchmark (Kuzmina et al., 26 Apr 2025).

The BCI fast-adaptability specification adds explicitly time-resolved adaptation metrics. If $B=10{,}000$7 is the query-set accuracy after $B=10{,}000$8 gradient steps, it defines a fast-adapt score

$B=10{,}000$9

and an adaptation-curve summary

$K \le 10$0

with $K \le 10$1. The edge-device template introduces a different formalism: a weighted deployment score

$K \le 10$2

thereby making latency and power first-class benchmark outputs rather than secondary implementation notes (Sharma et al., 2024, Bhardwaj et al., 2022).

5. Empirical findings and failure modes

The EEG TTA benchmark reports that standard gradient-based TTA methods yield inconsistent gains and often degrade performance, whereas the optimization-free T3A is more stable. On TUAB, Tent produces large negative balanced-accuracy deltas across all backbones, including $K \le 10$3 for CBraMod and $K \le 10$4 for REVE-Large. On CHB-MIT, by contrast, T3A yields strong gains, reaching $K \le 10$5 for REVE-Base and $K \le 10$6 for REVE-Large. SleepEDF-78 is broadly adverse to TTA, with SHOT reducing CBraMod balanced accuracy by $K \le 10$7 and $K \le 10$8 by $K \le 10$9. Under the Ear-EEG modality shift, T3A provides modest improvements, including balanced-accuracy gains of $\kappa$0 for CBraMod and $\kappa$1 for REVE-Base (Lee et al., 18 Apr 2026).

In connectomics, minimum-MMD source selection consistently outperforms scratch training across six targets and label budgets from 1 to 64 images. The paper reports that minimum-MMD selection yields the best performance in 99.2% of cases, and that adding active learning with Median-Uncertainty improves further in 83% of those cases. The largest relative gains occur at $\kappa$2 labeled images, with 25–67% reductions in VI versus scratch; representative improvements include MICrONS from $\kappa$3 to $\kappa$4 and C. elegans-1 from $\kappa$5 to $\kappa$6. A major failure mode is source mismatch: active transfer from a maximum-MMD source is consistently worse than from the optimal source and can approach or underperform scratch at higher budgets. Another failure mode is artifact attraction: highly uncertain images often contain white or black stripes, contrast distortions, or black tiles, with Mann–Whitney U test $\kappa$7-values between $\kappa$8 and $\kappa$9 for stripe-afflicted versus non-striped FlyWire images (Sawmya et al., 8 Mar 2025).

The channel-adaptation benchmark finds a sharp architecture dependence. Rigid-montage models require external channel adaptation, whereas flexible models can match or exceed external methods natively when fine-tuned but may benefit from external adapters in frozen-encoder probing. A central empirical result is the probe–SFT asymmetry: external adaptation can cause severe negative transfer during fine-tuning of flexible models. Examples include EEGPT on PhysioNet, where native probe reaches 53.6% while Conv1d SFT drops to 34.0%, and CBraMod on TUEV, where Riemannian re-centering collapses from 39.4% probe to 16.7% SFT, which is chance. The study also reports that the 5M-parameter CBraMod outperforms models up to 31$\pm$0 larger on 4/5 datasets, with native SFT reaching $\pm$1 on MDD and SSI SFT reaching $\pm$2 on PhysioNet MI (Kokate et al., 25 Apr 2026).

The fast-adaptability BCI specification reports a different pattern: simple transfer learning with layer normalization adapts faster than FO-MAML, often reaching near-maximal performance within at most five steps and, for Activity 2, attaining its maximum by step 2. The baseline labeled “Ours (TL+LN)” improves from 83.18 to 85.88 on Activity 1, from 85.91 to 86.28 on Activity 2, from 72.73 to 79.81 on Activity 3, and from 67.73 to 71.22 on Activity 4. The paper’s qualitative conclusion is that layer normalization improves few-step calibration whereas batch normalization tends to degrade or fail to improve in this regime (Sharma et al., 2024).

The edge-device and adaptive DBS formulations foreground deployment cost and control stability. On Jetson Xavier NX GPU, WRN-AM-50 with BN-Norm is the best balanced edge configuration at 0.315 s per batch, 2.96 J, and 15.21% error, but still incurs 213 ms and 1.9 J of overhead relative to No-Adapt. The highest accuracy overall comes from RXT-AM-200 with BN-Opt at 10.15% error, but its memory cost leads to OOM in several settings (Bhardwaj et al., 2022). In DBS-Gym, SAC is the strongest RL controller across increasing environment complexity: in Env0 it reduces $\pm$3 to approximately 27% while saving approximately 12% energy, and in Env2 it maintains approximately 37% $\pm$4 with approximately 89% energy, whereas DDPG deteriorates to approximately 94% $\pm$5 in the temporal-drift setting (Kuzmina et al., 26 Apr 2025).

6. Reporting standards, misconceptions, and future directions

A common misconception is that NeuroAdapt-Bench denotes one fixed benchmark with a single modality and protocol. The literature does not support that view. Only the EEG foundation-model study uses NeuroAdapt-Bench as an official benchmark title, while the other sources explicitly describe mappings, benchmark templates, or benchmark-ready specifications (Lee et al., 18 Apr 2026, Kuzmina et al., 26 Apr 2025, Sawmya et al., 8 Mar 2025, Sharma et al., 2024, Kokate et al., 25 Apr 2026, Bhardwaj et al., 2022). A second misconception is that source-free adaptation is synonymous with fully unsupervised adaptation. In fact, source-free means that source-domain data are unavailable during adaptation; connectomics still uses labeled target acquisitions under active learning, and the BCI fast-adaptation setting uses a labeled support set (Sawmya et al., 8 Mar 2025, Sharma et al., 2024).

The reporting norms proposed across these works are unusually explicit about hidden degrees of freedom. The connectomics blueprint requires disclosure of the 6×6 squared MMD matrix, the symmetrized matrix used for clustering, and any alternative kernels or feature layers, because the original paper does not specify kernel choice or bandwidth (Sawmya et al., 8 Mar 2025). The EEG TTA benchmark fixes patient-disjoint evaluation, held-out validation for model selection, five random seeds, and method-specific hyperparameters for Tent, SHOT, and T3A (Lee et al., 18 Apr 2026). The fast-calibration BCI specification requires per-step metrics, zero-shot performance, and wall-clock calibration time $\pm$6 (Sharma et al., 2024). The edge-device template emphasizes latency, energy, peak memory, and OOM events, and recommends standardizing reset-to-pretrained between corruption streams because the original paper does not explicitly state the reset policy (Bhardwaj et al., 2022). The channel-adaptation benchmark adds nonparametric significance testing through Wilcoxon signed-rank tests with Benjamini–Hochberg correction and Friedman tests across methods (Kokate et al., 25 Apr 2026).

The future directions identified by these sources are domain-specific but structurally aligned. EEG TTA calls for physiology-aware, montage-aware, and stability-first methods, especially under severe modality and dataset shifts (Lee et al., 18 Apr 2026). Connectomics explicitly encourages future work with other architectures and controlled modality comparisons, since the present analysis covers four species and a single 2D backbone (Sawmya et al., 8 Mar 2025). The edge-device study motivates algorithm–hardware co-design, mixed-precision exploration, and broader dataset coverage such as ImageNet-C (Bhardwaj et al., 2022). The adaptive DBS environment invites richer biomarkers, multi-symptom control, model-based RL, and more physiologically detailed extensions (Kuzmina et al., 26 Apr 2025). The channel-adaptation benchmark points to multi-montage pretraining strategies, parameter-efficient fine-tuning, and public benchmark suites for heterogeneous EEG deployment (Kokate et al., 25 Apr 2026). Taken together, these directions imply that NeuroAdapt-Bench is evolving toward a broader standard for evaluating adaptation in neural data systems under the coupled pressures of distribution shift, data scarcity, and deployment realism.