Open Brain Health Benchmark (OpenBHB)

• OpenBHB is a large-scale, multi-site benchmark for machine learning on T1-weighted MRI that focuses on brain age prediction and debiasing under inter-site heterogeneity.
• It aggregates data from thousands of subjects across numerous datasets, employing varied preprocessing pipelines (FreeSurfer, CAT12) and feature representations (ROI, VBM, SBM).
• Researchers use OpenBHB to evaluate models under strict cross-site generalization protocols, balancing predictive accuracy with site invariance through approaches like deep learning and variational autoencoders.

Open Brain Health Benchmark (OpenBHB) is a multi-site, publicly available neuroimaging benchmark for age prediction and debiasing based on T1-weighted brain MRI, with particular emphasis on robustness to site and scanner heterogeneity. In the published literature, the name refers to closely related benchmark configurations rather than a single immutable release. One line of work describes a healthy-control feature benchmark with 3965 subjects from 10 sources and more than 60 acquisition sites, uniformly processed into region-wise morphometric descriptors (Ahmed et al., 2023). Another line of work treats OpenBHB as a challenge-style resource with 5330 3D brain MRI scans across 71 sites, internal and external test sets, and an explicit site-leakage criterion for evaluating cross-site generalization (Barbano et al., 2022). Later studies use OpenBHB as the structural-MRI backbone for adversarial, variational, multitask, and deeply supervised models for biological brain age estimation (Usman et al., 2024, Rehman et al., 2024, Kanwal et al., 3 Aug 2025).

1. Definition, naming, and intended use

OpenBHB is presented as a large-scale multi-site benchmark for machine learning on structural neuroimaging, centered on brain age prediction and, more broadly, on debiasing under realistic inter-site distribution shift. In one study, the full name is given as Open Big Healthy Brain (OpenBHB) and the benchmark is described as a new multi-site, publicly available resource designed for brain age prediction and debiasing from T1-weighted MRI-derived region-wise features (Ahmed et al., 2023). In challenge-oriented work, it is described as a large multi-site T1-weighted MRI resource for brain-age modeling, with performance on unseen sites treated as the critical test of generalization (Barbano et al., 2022).

The benchmark’s scientific role is defined by heterogeneity. Published descriptions emphasize aggregation across many public cohorts, multiple acquisition sites and scanners, and demographic diversity. One report characterizes the benchmark as spanning 10 datasets and subjects of European-American, European, and Asian descent (Usman et al., 2024). This makes OpenBHB less a single dataset in the narrow sense than a standardized evaluation substrate for studying how age-prediction models behave under site effects, representation shifts, and demographic stratification.

A recurrent feature of OpenBHB-based research is that chronological age is not the sole modeling target in practice. Several methods incorporate sex information either as an auxiliary task or as an explicit conditioning variable, motivated by the claim that brain structural patterns differ significantly between sexes and that these differences affect aging trajectories and vulnerability to neurodegenerative diseases (Kanwal et al., 3 Aug 2025).

2. Reported dataset configurations and access layers

Published papers report different OpenBHB configurations, reflecting distinct benchmark layers, data-access regimes, and task formulations rather than a single universally adopted split.

Reported configuration	Description	Key counts
Feature benchmark (Ahmed et al., 2023)	Healthy controls from 10 sources with uniform preprocessing and semi-automatic QC	3965 subjects; age 6–86; 80–20 split into 3172 train and 793 test
Challenge resource (Barbano et al., 2022)	Unique-subject 3D MRI benchmark with internal and external private tests	5330 scans; 71 acquisition sites
Public-access subset (Rehman et al., 2024)	Publicly available sMRI portion used in sex-aware variational work	3984 public scans; 3227 train; 757 validation; 362 internal test; 390 external test

In the region-wise regression study, the 3965-subject cohort is described as healthy controls only, after excluding 18 duplicates, with a broadly uniform gender distribution of 2079 male and 1886 female participants and an equal distribution across 10 age bins (Ahmed et al., 2023). The contributing sources are listed as ABIDE-1/ABIDE-2, GSP, CoRR, IXI, Brainomics/Localizer, MPI-Leipzig, Narratives, Neuroimaging Predictors of Creativity, and the Reading Brain Project. That same study reports train and test ages of $25.2 \pm 14.6$ and $23.8 \pm 12.8$ years, respectively.

In the contrastive-learning study, OpenBHB is framed as a benchmark with 5330 3D brain MRI scans from 71 acquisition sites, each scan belonging to a unique subject (Barbano et al., 2022). That paper places primary emphasis on official private test sets partitioned into internal subjects from sites seen during training and external subjects from sites entirely unseen during training.

Later multimodal studies adopt the 5330/71-site description and further state that 3984 scans are publicly accessible (Rehman et al., 2024). One such paper reports an age range of 16 to 86 years and describes the sex distribution as balanced across age groups. Another gives the same train/validation counts but does not explain the discrepancy between 757 validation instances and the stated 362 internal plus 390 external tests, leaving the exact partitioning under-specified (Usman et al., 2024).

3. Preprocessing, modalities, and feature representations

OpenBHB does not correspond to a single representation level. Different studies consume the benchmark through markedly different data products.

In the region-wise feature formulation, T1-weighted MRI is uniformly preprocessed using FreeSurfer and CAT12, with semi-automatic quality control before feature extraction (Ahmed et al., 2023). The CAT12 VBM pipeline performs non-linear registration to a $1.5 \text{ mm}^3$ MNI template, GM/WM/CSF segmentation, bias correction, and Jacobian modulation, after which ROI volumes are averaged on the Neuromorphometrics atlas. The FreeSurfer pipeline includes intensity normalization, skull stripping, GM/WM segmentation, hemispheric tessellation, topology correction, surface inflation, and registration to fsaverage, with parcellations derived from the Desikan–Killiany and Destrieux atlases.

That study exposes three public feature families. CAT12 ROI volumetrics use GM and CSF volumes over 142 cortical and subcortical ROIs across both hemispheres, yielding $2 \times 142 = 284$ features. FreeSurfer Desikan ROI uses seven cortical measurements across 34 regions per hemisphere, giving $2 \times 7 \times 34 = 476$ features. FreeSurfer Destrieux ROI uses the same seven measurements across 74 regions per hemisphere, giving $2 \times 7 \times 74 = 1036$ features. The reported cortical measurements are surface area, GM volume, cortical thickness, thickness standard deviation, integrated rectified mean curvature, integrated rectified Gaussian curvature, and intrinsic curvature index.

In the challenge-style formulation, OpenBHB provides three modalities derived from the same T1-weighted MRI: voxel-based morphometry (VBM), surface-based morphometry (SBM), and quasi-raw (Barbano et al., 2022). The contrastive-learning paper uses VBM gray matter volumes and explicitly states that it does not add additional preprocessing beyond the benchmark preparation.

The multimodal variational papers move to a higher abstraction level. They do not operate on full volumetric MR images in the manuscript descriptions; instead, they apply a filter-style Random Forest feature-selection step to pre-extracted modality-specific feature vectors, denoted $X_{1n} \in \mathbb{R}^{m_1}$ for sMRI and $X_{2n} \in \mathbb{R}^{m_2}$ for fMRI (Rehman et al., 2024). Crucially, one of these papers explicitly states that OpenBHB is predominantly structural MRI and does not indicate that OpenBHB itself contains fMRI; for multimodal experiments, fMRI is imported from external datasets comprising 66 scans from Sunavsky and Poppenk and 315 scans from Narratives, for a total of 381 scans (Rehman et al., 2024). Another multimodal paper uses the same 381-scan merged subset but leaves ambiguous whether the fMRI-containing datasets are native components of OpenBHB or external additions (Usman et al., 2024). A practical implication is that “OpenBHB performance” is representation-dependent: region-wise ROI vectors, VBM gray-matter maps, full 3D volumes, and sMRI-plus-external-fMRI feature stacks are all reported under the same benchmark label.

4. Evaluation protocols and benchmark metrics

OpenBHB-based evaluation is structured around two related but non-identical paradigms: standard predictive accuracy on held-out subjects and explicit measurement of cross-site leakage.

The region-wise regression paper follows an 80–20 random split stratified by gender, with the test set further divided into male and female hold-outs and model validation performed by 10-fold cross-validation (Ahmed et al., 2023). It reports MAE, RMSE, and $R^2$, and also analyzes the brain-age delta

$\Delta = \hat{y} - y.$

The same paper notes, as a calibration issue, that predictions commonly overestimate age in younger subjects and underestimate age in older subjects, and it gives a standard linear bias correction

$23.8 \pm 12.8$0

while stating that this correction is not applied in that study.

The challenge-oriented protocol is more stringent. OpenBHB defines an internal test set composed of subjects from sites seen during training and an external test set composed of subjects from sites entirely unseen during training (Barbano et al., 2022). Cross-site robustness is then assessed not only by external age-prediction error but also by site leakage, quantified as the balanced accuracy of a post-hoc logistic regression that predicts site from learned embeddings. The final challenge score is

$23.8 \pm 12.8$1

This score makes the benchmark unusual among brain-age resources: age accuracy and site invariance are jointly optimized rather than treated as unrelated desiderata.

Metric choice varies across papers. The region-wise feature study uses MAE, RMSE, and $23.8 \pm 12.8$2 (Ahmed et al., 2023); the challenge paper focuses on internal MAE, external MAE, and site BAcc (Barbano et al., 2022); the multitask adversarial variational autoencoder reports MAE, RMSE, and Pearson’s correlation coefficient (Usman et al., 2024); and the sex-aware adversarial variational autoencoder returns to MAE, RMSE, and $23.8 \pm 12.8$3, with further stratification by sex and age group (Rehman et al., 2024). This suggests that published OpenBHB results are only directly comparable when the underlying split policy, modality definition, and metric suite are aligned.

5. Representative model families and reported empirical performance

OpenBHB has supported a wide methodological range, from linear models on public ROI summaries to deep 3D contrastive learning and multimodal adversarial-variational architectures.

The region-wise baseline of Ahmed and colleagues uses concatenated CAT12, Desikan, and Destrieux feature vectors with standard regression models (Ahmed et al., 2023). The best-performing configuration is a GLM on the “all region-wise” feature vector, yielding MAE 3.25 years, RMSE 4.73 years, and $23.8 \pm 12.8$4 on the 793-subject test set. Individual feature families perform worse: CAT12 ROI with RVR reaches MAE 3.94, Desikan ROI with GLM reaches 4.23, and Destrieux ROI with GLM reaches 3.90. Pairwise concatenation improves performance relative to single families, with CAT12 + Destrieux reaching MAE 3.33. The same study reports slightly lower MAE in the male hold-out than in the female hold-out and interprets the resulting correlations as biologically plausible, with GM volume decreasing and CSF volume increasing with age.

The challenge-winning deep-learning formulation in the contrastive-learning paper defines continuous-label contrastive losses over age-conditioned kernel weights rather than discrete class positives and negatives (Barbano et al., 2022). With a 3D ResNet-18 and the proposed $23.8 \pm 12.8$5 loss using an RBF kernel with $23.8 \pm 12.8$6, the model reaches internal MAE 2.55 \pm 0.00, external MAE 3.76 \pm 0.01, BAcc 5.10 \pm 0.10, and final score 1.54. On the OpenBHB leaderboard reported in that paper, this improves over the corresponding L1 baseline, which has external MAE 4.18 and BAcc 6.7. The same paper also shows that applying ComBat before training can reduce site BAcc further in some settings while degrading external MAE substantially, indicating that aggressive harmonization may suppress age-relevant signal together with site information.

Multimodal variational work shifts from challenge-style MRI volumes to a smaller sMRI+fMRI subset. The Multitask Adversarial Variational Autoencoder (M-AVAE) separates latent variables into shared generic and modality-specific unique codes, uses multitask learning with sex classification, and reports MAE 2.77 years on the 381-subject multimodal subset (Usman et al., 2024). That paper states that M-AVAE outperforms model-agnostic baselines such as Random Forest, SVR, Gaussian Process Regression, and PLSR, as well as model-based baselines such as Multiple Kernel Learning and Incomplete Multi-Source Fusion. For M-AVAE specifically, MAE improves from 3.15 in the unimodal sMRI setting to 2.77 in the multimodal setting.

The Sex-Aware Adversarial Variational Autoencoder (SA-AVAE) pushes this line further by feeding sex directly into the regressor and regularizing shared and distinct codes with adversarial, variational, cross-reconstruction, and shared–distinct distance ratio losses (Rehman et al., 2024). On the reported OpenBHB-based experiments, SA-AVAE obtains overall MAE $23.8 \pm 12.8$7, RMSE 3.039, and $23.8 \pm 12.8$8. Performance is nearly symmetric across sex strata: male MAE $23.8 \pm 12.8$9 and female MAE $1.5 \text{ mm}^3$0. The paper also reports that multimodal SA-AVAE outperforms its unimodal sMRI-only counterpart despite training on a much smaller multimodal cohort, improving from MAE $1.5 \text{ mm}^3$1 to $1.5 \text{ mm}^3$2.

The later Deeply Supervised Multitask Autoencoder (DSMT-AE) is described as jointly optimizing brain age prediction with auxiliary sex classification and image reconstruction, using deep supervision to stabilize optimization of deep 3D models (Kanwal et al., 3 Aug 2025). In the available abstract, OpenBHB is described as “the largest multisite neuroimaging cohort combining ten publicly available datasets,” and the model is said to achieve state-of-the-art performance and robustness across age and sex subgroups, with ablation showing that each component contributes substantially. The shared description does not provide cohort statistics, split definitions, preprocessing, or numerical benchmark values for this study, so its OpenBHB usage is presently documented at the level of framework design and qualitative outcome rather than full protocol detail.

6. Methodological significance, caveats, and recurrent misconceptions

The central methodological significance of OpenBHB lies in its treatment of site heterogeneity as a first-class evaluation variable. The benchmark is not merely large; it is large in a way that forces models to contend with differing acquisition sites, scanners, field strengths, and protocols. The challenge paper formalizes this through private internal and external tests and through site-prediction balanced accuracy on learned representations (Barbano et al., 2022). This moves the benchmark beyond simple in-distribution age regression and toward explicit measurement of representation leakage.

A common misconception is that all OpenBHB results can be ranked on a single universal scale. The published record does not support that interpretation. Some studies use public region-wise features from healthy controls only, others use VBM volumes under official challenge splits, and multimodal papers further restrict to a 381-subject sMRI+fMRI subset created by merging external fMRI sources with OpenBHB-derived sMRI (Ahmed et al., 2023, Rehman et al., 2024). As a result, a lower MAE in one paper does not automatically dominate a higher MAE in another unless data representation, cohort definition, and split policy are matched.

Another recurring issue is harmonization. Several OpenBHB studies do not apply explicit statistical harmonization such as ComBat (Ahmed et al., 2023, Rehman et al., 2024). The contrastive-learning paper directly examines this trade-off and reports that ComBat can reduce site BAcc while worsening external MAE, especially for DenseNet-121, where external MAE rises from 4.43 under the proposed contrastive loss to 10.48 under ComBat preprocessing (Barbano et al., 2022). This suggests that benchmark success depends on balancing site invariance against preservation of age-relevant neuroanatomical structure rather than minimizing site information unconditionally.

Age and sex stratification are also persistent design considerations. The region-wise regression study reports fewer elders and more outliers in older subjects, attributing this partly to right-skewed age distribution (Ahmed et al., 2023). The sex-aware variational study reports progressively lower MAE from AE to AAE to VAE to AVAE to SA-AVAE, with similar gains in male and female subgroups and improved robustness across age bins labeled “G1 under 25,” “G2 25–35,” “G3 35–45,” and “G4 45–55” (Rehman et al., 2024). A plausible implication is that OpenBHB has evolved from a pure age-regression benchmark into a platform for examining how nuisance structure, subgroup effects, and auxiliary biological variables interact in representation learning.

From a reproducibility standpoint, the benchmark is only as transparent as the surrounding paper. The regression-feature study is explicit about feature definitions, train/test counts, and preprocessing (Ahmed et al., 2023). The contrastive-learning study gives architecture, optimizer, batch size, and training duration but does not supply preprocessing scripts or weights (Barbano et al., 2022). The multimodal M-AVAE paper releases code at https://github.com/engrussman/MAVAE, whereas the sex-aware SA-AVAE paper does not provide code, seeds, or numerical loss weights $1.5 \text{ mm}^3$3 and $1.5 \text{ mm}^3$4 (Usman et al., 2024, Rehman et al., 2024). OpenBHB therefore functions both as a benchmark and as a test of reporting discipline: its cross-paper interpretability depends critically on how precisely authors specify which OpenBHB layer, split, modality definition, and evaluation protocol they actually use.