---
title: Open Brain Health Benchmark (OpenBHB)
url: https://www.emergentmind.com/topics/open-brain-health-benchmark-openbhb
type: topic
---

# Open Brain Health Benchmark (OpenBHB)

Open Brain Health Benchmark (OpenBHB) is a multi-site, publicly available neuroimaging benchmark for age prediction and debiasing based on T1-weighted brain MRI, with particular emphasis on robustness to site and scanner heterogeneity. In the published literature, the name refers to closely related benchmark configurations rather than a single immutable release. One line of work describes a healthy-control feature benchmark with 3965 subjects from 10 sources and more than 60 acquisition sites, uniformly processed into region-wise morphometric descriptors [2306.05514]. Another line of work treats OpenBHB as a challenge-style resource with 5330 3D brain MRI scans across 71 sites, internal and external test sets, and an explicit site-leakage criterion for evaluating cross-site generalization [2211.08326]. Later studies use OpenBHB as the structural-MRI backbone for adversarial, variational, multitask, and deeply supervised models for biological brain age estimation [2411.10100] [2412.05632] [2508.01565].

## 1. Definition, naming, and intended use

OpenBHB is presented as a large-scale multi-site benchmark for machine learning on structural neuroimaging, centered on brain age prediction and, more broadly, on debiasing under realistic inter-site distribution shift. In one study, the full name is given as **Open Big Healthy Brain (OpenBHB)** and the benchmark is described as a new multi-site, publicly available resource designed for brain age prediction and debiasing from T1-weighted MRI-derived region-wise features [2306.05514]. In challenge-oriented work, it is described as a large multi-site T1-weighted MRI resource for brain-age modeling, with performance on unseen sites treated as the critical test of generalization [2211.08326].

The benchmark’s scientific role is defined by heterogeneity. Published descriptions emphasize aggregation across many public cohorts, multiple acquisition sites and scanners, and demographic diversity. One report characterizes the benchmark as spanning 10 datasets and subjects of European-American, European, and Asian descent [2411.10100]. This makes OpenBHB less a single dataset in the narrow sense than a standardized evaluation substrate for studying how age-prediction models behave under site effects, representation shifts, and demographic stratification.

A recurrent feature of OpenBHB-based research is that chronological age is not the sole modeling target in practice. Several methods incorporate sex information either as an auxiliary task or as an explicit conditioning variable, motivated by the claim that brain structural patterns differ significantly between sexes and that these differences affect aging trajectories and vulnerability to neurodegenerative diseases [2508.01565].

## 2. Reported dataset configurations and access layers

Published papers report different OpenBHB configurations, reflecting distinct benchmark layers, data-access regimes, and task formulations rather than a single universally adopted split.

| Reported configuration | Description | Key counts |
|---|---|---|
| Feature benchmark [2306.05514] | Healthy controls from 10 sources with uniform preprocessing and semi-automatic QC | 3965 subjects; age 6–86; 80–20 split into 3172 train and 793 test |
| Challenge resource [2211.08326] | Unique-subject 3D MRI benchmark with internal and external private tests | 5330 scans; 71 acquisition sites |
| Public-access subset [2412.05632] | Publicly available sMRI portion used in sex-aware variational work | 3984 public scans; 3227 train; 757 validation; 362 internal test; 390 external test |

In the region-wise regression study, the 3965-subject cohort is described as healthy controls only, after excluding 18 duplicates, with a broadly uniform gender distribution of 2079 male and 1886 female participants and an equal distribution across 10 age bins [2306.05514]. The contributing sources are listed as ABIDE-1/ABIDE-2, GSP, CoRR, IXI, Brainomics/Localizer, MPI-Leipzig, Narratives, Neuroimaging Predictors of Creativity, and the Reading Brain Project. That same study reports train and test ages of \(25.2 \pm 14.6\) and \(23.8 \pm 12.8\) years, respectively.

In the contrastive-learning study, OpenBHB is framed as a benchmark with 5330 3D brain MRI scans from 71 acquisition sites, each scan belonging to a unique subject [2211.08326]. That paper places primary emphasis on official private test sets partitioned into **internal** subjects from sites seen during training and **external** subjects from sites entirely unseen during training.

Later multimodal studies adopt the 5330/71-site description and further state that 3984 scans are publicly accessible [2412.05632]. One such paper reports an age range of 16 to 86 years and describes the sex distribution as balanced across age groups. Another gives the same train/validation counts but does not explain the discrepancy between 757 validation instances and the stated 362 internal plus 390 external tests, leaving the exact partitioning under-specified [2411.10100].

## 3. Preprocessing, modalities, and feature representations

OpenBHB does not correspond to a single representation level. Different studies consume the benchmark through markedly different data products.

In the region-wise feature formulation, T1-weighted MRI is uniformly preprocessed using **FreeSurfer** and **CAT12**, with semi-automatic quality control before feature extraction [2306.05514]. The CAT12 VBM pipeline performs non-linear registration to a \(1.5 \text{ mm}^3\) MNI template, GM/WM/CSF segmentation, bias correction, and Jacobian modulation, after which ROI volumes are averaged on the Neuromorphometrics atlas. The FreeSurfer pipeline includes intensity normalization, skull stripping, GM/WM segmentation, hemispheric tessellation, topology correction, surface inflation, and registration to fsaverage, with parcellations derived from the Desikan–Killiany and Destrieux atlases.

That study exposes three public feature families. **CAT12 ROI volumetrics** use GM and CSF volumes over 142 cortical and subcortical ROIs across both hemispheres, yielding \(2 \times 142 = 284\) features. **FreeSurfer Desikan ROI** uses seven cortical measurements across 34 regions per hemisphere, giving \(2 \times 7 \times 34 = 476\) features. **FreeSurfer Destrieux ROI** uses the same seven measurements across 74 regions per hemisphere, giving \(2 \times 7 \times 74 = 1036\) features. The reported cortical measurements are surface area, GM volume, cortical thickness, thickness standard deviation, integrated rectified mean curvature, integrated rectified Gaussian curvature, and intrinsic curvature index.

In the challenge-style formulation, OpenBHB provides three modalities derived from the same T1-weighted MRI: **voxel-based morphometry (VBM)**, **surface-based morphometry (SBM)**, and **quasi-raw** [2211.08326]. The contrastive-learning paper uses VBM gray matter volumes and explicitly states that it does not add additional preprocessing beyond the benchmark preparation.

The multimodal variational papers move to a higher abstraction level. They do not operate on full volumetric MR images in the manuscript descriptions; instead, they apply a filter-style **Random Forest** feature-selection step to pre-extracted modality-specific feature vectors, denoted \(X_{1n} \in \mathbb{R}^{m_1}\) for sMRI and \(X_{2n} \in \mathbb{R}^{m_2}\) for fMRI [2412.05632]. Crucially, one of these papers explicitly states that OpenBHB is predominantly structural MRI and does **not** indicate that OpenBHB itself contains fMRI; for multimodal experiments, fMRI is imported from external datasets comprising 66 scans from Sunavsky and Poppenk and 315 scans from Narratives, for a total of 381 scans [2412.05632]. Another multimodal paper uses the same 381-scan merged subset but leaves ambiguous whether the fMRI-containing datasets are native components of OpenBHB or external additions [2411.10100]. A practical implication is that “OpenBHB performance” is representation-dependent: region-wise ROI vectors, VBM gray-matter maps, full 3D volumes, and sMRI-plus-external-fMRI feature stacks are all reported under the same benchmark label.

## 4. Evaluation protocols and benchmark metrics

OpenBHB-based evaluation is structured around two related but non-identical paradigms: standard predictive accuracy on held-out subjects and explicit measurement of cross-site leakage.

The region-wise regression paper follows an 80–20 random split stratified by gender, with the test set further divided into male and female hold-outs and model validation performed by 10-fold cross-validation [2306.05514]. It reports **MAE**, **RMSE**, and **\(R^2\)**, and also analyzes the brain-age delta
\[
\Delta = \hat{y} - y.
\]
The same paper notes, as a calibration issue, that predictions commonly overestimate age in younger subjects and underestimate age in older subjects, and it gives a standard linear bias correction
\[
\hat{y}_{\mathrm{corr}} = \hat{y} - (\alpha y + \beta),
\]
while stating that this correction is not applied in that study.

The challenge-oriented protocol is more stringent. OpenBHB defines an **internal** test set composed of subjects from sites seen during training and an **external** test set composed of subjects from sites entirely unseen during training [2211.08326]. Cross-site robustness is then assessed not only by external age-prediction error but also by **site leakage**, quantified as the balanced accuracy of a post-hoc logistic regression that predicts site from learned embeddings. The final challenge score is
\[
\mathcal{L}_c = \mathrm{BAcc}^{0.3} \cdot \mathrm{MAE}_{\mathrm{ext}}.
\]
This score makes the benchmark unusual among brain-age resources: age accuracy and site invariance are jointly optimized rather than treated as unrelated desiderata.

Metric choice varies across papers. The region-wise feature study uses MAE, RMSE, and \(R^2\) [2306.05514]; the challenge paper focuses on internal MAE, external MAE, and site **BAcc** [2211.08326]; the multitask adversarial variational autoencoder reports MAE, RMSE, and Pearson’s correlation coefficient [2411.10100]; and the sex-aware adversarial variational autoencoder returns to MAE, RMSE, and \(R^2\), with further stratification by sex and age group [2412.05632]. This suggests that published OpenBHB results are only directly comparable when the underlying split policy, modality definition, and metric suite are aligned.

## 5. Representative model families and reported empirical performance

OpenBHB has supported a wide methodological range, from linear models on public ROI summaries to deep 3D contrastive learning and multimodal adversarial-variational architectures.

The region-wise baseline of Ahmed and colleagues uses concatenated CAT12, Desikan, and Destrieux feature vectors with standard regression models [2306.05514]. The best-performing configuration is a **GLM** on the “all region-wise” feature vector, yielding **MAE 3.25 years, RMSE 4.73 years, and \(R^2 = 0.90\)** on the 793-subject test set. Individual feature families perform worse: CAT12 ROI with **RVR** reaches MAE 3.94, Desikan ROI with **GLM** reaches 4.23, and Destrieux ROI with **GLM** reaches 3.90. Pairwise concatenation improves performance relative to single families, with CAT12 + Destrieux reaching MAE 3.33. The same study reports slightly lower MAE in the male hold-out than in the female hold-out and interprets the resulting correlations as biologically plausible, with GM volume decreasing and CSF volume increasing with age.

The challenge-winning deep-learning formulation in the contrastive-learning paper defines continuous-label contrastive losses over age-conditioned kernel weights rather than discrete class positives and negatives [2211.08326]. With a 3D **ResNet-18** and the proposed \(\mathcal{L}^{exp}\) loss using an RBF kernel with \(\sigma = 2\), the model reaches **internal MAE 2.55 \pm 0.00**, **external MAE 3.76 \pm 0.01**, **BAcc 5.10 \pm 0.10**, and **final score 1.54**. On the OpenBHB leaderboard reported in that paper, this improves over the corresponding L1 baseline, which has external MAE 4.18 and BAcc 6.7. The same paper also shows that applying ComBat before training can reduce site BAcc further in some settings while degrading external MAE substantially, indicating that aggressive harmonization may suppress age-relevant signal together with site information.

Multimodal variational work shifts from challenge-style MRI volumes to a smaller sMRI+fMRI subset. The **Multitask Adversarial Variational Autoencoder (M-AVAE)** separates latent variables into shared generic and modality-specific unique codes, uses multitask learning with sex classification, and reports **MAE 2.77 years** on the 381-subject multimodal subset [2411.10100]. That paper states that M-AVAE outperforms model-agnostic baselines such as Random Forest, SVR, Gaussian Process Regression, and PLSR, as well as model-based baselines such as Multiple Kernel Learning and Incomplete Multi-Source Fusion. For M-AVAE specifically, MAE improves from **3.15** in the unimodal sMRI setting to **2.77** in the multimodal setting.

The **Sex-Aware Adversarial Variational Autoencoder (SA-AVAE)** pushes this line further by feeding sex directly into the regressor and regularizing shared and distinct codes with adversarial, variational, cross-reconstruction, and shared–distinct distance ratio losses [2412.05632]. On the reported OpenBHB-based experiments, SA-AVAE obtains **overall MAE \(2.722 \pm 1.351\)**, **RMSE 3.039**, and **\(R^2 = 0.936\)**. Performance is nearly symmetric across sex strata: **male MAE \(2.727 \pm 1.345\)** and **female MAE \(2.713 \pm 1.363\)**. The paper also reports that multimodal SA-AVAE outperforms its unimodal sMRI-only counterpart despite training on a much smaller multimodal cohort, improving from **MAE \(2.906 \pm 1.528\)** to **\(2.722 \pm 1.351\)**.

The later **Deeply Supervised Multitask Autoencoder (DSMT-AE)** is described as jointly optimizing brain age prediction with auxiliary sex classification and image reconstruction, using deep supervision to stabilize optimization of deep 3D models [2508.01565]. In the available abstract, OpenBHB is described as “the largest multisite neuroimaging cohort combining ten publicly available datasets,” and the model is said to achieve state-of-the-art performance and robustness across age and sex subgroups, with ablation showing that each component contributes substantially. The shared description does not provide cohort statistics, split definitions, preprocessing, or numerical benchmark values for this study, so its OpenBHB usage is presently documented at the level of framework design and qualitative outcome rather than full protocol detail.

## 6. Methodological significance, caveats, and recurrent misconceptions

The central methodological significance of OpenBHB lies in its treatment of **site heterogeneity as a first-class evaluation variable**. The benchmark is not merely large; it is large in a way that forces models to contend with differing acquisition sites, scanners, field strengths, and protocols. The challenge paper formalizes this through private internal and external tests and through site-prediction balanced accuracy on learned representations [2211.08326]. This moves the benchmark beyond simple in-distribution age regression and toward explicit measurement of representation leakage.

A common misconception is that all OpenBHB results can be ranked on a single universal scale. The published record does not support that interpretation. Some studies use public region-wise features from healthy controls only, others use VBM volumes under official challenge splits, and multimodal papers further restrict to a 381-subject sMRI+fMRI subset created by merging external fMRI sources with OpenBHB-derived sMRI [2306.05514] [2412.05632]. As a result, a lower MAE in one paper does not automatically dominate a higher MAE in another unless data representation, cohort definition, and split policy are matched.

Another recurring issue is harmonization. Several OpenBHB studies do **not** apply explicit statistical harmonization such as ComBat [2306.05514] [2412.05632]. The contrastive-learning paper directly examines this trade-off and reports that ComBat can reduce site BAcc while worsening external MAE, especially for DenseNet-121, where external MAE rises from 4.43 under the proposed contrastive loss to 10.48 under ComBat preprocessing [2211.08326]. This suggests that benchmark success depends on balancing site invariance against preservation of age-relevant neuroanatomical structure rather than minimizing site information unconditionally.

Age and sex stratification are also persistent design considerations. The region-wise regression study reports fewer elders and more outliers in older subjects, attributing this partly to right-skewed age distribution [2306.05514]. The sex-aware variational study reports progressively lower MAE from AE to AAE to VAE to AVAE to SA-AVAE, with similar gains in male and female subgroups and improved robustness across age bins labeled “G1 under 25,” “G2 25–35,” “G3 35–45,” and “G4 45–55” [2412.05632]. A plausible implication is that OpenBHB has evolved from a pure age-regression benchmark into a platform for examining how nuisance structure, subgroup effects, and auxiliary biological variables interact in representation learning.

From a reproducibility standpoint, the benchmark is only as transparent as the surrounding paper. The regression-feature study is explicit about feature definitions, train/test counts, and preprocessing [2306.05514]. The contrastive-learning study gives architecture, optimizer, batch size, and training duration but does not supply preprocessing scripts or weights [2211.08326]. The multimodal M-AVAE paper releases code at `https://github.com/engrussman/MAVAE`, whereas the sex-aware SA-AVAE paper does not provide code, seeds, or numerical loss weights \(\mu_k\) and \(\eta_k\) [2411.10100] [2412.05632]. OpenBHB therefore functions both as a benchmark and as a test of reporting discipline: its cross-paper interpretability depends critically on how precisely authors specify which OpenBHB layer, split, modality definition, and evaluation protocol they actually use.

Source: https://www.emergentmind.com/topics/open-brain-health-benchmark-openbhb