Brain MRI Foundation Models

• Brain MRI foundation models are large-scale machine learning frameworks that use self-supervised, multi-task, and cross-modal objectives to generate reusable neuroimaging representations.
• They combine architectures like 3D CNNs, Vision Transformers, and hybrid models with techniques such as masked autoencoding and contrastive learning for improved segmentation, classification, and regression.
• Extensive training on heterogeneous MRI datasets enhances data efficiency, robustness, and domain adaptability while addressing missing modalities and cross-protocol shifts.

Brain MRI foundation models are large-scale machine learning frameworks trained via self-supervised, multi-task, or cross-modal objectives to produce highly generalizable, reusable representations from brain magnetic resonance imaging (MRI) data. These models leverage massive, heterogeneous MRI corpora covering diverse scanners, patient populations, and acquisition protocols to encode anatomical, pathological, or functional priors that transfer with minimal supervision to a wide spectrum of downstream neuroimaging tasks. Contemporary research demonstrates that such models, when properly designed and pre-trained, substantially improve generalization, data efficiency, and robustness in classification, segmentation, regression, and cross-modal retrieval applications, often surpassing conventional supervised baselines—even under cross-domain clinical shift or extreme data scarcity (Kaczmarek et al., 12 Sep 2025, Wang et al., 11 Jun 2025, Wang et al., 26 Dec 2025, Mazher et al., 27 Oct 2025, Munk et al., 13 Apr 2026).

1. Foundation Model Objectives and Architectural Paradigms

Brain MRI foundation models employ a variety of architectural backbones, including 3D convolutional networks (UNet, ResNet), Vision Transformers (ViT/Swin), and hybrid CNN–ViT constructs (Ghamizi et al., 16 Jun 2025, Mazher et al., 27 Oct 2025). Their pretraining objectives fall into several canonical families:

• Masked autoencoding (MAE/MIM): Local patch- or voxel-wise masked reconstruction, denoted

$$\mathcal{L}_{\mathrm{MAE}} = \frac{1}{|\Omega_m|}\sum_{i\in\Omega_m}\|\hat x_i - x_i\|_2^2$$

where $\Omega_m$ is the set of masked patches/voxels (Mazher et al., 27 Oct 2025, Munk et al., 13 Apr 2026, Cox et al., 2024).

• Instance/contrastive learning: Global or local feature alignment via InfoNCE or NT-Xent, e.g.,

$$\mathcal{L}_\text{contrastive} = -\log \frac{\exp(s(z_i, z_j)/\tau)}{\sum_k \exp(s(z_i, z_k)/\tau)}$$

with $z_i$ global feature vectors and $s(\cdot,\cdot)$ a similarity function (Kaczmarek et al., 12 Sep 2025, Wang et al., 26 Dec 2025).

• Hybrid (reconstruction + contrastive): Weighted summing of MAE and contrastive losses, used to balance local detail with global invariance (Munk et al., 13 Apr 2026, Koutsouvelis et al., 14 Nov 2025).
• Task-specific predictive heads: Multitask decoders for segmentation, regression (brain age), and classification, with adaptive supervision for each target (Liu et al., 30 Aug 2025, Kayser et al., 21 Dec 2025, Farahani et al., 10 Mar 2025).

Architectural variants integrate learnable modality embeddings for dynamic sequence handling (Luu et al., 4 Nov 2025), dynamic adapters for domain adaptation (Deng et al., 1 May 2025), or multi-view attention for report alignment (Kayser et al., 21 Dec 2025). Token- or prompt-based adaptation enables efficient few-shot transfer (Chen et al., 26 Feb 2026, Wang et al., 11 Jun 2025).

2. Data Composition, Preprocessing, and Heterogeneity

Foundational performance and generalizability are contingent on scale and diversity of the pretraining datasets. Leading models utilize tens of thousands to hundreds of thousands of MRI volumes spanning T1, T2, FLAIR, DWI, SWI, and contrast-enhanced sequences from both healthy and pathological cohorts (tumor, stroke, neurodegeneration, psychiatric) (Ghamizi et al., 16 Jun 2025, Luu et al., 23 Oct 2025, Munk et al., 13 Apr 2026). For example, FOMO60K includes 60,529 scans from 16 public sources, explicitly retaining protocol heterogeneity, motion artifacts, and varying slice thickness (Munk et al., 13 Apr 2026).

Standardized preprocessing pipelines comprise:

• Intensity normalization (typically z-score)
• N4 bias-field correction
• Skull-stripping (e.g., FSL BET, HD-BET)
• Affine or non-linear registration to template space (MNI152)
• Resampling to isotropic 1mm³ or dataset-matched resolutions
• Modality harmonization to ensure both multi-channel and partial-modality input handling (Mazher et al., 27 Oct 2025, Luu et al., 4 Nov 2025)

Critical analysis shows that substantial inter-dataset covariate shift persists after harmonization, mandating domain-adaptive or preprocessing-aware training strategies for robust transfer (Luu et al., 23 Oct 2025).

3. Training Regimes and Self-Supervised Learning Strategies

Effective pretraining of foundation models leverages:

• Large batch sizes and multi-GPU scaling (e.g., 12× NVIDIA H100 for SimCLR, 64 A100s for BrainSegFounder) (Kaczmarek et al., 12 Sep 2025, Cox et al., 2024)
• Aggressive data augmentation: random 3D crops, flips, rotation, intensity perturbation, bias field simulation, elastic deformation, and contrast randomization to model realistic variability (Farahani et al., 10 Mar 2025, Munk et al., 13 Apr 2026)
• Long training schedules (50–250 epochs, or equivalently 100–500K gradient updates) with AdamW/cosine decay optimizers, and batch accumulation for stabilizing 3D SSL (Gordaliza et al., 19 Jan 2026, Mazher et al., 27 Oct 2025)

Salient innovations include:

• Multi-modal dynamic integration: learnable embeddings (and conditional normalization) to handle arbitrarily missing/novel modalities without retraining (Luu et al., 4 Nov 2025)
• Prompt- and adapter-based continual learning: parameter-efficient, task-separable adaptation to sequential downstream tasks with frozen backbone (0% catastrophic forgetting, <0.1% parameters per task with LoRA) (Chen et al., 26 Feb 2026)
• Saliency-adaptive preselection: for fMRI, two-stage pipelines such as SLIM-Brain initially extract salient temporal windows, then perform computationally intensive voxel-wise encoding only on these subsets (Wang et al., 26 Dec 2025)

4. Downstream Applications: Segmentation, Prediction, and Retrieval

Brain MRI foundation models are evaluated on varied tasks:

• Segmentation: Multi-class tissue (gray/white/CSF), tumor, infarct, and lesion segmentation, with foundation models such as BrainFound, SAM-Brain3D, MTS-UNET, and MedSAM achieving Dice coefficients up to 0.89 for fetal brain, 0.85 for tumor, and 0.8751 for white/gray matter (Sun et al., 31 Mar 2026, Kayser et al., 21 Dec 2025, Mazher et al., 27 Oct 2025, Cox et al., 2024, Deng et al., 1 May 2025, Farahani et al., 10 Mar 2025). Fine-tuned MedSAM and similar promptable frameworks require minimal structural changes for multi-class outputs (Sun et al., 31 Mar 2026, Putz et al., 2023).
• Classification & regression: Alzheimer's disease, MCI, dementia syndromes, tumor grading, molecular subtyping, and brain-age prediction; foundation models consistently outperform either supervised or ImageNet-pretrained baselines, often by large AUROC margins (e.g., 0.883 vs. 0.835 on NACC AD/controls for BrainFound) and with fewer labels (Mazher et al., 27 Oct 2025, Barbano et al., 2024, Farahani et al., 10 Mar 2025).
• Few-shot and domain-robust learning: Out-of-domain few-shot transfer, as in the FOMO25 challenge, shows that SSL foundation models can exceed in-domain supervised baselines by +17 Dice points (segmentation) or +20 AUROC points (classification), with strong model performance even for architectures as small as 20–50M parameters (Munk et al., 13 Apr 2026, Gordaliza et al., 19 Jan 2026).
• Multimodal and cross-modal retrieval: Models such as BRAT achieve high recall@k for text-image and image-text retrieval and facilitate report generation via multi-view alignment with clinical documentation (Kayser et al., 21 Dec 2025).

5. Generalization, Robustness, and Limitations

Foundation models display superior cross-protocol, cross-site, and data-scarce performance due to the diversity-normalizing effects of large-scale pretraining and regularization (Luu et al., 23 Oct 2025, Mazher et al., 27 Oct 2025, Munk et al., 13 Apr 2026). Notable findings include:

• Robustness to missing, unseen, and partially available modalities via shared encoder architectures conditioned with learned embeddings (Luu et al., 4 Nov 2025, Liu et al., 30 Aug 2025)
• Stable performance with aggressive voxel or patch masking ratios (up to 70%) enabling memory-efficient training for full-brain or time-resolved data (Wang et al., 26 Dec 2025, Wang et al., 11 Jun 2025)
• Domain-specific or pathology-aware priors can be encoded without introducing brittle task-specific adaptions, as evidenced by models such as AnatCL which infuse anatomical similarity into contrastive learning (Barbano et al., 2024)
• Zero-shot anomaly detection pipelines can be constructed using 2D pretrained encoders and volumetric patch aggregation, offering practical, truly prompt-free volumetric abnormality scoring (Le-Gia et al., 17 Feb 2026)

Limitations persist:

• Domain invariance is not always beneficial; e.g., strict modality-invariant pretraining can impair fine-grained segmentation due to suppression of contrast-dependent features (Koutsouvelis et al., 14 Nov 2025)
• Pretraining on predominantly healthy populations can bias representations away from rare or subtle pathologies; targeted sampling and adaptive loss weighting are required to mitigate this (Luu et al., 23 Oct 2025, Munk et al., 13 Apr 2026)
• Scaling backbone size or pretraining longer does not uniformly translate to improved out-of-domain/few-shot generalization, suggesting diminishing returns beyond moderate model capacities under practical compute constraints (Munk et al., 13 Apr 2026, Gordaliza et al., 19 Jan 2026, Mazher et al., 27 Oct 2025)

6. Current Challenges and Recommendations for Clinical Translation

Sustained progress in clinical translation of brain MRI foundation models requires:

• Diverse and representative pretraining datasets: Comprehensive curation and harmonization strategies to ensure balanced inclusion of pathologies, age groups, and imaging protocols (Luu et al., 23 Oct 2025)
• Preprocessing- and augmentation-aware networks: Incorporation of preprocessing-matched normalizers, spatially-aware encoding, and domain adversarial objectives to counteract covariate shift (Luu et al., 23 Oct 2025, Ghamizi et al., 16 Jun 2025)
• Architecture-task alignment: Choosing SSL objectives and decoder heads matched to the target downstream task; e.g., MAE for segmentation, hybrid contrastive for classification, and modular adapters for continual or multi-modal transfer (Munk et al., 13 Apr 2026, Deng et al., 1 May 2025)
• Reproducibility and evaluation protocols: Standardized, containerized evaluation on held-out, out-of-domain clinical data and public release of code, weights, and preprocessing recipes (Munk et al., 13 Apr 2026, Ghamizi et al., 16 Jun 2025, Kayser et al., 21 Dec 2025)
• Efficiency: Favoring lean architectures (≤50M parameters) and adaptive computation (saliency-based windowing, prompt tuning) for tractable deployment in real-world clinical settings (Wang et al., 26 Dec 2025, Gordaliza et al., 19 Jan 2026)

In summary, brain MRI foundation models, when architected and pretrained to capture multi-scale, multi-modal, and diagnosis-relevant priors, offer a scalable, sample-efficient, and domain-robust basis for a new generation of generalist and specialist neuroimaging tools. Ongoing work aims to resolve the remaining barriers to clinical adaptation, including handling missing modalities, improving sensitivity to rare pathology, and ensuring trustworthiness across the full diversity of MRI data encountered in practice (Munk et al., 13 Apr 2026, Luu et al., 4 Nov 2025, Wang et al., 26 Dec 2025, Kaczmarek et al., 12 Sep 2025, Mazher et al., 27 Oct 2025).