---
title: Brain Foundation Models
url: https://www.emergentmind.com/topics/brain-foundation-models
type: topic
---

# Brain Foundation Models

Searching arXiv for recent papers on brain foundation models, EEG/fMRI/MRI BFMs, and benchmarking/governance.
Searching arXiv for "brain foundation models self-supervised learning EEG fMRI MRI benchmark governance".
Brain foundation models (BFMs) are large-scale deep neural networks pretrained on extensive collections of neural recordings or brain images to learn reusable representations that can be adapted to downstream tasks with limited labeled data. In the neural-signal literature, BFMs are defined as large-scale deep neural networks pretrained on extensive collections of neural recordings such as EEG, fMRI, and intracranial signals using predominantly self-supervised objectives, with the goal of enabling zero- or few-shot transfer to tasks including decoding, disease diagnosis, brain simulation, and brain–computer interfaces [2503.00580]. In the adjacent brain-imaging literature, foundation models for brain MRI, CT, PET, and related modalities are surveyed as a broader class of pretrained models for segmentation, classification, regression, generation, registration, and question answering, with particular emphasis on multimodal data integration and heterogeneous clinical datasets [2506.13306]. Across these strands, the unifying premise is that large-scale pretraining on unlabeled or weakly labeled brain data can mitigate scarce labels, low signal-to-noise ratio, inter-subject variability, and cross-site heterogeneity while furnishing a common representational substrate for downstream neuroscience and clinical analysis [2506.16009].

## 1. Definitions, scope, and conceptual boundaries

The term “brain foundation model” was explicitly defined in a 2025 survey as referring to large-scale deep neural networks pretrained on extensive collections of neural recordings such as EEG, fMRI, and intracranial signals using predominantly self-supervised objectives [2503.00580]. A complementary survey framed BFMs as large-scale neural network backbones pretrained on broad collections of unlabeled brain recordings, including EEG, MEG, fNIRS, fMRI, and ECoG/iEEG, via self-supervised learning, and emphasized their use as reusable cortical representation learners for classification, regression, BCI, and cross-modal generation [2506.16009].

This scope is broader than non-invasive electrical recordings alone. EEG/iEEG benchmarks treat BFMs as self-supervised neural encoders trained on raw electrical brain signals [2602.11558], but other lines of work define BFMs over fMRI graphs and time series [2506.02044], 3D structural MRI [2510.23415], multimodal structural-plus-functional neuroimaging [2509.24693], diffusion-derived microstructural maps [2607.00573], and omnifunctional systems jointly pretrained on fMRI, EEG, and MEG [2602.23410]. Brain-imaging reviews further situate these models within a larger ecosystem of 86 foundation model architectures and 161 brain imaging datasets spanning MRI, CT, PET/SPECT, and ultrasound [2506.13306].

A recurring distinction separates BFMs from conventional task-specific pipelines. The surveys contrast BFMs with small, supervised models trained on narrow datasets and handcrafted features, and describe BFMs instead as systems that exploit thousands of hours of heterogeneous recordings or large-scale volumetric imaging corpora to generalize across subjects, tasks, modalities, and experimental paradigms [2503.00580]. This suggests that “foundation” in this domain refers less to a single architecture family than to a training regime: large-scale pretraining, broad transfer, and adaptation under distribution shift.

## 2. Self-supervised objectives and architectural principles

The most common pretraining objectives fall into masked modeling, autoregressive prediction, contrastive learning, and cross-modal alignment. For contrastive learning, a survey on self-supervised BFMs gives the standard InfoNCE form

$$
L_{\text{contrastive}}
= - \mathbb E_{(i,j)\in P}
\log
\frac{\exp(\mathrm{sim}(z_i,z_j)/\tau)}
{\sum_{k\in N(i)} \exp(\mathrm{sim}(z_i,z_k)/\tau)},
$$

where $z_i = \mathrm{Encoder}(\mathrm{Augment}(x_i))$ and $\mathrm{sim}(u,v)$ is cosine similarity [2506.16009]. Masked reconstruction is equally central, with a masked autoencoder loss written as

$$
L_{\text{MAE}}
= \mathbb E_{x\sim D}\bigl[\|x-\mathrm{Dec}(\mathrm{Enc}(M(x)))\|^2\bigr],
$$

and used for masked time-frequency patches, masked MRI volumes, masked graph nodes, or masked fMRI windows depending on modality [2506.16009]. A second survey formalizes the same design space through masked signal modeling, autoregressive prediction, contrastive learning, and cross-modal prediction, including $L_{\text{AR}}=-\sum_{t=1}^T \log p_\theta(x_t\mid x_{<t})$ and $L_{\text{cross}}=\mathbb E[\|f_A(x^A)-f_B(x^B)\|_2^2]$ [2503.00580].

Architecturally, transformers are common but not exclusive. The surveys enumerate transformers, CNNs, GNNs, and hybrid backbones, with spatial and temporal tokenization adapted to modality-specific structure [2506.16009]. In EEG, raw signals $X\in\mathbb R^{C\times T}$ are segmented into temporal patches or transformed into joint time-frequency representations; in fMRI graph models, nodes represent ROIs and edges encode correlation or connectivity; in MRI, volumetric crops are partitioned into non-overlapping 3D patches with positional embeddings [2510.23415]. Representative mechanisms include Graph Transformer encoders with Random Walk Structural Encoding for fMRI graphs [2506.02044], dual-domain cross-attention and topological embeddings for EEG [2509.24222], student–teacher self-distillation over global and local volumetric crops in 3D MRI [2510.23415], and Any-Resolution Neural Signal Sampler cross-attention that projects fMRI, EEG, and MEG signals into a fixed-length latent space [2602.23410].

The field has also diversified beyond plain masked autoencoding. BrainFound uses DINO-v2’s self-distillation loss rather than explicit masked-patch loss or InfoNCE [2510.23415]. BrainHarmonix combines 3D structural MAE with a JEPA-style functional encoder and a hub-token fusion transformer [2509.24693]. BrainFIBRE introduces Self-supervised Partial Information Decomposition, which uses a Mixture-of-Experts architecture and Counterfactual Candidate Construction to separate unique, redundant, and synergistic information in NODDI-derived maps [2607.00573]. Brain-OF adopts Masked Temporal-Frequency Modeling with a dual-domain reconstruction objective weighted as $\alpha=0.2$ and $\beta=0.8$ [2602.23410].

## 3. Modalities and representative model families

The current literature spans electrical recordings, hemodynamic signals, structural and multimodal MRI, and diffusion-derived microstructure. The following systems illustrate the breadth of the field.

| Domain | Representative systems | Reported characteristics |
|---|---|---|
| EEG and iEEG | BrainWave/Brant-2, Uni-NTFM, Brain4FMs benchmark models | Large-scale pretraining on electrical signals, time/frequency modeling, topology-aware encoding, cross-subject transfer [2402.10251], [2509.24222], [2602.11558] |
| fMRI and brain graphs | BrainGFM, SLIM-Brain, BrainLM/Brain-JEPA analyses | Multi-atlas graph pretraining, voxel-level JEPA, and critiques of cognition prediction from pretrained fMRI transformers [2506.02044], [2512.21881], [2606.04010] |
| 3D brain MRI | BrainFound, SAM-Brain3D, BrainFM-MRI, challenge-winning U-Net FM | Volumetric self-distillation, segmentation-oriented pretraining, dynamic modality integration, and lightweight CNN alternatives [2510.23415], [2505.00627], [2511.03014], [2601.13166] |
| Multimodal neuroimaging | BrainHarmonix, Brain-OF, BrainFIBRE | Structure–function fusion, omnifunctional fMRI/EEG/MEG pretraining, and microstructural disentanglement from NODDI maps [2509.24693], [2602.23410], [2607.00573] |

In EEG and intracranial recordings, BrainWave is presented as the first foundation model for both invasive and non-invasive neural recordings, pretrained on more than 40,000 hours of electrical brain recordings from approximately 16,000 individuals, with downstream performance reported on seizure detection, seizure prediction, sleep staging, emotion recognition, and motor imagery [2402.10251]. Uni-NTFM extends this line by arguing that EEG requires a decoupled treatment of waveform and rhythmic features, explicit topological embeddings for electrodes, and sparse Mixture-of-Experts routing. Its largest variant has 1.9B parameters and is pretrained on over 28,000 hours of diverse EEG data via a dual-domain masked reconstruction objective [2509.24222].

For fMRI, BrainGFM proposes graph contrastive learning plus graph masked autoencoding on weighted, correlation-based brain graphs. It is pretrained on 27 neuroimaging datasets spanning 25 common neurological and psychiatric disorders, 8 parcellations, over 25,000 subjects, 60,000 fMRI scans, and about 400,000 graph samples [2506.02044]. SLIM-Brain instead pursues atlas-free voxel-level representation learning, combining a lightweight temporal extractor that ranks windows by saliency with a 4D hierarchical JEPA that trains on only the top-$k$ windows and deletes about 70% masked patches [2512.21881].

MRI-oriented BFMs show substantial architectural heterogeneity. BrainFound adapts DINO-v2 to 3D brain MRI by treating a volumetric scan as an ordered sequence of slices and/or 3D patches, supporting single- and multimodal inputs and partial-modality inference through zero-filling of missing channels [2510.23415]. SAM-Brain3D fine-tunes SAM-Med3D on 66,280 brain image–label pairs across 14 MRI sub-modalities and couples the pretrained encoder to a Hypergraph Dynamic Adapter for downstream disease classification [2505.00627]. BrainFM-MRI uses one encoder with learnable modality embeddings, conditional layer normalization, masked autoencoding with modality dropout, and a VICReg-style variance–covariance regularizer for about 60,000 multi-center MRIs [2511.03014]. A challenge-winning technical report argues that a 3D U-Net CNN with anatomical priors and cross-contrast objectives trained 1-2 orders of magnitude faster and was 10 times smaller than competing transformer-based approaches in the first brain MRI foundation model challenges [2601.13166].

Multimodal systems broaden the notion of what counts as a BFM. BrainHarmonix is described as the first multimodal brain foundation model that unifies structural morphology and functional dynamics into compact 1D token representations, with pretraining on 64,594 T1-weighted structural MRI 3D volumes and 70,933 functional MRI time series [2509.24693]. Brain-OF jointly pretrains on fMRI, EEG, and MEG from 37 public datasets totaling 32,278 subjects and approximately 5.9 million samples [2602.23410]. BrainFIBRE focuses on brain microstructure rather than macroscopic anatomy or electrical activity, pretraining on NODDI-derived NDI, ODI, and FWF maps from 55,592 UK Biobank participants [2607.00573].

## 4. Adaptation, evaluation, and applications

The standard downstream workflow removes pretraining-specific heads, attaches a task-specific head, and then performs either linear probing with a frozen backbone or full fine-tuning with a smaller learning rate on pretrained weights [2506.16009]. The surveys formalize this as either frozen-backbone optimization of $\psi$ alone or joint optimization of $(\theta,\psi)$ for the backbone and task head [2506.16009]. This basic pattern has been extended by graph prompts and language prompts in BrainGFM [2506.02044], LoRA and Hub-LoRA for dynamic functional connectivity biomarkers [2604.22018], Hypergraph Dynamic Adapter for multimodal disease classification [2505.00627], and conditional layer normalization plus modality embeddings for missing-modality MRI [2511.03014].

Benchmarking practice has converged on a small set of protocols. A self-supervised BFM survey lists K-nearest neighbors on frozen embeddings, linear probing, full fine-tuning, and zero-shot/few-shot transfer as standard protocols, and reports common metrics including accuracy, balanced accuracy, F1-score, Cohen’s Kappa, AUROC, AUPRC, $R^2$, MSE, BLEU, ROUGE, and Top-1/Top-5 accuracy depending on task type [2506.16009]. For electrical signals, Brain4FMs consolidates 15 representative BFMs and 18 public EEG/iEEG datasets, while AdaBrain-Bench standardizes 13 public datasets across 7 non-invasive BCI applications with cross-subject, multi-subject, and few-shot evaluation settings [2602.11558], [2507.09882].

Applications are correspondingly diverse. The surveys identify BCIs, disease diagnosis, cognitive state decoding, brain simulation, segmentation, report generation, question answering, and image synthesis among downstream targets [2503.00580], [2506.13306]. Concrete examples include BrainFound’s AUROC values for Alzheimer’s disease versus controls across NACC, ADNI, OASIS, and AIBL, as well as Dice and HD95 results on FeTA 2021 and BraTS 2020 segmentation tasks [2510.23415]. BrainHarmonix reports gains on ABIDE-I, ABIDE-II, ADHD-200, PPMI, ADNI, and HCP-A benchmarks, including mixed-TR settings [2509.24693]. Brain-OF reports Balanced Accuracy of 82.87% on TUAB abnormality detection, 71.79% on ADNI Alzheimer’s classification, and MEG brain-age MAE of 7.87 years on CamCAN [2602.23410]. In MRI challenge settings, the U-Net-based foundation model report gives mean Dice of $0.82\pm0.03$ versus $0.80\pm0.04$ for transformer baselines in SSL3D, and mean classification accuracy of $0.78\pm0.05$ versus $0.72\pm0.06$ [2601.13166].

This suggests that adaptation strategy is not ancillary but constitutive. In several subfields, the practical value of BFMs lies less in zero-shot end use than in providing reusable encoders that can be efficiently specialized through prompts, low-rank adapters, linear heads, or modality-aware normalization.

## 5. Empirical strengths, limitations, and active controversies

The central empirical claim in favor of BFMs is improved transfer under data scarcity and heterogeneity. Surveys repeatedly cite gains in data efficiency, robustness to noise, and reduced need for labeled data [2506.16009], while AdaBrain-Bench reports macro-average cross-subject Balanced Accuracy of 64.61% for LaBraM and 62.66% for CBraMod versus 58.12% for the best traditional baseline across 13 datasets [2507.09882]. Brain4FMs further argues that generative SSL methods such as AE- and MAE-based models yield more separable latent geometry than contrastive-only approaches for many EEG/iEEG classification tasks [2602.11558].

At the same time, several papers challenge common assumptions about scale, architecture, and evaluation. A critique of fMRI BFMs reports that, across three state-of-the-art BFMs and every readout tested, cognition is predicted worse than by linear regression on the functional connectivity matrix, and that the gap widens with scale because pretraining preserves dominant variance but destroys third-order co-skewness relevant to cognition [2606.04010]. The same work concludes that the bottleneck is the pretraining objective rather than the backbone or model size, and shows that cumulant-aware fine-tuning can recover the raw-FC ceiling on BrainLM’s forward pass [2606.04010]. In a related but distinct critique, RE-CONFIRM argues that conventional predictive metrics are insufficient for evaluating the robustness of biomarkers identified by BFMs and reports that simply fine-tuning foundation models fails to capture regional hubs effectively, whereas Hub-LoRA improves hub sensitivity and neurobiological faithfulness [2604.22018].

Efficiency is another contested axis. Although transformers dominate many BFM narratives, the MRI challenge report states that its U-Net CNN models trained 1-2 orders of magnitude faster and were 10 times smaller than competing transformer-based approaches, while consistently ranking first in 5/5 FOMO25 tracks [2601.13166]. SLIM-Brain likewise targets both data- and training-efficiency, requiring only 4 thousand pre-training sessions and approximately 30% of GPU memory compared with traditional voxel-level methods [2512.21881]. These results argue against the misconception that brain foundation models are synonymous with very large transformer encoders.

Several limitations recur across surveys and model papers. These include heterogeneous electrode placements and sampling rates, lack of standardized preprocessing, inter-subject variability, true zero-shot performance remaining limited, high compute and energy requirements, missing modalities, and the need for neurophysiological grounding of latent features [2503.00580], [2506.16009]. BrainFM-MRI explicitly notes a trade-off between flexibility under missing modalities and single-modality reconstruction quality [2511.03014]. SAM-Brain3D+HyDA notes that all modalities must be present at inference in its current form and that class imbalance still depresses sensitivity and AUC in skewed tasks [2505.00627].

A plausible implication is that “generalization” in BFMs must be disaggregated. Transfer across subjects, across datasets, across atlases, across modalities, across scanner protocols, and across scientific claims such as biomarker discovery are not interchangeable achievements, and different pretraining objectives may improve one axis while degrading another.

## 6. Governance, ethics, and future research directions

Because BFMs are trained on body-derived neural data collected under clinical and research governance regimes, their development has prompted a parallel governance literature. A dedicated analysis of training-data governance argues that neural data carry stronger expectations of protection than text or images, yet foundation-model practice subjects them to large-scale repurposing, cross-context stitching, and open-ended downstream use [2602.02511]. The paper organizes concerns around privacy, consent, bias, benefit sharing, and legal governance, and proposes baseline safeguards such as membership-inference and memorization risk evaluations, explicit provenance documentation for modality combinations, controlled release defaults when uncertainty remains, stricter treatment of legacy consent regimes, subgroup performance reporting, tiered data-access models, and multistakeholder review bodies [2602.02511].

Technical future directions in the surveys are comparatively consistent. Proposed directions include graph- and topology-aware architectures, adaptive windowing, neuroscience-aware masking schemes, hybrid SSL objectives combining reconstruction, prediction, and contrastive terms, multimodal pretraining across EEG, fMRI, text, and vision, efficient pruning and distillation for real-time BCIs, synthetic augmentation, federated learning with secure aggregation, and explainable or neuro-symbolic AI [2506.16009], [2503.00580]. Brain imaging reviews add domain adaptation, harmonization across scanners, uncertainty quantification, and human-in-the-loop evaluation as unmet needs [2506.13306].

Some proposals extend beyond current benchmark practice. One perspective argues for training foundation models directly on human brain data as a complement to text and image corpora, and proposes reinforcement learning from human brain and chain of thought from human brain as targeted ways to use scarce neuroimaging data during post-training and inference [2601.12053]. This suggests a broader interpretation of BFMs: not only models for decoding brain data, but potentially models aligned by brain data. The current literature, however, still treats this as a strategic proposal rather than an established paradigm.

Taken together, the field presents BFMs as a convergence zone between self-supervised representation learning, multimodal neuroimaging, clinical machine learning, and neuroethics. The most mature evidence supports their utility as transferable pretrained backbones for heterogeneous downstream tasks. The most important unresolved questions concern objective design, robustness under domain shift, interpretability of latent structure, efficiency under realistic deployment constraints, and governance of highly sensitive training data [2506.16009], [2602.02511], [2606.04010].

Source: https://www.emergentmind.com/topics/brain-foundation-models