SIAM: Head and Brain MRI Segmentation from Few High-Quality Templates via Synthetic Training

Abstract: Synthetic training has recently advanced brain MRI segmentation by enabling contrast-agnostic models trained entirely on generated data. However, most existing approaches rely on hundreds of automatically labeled templates, introducing systematic biases and limiting their flexibility to incorporate new anatomical structures. We present the Segment It All Model (SIAM), a 3D whole-head segmentation framework for 16 anatomical structures, trained using only six high-quality, manually annotated templates. SIAM extends domain randomization to both intensity and shape domains: synthetic image generation ensures contrast variability, while high-resolution spatial transformations model anatomical differences in cortical thickness and deep nuclei morphology. Unlike prior synthetic models, SIAM simultaneously segments brain as well as extra-cerebral tissues, including cerebrospinal fluid, vessels, dura mater, skull, and skin, enabling fully automated, preprocessing-free analysis. Evaluation across eight heterogeneous datasets (N=301), that include multiple contrasts (T1-weighted, T2-weighted, CT) and span a wide range of ages, demonstrates that SIAM matches or outperforms state-of-the-art methods for brain structures, in addition to extending automated segmentation to non-brain structures. The model also exhibits superior consistency across contrasts and repeated acquisitions, together with improved sensitivity to subtle gray matter atrophy. We openly release the model and the label templates at https://github.com/romainVala/SIAM.

• The paper introduces SIAM, which achieves state-of-the-art segmentation using synthetic training on only six high-fidelity, manually curated templates.
• The methodology leverages joint intensity and shape domain randomization to generate contrast- and anatomy-agnostic training data, ensuring robust performance across varied MRI protocols.
• SIAM demonstrates superior consistency with high Dice scores and sensitivity to subtle morphometric changes, enhancing both clinical diagnostics and neuroimaging research.

SIAM: Synthetic Training for 3D Head and Brain MRI Segmentation from Minimal High-Quality Templates

Introduction

The "Segment It All Model" (SIAM) introduces a fundamentally distinct paradigm for whole-head tissue segmentation in neuroimaging, leveraging synthetic training on only six manually curated, high-fidelity label templates. SIAM addresses core limitations in current MRI segmentation approaches: dependence on large, automatically labeled template banks that propagate systematic errors, lack of flexibility in extending to new anatomical structures, and poor generalization to out-of-distribution image contrasts and resolutions. Integrating domain randomization in both intensity and shape, SIAM enables contrast-agnostic and anatomy-agnostic segmentation covering 16 brain and extra-cerebral classes.

Methods and Innovations

High-Quality Label Templates and Synthetic Data Generation

SIAM's development initiates with the assembly of an expert-curated template set, including the MIDA template and five additional skull and vasculature exemplars, exhaustively annotated or corrected at sub-millimeter resolution for cerebral and extra-cerebral structures.

Synthetic data generation comprises joint augmentation in the intensity and shape domains:

• Intensity Domain: Full randomization of tissue intensities via MRI signal models, extending the domain randomization (DR) principle to generate contrast-invariant training data. Partial volume effects at interfaces between tissues are faithfully modeled, avoiding interpolation artifacts.
• Shape Domain: High-resolution upsampling (via Sinc interpolation at 0.25mm) and subsequent controlled morphological dilations/erosions and elastic deformations introduce robust anatomical variability. This strategy modulates cortical thickness and deep nuclei morphology beyond what is attainable with statistical shape models derived from broad but lower-fidelity automated segmentations.

  Figure 1: Generative workflow for synthetic training—A: previous label-to-image pipeline; B: SIAM's enhanced pipeline; C: impact of upsampling choices; D: partial volume modeling yields realistic intensity transitions and label targets.

Network Design and Training

The resulting synthetic volumes and labels train a standard nnU-Net-based 3D residual encoder-decoder using 3D patches with no additional augmentations, with all data synthesized prior to network training. All augmentations are encoded in the generative process itself, yielding a bias-mitigated, high-variance training set from only six templates.

Experimental Evaluation

Datasets and Baselines

The evaluation benchmarks SIAM against conventional and synthetic SOTA approaches (FreeSurfer, FastSurfer, SynthSeg, SuperSynth, GOUHFI) across eight datasets (N=301): spanning contrast (T1w, T2w, CT), age (neonates to adults), anatomical distortions (hydrocephalus), and manual/silver-standard references. The scope includes test-retest and cross-contrast consistency as well as sensitivity to controlled morphometric changes (atrophy).

Anatomical Accuracy

SIAM achieves state-of-the-art or superior Dice scores for gray matter (GM) and subcortical segmentation on high-resolution datasets, notably excelling on UltraCortex (91.6%), HCP (93.8%), and dHCP (91.2%). In manual reference scenarios (MICCAI_2012), SIAM demonstrates highest accuracy for the putamen and robust performance for deep nuclei, even exceeding other synthetic approaches constrained by systematic reference errors.

Figure 2: Dice evaluation for GM and subcortical structures and skull across multiple datasets and input contrasts.

Qualitative analysis confirms SIAM's ability to alleviate inherited shape biases (e.g., exclusion of claustrum in putamen), with anatomically superior results even when quantitative metrics penalize deviations from systemically biased references.

Figure 3: Anatomical fidelity in SIAM segmentations—improved delineation in putamen and cerebellum; robust handling of outliers.

Consistency and Generalization

SIAM exhibits the highest test-retest and cross-contrast reproducibility in both GM and subcortical labels (Dice ≈95–97%), outperforming even models trained with hundreds of reference cases. This holds across data from repeated acquisitions, T1w/T2w pairs, and multi-contrast skull segmentation, for which SIAM achieves Dice scores above 90%.

Figure 4: Test-retest (T1w vs T1w_repeat) and cross-contrast (T1w vs T2w) consistency; accurate tracking of GM atrophy across controlled synthetic cortical thickness reductions.

Sensitivity to Morphometric Change

On simulated cortical atrophy, SIAM detects reductions in GM volume with lower error and less variance compared to other synthetic and classical models, indicating robust sensitivity to subtle anatomical change—arguably a central requirement for longitudinal neuroimaging biomarker research.

Whole-Head and Extra-cerebral Tissue Segmentation

By explicitly incorporating vessels, dura mater, skull, and other non-cerebral tissues, SIAM fully automates head segmentation without reliance on error-prone preprocessing (e.g., brain extraction), facilitating robust volumetric normalization and the potential for new applications in skull-based modeling and neurostimulation targeting.

Figure 5: SIAM enables anatomically precise, full-head segmentation encompassing extra-cerebral tissues (vessels, dura mater, skull, etc.) across imaging protocols.

Implications and Future Directions

The core contribution of SIAM is empirical validation that high-fidelity, low-number, high-quality templates—when coupled with principled, physics-informed synthetic generation and joint shape-intensity domain randomization—can outperform the conventional large-scale, silver-standard paradigm that dominates current segmentation pipelines. This raises important theoretical and practical implications:

• De-biasing segmentation: SIAM avoids systematic errors perpetuated by automated labeling tools, as proven by qualitative and quantitative assessments across reference protocols.
• Extensibility: The explicit generative model and modular template strategy enable rapid integration of additional anatomical targets without full data reannotation—paving the way for unified healthy/pathology/lesion segmentation frameworks (see also advances in joint anatomical/lesion modeling using synthetic pipelines [billot_joint_2021], [wu_tumorsynth_2026]).
• Clinical translation: The contrast and protocol invariance, together with preprocessing-free design, facilitates robust use on heterogeneous, unstandardized clinical data sets, increasing applicability for routine diagnostics and research studies.
• Morphometric research: SIAM's improved volumetric sensitivity and consistency meet key requirements for neurodegenerative and developmental studies, where submillimeter atrophy must be discerned robustly.

Remaining challenges include the need for even broader anatomical variability modeling (notably for gross deformations or rare pathologies), further validation for pathological tissues (e.g., tumor, stroke, demyelination), and continued integration with broader anatomical datasets and evolving MRI contrasts. The synthetic-low/high-quality tradeoff, identification of residual failure modalities, and optimization of label augmentation strategies for highly anisotropic or low-SNR input remain active areas for continued study.

Conclusion

SIAM demonstrates that a synthetic, domain-randomized training protocol derived from minimal, meticulously curated templates can achieve and often surpass the segmentation performance and clinical robustness of models trained on vast sets of lower-quality annotations. The work establishes a scalable, unbiased, and extensible approach for head and brain segmentation in MRI, with practical, reproducible benefits for both neuroimaging research and broader clinical applications.