MRSegmentator: Unified MRI & CT Segmentation

• MRSegmentator is an open-source deep-learning framework that performs unified multi-organ segmentation in MRI and CT, covering 40 anatomical classes including organs, muscles, bones, and vessels.
• It leverages a unified nnU-Net architecture with cross-modality transfer learning and a human-in-the-loop annotation workflow to achieve high Dice scores across diverse imaging sequences.
• The model has been extensively benchmarked and is applied as an anatomical prior in downstream tasks, enhancing segmentation in clinical pipelines despite challenges with small or low-contrast structures.

MRSegmentator is an open-source deep-learning system for whole-body multi-organ segmentation in both MRI and CT. In its 2024 formulation, it is a unified nnU-Net–based model that segments 40 anatomical classes in the thorax, abdomen, pelvis, and proximal thighs; it is trained on 1,200 manually annotated 3D axial MRI scans from the UK Biobank, 221 in-house MRI scans, and 1228 CT scans from the TotalSegmentator dataset, using a human-in-the-loop annotation workflow and cross-modality transfer learning from an existing CT segmentation model (Häntze et al., 2024).

1. Definition and scope

MRSegmentator was introduced as a multi-modality segmentation model rather than as a sequence-specific or organ-specific tool. Its stated scope is a single unified model for MRI and CT, with support for multiple MRI sequences, including T1, T2, T1-weighted fat saturated, Dixon in-phase, opposed-phase, fat-only, water-only, and T1-GRE opposed-phase, water, and fat images (Häntze et al., 2024).

The model is intended for automated multi-organ segmentation rather than lesion-specific delineation. Its label space spans organs, muscles, bones, and vessels, and its design explicitly targets routine clinical heterogeneity rather than a narrow benchmark protocol. This places it in the same methodological family as large-scale, “ready-to-use” segmentors such as TotalSegmentator CT, but with MRI as a primary target and CT retained as an additional modality (Häntze et al., 2024).

A recurring source of confusion is terminological. In contemporary body-imaging literature, “MRSegmentator” usually denotes the 2024 multi-modality 40-class model (Häntze et al., 2024). However, the name has also appeared in unrelated contexts, including a 2D fully automatic U-Net–based brain tumor segmentation project on T1-weighted slices (Bajwa, 2021). This suggests that the term should be interpreted by reference to the surrounding application domain rather than by name alone.

2. Training data and annotation workflow

The MRI training data combine two complementary sources. The UK Biobank component consists of 1,200 scans from 50 participants, with whole-body imaging divided into six regions and four Dixon T1-GRE sequences per region: in-phase, opposed-phase, fat-only, and water-only. The in-house MRI cohort contributes 221 scans from 177 patients with kidney tumors or cysts, acquired on Siemens MAGNETOM Avanto 1.5T and Siemens MAGNETOM Vida 3T systems, and includes T1, T2, and mostly post-contrast T1-weighted fat-saturated series (Häntze et al., 2024).

The CT component comprises 1,228 scans from the TotalSegmentator CT dataset. These scans provide overlapping organ, bone, and vessel labels, with vertebrae merged into a single spine class and lung lobes merged into left and right lung classes to match the MRI labeling scheme (Häntze et al., 2024).

Annotation was organized as a human-in-the-loop workflow. A CT-trained TotalSegmentator model was first applied directly to MRI after intensity inversion and histogram equalization. These pre-segmentations were then corrected or replaced in MONAI Label and 3D Slicer. After each batch of approximately 50 annotated examinations, an nnU-Net model was retrained and used to pre-segment the next batch, reducing manual workload over successive rounds (Häntze et al., 2024).

This workflow is important because MRSegmentator is not trained from an independently pre-existing MRI corpus of dense labels. Instead, its label set is built through iterative correction of cross-modality pseudo-labels, followed by a final nnU-Net training with 5-fold cross-validation on the full MRI+CT training set (Häntze et al., 2024).

3. Architecture and label taxonomy

MRSegmentator uses nnU-Net as its underlying architecture. In the paper’s description, it is a 3D encoder-decoder with skip connections, automatically configured by nnU-Net for resolution, patch size, and pooling depth. MRI and CT are handled by the same network and weights rather than by modality-specific encoders (Häntze et al., 2024).

The paper does not explicitly reprint the training loss. This suggests use of nnU-Net’s standard combined Dice and cross-entropy objective, but the explicit architectural claim in the source is the use of a unified nnU-Net framework trained jointly on MRI and CT (Häntze et al., 2024).

The 40 target classes are organized across five groups.

Group	Examples	Count
Chest	Heart, left lung, right lung, esophagus	4
Gastrointestinal tract	Liver, spleen, pancreas, stomach, duodenum, small bowel, colon	8
Retroperitoneum / urinary tract	Kidneys, adrenal glands, urinary bladder	5
Musculoskeletal	Spine, sacrum, hip bones, femurs, gluteal, paraspinal, iliopsoas muscles	16
Vessels	Aorta, IVC, portal/splenic vein, iliac arteries and veins	7

Two label-design choices are especially consequential. First, all vertebrae are merged into a single spine class because of anisotropic axial spacing in UK Biobank MRI. Second, lung lobes are not separated; only left and right lung are labeled, reflecting poor MRI contrast in lung parenchyma and lobar boundaries (Häntze et al., 2024).

4. Evaluation protocol and reported performance

MRSegmentator is evaluated with the Dice Similarity Coefficient and the 95th percentile Hausdorff Distance. The Dice coefficient is defined as

$DSC = \frac{2 |P \cap G|}{|P| + |G|}$

where $P$ is the predicted segmentation and $G$ the reference segmentation (Häntze et al., 2024).

Internal results are reported by sequence type. In 5-fold cross-validation across the training data, mean Dice and HD95 are 0.84 and 5.05 mm for T1, 0.77 and 6.91 mm for T2, 0.85 and 4.63 mm for T1-weighted fat-saturated MRI, 0.76 and 8.12 mm for Dixon in-phase, 0.79 and 7.34 mm for opposed-phase, 0.82 and 7.07 mm for water-only, 0.78 and 7.71 mm for fat-only, and 0.91 and 7.82 mm for CT (Häntze et al., 2024).

External MRI evaluation is performed on the German National Cohort (NAKO), where the overall MRI mean is 0.85 ± 0.13 Dice and 6.55 ± 11.9 mm HD95. Sequence-specific NAKO means are 0.86 for T1-GRE opposed-phase, 0.84 for T1-GRE water, and 0.86 for T1-GRE fat. Representative per-organ NAKO values include liver 0.95, heart 0.95, left and right lungs 0.97, kidneys 0.88 and 0.90, bladder 0.95, spine 0.89, and aorta 0.90, whereas adrenal glands and portal/splenic vein are substantially harder, at 0.65/0.61 and 0.54 respectively (Häntze et al., 2024).

External AMOS22 results are reported for 13 overlapping abdominal classes. On AMOS22 MRI, the mean is 0.79 ± 0.11 Dice and 8.29 ± 7.35 mm HD95; on AMOS22 CT, 0.84 ± 0.11 Dice and 7.86 ± 11.12 mm HD95. Large organs such as liver, spleen, and kidneys remain near 0.95 Dice, whereas duodenum and adrenal glands are markedly lower (Häntze et al., 2024).

These results establish a consistent pattern. MRSegmentator performs strongly on large, well-defined organs, bones, and muscle groups, shows intermediate performance on anatomically variable gastrointestinal structures such as pancreas, stomach, and bowel, and is weakest on small or tubular structures such as adrenal glands, portal/splenic vein, and iliac vessels (Häntze et al., 2024).

5. Position in subsequent benchmarking literature

Later benchmarking studies position MRSegmentator as a strong general-purpose abdominal MRI segmentor, but the conclusions are dataset- and label-set dependent.

Study	Setting	Reported position of MRSegmentator
(Tran et al., 10 Apr 2025)	40 Duke Liver T1w volumes across pre, arterial, venous, delayed phases; 10 abdominal structures	Best among MRSeg, TotalSegmentator MRI, and TotalVibeSegmentator; Dice 80.7 ± 18.6 and HD 8.9 ± 10.4 mm
(Krishnaswamy et al., 23 Jul 2025)	Unseen abdominal MRI datasets AMOS, CHAOS, LiverHCCSeg	Best performance and most generalizable among MRSegmentator, MRISegmentator-Abdomen, TotalSegmentator MRI, and ABDSynth
(D'Antonoli et al., 2024)	TotalSegmentator MRI internal mixed-sequence comparison	TotalSegmentator MRI better on 40 shared structures overall, but MRSegmentator slightly better on 13 abdominal structures

On a curated multi-parametric T1-weighted abdominal MRI benchmark derived from the Duke Liver Dataset, MRSegmentator obtained a Dice score of 80.7 ± 18.6 and a Hausdorff Distance error of 8.9 ± 10.4 mm, and it fared the best across the different sequence types in contrast to TotalSegmentator MRI and TotalVibeSegmentator (Tran et al., 10 Apr 2025). The strongest phase-specific result in that study occurred on venous T1-weighted MRI, where MRSegmentator reached 84.1 ± 16.7 Dice and 6.8 ± 7.6 mm HD (Tran et al., 10 Apr 2025).

A broader abdominal benchmark on three public datasets not seen during training concluded that MRSegmentator achieves the best performance and is most generalizable. In that study, its strengths were high Dice and consistently low HD95 on large organs and stable cross-sequence behavior, while pancreas, gallbladder, duodenum, and adrenal glands remained challenging (Krishnaswamy et al., 23 Jul 2025).

At the same time, comparative claims are not uniformly one-sided. In the TotalSegmentator MRI study, TotalSegmentator MRI outperformed MRSegmentator on 40 shared structures in the authors’ mixed-sequence internal MR test set, but MRSegmentator was slightly better on 13 abdominal structures and on the AMOS MRI validation set. This suggests that “better” depends materially on anatomical scope, sequence mix, and whether evaluation emphasizes broad whole-body coverage or abdominal MRI specifically (D'Antonoli et al., 2024).

A further clarification is necessary regarding adjacent literature. According to the detailed notes provided for “MRAnnotator,” that paper does not mention MRSegmentator and does not provide a direct comparison, despite addressing a neighboring problem class of MRI multi-anatomy segmentation (Zhou et al., 2024).

6. Downstream use, limitations, and interpretive cautions

MRSegmentator has already been reused as an anatomical prior in downstream systems. In automated neurofibroma segmentation on fat-suppressed T2-weighted whole-body MRI, it serves as the stage-1 anatomy segmenter; its output is filtered into a reduced anatomy mask and an additional “NF high-risk zone,” then concatenated with the input MRI for anatomy-informed tumor segmentation. In that pipeline, integrating anatomy information produced a 68% improvement in per-scan Dice Similarity Coefficient, a 21% increase in per-tumor DSC, and a two-fold improvement in F1 score for tumor detection in high tumor burden cases (Kolokolnikov et al., 21 Feb 2025). This suggests that MRSegmentator functions not only as an endpoint segmentor but also as a reusable anatomical context engine.

The original paper identifies several limitations. Small and low-contrast structures remain difficult; adrenal glands, portal/splenic vein, and some iliac vessels show low Dice and larger HD95. Sequence coverage is broad but not exhaustive, so certain specialized MRI protocols may require fine-tuning. The training and internal validation labels were produced in a human-in-the-loop workflow centered on a single radiologist, which may introduce annotation bias even when external performance is good. The authors also note occasional left-right confusion in the pelvis and edge-of-field-of-view failures for structures truncated by scan coverage (Häntze et al., 2024).

These limitations should moderate two common misconceptions. First, MRSegmentator is not a lesion segmentor; it is a multi-organ anatomical model whose outputs are often used to support downstream disease-specific systems. Second, its reported Dice values are not directly comparable across papers without close attention to class overlap, MRI sequence type, and evaluation protocol. The literature after 2024 shows both that MRSegmentator is highly competitive and that different benchmarks can reverse pairwise rankings with other tools depending on scope (Tran et al., 10 Apr 2025).