Medal S Medical Segmentation Model

• Medal S is a foundation model that integrates native-resolution spatial and textual prompts to achieve accurate, multi-class 3D segmentation across CT, MRI, PET, ultrasound, and microscopy.
• It employs a 3D U-Net–style image encoder and transformer-based text decoder to align full-resolution volumetric features with semantic queries in a unified, end-to-end trainable framework.
• Its dual-mode prompting, offering both automatic self-refinement and interactive guidance, substantially reduces inference time while enhancing segmentation precision.

Searching arXiv for Medal S and directly related methods to support the encyclopedia entry. Tool call: arxiv_search(query="(Shi et al., 17 Nov 2025) OR \"Medal S\" medical segmentation foundation model SAT nnU-Net BiomedParse-V", max_results=10, sort_by="relevance") Medal S is a medical segmentation foundation model that supports native-resolution spatial and textual prompts within an end-to-end trainable framework. It is designed to address a limitation of text-only methods lacking spatial awareness by achieving channel-wise alignment between volumetric prompts and text embeddings, thereby mitigating inaccuracies from resolution mismatches. By preserving full 3D context, it efficiently processes multiple native-resolution masks in parallel and supports up to 243 classes across CT, MRI, PET, ultrasound, and microscopy modalities in the BiomedSegFM dataset. The model further combines a lightweight 3D convolutional module, dynamic resampling, a two-stage inference strategy, and post-processing to improve memory efficiency, precision, and inference speed (Shi et al., 17 Nov 2025).

1. Conceptual position and problem setting

Medal S is formulated as a prompt-driven segmentation system in which textual prompts specify semantic targets and spatial prompts provide voxel-space guidance. Its defining claim is that native-resolution spatial prompting and textual prompting are not treated as separate pathways but are harmonized within a single trainable model. This contrasts, in the paper’s framing, with text-only methods lacking spatial awareness and with sequential prompt-based approaches that do not process multiple classes in parallel (Shi et al., 17 Nov 2025).

The architecture is intended for multi-class medical segmentation under heterogeneous imaging conditions. The reported scope includes CT, MRI, PET, ultrasound, and microscopy, with support for up to 243 classes in BiomedSegFM. Two inference modes are provided. In a text-only mode, model predictions serve as spatial prompts for self-refinement without human input. In a hybrid mode, manual annotations are incorporated for enhanced flexibility. A plausible implication is that Medal S is positioned simultaneously as a fully automatic system and as an interactive system, depending on how spatial prompts are initialized.

2. Architectural composition

Medal S consists of three main modules. The first is an image encoder implemented as a 3D U-Net–style backbone that extracts multi-scale volumetric features and produces a per-voxel feature map $F \in \mathbb{R}^{C \times H \times W \times D}$ via progressive up-sampling and skip connections. The second is a text encoder: a frozen SAT text encoder $\Phi_{\text{text}}$ maps each anatomical prompt $t_j$ to a $d$-dim embedding $z_j \in \mathbb{R}^d$, and a transformer-based query decoder $\Phi_{\text{query}}$ fuses $z_j$ with multi-scale visual features $V$ to produce adapted query embeddings $T \in \mathbb{R}^{N \times C}$, where $N$ is the number of semantic targets. The third is the query decoder plus spatial prompt refinement stage, which integrates native-resolution volumetric prompts $\Phi_{\text{text}}$0 with textual queries $\Phi_{\text{text}}$1 and yields final per-voxel probability maps $\Phi_{\text{text}}$2 (Shi et al., 17 Nov 2025).

The framework is end-to-end trainable. The image encoder, the query decoder, including the light 3D convolutional module, and prompt-generation data augmentation are trained jointly with a BCE+Dice loss. The model therefore couples semantic target specification, spatial conditioning, and voxel-wise refinement within a single optimization pipeline rather than delegating these stages to separate systems.

3. Channel-wise alignment and native-resolution refinement

The central technical mechanism is channel-wise alignment between full-resolution volumetric prompts and adapted text queries. Let $\Phi_{\text{text}}$3 denote visual features, $\Phi_{\text{text}}$4 the adapted text queries, and $\Phi_{\text{text}}$5 the spatial prompts. Medal S computes spatio-textual aligned features via per-voxel query correlation:

$\Phi_{\text{text}}$6

with $\Phi_{\text{text}}$7 and $\Phi_{\text{text}}$8, yielding $\Phi_{\text{text}}$9. The model then concatenates $t_j$0 and applies a light-weight 3D convolutional block,

$t_j$1

followed by a per-voxel inner product with queries and sigmoid activation,

$t_j$2

which yields $t_j$3 (Shi et al., 17 Nov 2025).

Two implementation choices are decisive. First, Medal S keeps volumetric prompts at native resolution and does not downsample $t_j$4, preserving every voxel’s spatial fidelity. Second, all $t_j$5 classes are processed in a single forward pass. The paper attributes to this design a more than 90% inference-time reduction for 24-class segmentation relative to sequential prompting. The lightweight 3D convolutional refinement is described as a single or small stack of $t_j$6 convolutions inspired by nnU-Net skip connections, operating directly in voxel space while maintaining channel alignment.

4. Prompting modes and iterative decoding

Medal S supports two prompting modes at inference. In text-only self-refinement, the initialization is $t_j$7, corresponding to empty prompts. The model uses text queries $t_j$8 alone to produce an initial prediction $t_j$9, then feeds $d$0 back as a spatial prompt and repeats the $d$1–$d$2 interaction for $d$3 iterations. The paper characterizes this as fully automatic and “zero-click” refinement (Shi et al., 17 Nov 2025).

In hybrid mode, users supply initial spatial cues at native resolution, including points, scribbles, and boxes. These manual $d$4 guide the first pass, and subsequent predictions $d$5 serve as refined prompts. This combines human correction with text guidance for flexible interactive use. The distinction between the two modes is operational rather than architectural: the same spatio-textual alignment mechanism is reused, but the source of the initial spatial prompt differs.

The iterative query decoder inference procedure includes random masking. For $d$6 iterations and $d$7 random masking rounds per iteration, the method performs complementary masked and unmasked predictions and averages them to improve robustness. This suggests that prompt refinement is not merely recursive reuse of prior predictions, but a regularized inference procedure intended to stabilize performance under imperfect prompt conditions.

5. Data augmentation, resampling, and inference pipeline

The paper introduces dynamic resampling to address target-patch ratio imbalance in medical volumes, where object size can vary substantially relative to a fixed patch size. Voxel spacing $d$8 is adapted to target spacing $d$9 based on the smallest foreground component size $z_j \in \mathbb{R}^d$0 and patch size $z_j \in \mathbb{R}^d$1:

$z_j \in \mathbb{R}^d$2

where $z_j \in \mathbb{R}^d$3 is a reference spacing, $z_j \in \mathbb{R}^d$4 is a scale factor, and $z_j \in \mathbb{R}^d$5 is clamped to practical bounds. The stated purpose is to ensure that small targets are not lost, reducing false negatives, and that large targets do not dominate, reducing false positives. The implementation uses an efficient CPU-based resampling module borrowed from nnU-Net (Shi et al., 17 Nov 2025).

Spatial prompt generation is also synthesized during training. Algorithm 1 generates realistic, noisy prompts $z_j \in \mathbb{R}^d$6 and $z_j \in \mathbb{R}^d$7 from ground-truth masks $z_j \in \mathbb{R}^d$8 by random block drop/add, channel dropout, and empty-prompt sampling. This places prompt corruption and incompleteness directly within the training distribution.

Inference is organized as a two-stage coarse-to-fine procedure. Stage 1 uses voxel spacing $z_j \in \mathbb{R}^d$9 and patch size $\Phi_{\text{query}}$0, corresponding to an approximately $\Phi_{\text{query}}$1 mm field of view, and applies fast sliding-window inference to detect ROI; if no foreground is found, the system falls back to a full high-resolution pass. Stage 2 resamples the ROI to $\Phi_{\text{query}}$2 with patch size $\Phi_{\text{query}}$3, enlarges the crop by $\Phi_{\text{query}}$4–$\Phi_{\text{query}}$5 to include context, and imposes the physical-volume constraint

$\Phi_{\text{query}}$6

If this constraint is exceeded, $\Phi_{\text{query}}$7 is adjusted to satisfy memory $\Phi_{\text{query}}$8 GB. The Stage 1 output is then used as a spatial prompt for fine segmentation.

Post-processing is class-wise component filtering. For each class $\Phi_{\text{query}}$9, the method thresholds $z_j$0 to obtain a binary mask $z_j$1, labels 6-connected components, and computes each component’s size and mean probability. It retains, among the top-3 components by size, those with mean probability within $z_j$2 of the maximum and greater than $z_j$3. If none satisfy this condition, it keeps the largest component, or the second if its size is at least $z_j$4 the largest. All other components are removed.

6. Optimization, training configuration, and evaluation protocol

The loss function is the sum of a binary cross-entropy term and a Dice term:

$z_j$5

with

$z_j$6

and

$z_j$7

This formulation reflects the dual objective of per-voxel classification and overlap-sensitive segmentation quality (Shi et al., 17 Nov 2025).

Two training protocols are reported. The Stage 1 model uses a pre-trained SAT backbone, batch size $z_j$8, patch size $z_j$9 at spacing $V$0, $V$1k steps, AdamW, learning rate $V$2 with cosine decay, $V$3M parameters, and $V$4 hours on $V$5 H100 GPUs. The Stage 2 model uses batch size $V$6, patch size $V$7 at $V$8, $V$9k steps, the same optimizer, and $T \in \mathbb{R}^{N \times C}$0 hours on $T \in \mathbb{R}^{N \times C}$1 H100 GPUs.

The evaluation protocol distinguishes semantic and instance metrics. Semantic performance is measured by Dice Similarity Coefficient (DSC) and Normalized Surface Distance (NSD). Instance performance is measured by F1 at $T \in \mathbb{R}^{N \times C}$2 overlap and DSC over true positive instances (DSC TP). The runtime limit is $T \in \mathbb{R}^{N \times C}$3 s per class.

7. Reported results, comparative standing, and interpretation

On the validation set, using the five-modality average in the all-data track, Medal S reports DSC $T \in \mathbb{R}^{N \times C}$4 versus $T \in \mathbb{R}^{N \times C}$5 for SAT, NSD $T \in \mathbb{R}^{N \times C}$6 versus $T \in \mathbb{R}^{N \times C}$7, F1 $T \in \mathbb{R}^{N \times C}$8 versus $T \in \mathbb{R}^{N \times C}$9, and DSC TP $N$0 versus $N$1 (Shi et al., 17 Nov 2025). The same table lists CAT at DSC $N$2, NSD $N$3, F1 $N$4, and DSC TP $N$5, while BiomedParse-V is reported at DSC $N$6, NSD $N$7, F1 $N$8, and DSC TP $N$9.

Modality-specific comparisons against SAT are heterogeneous. For CT, Medal S reports DSC $\Phi_{\text{text}}$00 versus $\Phi_{\text{text}}$01, NSD $\Phi_{\text{text}}$02 versus $\Phi_{\text{text}}$03, F1 $\Phi_{\text{text}}$04 versus $\Phi_{\text{text}}$05, and DSC TP $\Phi_{\text{text}}$06 versus $\Phi_{\text{text}}$07. For MRI, the corresponding values are DSC $\Phi_{\text{text}}$08 versus $\Phi_{\text{text}}$09, NSD $\Phi_{\text{text}}$10 versus $\Phi_{\text{text}}$11, F1 $\Phi_{\text{text}}$12 versus $\Phi_{\text{text}}$13, and DSC TP $\Phi_{\text{text}}$14 versus $\Phi_{\text{text}}$15. For microscopy, the reported instance metrics are F1 $\Phi_{\text{text}}$16 versus $\Phi_{\text{text}}$17 and DSC TP $\Phi_{\text{text}}$18 versus $\Phi_{\text{text}}$19. PET is an exception in the reported results: SAT is slightly higher in instance metrics, with F1 $\Phi_{\text{text}}$20 versus $\Phi_{\text{text}}$21 and DSC TP $\Phi_{\text{text}}$22 versus $\Phi_{\text{text}}$23. Ultrasound is also mixed: SAT is higher in DSC, $\Phi_{\text{text}}$24 versus $\Phi_{\text{text}}$25, while Medal S is slightly higher in NSD, $\Phi_{\text{text}}$26 versus $\Phi_{\text{text}}$27.

On the test set in the all-data track, Medal S outperforms SAT by approximately $\Phi_{\text{text}}$28 points in DSC, $\Phi_{\text{text}}$29 versus $\Phi_{\text{text}}$30, and approximately $\Phi_{\text{text}}$31 points in NSD, $\Phi_{\text{text}}$32 versus $\Phi_{\text{text}}$33. The paper therefore presents Medal S as superior to SAT in overall multi-class medical segmentation efficiency and accuracy, especially when parallel spatial prompting is compared with sequential prompting.

These results also delimit the model’s comparative standing. A common overgeneralization would be to read Medal S as uniformly dominant across all reported benchmarks. The reported validation averages do not support that reading, since BiomedParse-V has higher average DSC, NSD, F1, and DSC TP, and SAT remains higher on PET F1 and DSC TP and on ultrasound DSC. What the reported evidence does support is a specific claim: Medal S improves substantially over SAT while harmonizing spatial precision with semantic textual guidance and reducing the inference burden associated with sequential prompt-based approaches. The model is reported as being publicly available at https://github.com/yinghemedical/Medal-S.