MedSeg-TTA: Test-Time Adaptation in Segmentation

• The paper introduces a large-scale benchmark for test-time adaptation in medical image segmentation under source-free, domain shift conditions.
• MedSeg-TTA organizes methods into four paradigms—input-level, feature-level, output-level, and prior estimation—to address challenges across various imaging modalities.
• It reveals that no single adaptation strategy excels in all conditions, emphasizing the need for hybrid approaches tailored to specific modality shifts.

MedSeg-TTA denotes test-time adaptation for medical image segmentation under domain shift and, more specifically, the benchmark introduced in “A Large Scale Benchmark for Test Time Adaptation Methods in Medical Image Segmentation” (Yu et al., 2 Dec 2025). The setting is source-free: a segmentation model trained on a source domain is deployed on unlabeled target data from new centers, scanners, devices, protocols, or populations, and adaptation is carried out at inference time without access to source images or target annotations. Recent work has expanded this setting from classical entropy minimization to anchor-guided pseudo-labeling, inter-sequence MRI adaptation, prompt and LoRA tuning for SAM-like models, and training-free augmentation ensembles, making MedSeg-TTA a distinct subfield at the intersection of medical image segmentation, domain adaptation, and foundation-model deployment (Yu et al., 2 Dec 2025).

1. Problem formulation and scope

In MedSeg-TTA, the central objective is to reduce target-domain risk using only unlabeled target inputs observed at deployment time. A representative formulation appears in multi-sequence MRI adaptation, where a co-registered target volume is written as $$\mathbf{x} \in \mathbb{R}^{S \times H \times W \times D}$$, logits are $\mathbf{z} = f_\theta(\mathbf{x})$, probabilities are $\mathbf{p} = \sigma(\mathbf{z})$, and the model is updated online by minimizing an unsupervised loss $\mathcal{L}(\theta; \mathbf{x}_t)$ on each incoming case (Deng et al., 17 May 2026). This source-free, online, continual setting is now common across recent MedSeg-TTA work, although the adaptable parameters vary substantially: some methods update only BatchNorm affine parameters, some adapt prompt embeddings, some optimize LoRA modules, and some avoid parameter updates altogether by transforming inputs or aggregating test-time views (Deng et al., 17 May 2026).

The benchmark formalization is intentionally strict. MedSeg-TTA evaluates twenty representative methods across seven imaging modalities—MRI, CT, ultrasound, pathology, dermoscopy, OCT, and chest X-ray—under fully unified data preprocessing, backbone configuration, and test-time protocols (Yu et al., 2 Dec 2025). All methods share the same UNet backbone family, identical source-free constraints, and the same test-time evaluation rules, so performance differences are attributable to the adaptation strategy rather than to heterogeneous implementations or backbones (Yu et al., 2 Dec 2025).

This problem formulation is broader than simple test-time augmentation. Some related work uses augmentation at inference without online learning, while other methods perform actual optimization on target samples. A plausible implication is that “MedSeg-TTA” now names a family of deployment-time strategies rather than a single algorithmic recipe.

2. Benchmark structure and paradigm taxonomy

The benchmark organizes MedSeg-TTA methods into four paradigms: Input-level Transformation, Feature-level Alignment, Output-level Regularization, and Prior Estimation (Yu et al., 2 Dec 2025). This taxonomy is important because recent results show that no single paradigm performs best in all conditions (Yu et al., 2 Dec 2025).

Paradigm	Typical mechanism	Representative methods
Input-level Transformation	Transform target images toward source-style appearance before segmentation	SFDA-FSM, RSA, DL-TTA, STDR, AIF-SFDA
Feature-level Alignment	Align latent feature distributions or stabilize feature-space updates	DANN, TestFit, GraTa, UDA-MIMA, DeTTA
Output-level Regularization	Use entropy minimization, pseudo-labeling, or consistency on predictions	TENT, DG-TTA, SaTTCA, UPL-SFDA, SmaRT
Prior Estimation	Inject shape, prompt, prototype, or structural priors	AdaMI, PASS, ProSFDA, ExploringTTA, VPTTA

The seven source–target pairs span cross-center, cross-device, and cross-population shifts. They include BraTS-GLI2024 to BraTS-SSA for brain tumor MRI, LiTS to 3D-IRCADB for liver CT, RIGA+(MES) to RIGA+(MB) for fundus/OCT-style optic disc and cup segmentation, TN3K to DDTI for thyroid ultrasound, CRAG to GlaS for pathology, ISIC-2017 to PH² for dermoscopy, and Shenzhen to Montgomery for chest X-ray lung segmentation (Yu et al., 2 Dec 2025). Volumes are standardized to $128 \times 128 \times 128$ for 3D tasks, and 2D images or pathology patches to $256 \times 256$ (Yu et al., 2 Dec 2025).

A notable aspect of this benchmark design is that it evaluates both overlap and boundary fidelity. Dice, Jaccard, sensitivity, positive predictive value, and HD95 are all reported, which matters because many TTA methods improve average overlap while worsening boundary behavior or producing clinically undesirable false positives (Yu et al., 2 Dec 2025).

3. Core algorithmic patterns

One major MedSeg-TTA branch operates through image-space transformation or augmentation. “S$^3$-TTA: Scale-Style Selection for Test-Time Augmentation in Biomedical Image Segmentation” selects the suitable image scale and style for each test image using a transformation consistency metric and reports improvements over prior art by 3.4% on cell segmentation and 1.3% on lung segmentation (Xie et al., 2023). “SegTTA: Training-Free Test-Time Augmentation for Zero-Shot Medical Imaging Segmentation” combines Gamma correction, Contrast enhancement, Gaussian blur, and Gaussian noise with weighted voting across multiple MedSAM2 checkpoints, and on a multiclass hepatic vessel dataset reports an increase of 1.6 in mIoU and 1.9 in aIoU together with a reduction of approximately 2.0 in HD95 (Yao et al., 19 Apr 2026). A related TTA-centric line uses test-time augmentation as a quality-estimation backbone rather than as deployment-time model adaptation: “Test-time augmentation-based active learning and self-training for label-efficient segmentation” performs 3D TTA with 16 variants, computes a TTA-based estimated Dice $\hat{D}$, and uses it jointly for self-training and active learning (Specktor-Fadida et al., 2023).

A second branch centers on pseudo-labeling, consistency, and feature refinement. “A3-TTA: Adaptive Anchor Alignment Test-Time Adaptation for Image Segmentation” identifies well-predicted target-domain images using a class compact density metric, stores them as anchors, aligns target bottleneck features to the closest anchor, and regularizes learning with semantic consistency, boundary-aware entropy minimization, and a self-adaptive EMA teacher; on multi-domain medical images it improves average Dice scores by 10.40 to 17.68 percentage points compared to the source model (Wu et al., 3 Feb 2026). “VISTA: Variance-Gated Inter-Sequence Test-Time Adaptation for Multi-Sequence MRI Segmentation” addresses a dual-shift problem in multi-sequence MRI by generating inter-sequence intervention views and optimizing

$$\mathcal{L} = \mathcal{L}_{\mathrm{PL}} + \lambda \mathcal{L}_{\mathrm{CONS}},$$

with voxel-wise variance gating to suppress unreliable pseudo-labels (Deng et al., 17 May 2026). “HD-TTA: Hypothesis-Driven Test-Time Adaptation for Safer Brain Tumor Segmentation” reformulates adaptation as a dynamic decision process, generating compaction and inflation hypotheses and using a Gatekeeper plus a representation-guided selector to prioritize safer outcomes, with reported reductions of approximately 6.4 mm in HD95 and improvements of over 4% in Precision on challenging safety-oriented brain tumor transfer settings (Jhawar et al., 23 Feb 2026).

These methods differ in where they locate reliability. Augmentation-based systems assume some transformed views are closer to the source domain; anchor-based systems assume a subset of target samples is already source-like; inter-sequence MRI systems assume predictions that remain stable under anatomy-preserving interventions are more trustworthy; safety-oriented systems assume adaptation should be selective rather than universally applied. This suggests that MedSeg-TTA is increasingly organized around reliability estimation, not just unsupervised loss design.

4. Foundation-model and vision-language variants

Foundation-model adaptation has become a major MedSeg-TTA frontier. “SAM-aware Test-time Adaptation for Universal Medical Image Segmentation” introduces Self-adaptive Bezier Curve-based Transformation (SBCT) for input adaptation and Dual-scale Uncertainty-driven Mean Teacher adaptation (DUMT) for semantic alignment, while keeping the SAM mask decoder frozen and updating only LoRA modules in the image encoder, the prompt encoder, and the 12 SBCT parameters (Wu et al., 5 Jun 2025). Across five public datasets, SAM-TTA outperforms existing TTA approaches and even surpasses fully fine-tuned models such as MedSAM in certain scenarios (Wu et al., 5 Jun 2025).

“Concept Alignment Contrast and Long-Short Prompt Memory for Test-Time Adaptation of SAM3 in Medical Image Segmentation” pushes this further into vision-language adaptation. It updates only learnable text prompt token embeddings $T \in \mathbb{R}^{N \times D}$ with $\mathbf{z} = f_\theta(\mathbf{x})$0 and $\mathbf{z} = f_\theta(\mathbf{x})$1, uses Concept Alignment Contrast (CAC) to select the best augmented view, and stabilizes continual one-pass adaptation with Long-Short Prompt Memory and Densely Supervised Prompt Update (Zhou et al., 22 Jun 2026). On PROMISE12, CM-TTA reaches 73.96 ± 24.36 Dice and 10.91 ± 17.71 ASSD; on ISIC2018 it reaches 82.34 ± 20.42 Dice and 9.39 ± 14.38 ASSD, outperforming ZERO, TPT, HisTPT, TENT, and TTL (Zhou et al., 22 Jun 2026).

A closely related but not strictly test-time-updating line is “SegMoTE: Token-Level Mixture of Experts for Medical Image Segmentation” (Lu et al., 22 Feb 2026). That work does not perform online adaptation or self-training at test time, but it uses token-level expert routing as an input-conditional adaptation mechanism inside SAM’s decoder and is explicitly discussed as conceptually related to MedSeg-TTA (Lu et al., 22 Feb 2026).

Open-vocabulary dense prediction has also entered the field. “Test-Time Adaptation of Vision-LLMs for Open-Vocabulary Semantic Segmentation” proposes Multi-Level and Multi-Prompt entropy minimization, adapting only LayerNorm parameters and showing that segmentation-specific TTA benefits from intermediate features and prompt diversity (Noori et al., 28 May 2025). “Test-Time Optimization for Domain Adaptive Open Vocabulary Segmentation” introduces Seg-TTO, which performs pixel-level self-supervised optimization, visual feature aggregation, and per-category textual embedding tuning; it reports gains on several medical tasks in the MESS benchmark, including retinal, histology, endoscopy, and chest X-ray segmentation (Silva et al., 8 Jan 2025).

5. Empirical regularities across modalities and shifts

The clearest benchmark conclusion is that no single paradigm performs best in all conditions (Yu et al., 2 Dec 2025). Input-level methods are more stable under mild appearance shifts, especially in fundus/OCT, dermoscopy, and chest X-ray, where style and intensity differences dominate (Yu et al., 2 Dec 2025). Feature-level and Output-level methods offer greater advantages in boundary-related metrics, particularly HD95, in strong-shift 3D settings such as MRI and CT (Yu et al., 2 Dec 2025). Prior-based methods exhibit strong modality dependence, often helping when anatomy is stable but degrading in settings with large structural variability such as pathology and some lesion-centric tasks (Yu et al., 2 Dec 2025).

Individual methods reinforce this picture. VISTA reports absolute Dice improvements of +1.89% on BraTS-SSA and +2.82% on BraTS-PED over the source model, specifically by modeling modality-interaction shifts in multi-sequence MRI (Deng et al., 17 May 2026). A3-TTA shows that anchor-guided pseudo-labeling and adaptive EMA yield both strong per-domain gains and improved continual TTA behavior across sequential target domains (Wu et al., 3 Feb 2026). SAM-TTA shows that preserving SAM’s generalization while adapting only small parameter subsets can outperform heavier medical fine-tuning on unseen grayscale targets (Wu et al., 5 Jun 2025). By contrast, the benchmark documents that several methods degrade significantly under large inter-center and inter-device shifts, which highlights the importance of principled method selection for clinical deployment (Yu et al., 2 Dec 2025).

A further regularity is that boundary-sensitive metrics are often more discriminative than Dice alone. This is especially visible in safety-oriented brain tumor TTA, multiclass hepatic vessel segmentation, and pathology patch segmentation, where a method may improve average overlap while harming HD95 or structural plausibility (Jhawar et al., 23 Feb 2026). The benchmark’s emphasis on Dice and HD95 together therefore aligns with current practice in clinically relevant MedSeg-TTA evaluation (Yu et al., 2 Dec 2025).

6. Limitations and future directions

Current MedSeg-TTA methods remain constrained by modality assumptions, optimization stability, and architectural scope. The benchmark identifies stable adaptation and streaming forgetting as an unresolved challenge, alongside style–content disentanglement, test-time hyperparameter adaptation, representation alignment and class prototypes, prior estimation and weak prompts, and semantic anchors with retrieval-enhanced TTA (Yu et al., 2 Dec 2025). These issues are not merely engineering details; they determine whether a method remains reliable under continual deployment.

Method-specific limitations are equally instructive. VISTA assumes co-registered and aligned sequences, tunes thresholds only on a held-out 20% source validation set, and does not study abrupt domain switches within a stream (Deng et al., 17 May 2026). HD-TTA is a proof-of-concept on binary whole-tumor segmentation and carries a non-trivial runtime cost for flagged cases (Jhawar et al., 23 Feb 2026). CM-TTA is currently 2D and single-class, and its prompt-based adaptation depends on prompt quality and CAC ranking (Zhou et al., 22 Jun 2026). SAM-TTA is also 2D, depends on SAM’s base capability, and still requires prompts such as bounding boxes (Wu et al., 5 Jun 2025). SegTTA is training-free but multiplies inference cost through multiple augmentations and multiple MedSAM2 checkpoints (Yao et al., 19 Apr 2026).

A plausible implication is that the next phase of MedSeg-TTA will be hybrid. Benchmark evidence favors paradigm selection by modality and shift regime rather than universal prescriptions (Yu et al., 2 Dec 2025), while recent methods increasingly combine complementary mechanisms: input transformation with lightweight parameter updates, pseudo-labeling with uncertainty or anchor selection, dual-scale or inter-view consistency with structural priors, and foundation-model prompting with selective adaptation (Wu et al., 5 Jun 2025). In that sense, MedSeg-TTA has evolved from a narrow source-free optimization trick into a broader framework for clinically constrained, deployment-time model correction in medical image segmentation.