Segmentation–Regression Consistency Learning

• Segmentation–regression consistency learning is a multi-task approach that enforces agreement between categorical segmentation maps and continuous regression outputs to capture anatomical and geometric details.
• It employs both intra-task and cross-task consistency constraints, such as CAM-based attention and ROI enhancement, to refine predictions in applications like scoliosis assessment and liver tumor analysis.
• Empirical studies show that integrating boundary-aware regression and uncertainty-guided filtering significantly improves segmentation quality and clinical prediction accuracy.

Searching arXiv for the cited papers to ground the article in the relevant literature. Segmentation–regression consistency learning denotes a class of multi-task formulations in which a dense segmentation output and a continuous regression output are trained to remain mutually compatible, so that geometric, anatomical, or clinical information carried by one task regularizes the other. In the cited medical-imaging literature, this pattern appears in semi-supervised segmentation through joint mask prediction and signed distance map regression, in scoliosis assessment through joint spine segmentation and Cobb angle regression, and in liver tumor analysis through coupling tumor segmentation with dynamic enhancement regression (Zhang et al., 2021, Lin et al., 2022, Xiao et al., 25 Nov 2025). A related consistency-regularization line, although not itself a segmentation–regression formulation, shows that consistency constraints can also improve calibration by suppressing overconfident predictions near ambiguous boundaries (Karani et al., 2023).

1. Conceptual scope and problem setting

Across these works, segmentation and regression are not treated as isolated outputs. The segmentation branch supplies spatial support, shape, or region-of-interest structure, while the regression branch supplies continuous geometric or clinical information that is difficult to encode with categorical labels alone (Zhang et al., 2021, Lin et al., 2022, Xiao et al., 25 Nov 2025). The resulting interaction can be local, as when a mask gates the input to a regressor, or global, as when two inference paths are required to agree after passing through different intermediate representations.

A central distinction is between intra-task consistency and cross-task consistency. In UG-MCL, intra-task consistency refers to student–teacher agreement within the segmentation task and within the signed distance regression task, whereas cross-task consistency aligns segmentation probabilities with the inverse-transformed signed distance output (Zhang et al., 2021). In Seg4Reg+, consistency also has two directions, but the emphasis is bidirectional transfer: segmentation regularizes regression through class activation maps, and regression-derived attention refines segmentation (Lin et al., 2022). In MTI-Net, consistency is expressed structurally by making dynamic enhancement regression depend on the predicted tumor region and by adding a segmentation-derived regression loss (Xiao et al., 25 Nov 2025).

This literature therefore supports a narrow technical reading of the term: segmentation–regression consistency learning is not generic multi-task learning, but multi-task learning in which the two tasks are explicitly constrained to agree in representation space, attention space, or task-conditioned input space.

2. Recurrent mathematical patterns

A common pattern is the combination of task-specific supervision with an additional cross-task coupling term. In UG-MCL, the supervised objective on labeled data is

$$\mathcal{L}_{sup}(\theta;\mathcal{D}_{L}) = \mathcal{L}_{dice}(f_{seg},y) + \mathcal{L}_{ce}(f_{seg},y) + \mathcal{L}_{dis}(f_{dis},\mathcal{T}(y)),$$

and the full objective is

$$\min \limits_{\theta} \mathcal{L}_{sup}(\theta;\mathcal{D}_{L}) + \lambda_{i} \mathcal{L}_{itc}(\theta;\theta^{'};\mathcal{D}) + \lambda_{c} \mathcal{L}_{ctc}(\theta;\mathcal{D}),$$

with the cross-task term

$$\mathcal{L}_{ctc}(\theta;\mathcal{D}) = \| f_{seg} - \mathcal{T}^{-1}( f_{dis}) \| ^{2}.$$

Here the regression target is a signed distance map, and consistency is enforced after mapping the regression output back into a segmentation-like space (Zhang et al., 2021).

Seg4Reg+ uses a different coupling. The segmentation network $N_S$ predicts a spine mask $s(x)$, while the regression network $N_R$ predicts three Cobb angles: proximal thoracic, main thoracic, and thoracolumbar/lumbar. The method combines segmentation loss, SMAPE regression loss, and an attention regularization loss

$$\mathcal{L}_{\mathrm{AR}} = \| C(x, s(x)) - C(s(x))\|_1,$$

where CAMs computed from image-segmentation input pairs and from segmentation-only inputs are forced to agree (Lin et al., 2022). The regression-to-segmentation direction is implemented by the ROIE gate,

$f'_m(x) = \alpha(C(r(x))\circ f_m(x)) + f_m(x),$

which injects regression-derived CAMs into a middle-layer segmentation feature map (Lin et al., 2022).

MTI-Net expresses the coupling through task-conditioned masking and loss-based alignment. TIM performs element-wise multiplication between the predicted segmentation mask and each image in the dynamic MRI sequence, conceptually

$$\mathcal{X}^{t}_{\text{masked}} = \hat{\mathcal{Y}}_S \odot \mathcal{X}^{t},$$

so that regression is computed from segmented tumor regions (Xiao et al., 25 Nov 2025). It also introduces

$\mathcal{L}^{1}_{seg}(\hat{\mathcal{Y}_{SI}},\mathcal{Y}_I) = \frac{1}{N}\sum_{n=1}^{N}\left|\hat{\mathcal{Y}_{SI}^n - \mathcal{Y}_I^n\right|,$

which makes the segmentation output itself useful for predicting enhancement and thereby enforces compatibility between segmentation and regression (Xiao et al., 25 Nov 2025).

These designs differ operationally, but they share a structural principle: supervision for each task is augmented by a mechanism that penalizes disagreement between what the segmentation implies and what the regression predicts.

3. Dual-task semi-supervised segmentation via signed distance regression

UG-MCL provides a direct and explicit instance of segmentation–regression consistency learning in semi-supervised medical image segmentation (Zhang et al., 2021). The model is built on a Mean Teacher backbone with a student network and an exponential-moving-average teacher,

$$\min \limits_{\theta} \mathcal{L}_{sup}(\theta;\mathcal{D}_{L}) + \lambda_{i} \mathcal{L}_{itc}(\theta;\theta^{'};\mathcal{D}) + \lambda_{c} \mathcal{L}_{ctc}(\theta;\mathcal{D}),$$0

Its dual-task backbone has two output branches: $$\min \limits_{\theta} \mathcal{L}_{sup}(\theta;\mathcal{D}_{L}) + \lambda_{i} \mathcal{L}_{itc}(\theta;\theta^{'};\mathcal{D}) + \lambda_{c} \mathcal{L}_{ctc}(\theta;\mathcal{D}),$$1 for segmentation probabilistic maps and $$\min \limits_{\theta} \mathcal{L}_{sup}(\theta;\mathcal{D}_{L}) + \lambda_{i} \mathcal{L}_{itc}(\theta;\theta^{'};\mathcal{D}) + \lambda_{c} \mathcal{L}_{ctc}(\theta;\mathcal{D}),$$2 for signed distance maps. The regression branch is implemented as a 3D convolution block followed by a hyperbolic tangent activation (Zhang et al., 2021).

The signed distance map is a continuous geometric representation of the object boundary. The paper transforms the binary ground truth into a signed distance target $$\min \limits_{\theta} \mathcal{L}_{sup}(\theta;\mathcal{D}_{L}) + \lambda_{i} \mathcal{L}_{itc}(\theta;\theta^{'};\mathcal{D}) + \lambda_{c} \mathcal{L}_{ctc}(\theta;\mathcal{D}),$$3, so that the auxiliary regression task carries boundary and shape priors that standard pixel-wise segmentation does not capture directly (Zhang et al., 2021). This is the defining segmentation–regression pairing in the method: one branch predicts voxel-wise class probabilities, the other regresses a boundary-aware continuous field.

UG-MCL distinguishes two forms of consistency. The first is intra-task consistency, enforced between student and teacher outputs for both branches. The second is cross-task consistency, enforced between the segmentation output and the inverse-transformed distance output. The inverse transform is defined as

$$\min \limits_{\theta} \mathcal{L}_{sup}(\theta;\mathcal{D}_{L}) + \lambda_{i} \mathcal{L}_{itc}(\theta;\theta^{'};\mathcal{D}) + \lambda_{c} \mathcal{L}_{ctc}(\theta;\mathcal{D}),$$4

so that the continuous SDF prediction can be compared directly with a segmentation-like probability map (Zhang et al., 2021).

A further element is uncertainty guidance. Teacher uncertainty is estimated with Monte Carlo Dropout using $$\min \limits_{\theta} \mathcal{L}_{sup}(\theta;\mathcal{D}_{L}) + \lambda_{i} \mathcal{L}_{itc}(\theta;\theta^{'};\mathcal{D}) + \lambda_{c} \mathcal{L}_{ctc}(\theta;\mathcal{D}),$$5 stochastic forward passes, and predictive entropy

$$\min \limits_{\theta} \mathcal{L}_{sup}(\theta;\mathcal{D}_{L}) + \lambda_{i} \mathcal{L}_{itc}(\theta;\theta^{'};\mathcal{D}) + \lambda_{c} \mathcal{L}_{ctc}(\theta;\mathcal{D}),$$6

is used as voxel-wise uncertainty. Only voxels satisfying $$\min \limits_{\theta} \mathcal{L}_{sup}(\theta;\mathcal{D}_{L}) + \lambda_{i} \mathcal{L}_{itc}(\theta;\theta^{'};\mathcal{D}) + \lambda_{c} \mathcal{L}_{ctc}(\theta;\mathcal{D}),$$7 are retained for consistency learning (Zhang et al., 2021). The paper reports that the threshold is ramped from $$\min \limits_{\theta} \mathcal{L}_{sup}(\theta;\mathcal{D}_{L}) + \lambda_{i} \mathcal{L}_{itc}(\theta;\theta^{'};\mathcal{D}) + \lambda_{c} \mathcal{L}_{ctc}(\theta;\mathcal{D}),$$8 to $$\min \limits_{\theta} \mathcal{L}_{sup}(\theta;\mathcal{D}_{L}) + \lambda_{i} \mathcal{L}_{itc}(\theta;\theta^{'};\mathcal{D}) + \lambda_{c} \mathcal{L}_{ctc}(\theta;\mathcal{D}),$$9, with the explicit purpose of ignoring unreliable teacher predictions on unlabeled data.

The method is therefore not merely a self-ensembling model with an auxiliary head. Its distinctive property is that boundary-aware regression and segmentation are trained to agree, while uncertainty estimates determine which unlabeled voxels are trustworthy enough to participate in that agreement.

4. Bidirectional and global consistency in clinical prediction pipelines

Seg4Reg+ moves the segmentation–regression consistency idea from geometric auxiliary supervision to a clinically motivated estimation task: Cobb angle prediction for scoliosis assessment (Lin et al., 2022). Its framework contains two networks, $$\mathcal{L}_{ctc}(\theta;\mathcal{D}) = \| f_{seg} - \mathcal{T}^{-1}( f_{dis}) \| ^{2}.$$0 for spine segmentation and $$\mathcal{L}_{ctc}(\theta;\mathcal{D}) = \| f_{seg} - \mathcal{T}^{-1}( f_{dis}) \| ^{2}.$$1 for regression of the three Cobb angles. The paper explicitly argues that segmentation and regression should not be treated as independent or merely cascaded tasks. Instead, the segmentation mask should help the regressor focus on the spine, while the regressor’s CAM should refine segmentation in return.

The local coupling mechanism has two parts. First, the attention regularization module expands the regression network into a shared-weight Siamese structure with one branch taking the concatenation of raw image and segmentation mask and the other taking the segmentation mask alone. Consistency between the two CAMs is enforced by $$\mathcal{L}_{ctc}(\theta;\mathcal{D}) = \| f_{seg} - \mathcal{T}^{-1}( f_{dis}) \| ^{2}.$$2, so that the regressor is encouraged to focus on the spine region rather than spurious image areas (Lin et al., 2022). Second, the region-of-interest enhancement gate uses regression CAMs as an attention map for segmentation features, with the stated effect of improving intra-class compactness and semantic consistency (Lin et al., 2022).

The paper’s global mechanism is triangle consistency learning, inspired by inference-path invariance. The three corners of the triangle are the input image $$\mathcal{L}_{ctc}(\theta;\mathcal{D}) = \| f_{seg} - \mathcal{T}^{-1}( f_{dis}) \| ^{2}.$$3, the segmentation mask $$\mathcal{L}_{ctc}(\theta;\mathcal{D}) = \| f_{seg} - \mathcal{T}^{-1}( f_{dis}) \| ^{2}.$$4, and the Cobb angle prediction $$\mathcal{L}_{ctc}(\theta;\mathcal{D}) = \| f_{seg} - \mathcal{T}^{-1}( f_{dis}) \| ^{2}.$$5. The five-process training schedule is: train basic $$\mathcal{L}_{ctc}(\theta;\mathcal{D}) = \| f_{seg} - \mathcal{T}^{-1}( f_{dis}) \| ^{2}.$$6, train $$\mathcal{L}_{ctc}(\theta;\mathcal{D}) = \| f_{seg} - \mathcal{T}^{-1}( f_{dis}) \| ^{2}.$$7 with AR, fine-tune $$\mathcal{L}_{ctc}(\theta;\mathcal{D}) = \| f_{seg} - \mathcal{T}^{-1}( f_{dis}) \| ^{2}.$$8 with ROIE, fine-tune $$\mathcal{L}_{ctc}(\theta;\mathcal{D}) = \| f_{seg} - \mathcal{T}^{-1}( f_{dis}) \| ^{2}.$$9 by SMAPE loss on $N_S$0 and $N_S$1, and fine-tune $N_S$2 with refined segmentation, then repeat (Lin et al., 2022). This establishes a closed optimization loop in which segmentation and regression supervise each other iteratively.

MTI-Net extends the same general logic to a three-task setting involving liver tumor segmentation, dynamic enhancement regression, and classification (Xiao et al., 25 Nov 2025). Its core interaction mechanism is the Task Interaction Module, motivated by the claim that dynamic enhancement regression is computed based on segmented tumor regions. TIM takes the segmentation prediction and gates the dynamic MRI sequence by element-wise multiplication, so regression is conditioned on the predicted lesion extent rather than the whole image. The paper describes this as higher-order consistency between segmentation and regression labels and as an additional constraint strategy that fosters synergy between tasks (Xiao et al., 25 Nov 2025).

The model supplements TIM with two additional modules. MdIEF performs Multi-domain Information Entropy Fusion by combining spatial-domain and spectral-domain features, and a shallow Transformer performs positional encoding to capture relationships within dynamic MRI sequences (Xiao et al., 25 Nov 2025). TDD, a task-driven discriminator, models high-order semantic dependencies, particularly between regression and classification, through an adversarial consistency constraint. These components are not themselves the direct segmentation–regression consistency term, but they define the broader multi-task setting in which that term operates.

5. Reported empirical behavior

The reported literature associates segmentation–regression consistency with gains in segmentation quality, regression accuracy, or both. In UG-MCL, experiments on the Left Atrium and BraTS 2019 datasets show that leveraging unlabeled data with signed distance regression and uncertainty-guided consistency improves performance over supervised training and over other semi-supervised methods such as MT, UA-MT, DTC, and URPC under the same backbone (Zhang et al., 2021). In Seg4Reg+, the AASCE Challenge experiments indicate that local CAM-based interaction and triangle consistency improve SMAPE and segmentation quality relative to simpler variants and prior methods (Lin et al., 2022). In MTI-Net, the synergy experiments are presented as direct evidence that adding regression improves segmentation and that full multi-task interaction improves regression and classification (Xiao et al., 25 Nov 2025).

System	Setting	Reported outcome
UG-MCL	Left Atrium, 16 labeled and 64 unlabeled scans	Dice 90.16%, Jaccard 82.18%, ASD 1.98, 95HD 6.50; supervised baseline Dice 86.03%
UG-MCL	BraTS 2019, 25 labeled and 225 unlabeled scans	Dice 82.82%, Jaccard 72.77%, ASD 2.30, 95HD 11.29
Seg4Reg+	AASCE Challenge, final Seg4Reg+ model	SMAPE = 7.32%, MAE = 3.73
Seg4Reg+	ROIE segmentation comparison	JA 77.86 and Dice 87.55 versus JA 75.47 and Dice 86.02 w/o CAMs
MTI-Net	Synergy experiment	Seg-only DSC 85.21; Seg + Reg DSC 86.34, MAE 48.72; MTI-Net MAE 44.35, Accuracy 92.8

Ablation studies further clarify what is being gained. In UG-MCL on Left Atrium segmentation, the segmentation-only supervised baseline reaches 86.03 Dice, adding distance supervision raises this to 87.88, adding intra-task consistency yields 88.68, and combining intra-task and cross-task consistency yields 90.16 (Zhang et al., 2021). The paper therefore attributes the best result not to the auxiliary regression branch alone, but to the combination of distance supervision, self-ensembling, cross-task agreement, and uncertainty-guided filtering.

Seg4Reg+ reports a staged ablation in which raw-image-only regression starts at SMAPE 12.32, adding AR reduces this to 9.39, adding ROIE gives 9.32, and adding TCL gives 9.17; when all parts are combined with raw image and segmentation mask together, SMAPE becomes 8.47, while the final model in Table 3 reaches 7.32% (Lin et al., 2022). The paper also states that Seg4Reg+ achieves the best SMAPE among compared methods, which matters because SMAPE is the official challenge metric.

For MTI-Net, the most direct evidence is the cross-task synergy table: adding regression to segmentation increases DSC from 85.21 to 86.34, and the full multi-task setting improves regression MAE from 48.72 to 44.35 while also increasing classification accuracy to 92.8 (Xiao et al., 25 Nov 2025). This suggests that segmentation–regression consistency is not only a regularizer on segmentation, but part of a larger inter-task optimization structure.

6. Boundary ambiguity, calibration, and limits of the paradigm

The cited literature also shows that consistency learning should not be reduced to a single mechanism or a single objective. UG-MCL explicitly separates temporal self-ensembling from cross-task alignment (Zhang et al., 2021). Seg4Reg+ explicitly rejects the view that segmentation and regression should merely coexist or be arranged in a simple cascade (Lin et al., 2022). MTI-Net frames its interaction as higher-order consistency rather than feature sharing alone (Xiao et al., 25 Nov 2025). A plausible implication is that “consistency” in this area is best understood as a family of coupling constraints rather than a fixed algorithmic template.

BWCR provides an adjacent but instructive case (Karani et al., 2023). It is a segmentation consistency-regularization method rather than a segmentation–regression model, yet it demonstrates a closely related principle: consistency penalties act as spatially varying regularizers, particularly near ambiguous boundaries where hard labels are unreliable. The method weights logit consistency by distance to the closest boundary,

$N_S$3

and reports improved calibration on prostate and cardiac MRI segmentation. On NCI with $N_S$4, BWCR gives Dice $N_S$5, ECE $N_S$6, and TACE $N_S$7, compared with CR at Dice $N_S$8, ECE $N_S$9, and TACE $s(x)$0 (Karani et al., 2023). This result indicates that consistency pressure may alter not only overlap metrics but also confidence calibration.

The main explicit limitations in the surveyed segmentation–regression work are stated in UG-MCL: the method focuses only on single-class segmentation tasks and experiments on relatively small datasets (Zhang et al., 2021). The authors suggest future work on multi-class segmentation and more diverse datasets. More broadly, the reported evidence is domain-specific: left atrium segmentation, brain tumor segmentation, Cobb angle estimation from spinal X-rays, and liver tumor analysis define different operational meanings of “regression,” from boundary-aware distance fields to anatomical angles and dynamic enhancement values (Zhang et al., 2021, Lin et al., 2022, Xiao et al., 25 Nov 2025). This suggests that segmentation–regression consistency learning is a transferable design principle, but one whose exact semantics depend strongly on what continuous quantity the regression branch represents.

Taken together, these works establish segmentation–regression consistency learning as a structured form of multi-task regularization in which categorical masks and continuous targets are forced into mutual agreement. The agreement may be imposed through inverse transforms between task spaces, bidirectional CAM supervision, task-conditioned masking, auxiliary regression losses derived from segmentation, or related consistency terms that regulate confidence near ambiguous boundaries (Zhang et al., 2021, Lin et al., 2022, Xiao et al., 25 Nov 2025, Karani et al., 2023).