Confidence Separable Learning (CSL)

• Confidence Separable Learning (CSL) is a framework for semi-supervised segmentation that uses a two-dimensional confidence-distribution descriptor to distinguish reliable from unreliable predictions.
• It reformulates pseudo-label selection as a convex optimization problem over per-pixel features, improving spatial continuity and mitigating overconfidence in segmentation networks.
• CSL integrates Trusted Mask Perturbation (TMP) to enforce contextual learning in under-supervised regions, leading to significant improvements in segmentation metrics.

Confidence Separable Learning (CSL) is a pseudo-label selection framework for semi-supervised semantic segmentation that replaces fixed confidence-threshold filtering with per-image separability learning in a two-dimensional confidence-distribution feature space. It was introduced to address a specific failure mode of weak-to-strong self-training pipelines: modern segmentation networks can be overconfident, so correct and incorrect pseudo-labels overlap substantially in high-confidence regions, while directly discarding low-confidence pixels breaks spatial-semantic continuity and removes useful context. CSL formulates reliable-versus-unreliable pseudo-label discrimination as a convex optimization problem over per-pixel confidence features, then complements this selection stage with Trusted Mask Perturbation (TMP), which forces the network to learn contextual relations from regions that would otherwise be under-supervised (Liu et al., 20 Sep 2025).

1. Problem setting and motivation

CSL is defined in the standard semi-supervised semantic segmentation setting with a labeled set

$$D_l=\{(x_i^l,y_i^l)\}_{i=1}^{N_l}, \qquad D_u=\{x_i^u\}_{i=1}^{N_u}, \qquad N_l \ll N_u,$$

where each image satisfies $x_i \in \mathbb{R}^{3\times H\times W}$ and each segmentation map is one-hot encoded as $y_i \in \mathbb{R}^{K\times H\times W}$. A segmentation network $\mathcal F$ is trained with supervised loss on labeled images and pseudo-label-based consistency loss on unlabeled images (Liu et al., 20 Sep 2025).

The supervised term is

$$\mathcal L_S = \frac{1}{B_L}\sum_{i=1}^{B_L} CE\!\left(y_i^l,\mathcal F(A^\omega(x_i^l))\right),$$

with weak augmentation $A^\omega(\cdot)$. For unlabeled data, the baseline weak-to-strong formulation predicts pseudo-labels on weakly augmented inputs and supervises strongly augmented versions:

$$\mathcal L_u^a = \frac{1}{B_U}\sum_{i=1}^{B_U} M_i \odot CE\!\left(\hat y_i^u,\mathcal F(A^s(A^\omega(x_i^u)))\right),$$

where $A^s(\cdot)$ is strong augmentation, $\hat y_i^u$ is the pseudo-label from the weak prediction, and $M_i$ is a pixelwise selection mask. In conventional threshold-based pipelines,

$x_i \in \mathbb{R}^{3\times H\times W}$0

The motivation for CSL is that this scalar-threshold rule is structurally inadequate when the network is overconfident. The paper identifies three linked failure modes. First, correct and incorrect predictions can overlap significantly in the high-confidence region, making fixed-threshold separation unreliable. Second, retaining high-confidence but wrong pixels amplifies confirmation bias during self-training. Third, discarding low-confidence pixels removes supervision around boundaries, small structures, occlusions, and ambiguous transitions, thereby disrupting spatial-semantic continuity (Liu et al., 20 Sep 2025).

Within this framing, CSL shifts the central question from “is the maximum softmax score above a global threshold?” to “are reliable and unreliable predictions separable within the confidence distribution of this particular image?” That change defines both the method and the name.

2. Confidence-distribution feature space

CSL represents each unlabeled pixel by a two-dimensional feature built from the model’s full class-probability distribution rather than from the maximum probability alone. For pixel $x_i \in \mathbb{R}^{3\times H\times W}$1, let

$x_i \in \mathbb{R}^{3\times H\times W}$2

be the maximum predicted class probability, where $x_i \in \mathbb{R}^{3\times H\times W}$3 is the predicted class. The pseudo-label is the corresponding argmax class. CSL then augments $x_i \in \mathbb{R}^{3\times H\times W}$4 with a second quantity, residual dispersion, and defines the per-pixel feature vector

$x_i \in \mathbb{R}^{3\times H\times W}$5

Residual dispersion is derived from an entropy-minimization analysis around an ideal unimodal target distribution

$x_i \in \mathbb{R}^{3\times H\times W}$6

For the non-maximum classes, the paper writes

$x_i \in \mathbb{R}^{3\times H\times W}$7

with

$x_i \in \mathbb{R}^{3\times H\times W}$8

A second-order Taylor expansion of $x_i \in \mathbb{R}^{3\times H\times W}$9 around $y_i \in \mathbb{R}^{K\times H\times W}$0 yields the approximation

$y_i \in \mathbb{R}^{K\times H\times W}$1

where residual dispersion is

$y_i \in \mathbb{R}^{K\times H\times W}$2

The intended consequence is that reliable predictions are not characterized solely by large $y_i \in \mathbb{R}^{K\times H\times W}$3; they are better distinguished when the residual structure of the non-maximum probabilities is also considered. The method therefore treats

$y_i \in \mathbb{R}^{K\times H\times W}$4

as the operative “confidence-distribution” descriptor (Liu et al., 20 Sep 2025).

The paper’s terminology is precise on this point: predictions are “confidence separable” when reliable and unreliable pixels become distinguishable in the two-dimensional space formed by maximum confidence and residual dispersion. This is the central conceptual move in CSL. It does not attempt to recalibrate confidence globally; instead, it constructs a feature space in which overconfident false positives are more separable from genuinely reliable predictions.

3. Separability optimization and reliability weighting

For an unlabeled image, CSL stacks all pixel features into the confidence-distribution feature matrix

$y_i \in \mathbb{R}^{K\times H\times W}$5

Pseudo-label selection is then formulated as a binary partitioning problem over the pixels of that image. Let

$y_i \in \mathbb{R}^{K\times H\times W}$6

be the selection matrix with one-hot assignment constraint

$y_i \in \mathbb{R}^{K\times H\times W}$7

The optimization objective is

$y_i \in \mathbb{R}^{K\times H\times W}$8

In the supplementary derivation, this trace maximization is obtained from an intra-class consistency objective

$y_i \in \mathbb{R}^{K\times H\times W}$9

which makes the selection problem equivalent to maximizing a trace form. The resulting discrete program is relaxed spectrally. Using the Ky Fan theorem, the relaxed solution is approximated by

$\mathcal F$0

where $\mathcal F$1 are eigenvectors of $\mathcal F$2, $\mathcal F$3 is the absolute value of the $\mathcal F$4-th element of eigenvector $\mathcal F$5, and $\mathcal F$6 is the Kronecker delta (Liu et al., 20 Sep 2025).

For implementation, the paper proposes an SVD-based route. Writing

$\mathcal F$7

one has

$\mathcal F$8

so the columns of $\mathcal F$9 are the eigenvectors of $$\mathcal L_S = \frac{1}{B_L}\sum_{i=1}^{B_L} CE\!\left(y_i^l,\mathcal F(A^\omega(x_i^l))\right),$$0. This avoids explicit eigendecomposition of a large $$\mathcal L_S = \frac{1}{B_L}\sum_{i=1}^{B_L} CE\!\left(y_i^l,\mathcal F(A^\omega(x_i^l))\right),$$1 matrix.

After partitioning, the subset with stronger statistics in the two confidence-distribution coordinates is treated as the reliable component. CSL then computes Gaussian-smoothed reliability weights

$$\mathcal L_S = \frac{1}{B_L}\sum_{i=1}^{B_L} CE\!\left(y_i^l,\mathcal F(A^\omega(x_i^l))\right),$$2

where $$\mathcal L_S = \frac{1}{B_L}\sum_{i=1}^{B_L} CE\!\left(y_i^l,\mathcal F(A^\omega(x_i^l))\right),$$3 and $$\mathcal L_S = \frac{1}{B_L}\sum_{i=1}^{B_L} CE\!\left(y_i^l,\mathcal F(A^\omega(x_i^l))\right),$$4 are the mean and variance of the reliable subset for component $$\mathcal L_S = \frac{1}{B_L}\sum_{i=1}^{B_L} CE\!\left(y_i^l,\mathcal F(A^\omega(x_i^l))\right),$$5, and the paper fixes $$\mathcal L_S = \frac{1}{B_L}\sum_{i=1}^{B_L} CE\!\left(y_i^l,\mathcal F(A^\omega(x_i^l))\right),$$6. The final reliability map is

$$\mathcal L_S = \frac{1}{B_L}\sum_{i=1}^{B_L} CE\!\left(y_i^l,\mathcal F(A^\omega(x_i^l))\right),$$7

This soft weighting is one of the method’s practical distinctions from thresholding. Pixels that are clearly in the reliable region receive full weight, while the remainder are downweighted rather than discarded outright. The method therefore combines a binary separability objective with a continuous reliability weighting stage (Liu et al., 20 Sep 2025).

4. Training pipeline and Trusted Mask Perturbation

CSL is built on a single-network weak-to-strong consistency framework. For an unlabeled image $$\mathcal L_S = \frac{1}{B_L}\sum_{i=1}^{B_L} CE\!\left(y_i^l,\mathcal F(A^\omega(x_i^l))\right),$$8, the pipeline begins with weak augmentation

$$\mathcal L_S = \frac{1}{B_L}\sum_{i=1}^{B_L} CE\!\left(y_i^l,\mathcal F(A^\omega(x_i^l))\right),$$9

followed by the weak prediction

$A^\omega(\cdot)$0

Pseudo-labels are obtained by argmax,

$A^\omega(\cdot)$1

and the per-pixel confidence-distribution features $A^\omega(\cdot)$2 are computed from the resulting probability map. Solving the separability objective yields the weight map $A^\omega(\cdot)$3, which is then used in the standard weak-to-strong unsupervised loss

$A^\omega(\cdot)$4

Because the decision boundary is recomputed from the current prediction distribution, pseudo-label reliability is effectively re-estimated each time pseudo-labels are generated (Liu et al., 20 Sep 2025).

The second component of CSL is Trusted Mask Perturbation (TMP). Its purpose is to counter the under-supervision of low-reliability regions. Instead of discarding those regions and training only on easy reliable pixels, TMP masks reliable pixels in the input while preserving their pseudo-label supervision, forcing the network to infer them from surrounding context. The mask is defined as

$A^\omega(\cdot)$5

where the paper uses patch size $A^\omega(\cdot)$6, masking ratio parameter $A^\omega(\cdot)$7, and $A^\omega(\cdot)$8. The perturbed image is

$A^\omega(\cdot)$9

The corresponding masked-input loss is

$$\mathcal L_u^a = \frac{1}{B_U}\sum_{i=1}^{B_U} M_i \odot CE\!\left(\hat y_i^u,\mathcal F(A^s(A^\omega(x_i^u)))\right),$$0

The full objective is

$$\mathcal L_u^a = \frac{1}{B_U}\sum_{i=1}^{B_U} M_i \odot CE\!\left(\hat y_i^u,\mathcal F(A^s(A^\omega(x_i^u)))\right),$$1

with default weights

$$\mathcal L_u^a = \frac{1}{B_U}\sum_{i=1}^{B_U} M_i \odot CE\!\left(\hat y_i^u,\mathcal F(A^s(A^\omega(x_i^u)))\right),$$2

TMP is not a generic masking branch. The paper reports that random patch masking is better than block-wise masking, that masking only reliable pixels works best, and that masking unreliable pixels or both reliable and unreliable pixels degrades performance. Large patches, such as $$\mathcal L_u^a = \frac{1}{B_U}\sum_{i=1}^{B_U} M_i \odot CE\!\left(\hat y_i^u,\mathcal F(A^s(A^\omega(x_i^u)))\right),$$3, are also reported to hurt results because they destroy too much local context (Liu et al., 20 Sep 2025).

5. Empirical performance and ablation evidence

CSL is evaluated on three standard semi-supervised segmentation benchmarks: Pascal VOC 2012, Cityscapes, and MS COCO. On Pascal VOC original at $$\mathcal L_u^a = \frac{1}{B_U}\sum_{i=1}^{B_U} M_i \odot CE\!\left(\hat y_i^u,\mathcal F(A^s(A^\omega(x_i^u)))\right),$$4, the reported mIoU values are $$\mathcal L_u^a = \frac{1}{B_U}\sum_{i=1}^{B_U} M_i \odot CE\!\left(\hat y_i^u,\mathcal F(A^s(A^\omega(x_i^u)))\right),$$5 for the $$\mathcal L_u^a = \frac{1}{B_U}\sum_{i=1}^{B_U} M_i \odot CE\!\left(\hat y_i^u,\mathcal F(A^s(A^\omega(x_i^u)))\right),$$6 split, $$\mathcal L_u^a = \frac{1}{B_U}\sum_{i=1}^{B_U} M_i \odot CE\!\left(\hat y_i^u,\mathcal F(A^s(A^\omega(x_i^u)))\right),$$7 for $$\mathcal L_u^a = \frac{1}{B_U}\sum_{i=1}^{B_U} M_i \odot CE\!\left(\hat y_i^u,\mathcal F(A^s(A^\omega(x_i^u)))\right),$$8, $$\mathcal L_u^a = \frac{1}{B_U}\sum_{i=1}^{B_U} M_i \odot CE\!\left(\hat y_i^u,\mathcal F(A^s(A^\omega(x_i^u)))\right),$$9 for $A^s(\cdot)$0, $A^s(\cdot)$1 for $A^s(\cdot)$2, and $A^s(\cdot)$3 for full supervision. Relative to UniMatch on the original Pascal setup, the paper reports gains of $A^s(\cdot)$4 at $A^s(\cdot)$5 and $A^s(\cdot)$6 at $A^s(\cdot)$7. On Pascal blender, CSL reaches $A^s(\cdot)$8, $A^s(\cdot)$9, and $\hat y_i^u$0 at $\hat y_i^u$1 for $\hat y_i^u$2, $\hat y_i^u$3, and $\hat y_i^u$4, and $\hat y_i^u$5, $\hat y_i^u$6, and $\hat y_i^u$7 at $\hat y_i^u$8. With U2PL splits, the reported values are $\hat y_i^u$9, $M_i$0, and $M_i$1 for $M_i$2, $M_i$3, and $M_i$4. On Cityscapes, CSL attains $M_i$5, $M_i$6, $M_i$7, and $M_i$8 for the $M_i$9, $x_i \in \mathbb{R}^{3\times H\times W}$00, $x_i \in \mathbb{R}^{3\times H\times W}$01, and $x_i \in \mathbb{R}^{3\times H\times W}$02 splits. On COCO, the corresponding values are $x_i \in \mathbb{R}^{3\times H\times W}$03, $x_i \in \mathbb{R}^{3\times H\times W}$04, $x_i \in \mathbb{R}^{3\times H\times W}$05, $x_i \in \mathbb{R}^{3\times H\times W}$06, and $x_i \in \mathbb{R}^{3\times H\times W}$07 for the $x_i \in \mathbb{R}^{3\times H\times W}$08, $x_i \in \mathbb{R}^{3\times H\times W}$09, $x_i \in \mathbb{R}^{3\times H\times W}$10, $x_i \in \mathbb{R}^{3\times H\times W}$11, and $x_i \in \mathbb{R}^{3\times H\times W}$12 splits (Liu et al., 20 Sep 2025).

The ablation results isolate the contribution of each component. On Pascal original, the baseline reports $x_i \in \mathbb{R}^{3\times H\times W}$13 and $x_i \in \mathbb{R}^{3\times H\times W}$14 mIoU at the $x_i \in \mathbb{R}^{3\times H\times W}$15 and $x_i \in \mathbb{R}^{3\times H\times W}$16 splits. Adding only the optimization strategy $x_i \in \mathbb{R}^{3\times H\times W}$17 raises these to $x_i \in \mathbb{R}^{3\times H\times W}$18 and $x_i \in \mathbb{R}^{3\times H\times W}$19; adding smooth weighting $x_i \in \mathbb{R}^{3\times H\times W}$20 further improves them to $x_i \in \mathbb{R}^{3\times H\times W}$21 and $x_i \in \mathbb{R}^{3\times H\times W}$22; and adding TMP yields $x_i \in \mathbb{R}^{3\times H\times W}$23 and $x_i \in \mathbb{R}^{3\times H\times W}$24. TMP alone, when added to the baseline, improves performance by $x_i \in \mathbb{R}^{3\times H\times W}$25 on $x_i \in \mathbb{R}^{3\times H\times W}$26 and $x_i \in \mathbb{R}^{3\times H\times W}$27 on $x_i \in \mathbb{R}^{3\times H\times W}$28 (Liu et al., 20 Sep 2025).

The paper also reports direct evidence that the separability formulation improves pseudo-label quality. On the Pascal $x_i \in \mathbb{R}^{3\times H\times W}$29 split, using the same model predictions, CSL improves pseudo-label accuracy by $x_i \in \mathbb{R}^{3\times H\times W}$30 while keeping recall competitive, with $x_i \in \mathbb{R}^{3\times H\times W}$31 mean recall reported in the supplement. Replacing the optimization strategy with sample-wise mean thresholds of the SoftMatch type consistently degrades performance, which the authors use as evidence that sample-specific boundaries matter. Metric ablations further show that residual dispersion is preferable to entropy and margin under overconfidence, and that combining multiple related confidence metrics can hurt because of redundancy and covariance conflict; the best configuration uses only $x_i \in \mathbb{R}^{3\times H\times W}$32 (Liu et al., 20 Sep 2025).

These results support the paper’s narrower claim. CSL is not presented as a new pseudo-label generator; it is presented as a new pseudo-label selector. Its empirical gains are tied to that distinction.

6. Scope, related approaches, and acronym ambiguity

Within semi-supervised learning, CSL belongs to a line of work that tries to separate reliable from unreliable pseudo-labels rather than using all pseudo-labels indiscriminately. A related method, “Semi-supervised learning by selective training with pseudo labels via confidence estimation,” performs hard pseudo-labeling, screens candidates by max-probability confidence, and then selects small-loss examples after augmentation; it also introduces MixConf for calibration (Ishii, 2021). CSL differs from that formulation in two important ways: it targets dense per-pixel segmentation rather than image-level classification, and it replaces scalar confidence thresholding with sample-specific separability in a two-dimensional confidence-distribution space (Liu et al., 20 Sep 2025).

CSL is also related, at a methodological level, to work that aligns confidence with internal representation geometry. “Confidence-aware Contrastive Learning for Selective Classification” shows that feature-layer optimization can improve selective classification, and introduces a confidence-weighted contrastive objective to align confidence with correctness (Wu et al., 2024). The connection is conceptual rather than nominal: both methods treat confidence as something that should be structured geometrically rather than merely thresholded at the output layer.

The acronym itself is overloaded in the literature. In several vision papers, CSL denotes “contrastive self-supervised learning,” including framework papers that unify AMDIM, CPC, and SimCLR (Falcon et al., 2020). In segmentation, CSL can also denote “Class-Agnostic Structure-Constrained Learning” (Zhang et al., 2023). In cosmology and foundations of quantum mechanics, CSL usually refers to “continuous spontaneous localization” (Palermo et al., 2022, León et al., 2021). In the present context, however, CSL specifically names “Confidence Separable Learning,” a pseudo-label selection framework for semi-supervised semantic segmentation (Liu et al., 20 Sep 2025).

As formulated in its original paper, Confidence Separable Learning is therefore best understood as a redefinition of pseudo-label selection under overconfidence: pseudo-label reliability is inferred from separability in $x_i \in \mathbb{R}^{3\times H\times W}$33 space, converted into soft per-pixel supervision weights, and coupled to a masking strategy that preserves contextual learning where fixed-threshold methods would remove it.