CW-BASS v2: Saturation-Aware Pseudo-Label Selection for Semi-Supervised Segmentation under Foundation-Model Teachers

Abstract: Semi-supervised semantic segmentation has long turned on one question, which pseudo-labels to trust, and a generation of selection rules, dynamic thresholds, per-class curricula, soft confidence weights, answered it for the noisy, under-confident ResNet teachers of their day. Self-supervised foundation encoders change the regime: with a DINOv2 teacher, confidence saturates, so the filtering that helped a weak teacher can hurt a strong one. We propose CW-BASS v2, a saturation-aware pseudo-label selection method that reads the teacher's confidence regime rather than committing to one rule. It pairs held-out calibration, an unbiased per-class noise estimate, with a self-adaptive confidence floor that provably bounds retention away from 1, and combines them in a one-pass gate: measure the reliability of the teacher's confident set, pi_kept = Pr[correct | c >= tau], on a held-out slice, and filter strictly when it meets the confidence demanded (pi_kept >= tau), falling back to the adaptive floor otherwise. The boundary is the pre-existing operating threshold, not a value tuned to mIoU, and across six DINOv2 teachers it makes the correct strict-vs-floor call blind. CW-BASS v2 thus recovers the UniMatch V2 operating point on the saturated benchmarks by selecting strict (Pascal VOC 1/8 87.4 against its reported 87.9; Cityscapes within 0.5), and improves on it where the confident set is unreliable (pi_kept ~ 89%, ADE20K), where the floor edges ahead (+1.5 mIoU, single seed). The gate is principled because the failure it avoids is measured, not assumed: on a reliable, saturated teacher the confidence distribution's dynamic range collapses (98% of Pascal pixels >= 0.95), so an adaptive cutoff floods the retention mask and self-training decays into confirmation bias.

• The paper shows that adaptive thresholding can fail under saturated DINOv2 teachers: strict filtering reaches 86.19 ± 1.82 mIoU on Pascal VOC 1/8, while no adaptive method exceeds 84.52.
• The method uses held-out confident-set reliability to select strict filtering when pseudo-label accuracy is high and a self-adaptive floor when the teacher is confidently unreliable, improving ADE20K to 50.58 mIoU, or 1.48 points above strict filtering.
• The audit links confidence saturation to threshold collapse, mask flooding, and early training decline, while noting that results remain limited by DINOv2-only testing, single-seed ADE20K evidence, and unmatched training recipes.

CW-BASS v2 is a saturation-aware pseudo-label selection method for semi-supervised semantic segmentation (SSSS) under foundation-model teachers. The paper's central claim is that a generation of adaptive thresholding rules—dynamic global thresholds, per-class curricula, soft confidence weighting—was designed for the noisy, under-confident ResNet teachers of an earlier era, and that under a saturated DINOv2 teacher these rules not only stop helping but actively harm. The method's response is not a new threshold rule but a gate: measure the reliability of the teacher's confident set on a held-out slice, deploy strict filtering when that reliability meets the demanded confidence, and engage a self-adaptive floor otherwise. The paper is as much a controlled audit of adaptive thresholding as it is a method contribution, and it is notably candid about the limits of its own affirmative results.

The regime shift and its two failure modes

The paper's premise is quantified rather than asserted. On Pascal VOC 1/8, a DINOv2-Base EMA teacher places 98% of valid pixels at confidence ≥ 0.95, versus 53% for a ResNet-50 teacher, and does so with the lower expected calibration error (0.007 vs. 0.024). The obstacle to adaptive thresholding is therefore not aggregate miscalibration but dynamic-range collapse: the confidence distribution has almost no spread left to threshold on, and what little spread survives below the saturated mass is precisely where the teacher's errors concentrate. The paper is careful to note that the ECE comparison is bin-sensitive at these magnitudes and confounded by training state (the DINOv2 teacher is converged, the ResNet one partially trained); the load-bearing statistic is the bin-free fraction of pixels above cutoff.

Two failure modes are formalized. First, in-batch noise estimation is downward-biased (Proposition 1): any estimator of pseudo-label noise computed on labeled training pixels measures training error, which is optimistic by the teacher's generalization gap on confident pixels. The bias is not hypothetical—the ADE20K teacher's confident-set reliability reads 98.4% in-batch but only 89.3% held out, a 9-point optimism that would cause a gate reading in-batch statistics to wrongly select strict filtering. The fix, a 5% held-out calibration slice, is proven conditionally unbiased; the paper deliberately attaches no concentration bound, since spatially correlated pixels reduce the effective sample size to roughly 9 images on the Pascal 1/8 split.

Second, the dynamic threshold collapses (Corollary 1). The original CW-BASS sigmoid threshold is algebraically bounded by $\tau_0\sigma(\beta/2)\approx0.34$ for any input, so under a saturated teacher the cutoff sits beneath essentially the entire confidence mass and retention floods to 1. The measured cutoff indeed stays pinned in [0.300, 0.331] across a full matched-batch run. The paper is precise about what this confirms: the ceiling is a property of the rule's hyperparameters, not a law about foundation models; the genuinely empirical content is the downstream mask flooding, which the run bears out. The proposed self-adaptive floor scales with the teacher's mean confidence and, under an explicit scale-family assumption on the confidence distribution (Theorem 1), pins retention to a fixed quantile bounded away from 1. The paper is equally precise that this theorem has no bearing on accuracy: the floor buys bounded retention and nothing more.

The controlled comparison

The decisive evidence is a three-seed, matched-batch-16 comparison on Pascal VOC 1/8 in which the adaptive rules differ from one another in the threshold rule alone, while the strict arm is the UniMatch V2 recipe. Strict reaches $86.19\pm1.82$ mIoU and attains the ~87.4 mode on two of three seeds; no adaptive run exceeds 84.52. The paper does not claim a clean sweep: strict's own seed 1 stalls at 84.09, below the dynamic rule's 84.46 on that seed—the single cell in the grid where an adaptive rule wins—and the margins over the dynamic, per-class, and SoftMatch rules are not significant at three seeds (Welch $p$ = 0.22/0.14/0.08); only the margins over the floor ($p$=0.044) and FreeMatch ($p$=0.017) clear 0.05. What is unambiguous is that strict uniquely reaches the ~87.4 mode.

The mechanism behind the failure is dissected link by link from training logs: saturation (98% ≥ 0.95) → dynamic-range collapse (cutoff pinned near 0.33) → mask flooding (retention crosses 0.95 by epoch 7, admitting 99.9% of teacher errors vs. 63% for strict) → early peak (epochs 4–20 for adaptive vs. 42 for strict) → decline, with the per-class EMA teacher shedding 6.14 mIoU from its own peak (84.07 → 77.93). A parameter sweep rules out the stale-constants explanation: raising the ceiling knobs leaves the dynamic rule in the same 81–85 band, and the only configuration that would close the gap is the degenerate one where the clamp equals the strict threshold. A confound analysis bounds the two known recipe differences (strict's different unlabeled loss, the 5% calibration holdout) at ≈1.2 mIoU of the 3.3–5.1 gap, though the dual-strong-view difference is not ablated and is flagged as the most valuable experiment left undone.

The gate and its generality

The gate itself is a one-pass calibration test: compute $$\pi_{\mathrm{kept}}=\Pr[\text{correct}\mid c\ge0.95]$$ on held-out labels; use strict when $\pi_{\mathrm{kept}}\ge\tau$, the adaptive floor otherwise. Its boundary is the pre-existing operating threshold, not a value tuned on mIoU. Across six DINOv2 teachers (S/B × three datasets), all of which are highly saturated (≥82% of pixels ≥ 0.95), $\pi_{\mathrm{kept}}$ alone tracks the sign of the adaptive-vs-strict gap: ≈98% on Pascal and Cityscapes, where strict wins or ties; ≈89% on ADE20K, where the floor reaches 50.58 mIoU, +1.48 over strict and above UniMatch V2-B's reported 49.8. The paper explicitly rejects the tempting story that ADE20K's teacher is merely under-confident—it is confidently unreliable, which is exactly the quantity the gate reads.

On the saturated benchmarks the gate returns strict, so the deployed Pascal and Cityscapes rows (87.40 and 83.96 at 1/8) are UniMatch V2 reproductions trailing the reported numbers by 0.3–0.5, evidence of harness fidelity rather than of a new state of the art. The paper states this plainly.

Limitations and open questions

The paper is unusually explicit about scope. Every teacher tested is DINOv2-family, so the gate is demonstrated only there, not for CLIP- or SAM-style encoders. The affirmative ADE20K result is single-seed; its sign reproduces across S and B, but the +1.5 magnitude is within plausible seed noise. The gate boundary is validated post-hoc on validation splits from converged teachers rather than measured live on the calibration slice mid-run, because a 9-image slice cannot resolve 98% from 89%—the criterion is validated, not the deployed system. The strict arm is not loss-matched to the adaptive arms. The bounded-retention theorem rests on a scale-family idealization whose operative content is the empirical mask-ratio prediction. The CAFS/ENCORE mechanisms are audited through reimplementation rather than released code. The DINOv2-L cross-check is partial (strict L-ADE exceeded GPU memory; L-Cityscapes was not run), and the batch-4 severity trend is reported only qualitatively.

Conclusion

CW-BASS v2's contribution is a decision rule grounded in a measured mechanism: confidence saturation destroys the dynamic range on which every adaptive threshold rule depends, regardless of aggregate calibration, so the correct rule depends on confident-set reliability rather than on backbone strength alone. The controlled audit establishes that no setting of an adapt-downward rule beats a fixed high cutoff once the confidence range has collapsed, and the gate operationalizes this as a one-pass, forward-only test that makes the correct call on all six tested teachers. The residual questions the paper leaves open—whether the gate boundary generalizes beyond DINOv2-family teachers, whether the floor's ADE20K advantage survives multiple seeds, and whether a loss-matched strict arm inside the CW-BASS v2 loop would confirm the single-factor attribution—are precisely stated and, for the most part, cheap to test.