PixCon: Clean-Positive Contrastive Learning for Foundation-Model Semi-Supervised Segmentation

Abstract: Semi-supervised semantic segmentation (SSSS) has long turned on one question, which pseudo-labels to trust, and answered it with ever more careful confidence filtering. Foundation backbones change the regime: with a DINOv2 teacher a strict threshold already retains a measured 98%-clean pseudo-label set, so the accuracy that remains lives not in the filter but in how the embedding space is structured by class. We propose PixCon, a clean-positive pixel-contrastive framework. PixCon maintains a per-class memory bank that admits only labeled pixels the student already classifies correctly, guaranteeing a contamination-free positive set ($ρ_F=0$) by construction, unlike prior contrastive SSSS banks (ReCo, U$<sup>2$PL) built from confidence-filtered pseudo-labels. It is a single branch over a consistency backbone, adds no inference-time parameters, and needs no bank-specific threshold. A first-order analysis of the supervised-InfoNCE gradient explains why contamination hurts: its false-positive term scales as $ρ_F/(1-ρ_F)$, which we measure (0.018 on Pascal, 0.106 on ADE20K) rather than assume. Across Pascal VOC, Cityscapes, and ADE20K, PixCon matches or improves a strong DINOv2-based UniMatch V2 baseline in a compute-matched one-switch protocol: it improves every Pascal-1/8 seed (a per-seed gain of about +0.2 mIoU) and its three-seed mean reaches 87.90, the published UniMatch V2-B figure. Because contamination is already rare under foundation-model teachers, our analysis indicates the $ρ_F=0$ guarantee acts chiefly as robustness as teachers weaken, while the accuracy gain comes from cleaner positive supervision, making clean-positive contrast a robust, low-cost default for foundation-model SSSS.

• The paper introduces PixCon, which employs a clean-positive contrastive branch with per-class memory banks to eliminate label noise in semi-supervised segmentation.
• It leverages a dual-branch architecture over a DINOv2-based backbone, ensuring robust embedding-space regularization without relying on confidence thresholds.
• Empirical results on Pascal VOC, Cityscapes, and ADE20K demonstrate that PixCon maintains state-of-the-art performance with improved region boundary accuracy.

PixCon: Clean-Positive Contrastive Learning for Foundation-Model Semi-Supervised Segmentation

Motivation and Foundation-Model Regime Analysis

Recent advances in semi-supervised semantic segmentation (SSSS) have marginalised the value of pseudo-label selection refinements due to the increasingly clean pseudo-labels enabled by foundation models, particularly DINOv2-based ViTs. Historically, SSSS advances focused on filtering unreliable pseudo-labels (e.g., via confidence thresholds) and weak-to-strong consistency regularization. However, with the adoption of powerful foundation backbones, such as DINOv2-B, the pseudo-labels filtered at high threshold (e.g., $\tau = 0.95$) have contamination rates $\rho_\mathrm{F}<2\%$ on Pascal VOC and only moderate ($\sim 10\%$) on ADE20K.

Figure 1: One switch, cleaner supervision: PixCon's clean-positive branch sharply reduces errors vs UniMatch~V2, while maintaining aggregate performance at no cost.

UniMatch~V2 combined a DINOv2 backbone with strong augmentations and thresholding, establishing a new operating regime for SSSS where the bulk of progress derives from representation quality rather than advances in pseudo-label selection. In this regime, the bottleneck has transitioned: further accuracy improvements now depend on structuring the embedding space by class, not further filtering. PixCon is designed explicitly for this realization.

Clean-Positive Contrastive Learning: Methodology

PixCon introduces a clean-positive contrastive branch over a shared encoder–decoder architecture, orthogonal to consistency regularization. The core methodology is the maintenance of per-class memory banks that admit only labeled, correctly classified pixels as positives. This negates label noise by construction ($\rho_\mathrm{F}=0$), in contrast to confidence-filtered banks (e.g., ReCo, U$^2$PL) that can propagate confidently-wrong pseudo-label contamination into the representation.

Figure 2: The PixCon architecture: a dual-branch structure over DINOv2-B and DPT, coupling a clean-positive contrastive branch to a strong consistency branch (from UniMatch~V2).

The supervised InfoNCE loss is applied over these clean anchors, with class-balanced sampling and limited bank sizes. The total training objective aggregates cross-entropy on labeled data, consistency loss on pseudo-labeled data, and the clean-positive contrastive objective—parameterized by a single $\lambda_\mathrm{pix}$.

Theoretical analysis yields that the false-positive term in the InfoNCE gradient scales as $\rho_\mathrm{F}/(1-\rho_\mathrm{F})$. With measured contamination rates below $2\%$, this term is negligible under foundation teachers, but exact $\rho_\mathrm{F}=0$ ensures robustness as teacher error increases—making this guarantee primarily a safeguard for weaker regimes.

Figure 3: Schematic—confidence-filtered banks (above) yield false positive gradients, while PixCon (below) ensures pure, true-positive signal by construction.

Empirical Results

The empirical evaluation employs Pascal VOC, Cityscapes, and ADE20K, and isolates the clean-positive contrastive branch as the only difference from the UniMatch~V2 baseline (identical codebase, compute-matched). The most salient finding is a consistent per-seed mIoU improvement of $\sim +0.2$ (mean $\rho_\mathrm{F}<2\%$0, partly variance-driven) on Pascal-1/8, where DINOv2-B is already near the fully-supervised ceiling. Other cells (Cityscapes 1/16, 1/8, ADE20K) show parity between PixCon and UniMatch~V2, consistent with theoretical predictions given low contamination rates.

Figure 4: Consistent per-epoch mIoU improvement during training: PixCon outperforms UniMatch~V2 across all seeds/epochs on Pascal 1/8.

Qualitative evaluation corroborates these findings. In Pascal/ADE20K, PixCon consistently resolves part-level confusions, reducing error contours (red) versus UniMatch~V2, and yielding cleaner, more coherent mask predictions—especially on difficult classes.

Figure 5: Qualitative wins on Pascal 1/8: PixCon corrects fine-grained errors and minimizes spurious region assignments.

Figure 6: ADE20K: even in aggregate ties, PixCon clearly corrects large-scale misclassifications present in UniMatch~V2.

A controlled ablation varying only the bank admission rule shows clean-positive and confidence-filtered banks tie within $\rho_\mathrm{F}<2\%$1~mIoU under foundation teacher strength; thus, the correctness condition (student prediction matches label), not the removal of rare contamination, is responsible for the margin.

Analyses: Error Structure and Regularization

Per-class IoU analysis demonstrates that PixCon's gains are broad and not concentrated on rare or underrepresented classes. The main effect is a diffuse regularization of the embedding space, marginally increasing intra-class compactness and reducing mean/max inter-class cosine similarity.

Figure 7: Per-class IoU improvements are diffuse, with largest gains and regressions spread across the frequency spectrum; the effect is not driven by rare-class-specific improvements.

Further, PixCon's gains are more pronounced on boundary regions than interiors—doubling the improvement rate—suggesting the method enhances hard pixel classification, not merely "easy" pixels.

Theoretical Implications, Practical Adoption, and Future Directions

PixCon's by-construction guarantee ($\rho_\mathrm{F}<2\%$2) eliminates the need for confidence threshold tuning, providing robust, default-safe contrastive supervision. This is especially relevant as teacher quality degrades in lower-resource or cross-domain scenarios. The lack of per-dataset meta-parameterization is also significant for practical deployments.

However, the magnitude of measurable accuracy gains is modest, as expected under regimes where pseudo-label noise has already been largely abated by foundation teachers. The main benefit thus shifts to robustness, parameter elimination, and ensuring clean supervision by design, rather than boosting best-case aggregate mIoU on strong backbones.

Limitations include potential slow bank fill in ultra-low-label regimes and inability to exploit soft-correctness for near-miss predictions. Future research should focus on:

• Explicit attribution of gain between strict correctness vs labeled-only admission.
• Evaluation under high contamination, e.g., with weak teachers or injected label noise.
• Exploring soft or probabilistic bank admission criteria to complement the hard predicate.

Conclusion

PixCon provides a principled, robust, and low-cost mechanism for embedding-space regularization in foundation-model SSSS, attaining parity or marginal gains over state-of-the-art UniMatch~V2 in strong-teacher regimes while certifying clean, contamination-free contrastive supervision. Its practical advantage is as a safe, threshold-free, and reliable default for semi-supervised segmentation in the foundation era, with guaranteed robustness against the reintroduction of label noise, and its conceptual contribution is to clarify the locus of remaining gains in the modern SSSS pipeline.