Pseudo-Label Guided Contrastive Learning (PLGCL)

• Pseudo-Label Guided Contrastive Learning (PLGCL) is a design pattern that uses pseudo-labels to guide semantic contrastive objectives and reduce false negatives.
• It employs a staged process of initialization, pseudo-label generation, semantic relation definition, and joint optimization with task and contrastive losses.
• Empirical evaluations in medical segmentation, image classification, and point cloud analysis demonstrate enhanced performance, robustness, and efficiency under limited supervision.

Pseudo-Label Guided Contrastive Learning (PLGCL) denotes a class of contrastive learning methods in which pseudo-labels are used to guide the construction of positives, negatives, prototypes, anchors, or sampling constraints. In contrast to purely instance-discriminative objectives, these methods inject task-relevant semantic structure into representation learning under limited supervision. Closely related formulations appear under names such as “Local contrastive loss with pseudo-label based self-training,” “Semantic Positives via Pseudo-Labels,” “guided point contrastive loss,” “Pseudo Contrastive Learning,” and “Pseudo-label Guided Pixel Contrast,” spanning semi-supervised classification, dense prediction, point cloud understanding, graph learning, weakly supervised detection, and domain adaptation (Chaitanya et al., 2021, Bošnjak et al., 2023, Jiang et al., 2021, Lu et al., 2023, Xiang et al., 15 Jan 2025).

1. Conceptual scope and historical emergence

Early formulations already expressed the central PLGCL intuition: pseudo-labels can reduce semantically incorrect repulsion in contrastive objectives. “G-SimCLR” constructs pseudo categories by clustering denoising-autoencoder embeddings with k-means and then forms batches such that no two images from the same pseudo-category appear together, thereby reducing false negatives under NT-Xent without changing the loss itself (Chakraborty et al., 2020). Later work moved from batch construction to directly pseudo-label-guided objectives.

In semi-supervised medical image segmentation, “Local contrastive loss with pseudo-label based self-training” defines similar pairs as pixels with the same label or pseudo-label and dissimilar pairs as pixels with different label or pseudo-label, with joint optimization of segmentation loss on labeled data and local contrastive loss on labeled plus unlabeled data (Chaitanya et al., 2021). In semi-supervised image classification, “SemPPL” expands the positive set beyond two augmented views by predicting pseudo-labels through a $k$-nearest neighbours classifier on learned embeddings and treating datapoints with the same pseudo-label as semantic positives (Bošnjak et al., 2023). In point cloud segmentation, “Guided Point Contrastive Learning” uses semantic predictions on unlabeled point clouds as pseudo-label guidance to avoid negative pairs in the same category, adds confidence guidance, and introduces category-balanced sampling (Jiang et al., 2021).

Related developments broadened the pattern rather than preserving a single canonical algorithm. “Contrastive Regularization for Semi-Supervised Learning” groups strongly augmented samples into pseudo-label clusters and applies a pseudo-label-guided supervised-contrastive term to accelerate label propagation (Lee et al., 2022). “Pseudo Contrastive Learning for Graph-based Semi-supervised Learning” transforms pseudo-labeling into reliable negative-pair mining and separates anchors with positive pseudo-labels from nodes carrying negative pseudo-labels for the same class (Lu et al., 2023). “Pseudolabel Guided Pixel Contrast” extends the idea to unsupervised domain adaptation by using teacher pseudo-labels to select confident target anchors, source class prototypes as positives, and source/target pixels as negatives (Xiang et al., 15 Jan 2025).

This suggests PLGCL is best understood as a design pattern rather than a single architecture. Across the literature, the common invariant is that pseudo-labels do not merely provide extra classification targets; they alter the geometry of the contrastive objective itself.

2. Core objective structure

A recurring pipeline has four stages. First, a model is warm-started or otherwise initialized so that pseudo-labels are nontrivial. Second, pseudo-labels are generated for unlabeled data, either periodically as in self-training or online per batch. Third, pseudo-labels define semantic relations for contrastive learning. Fourth, the model is optimized jointly with a task loss, a contrastive loss, or both (Chaitanya et al., 2021, Bošnjak et al., 2023, Lee et al., 2022).

A representative dense-prediction formulation appears in semi-supervised medical segmentation. Let $$z(x)=h_\phi(c_\theta(x)) \in \mathbb{R}^{H \times W \times D}$$ be per-pixel embeddings and let $S_c(x)$ denote the pixel set of class $c$, where class membership comes from ground-truth on labeled images and pseudo-labels on unlabeled images. The class prototype is

$$\bar{z}_c(x) = \frac{1}{|S_c(x)|} \sum_{i \in S_c(x)} [z(x)]_i.$$

With cosine similarity and temperature $\tau=0.1$,

$$\mathrm{sim}(a,b) = \frac{a^\top b}{\|a\| \, \|b\|},$$

the per-pixel loss attracts a pixel embedding toward its class prototype and repels it from other-class prototypes (Chaitanya et al., 2021):

$$L_{i,c}\Big([z(x)]_i, \{\bar{z}_k(x')\}\Big) = - \log \frac{\exp\big(\mathrm{sim}([z(x)]_i, \bar{z}_c(x'))/\tau\big)} {\exp\big(\mathrm{sim}([z(x)]_i, \bar{z}_c(x'))/\tau\big) + \sum_{k \neq c} \exp\big(\mathrm{sim}([z(x)]_i, \bar{z}_k(x'))/\tau\big)}.$$

The full objective combines labeled-only segmentation loss with contrastive loss over labeled and pseudo-labeled images. A notable design choice is that pseudo-labels guide only the contrastive term, not the segmentation term, in order to mitigate error propagation (Chaitanya et al., 2021).

Image-level PLGCL often uses multi-positive InfoNCE. In SemPPL, the anchor’s positive set contains both the augmentation-positive and semantic positives sampled from a queue of labeled embeddings with the same label or pseudo-label. Pseudo-labels for unlabeled queries are predicted by $k$NN over a labeled FIFO queue, with $k=1$ as the default and majority voting across multiple views for stabilization (Bošnjak et al., 2023). By contrast, graph PCL emphasizes negative supervision. For node embedding $$z(x)=h_\phi(c_\theta(x)) \in \mathbb{R}^{H \times W \times D}$$0, with class-specific negative set $$z(x)=h_\phi(c_\theta(x)) \in \mathbb{R}^{H \times W \times D}$$1 derived from opposite pseudo-labels and topological weights $$z(x)=h_\phi(c_\theta(x)) \in \mathbb{R}^{H \times W \times D}$$2, the loss is

$$z(x)=h_\phi(c_\theta(x)) \in \mathbb{R}^{H \times W \times D}$$3

where $$z(x)=h_\phi(c_\theta(x)) \in \mathbb{R}^{H \times W \times D}$$4 is a temperature-scaled sigmoid discriminator and the weights are derived from Random Walk with Restart relevance scores (Lu et al., 2023).

These formulations differ in mechanics but share a common principle: pseudo-labels determine which representation alignments are encouraged, which separations are enforced, and how much trust is placed in each relation.

3. Guidance mechanisms and representative variants

The literature implements pseudo-label guidance through several distinct but compatible mechanisms.

Representative paper	Domain	Guidance mechanism
“Local contrastive loss with pseudo-label based self-training” (Chaitanya et al., 2021)	Semi-supervised medical image segmentation	Pseudo-labels define same-class pixels, other-class negatives, and class prototypes within and across images
“SemPPL” (Bošnjak et al., 2023)	Semi-supervised image classification	$$z(x)=h_\phi(c_\theta(x)) \in \mathbb{R}^{H \times W \times D}$$5NN pseudo-labels expand semantic positives from a labeled memory queue
“Guided Point Contrastive Learning” (Jiang et al., 2021)	Semi-supervised point cloud segmentation	Pseudo-label guidance masks false negatives; confidence guidance gates positive pulls
“Contrastive Regularization for Semi-Supervised Learning” (Lee et al., 2022)	Semi-supervised image classification	Confident pseudo-label clusters define pseudo-positives among strong views
“Pseudo Contrastive Learning” (Lu et al., 2023)	Graph semi-supervised learning	Positive pseudo-label anchors are separated from class-specific negative pseudo-label nodes with topological weighting
“Pseudolabel Guided Pixel Contrast” (Xiang et al., 15 Jan 2025)	Domain-adaptive semantic segmentation	Teacher pseudo-labels select confident target anchors; source prototypes act as positives; source/target pixels serve as negatives

Additional variants show that PLGCL is not restricted to semantic-positive expansion. In weakly supervised object detection, “Negative Prototypes Guided Contrastive Learning” constructs an online updated global feature bank containing both positive prototypes and negative prototypes, where a negative prototype is the highest-confidence proposal for a class absent from the image label, and a pseudo-label sampling module discards easily misclassified instances based on similarity to those prototypes (Zhang et al., 2024). In graph representation learning, ComGRL uses pseudo-labels from a global multi-head self-attention module to guide a triple sampling strategy and reliable Mixup augmentation across attributes and structure, after which the augmented graph is fed back into local graph contrastive learning (Wang et al., 30 Jan 2025). In breast ultrasound segmentation, a dual-teacher framework uses uncertainty–entropy weighted fusion to refine pseudo labels and then applies adaptive uncertainty-guided reverse contrastive learning on hard boundary regions, with patch features pooled using pseudo probabilities and reversed pseudo probabilities (Li et al., 6 Mar 2026).

The diversity of these mechanisms is substantial. Some methods use pseudo-labels to create additional positives, some to avoid false negatives, some to define prototype targets, and some to identify reliable negatives. PLGCL therefore spans both attraction-based and repulsion-based semantics.

4. Empirical behavior across tasks

Empirical evidence consistently shows that pseudo-label-guided contrast can improve low-label performance, although the magnitude and failure modes depend strongly on pseudo-label quality. In semi-supervised medical segmentation, PLGCL improves mean Dice on all three evaluated datasets. On ACDC with $$z(x)=h_\phi(c_\theta(x)) \in \mathbb{R}^{H \times W \times D}$$6, the baseline reaches $$z(x)=h_\phi(c_\theta(x)) \in \mathbb{R}^{H \times W \times D}$$7, self-training reaches $$z(x)=h_\phi(c_\theta(x)) \in \mathbb{R}^{H \times W \times D}$$8, and PLGCL intra reaches $$z(x)=h_\phi(c_\theta(x)) \in \mathbb{R}^{H \times W \times D}$$9; with $S_c(x)$0, the baseline is $S_c(x)$1 and PLGCL intra reaches $S_c(x)$2 (Chaitanya et al., 2021). The same study reports that t-SNE plots of pixel embeddings show tighter intra-class clusters and clearer inter-class separation for PLGCL than for self-training alone.

SemPPL reports state-of-the-art ImageNet performance at publication. With a ResNet-50, it reaches $S_c(x)$3 top-$S_c(x)$4 accuracy using $S_c(x)$5 labels and $S_c(x)$6 using $S_c(x)$7 labels; with Selective Kernels, it reaches $S_c(x)$8 and $S_c(x)$9 for the same label fractions, improving absolute $c$0 and $c$1 over previous work (Bošnjak et al., 2023). The paper also reports strong robustness, out-of-distribution, and transfer performance, indicating that semantic-positive expansion can affect not only in-domain top-$c$2 accuracy but also representation portability.

In 3D point cloud semantic segmentation, the guided point contrastive framework improves supervised baselines on ScanNet V2, S3DIS, and SemanticKITTI. With $c$3 labeled data, ScanNet V2 improves from $c$4 to $c$5 mIoU, S3DIS from $c$6 to $c$7 mIoU, and SemanticKITTI from $c$8 to $c$9 mIoU (Jiang et al., 2021). The ablation sequence on ScanNet V2, from “Sup-only” to “PointInfoNCE” to “+ Pseudo-label guidance” to “+ Confidence guidance” to “+ CBS,” isolates gains from each PLGCL component.

Contrastive regularization for semi-supervised classification shows that pseudo-label-guided clustering can also improve training efficiency. On ImageNet with a ResNet-50 and $$\bar{z}_c(x) = \frac{1}{|S_c(x)|} \sum_{i \in S_c(x)} [z(x)]_i.$$0 epochs, FixMatch reaches $$\bar{z}_c(x) = \frac{1}{|S_c(x)|} \sum_{i \in S_c(x)} [z(x)]_i.$$1 top-$$\bar{z}_c(x) = \frac{1}{|S_c(x)|} \sum_{i \in S_c(x)} [z(x)]_i.$$2 accuracy with $$\bar{z}_c(x) = \frac{1}{|S_c(x)|} \sum_{i \in S_c(x)} [z(x)]_i.$$3 labels, whereas FixMatch+CR reaches $$\bar{z}_c(x) = \frac{1}{|S_c(x)|} \sum_{i \in S_c(x)} [z(x)]_i.$$4; with $$\bar{z}_c(x) = \frac{1}{|S_c(x)|} \sum_{i \in S_c(x)} [z(x)]_i.$$5 labels, the scores are $$\bar{z}_c(x) = \frac{1}{|S_c(x)|} \sum_{i \in S_c(x)} [z(x)]_i.$$6 and $$\bar{z}_c(x) = \frac{1}{|S_c(x)|} \sum_{i \in S_c(x)} [z(x)]_i.$$7 (Lee et al., 2022). On CIFAR-100, the same work reports that FixMatch+CR needs only approximately $$\bar{z}_c(x) = \frac{1}{|S_c(x)|} \sum_{i \in S_c(x)} [z(x)]_i.$$8 of FixMatch total training time to achieve superior performance and approximately $$\bar{z}_c(x) = \frac{1}{|S_c(x)|} \sum_{i \in S_c(x)} [z(x)]_i.$$9 of FixMatch time to match FixMatch’s best accuracy.

In domain-adaptive segmentation, PGPC raises DAFormer on GTA5$\tau=0.1$0Cityscapes from $\tau=0.1$1 to $\tau=0.1$2 mIoU and on SYNTHIA$\tau=0.1$3Cityscapes from $\tau=0.1$4 to $\tau=0.1$5; it also improves several DAFormer, HRDA, and MIC configurations on night and adverse-condition benchmarks, although Cityscapes$\tau=0.1$6ACDC with HRDA is a reported exception, where HRDA drops from $\tau=0.1$7 to $\tau=0.1$8 under PGPC (Xiang et al., 15 Jan 2025). This counterexample underscores that pseudo-label-guided contrast is not uniformly beneficial under all teacher–student and domain-gap conditions.

In extremely low-label medical segmentation, the 2026 breast ultrasound framework reports BUSI Dice $\tau=0.1$9, IoU $$\mathrm{sim}(a,b) = \frac{a^\top b}{\|a\| \, \|b\|},$$0, and Acc $$\mathrm{sim}(a,b) = \frac{a^\top b}{\|a\| \, \|b\|},$$1 with only $$\mathrm{sim}(a,b) = \frac{a^\top b}{\|a\| \, \|b\|},$$2 labeled data, and UBB Dice $$\mathrm{sim}(a,b) = \frac{a^\top b}{\|a\| \, \|b\|},$$3, IoU $$\mathrm{sim}(a,b) = \frac{a^\top b}{\|a\| \, \|b\|},$$4, and Acc $$\mathrm{sim}(a,b) = \frac{a^\top b}{\|a\| \, \|b\|},$$5 under the same setting (Li et al., 6 Mar 2026). The ablation sequence attributes large gains to training-free pseudo-label generation and smaller but measurable gains to uncertainty–entropy weighted fusion and adaptive uncertainty-guided reverse contrastive learning.

5. Design choices, ablations, and recurrent technical issues

A first recurrent issue is whether pseudo-labels should supervise the main task loss directly. Several papers explicitly avoid that choice. In the medical segmentation formulation, pseudo-labels guide only the contrastive loss and not the Dice loss, and joint training that also uses pseudo-labels in the segmentation loss significantly underperforms (Chaitanya et al., 2021). SemPPL similarly does not add a cross-entropy term during pretraining; classification is used only during finetuning (Bošnjak et al., 2023). Graph PCL goes further by arguing that the supervision “two nodes do not belong to the same class” is more fault-tolerant than directly fitting a pseudo class assignment (Lu et al., 2023). A common misconception is therefore that PLGCL is merely pseudo-label cross-entropy plus a contrastive regularizer. In a substantial part of the literature, pseudo-labels are deliberately restricted to the representation-learning channel.

A second issue is confidence handling. The approaches are heterogeneous. Guided point contrastive learning gates positive pulls using a confidence threshold $$\mathrm{sim}(a,b) = \frac{a^\top b}{\|a\| \, \|b\|},$$6 and computes confidence as the maximum softmax score (Jiang et al., 2021). PGPC selects target anchors by global entropy, with the best setting at reliable pixel fraction $$\mathrm{sim}(a,b) = \frac{a^\top b}{\|a\| \, \|b\|},$$7, and reports that class-wise confidence performs worst because it injects more noise for minority classes (Xiang et al., 15 Jan 2025). By contrast, the medical segmentation PLGCL study finds that heavy filtering did not consistently outperform using all pseudo-labels, and SemPPL does not require explicit confidence thresholding in training, studying precision and recall of votes mainly as an analysis tool (Chaitanya et al., 2021, Bošnjak et al., 2023). The literature therefore does not support a universal rule that aggressive pseudo-label filtering is always necessary.

A third issue concerns pairwise versus prototype-based contrast. Prototype-based losses are repeatedly justified by efficiency and robustness. The medical segmentation method explicitly states that prototype-based loss avoids quadratic pixel-pair costs and stabilizes training with noisy pseudo-labels (Chaitanya et al., 2021). PGPC uses class prototypes from a source memory bank as positives while retaining pixel features as negatives, specifically to address both class centers and intra-class diversity (Xiang et al., 15 Jan 2025). At the same time, queue-based or bank-based designs remain important in image-level and detection settings: SemPPL uses a FIFO queue of labeled target embeddings with capacity $$\mathrm{sim}(a,b) = \frac{a^\top b}{\|a\| \, \|b\|},$$8, while NPGC maintains per-class positive and negative prototype banks with capacity $$\mathrm{sim}(a,b) = \frac{a^\top b}{\|a\| \, \|b\|},$$9 (Bošnjak et al., 2023, Zhang et al., 2024).

A fourth issue is the granularity of matching. In medical segmentation, inter-image matching often outperforms intra-image matching when pseudo-label quality is sufficient, but in extremely low-label regimes intra-image matching can be safer (Chaitanya et al., 2021). In point clouds, category-balanced sampling is introduced precisely because random sampling under severe class imbalance yields weaker contrastive signals (Jiang et al., 2021). In graph settings, topological distance enters the loss explicitly: PCL spends more effort separating reliable negative pairs with smaller topological distances (Lu et al., 2023).

6. Relation to adjacent paradigms and open directions

PLGCL overlaps with but is not identical to several adjacent paradigms. Relative to SimCLR and MoCo, PLGCL methods do not restrict positives to two views of the same instance; they use pseudo-labels to form semantic positives or to suppress semantically invalid negatives (Bošnjak et al., 2023, Chakraborty et al., 2020). Relative to FixMatch and MixMatch, several PLGCL methods use pseudo-labels to choose contrastive relations rather than to define cross-entropy targets during pretraining (Bošnjak et al., 2023, Lee et al., 2022). Relative to supervised contrastive learning, the supervisory signal is partial, predicted, confidence-gated, or structurally constrained rather than ground-truth complete (Lee et al., 2022, Lu et al., 2023).

The relation to teacher–student learning is similarly non-uniform. Some PLGCL systems are deliberately simple and omit teacher networks, memory banks, or entropy minimization, as noted in semi-supervised medical segmentation (Chaitanya et al., 2021). Others depend centrally on teacher or EMA machinery, including SemPPL’s EMA target networks, PGPC’s EMA teacher with momentum $$L_{i,c}\Big([z(x)]_i, \{\bar{z}_k(x')\}\Big) = - \log \frac{\exp\big(\mathrm{sim}([z(x)]_i, \bar{z}_c(x'))/\tau\big)} {\exp\big(\mathrm{sim}([z(x)]_i, \bar{z}_c(x'))/\tau\big) + \sum_{k \neq c} \exp\big(\mathrm{sim}([z(x)]_i, \bar{z}_k(x'))/\tau\big)}.$$0, and the breast ultrasound framework’s static teacher plus EMA teacher with uncertainty–entropy weighted fusion (Bošnjak et al., 2023, Xiang et al., 15 Jan 2025, Li et al., 6 Mar 2026). PLGCL is therefore compatible with both lightweight and teacher-based SSL recipes.

Open directions are explicitly identified across the surveyed works. The medical segmentation study lists multi-scale contrast, cross-image hard positive mining, teacher-student consistency or temporal ensembling, uncertainty-aware weighting of pseudo-labels, and a memory bank of class prototypes as potential extensions (Chaitanya et al., 2021). PGPC argues that multi-prototype per class could better model intra-class sub-modes and reduce sub-mode collisions (Xiang et al., 15 Jan 2025). The breast ultrasound framework emphasizes extensibility to other imaging modalities or diseases when only a global appearance description is available for training-free pseudo-label generation (Li et al., 6 Mar 2026). ComGRL points toward further coordination between local and global representation learning under pseudo-label-assisted augmentation (Wang et al., 30 Jan 2025).

A plausible implication is that future PLGCL systems will continue to move away from uniform pseudo-label usage toward selective semantic deployment: pseudo-labels will be trusted differently for anchor selection, positive construction, negative mining, prototype updates, and auxiliary task losses. The literature already shows that pseudo-label-guided contrast is most effective when the geometry of trust is designed as carefully as the geometry of the embedding space itself.