---
title: Cross-Pseudo-Labeling in Semi-supervised Learning
url: https://www.emergentmind.com/topics/cross-pseudo-labeling
type: topic
---

# Cross-Pseudo-Labeling in Semi-supervised Learning

Cross-pseudo-labeling is a general paradigm in semi-supervised and weakly supervised learning in which multiple models, branches, or agents mutually select or refine pseudo-labels for each other, rather than relying solely on self-generated pseudo-labels. This setup strategically exploits model or view diversity to reduce confirmation bias and accumulative error, and has been instantiated across classification, detection, segmentation, language modeling, in-context learning, audio-visual correspondence, action recognition, and cross-lingual speech recognition. The cross-pseudo-labeling motif appears under distinct naming conventions (e.g., cross-model pseudo-labeling, cross selection, cross-labeling supervision), but all implementations share the foundational property of model mutualism in the pseudo-labeling step.

## 1. Core Principles and Definitions

Cross-pseudo-labeling distinguishes itself from canonical self-training by leveraging a collaborative or competitive structure: at least two predictive functions (models, subnetworks, branches, or architectures) alternately assign supervision in the form of pseudo-labels to the other. The predictive entities may be identical in architecture or purposely diversified to amplify representational complementarity. The principal rationale is to mitigate undetectable self-errors and confirmation bias that arises when a single model trains exclusively on its own high-confidence outputs [2201.10734, 2202.08502, 2112.09690].

Formally, given unlabeled data $U$, two models $f_1, f_2$ generate pseudo-labels $y^*_1, y^*_2$ for each input, typically accepting only those predictions above a confidence threshold. Each model trains on the other's pseudo-labeled data, optionally fusing this with labeled data if available. This exchange can be symmetrically enforced or selectively filtered for further robustness.

A related strategy is cross-task or cross-lingual pseudo-labeling, where a model trained on one domain or language is used to generate initial pseudo-labels on a new domain (as in transfer learning for ASR [2305.13330] or in LLM label transfer for ICL [2510.24528]).

## 2. Algorithmic Instantiations

Canonical cross-pseudo-labeling is instantiated via multiple architectures:

- **Dual-network cross-selection (CroSel):** Maintains two identical networks with memory banks of historical softmax outputs, using cross-selection criteria (candidate-label set check, stability, confidence) to select pseudo-labels for each other; additionally employs a MixUp-based co-mix consistency term [2303.10365].

- **Cross-rectifying in object detection (CrossRectify):** Parallel detectors cross-validate proposed bounding boxes using IoU matching and select the higher-confidence peer prediction in case of disagreement, replacing erroneous proposals and preventing reinforcement of self-errors [2201.10734].

- **Co-training with within- and cross-group supervision (DFCPS):** Two groups of FixMatch-inspired subnetworks, each with weak and strong augmentation branches, iteratively supervise strong augmentations within the group and the weak branch of the other group for medical image segmentation [2402.11273].

- **Cross-labeling supervision for semi-supervised learning (CLS):** Dual networks generate both positive and negative pseudo-labels (pseudo and complementary), with adaptive weighting based on prediction entropy; each network is additionally trained using its peer’s high-confidence labels [2202.08502].

- **Branchwise pseudo-label exchange in video anomaly detection (CPL-VAD):** Binary anomaly localization and category classification branches with distinct strengths exchange refined snippet-level pseudo-labels, each branch guiding the other toward better temporal or semantic localization [2602.17077].

- **Cross-refine mechanism in audio-visual source localization (XPL):** Two models with distinct backbones and augmentation schemes generate soft pseudo-labels with exponential moving average smoothing that are used to supervise each other's predictions, enhanced by a curriculum data selection phase based on prediction similarity [2403.03095].

- **Cross-model pseudo-labeling in action recognition (CMPL):** Two networks (primary backbone and lightweight auxiliary) with different frame rates or spatial configurations use their own confident predictions to supply pseudo-labels for the other's training on strongly augmented data [2112.09690].

- **Cross-lingual pseudo-labeling in ASR (CLPL):** An acoustic model trained on a source language pseudo-labels unlabeled audio in a target language, using target-language LM rescoring; the pseudo-labeled target data then trains a target AM, with iterative teacher updates [2305.13330].

- **Cross-task pseudo-label transfer for language model ICL (GraphSim, GLIP):** Cross-task labeled data prompt an LLM to pseudo-label a seed subset of a novel task; label propagation via a graph neural network extends pseudo-labels to the full target pool without additional LLM calls [2510.24528].

## 3. Pseudo-Label Filtering, Selection, and Quality Assurance

Effective cross-pseudo-labeling requires rigorous strategies to select and filter exchanged pseudo-labels, aiming to maximize precision while maintaining coverage:

- **Confidence-Based Thresholding:** Most frameworks select pseudo-labels only if maximum per-class prediction confidence surpasses a threshold (e.g., τ=0.9 in [2112.09690, 2202.08502]).
- **Prediction Stability:** Memory banks record sequence of predictions to ensure selection only if outputs remain stable across epochs [2303.10365].
- **Entropy/Uncertainty Weighting:** Sample weights derived from normalized prediction entropy soften the impact of low-confidence or ambiguous pseudo-labels [2202.08502].
- **Peer Disagreement Analysis:** CrossRectify leverages pairwise detector disagreements, preferring predictions with higher maximum confidence among matched proposals [2201.10734].
- **Structural/Graph-based Similarity:** Cross-domain pseudo-labeling employs graph or GNN-based structural similarity to select the closest source examples for seed pseudo-labels and then propagates via label propagation [2510.24528].
- **Soft Labeling and Smoothing:** Soft pseudo-labels obtained through label sharpening and exponential moving average mechanisms stabilize training and guard against abrupt erroneous label updates [2403.03095].

## 4. Training Objectives and Regularization

Most cross-pseudo-labeling methods synthesize loss terms that combine peer-provided pseudo-label supervision with additional regularization, supervised loss, or data consistency constraints:

- **Cross-Entropy on Cross-Selected or Peer Labels:** Each model or branch minimizes the cross-entropy between its output on differently-augmented or new data and the pseudo-label selected by its peer [2202.08502, 2303.10365].
- **Consistency Regularization:** Techniques such as MixUp or strong/weak augmentation enforce robustness of predictions under intra-sample transformations and between mutually supervising models [2303.10365, 2402.11273].
- **Negative/Complementary Learning:** CLS integrates negative supervision via complementary labels (argmin channels) to actively push the model away from incorrect predictions [2202.08502].
- **Auxiliary Losses and Refinements:** Additional terms encourage model similarity, contrastive learning between modalities (XPL), or exploit label propagation consistency (GLIP) [2403.03095, 2510.24528].

Loss weights for cross-supervision and consistency may be annealed or dynamically adjusted depending on the fraction of high-quality selected pseudo-labels, with late-stage training reducing regularization in favor of direct hard-label supervision [2303.10365].

## 5. Empirical Results and Effects

Across a broad array of domains, cross-pseudo-labeling consistently outperforms corresponding single-model or self-pseudo-labeling baselines, with gains attributable to improved pseudo-label quality, diversity, and confirmation bias mitigation:

- **Image Classification:** CLS yields 2–4% gains in accuracy over FixMatch in low-label regimes on CIFAR-10/100 [2202.08502].
- **Object Detection:** CrossRectify achieves absolute mAP boosts up to +4.7% (Faster-R-CNN on VOC split), surpassing both self-training and state-of-the-art single-teacher frameworks [2201.10734].
- **Medical Image Segmentation:** DFCPS shows sustained mIoU increases versus single-view consistency and other state-of-the-art segmentation models, particularly under scarce label conditions [2402.11273].
- **Video Anomaly Detection:** CPL-VAD attains frame-level AP=88.53% (versus 84.51% previously) and higher segmental mAP at all IoU thresholds [2602.17077]. 
- **Audio-Visual Source Localization:** XPL improves CIoU by 5.3 points on Flickr-SoundNet, outperforming both supervised and vanilla hard pseudo-labeling baselines [2403.03095].
- **Action Recognition:** CMPL improves Top-1 accuracy by 9–10% over single-model FixMatch in 1%-labeled UCF-101 and Kinetics-400 [2112.09690].
- **Unsupervised ASR:** Cross-lingual pseudo-labeling reduces WER by 15% absolute compared to end-to-end unsupervised baselines when transferring from German to LJSpeech [2305.13330].
- **In-Context Learning:** Cross-task label transfer with graph label propagation for ICL reduces LLM labeling cost by 80% while maintaining performance at the in-task upper bound [2510.24528].

## 6. Domain-Specific Adaptations and Extensions

Cross-pseudo-labeling has been tailored to address unique challenges across domains:

- **Partial-Label Learning:** In CroSel, cross-selection exploits memory bank filtration designed to resolve ambiguous candidate-label sets. The integration with MixUp-based soft regularization addresses false selection noise [2303.10365].
- **Action and Temporal Domains:** Cross-model pseudo-labelers employ differential designs (frame rates, temporal offsets, network width/depth) to maximize the diversity and complementary strengths of peer models [2112.09690].
- **Video and Audio-Visual Streams:** Cross-pseudo-labeling coordinates between temporally or semantically enriched branches, allowing for accurate boundary localization and semantic disambiguation in weakly-labeled sequence settings [2602.17077, 2403.03095].
- **Cross-Task/Language Transfer:** Aligns source and target domains or languages using structural graph similarity mechanisms or language model constraints to propagate cross-domain pseudo-labels, minimizing data and compute expense [2305.13330, 2510.24528].

## 7. Limitations, Challenges, and Future Directions

While cross-pseudo-labeling demonstrates clear empirical benefits, reported limitations include sensitivity to pseudo-label quality (especially in cross-lingual or cross-task settings), reliance on model diversity, and possible failure modes where both models produce correlated errors or where domain transfer is weak (e.g., highly dissimilar languages [2305.13330]). Advances may lie in refining selection/filtration heuristics, further architectural diversification, integrating structured label propagation, and extending to more general multi-agent or federated systems.

Additionally, research into adaptive exchange protocols, robustness to adversarial labeling errors, and scalable applications to foundation models or multi-modal, structured output spaces remains ongoing. The paradigm remains central in modern semi-supervised, weakly supervised, and transfer learning systems due to its consistent improvements in label efficiency and generalization robustness.

Source: https://www.emergentmind.com/topics/cross-pseudo-labeling