Iterative Pseudo-Label Refinement

Updated 8 July 2026

Iterative pseudo-label refinement is a process that repeatedly updates model-generated labels on unlabeled data using correction operators to reduce error accumulation and confirmation bias.
It employs diverse strategies—such as confidence filtering, clustering consensus, and generative denoising—to convert noisy supervisory signals into improved labels across domains.
Integrating robust training curricula and stage alternation, this method has shown significant performance gains in applications from speech recognition to semantic segmentation.

Iterative pseudo-label refinement is a family of procedures in which model-generated labels on unlabeled or weakly labeled data are not treated as a one-shot surrogate for annotation, but are repeatedly regenerated, corrected, filtered, or reweighted as the model and auxiliary estimators improve. Across semi-supervised learning, unsupervised domain adaptation, noisy-label learning, weak supervision, retrieval, and automatic speech recognition, the central objective is to use successive refinement cycles to reduce confirmation bias, suppress error accumulation, and convert imperfect pseudo-labels into progressively stronger supervisory signals (Xu et al., 2020, Kwon et al., 8 Apr 2026, Teh et al., 2021).

1. Canonical formulation and scope

The canonical loop has three elements: an initial model trained from limited supervision, a pseudo-label generation step on unlabeled data, and a refinement-and-retraining step that updates either the same model or a student model. In "Iterative Pseudo-Labeling for Speech Recognition" (Xu et al., 2020), this is expressed as drawing a subset of unlabeled data, decoding it to obtain pseudo-labels, and fine-tuning the current acoustic model on labeled and pseudo-labeled data. The associated objective is

$\mathscr{L} = \mathscr{L}_L + \lambda \mathscr{L}_U,$

with pseudo-labels typically produced by

$\hat{y} = \arg\max_y p_\theta(y|x),$

or by beam search with an external LLM.

Many later systems retain this basic structure while changing the refinement operator. The wheat head segmentation framework uses an iterative teacher-student loop in which the teacher generates masks, the student is trained first on pseudo-labeled data and then fine-tuned on real labels, and the student becomes the next teacher (Jiang et al., 7 Dec 2025). The Mandarin-English code-switching ASR method likewise alternates pseudo-label generation, two-stage bilingual model training, and iterative improvements, with the pseudo-labeled set

$P = \{(x_j, M(x_j)) \mid x_j \in U\}$

serving as the intermediate training target (Yang et al., 6 Jul 2026). ReHear replaces direct reuse of ASR hypotheses with a correction step mediated by an audio-aware LLM, yielding corrected pseudo-labels before the ASR model is retrained (Liu et al., 21 Feb 2026).

This breadth indicates that iterative pseudo-label refinement is not a single algorithm but a reusable control structure. A plausible implication is that the term is best understood operationally: pseudo-labels are treated as latent supervisory variables that are repeatedly updated under model, data, or auxiliary-prior changes rather than fixed once and consumed passively.

The distinguishing feature of a given method is the operator used to improve pseudo-labels between iterations. The literature represented here includes confidence filtering, neighborhood or cluster consensus, reconstruction-based correction, generative denoising, geometry-aware filtering, and multimodal correction.

Confidence-based selection is the simplest form. The wheat head segmentation system keeps only predictions above a confidence threshold for the next phase (Jiang et al., 7 Dec 2025). The instance-dependent label-noise framework updates labels only for samples that are confident across multiple stages, using the criterion

$\mathcal{L}_{CE}(f_\theta(x_i), y_i) < \tau,$

with $\tau = 1$ , and requiring multistage consensus before relabeling (Bala et al., 2024). In 3D weakly supervised semantic segmentation, the Class-Aware Label Refinement module keeps the top- $V\%$ confident points for each class and marks the remainder as unlabeled, explicitly countering class imbalance (Xu et al., 17 Oct 2025).

Other methods refine pseudo-labels by exploiting neighborhood structure or inter-generation consistency. ProtoCon refines pseudo-labels with nearest neighbours identified by online clustering in an embedding space trained with a prototypical loss, and uses the label history of the entire dataset across training cycles without storing image embeddings (Nassar et al., 2023). RLCC estimates similarities between pseudo-labels in consecutive generations via an IoU-based clustering consensus matrix and propagates soft labels across generations before temporal ensembling (Zhang et al., 2021). iLPC constructs a $k$ -NN graph, performs label propagation with

$Z = (I - \alpha W)^{-1}Y,$

balances class marginals via Sinkhorn normalization, and then keeps only the cleanest pseudo-labels according to the loss distribution of a limited-capacity classifier (Lazarou et al., 2020). PCL for semi-supervised object detection iteratively refines pseudo boxes through multi-round refining and then performs multi-vote weighting over jittered boxes to correct localization noise (He et al., 2023).

Reconstruction-based and generative correctors introduce a stronger denoising mechanism. RePL identifies unreliable voxels using agreement and confidence from teacher and student, masks these regions, and reconstructs improved pseudo-labels with a dedicated refiner (Kwon et al., 8 Apr 2026). The UDA method based on conditional GANs uses a jointly trained classifier and cGAN, arguing that cGANs trained with noisy pseudo-labels can generate cleaner target samples and thereby refine pseudo-labels iteratively (Morerio et al., 2020). EReCu combines teacher-student pseudo-label evolution fusion with local pseudo-label refinement driven by attention diversity and multi-cue native perception (Jiang et al., 12 Mar 2026). ReHear conditions an instruction-tuned corrector on both the ASR hypothesis and the source audio, so that pseudo-label refinement is grounded in the acoustic signal rather than text alone (Liu et al., 21 Feb 2026).

Refinement operator	Representative mechanism	Example papers
Confidence filtering	Thresholds or consensus over stages	(Jiang et al., 7 Dec 2025, Bala et al., 2024)
Cluster or graph consensus	Online clustering, IoU mapping, label propagation	(Nassar et al., 2023, Zhang et al., 2021, Lazarou et al., 2020)
Reconstruction or denoising	Masked reconstruction, generative cleaning, multimodal correction	(Kwon et al., 8 Apr 2026, Morerio et al., 2020, Liu et al., 21 Feb 2026)
Structure-aware filtering	Geometry, superpoints, local detail priors	(Xu et al., 17 Oct 2025, Jiang et al., 12 Mar 2026)
Localization correction	Multi-round box refinement and weighted box voting	(He et al., 2023)

Taken together, these methods show that refinement often targets a specific failure mode: semantic ambiguity, localization error, class imbalance, temporal inconsistency, or modality-specific corruption. This suggests that successful refinement depends less on the generic self-training loop than on aligning the correction operator with the dominant error structure of the task.

3. Training curricula and optimization strategies

Iterative pseudo-label refinement is frequently coupled with a curriculum that separates noisy large-scale supervision from scarce high-quality supervision. The wheat head segmentation framework adopts a two-stage hybrid strategy in which the student is trained from scratch on a large pseudo-labeled dataset at $512 \times 512$ for 40 epochs with batch size 8 and learning rate $6 \times 10^{-5}$ , then fine-tuned on 99 hand-labeled wheat images at $\hat{y} = \arg\max_y p_\theta(y|x),$ 0 for 25 epochs per fold with batch size 1, learning rate $\hat{y} = \arg\max_y p_\theta(y|x),$ 1, and AdamW (Jiang et al., 7 Dec 2025). The code-switching ASR method uses an analogous pre-train/fine-tune structure, first training on pseudo-labeled monolingual and code-switching speech and then fine-tuning on human-labeled monolingual and code-switching data; the paper reports that the fine-tuning sequence matters, and that best results are achieved when first fine-tuning on semi-supervised code-switching data and then on supervised human-labeled data (Yang et al., 6 Jul 2026).

Some works replace mixture training with stage alternation. "The GIST and RIST of Iterative Self-Training for Semi-Supervised Segmentation" (Teh et al., 2021) reports that iterative self-training degrades when a fixed ratio of human-labeled to pseudo-labeled data is used naively across stages. GIST and RIST instead choose $\hat{y} = \arg\max_y p_\theta(y|x),$ 2 at each stage, training on only human-labeled or only pseudo-labeled data during that stage, with the stage loss

$\hat{y} = \arg\max_y p_\theta(y|x),$ 3

The purpose is to avoid competing objectives and pseudo-label bloat.

Other curricula rely on robust optimization after sample splitting. PCSR first divides samples into clean and noisy pairs with a Gaussian mixture model over matching losses, then subdivides noisy pairs into ambiguous and refinable subsets by the Pseudo-label Consistency Score, and finally applies Adaptive Pair Optimization: caption rematching for refinable pairs and Generalized Cross-Entropy for ambiguous pairs (Liu et al., 19 Sep 2025). The UU-learning approach for LLM-generated pseudo-labels iteratively retrains a classifier using two unlabeled corpora with different positive-class ratios and then relabels the full corpus, using the robust UU risk

$\hat{y} = \arg\max_y p_\theta(y|x),$ 4

to stabilize training under noisy pseudo-labels (Asano et al., 18 Feb 2025).

A recurring pattern is that refinement quality depends on how noisy pseudo-labels are introduced into optimization. The pseudo-labels are rarely used uniformly; instead, papers repeatedly separate broad noisy supervision from narrower trusted supervision, either temporally, by sample type, or by loss design.

4. Theoretical views and failure modes

Several papers make the theoretical claim that refinement can be easier than direct prediction, but only under identifiable conditions. RePL formalizes refinement as predicting $\hat{y} = \arg\max_y p_\theta(y|x),$ 5 from both the input and an initial pseudo-label, yielding

$\hat{y} = \arg\max_y p_\theta(y|x),$ 6

and then states an explicit condition for beneficial correction:

$\hat{y} = \arg\max_y p_\theta(y|x),$ 7

equivalently

$\hat{y} = \arg\max_y p_\theta(y|x),$ 8

Here $\hat{y} = \arg\max_y p_\theta(y|x),$ 9 is the precision of the error mask on unreliable voxels, $P = \{(x_j, M(x_j)) \mid x_j \in U\}$ 0 is the correction rate, and $P = \{(x_j, M(x_j)) \mid x_j \in U\}$ 1 is the error introduction rate (Kwon et al., 8 Apr 2026). The reported empirical example for SemanticKITTI 1% labeled gives $P = \{(x_j, M(x_j)) \mid x_j \in U\}$ 2, under which refinement remains beneficial over a broad region.

The reduction-based pseudo-label generation method for instance-dependent partial label learning gives a related but distinct argument. It states that reduction-based pseudo-labels produced by a multi-branch auxiliary model are more consistent with the Bayes optimal classifier than pseudo-labels directly generated by the predictive model, particularly for instances affected by disturbing incorrect labels (Qiao et al., 2024). The theorem is a formal response to a common pathology of iterative self-training: if the same model both overfits and then re-supervises the same mistakes, the loop can harden the wrong latent assignment.

This failure mode is repeatedly observed empirically. Naive multi-stage self-training for semantic segmentation degrades under a fixed labeled-to-pseudo-labeled ratio (Teh et al., 2021). In 3D weakly supervised semantic segmentation, performance increases with more self-training cycles until $P = \{(x_j, M(x_j)) \mid x_j \in U\}$ 3, after which overfitting or error propagation can begin (Xu et al., 17 Oct 2025). ReHear reports that standard iterative pseudo-labeling can perform worse than iterative supervised learning without pseudo-labeling, with WER increasing from 9.66 to 16.52 on AMI test under a baseline IPL configuration, whereas the corrected-pseudo-label framework reverses that pattern (Liu et al., 21 Feb 2026).

A common misconception is therefore that more refinement rounds are intrinsically beneficial. The cited works indicate the opposite: iteration is advantageous only when the refinement operator improves target quality faster than the loop amplifies preexisting errors. Another misconception is that pseudo-label refinement is synonymous with confidence thresholding; the surveyed methods show that thresholds are only one special case within a much larger design space.

5. Empirical behavior across application domains

Empirical results span speech, segmentation, detection, retrieval, re-identification, and noisy-label classification. In ASR, IPL reaches 4.01% test-other WER on LibriSpeech LS-960 with 4-gram plus Transformer LM rescoring, and reports up to a factor-of-5 computational saving by fine-tuning rather than retraining from scratch at each round (Xu et al., 2020). Online refinement for BEST-RQ improves test-other WER from 10.1% to 8.8% when PCA projection, iterative codebook refinement, and codebook distillation are all enabled (Xu et al., 24 Jun 2026). In Mandarin-English code-switching ASR, iterative pseudo-labeling reduces Mix Error Rate to 12.88% on SEAME devman and 18.89% on devsge, corresponding to absolute reductions of 6.35% and 8.29% (Yang et al., 6 Jul 2026).

In vision, the wheat head segmentation system reports mIoU of 0.7480 on the Development Phase and 0.7099 on the Testing Phase (Jiang et al., 7 Dec 2025). RePL reaches 60.0 mIoU on nuScenes-lidarseg with 1% labels and 75.8 mIoU with 50% labels, and 54.7 mIoU and 65.9 mIoU on SemanticKITTI at the same label ratios (Kwon et al., 8 Apr 2026). PCL reports gains over the supervised baseline on MS COCO of 12.16, 12.11, and 9.57 mAP at 1%, 5%, and 10% labeling ratios, and gains of 3.90, 2.54, and 2.43 mAP over SoftTeacher (He et al., 2023). In unsupervised object re-identification, RLCC improves MSMT17 mAP from 19.1 to 27.9, a gain of 8.8 (Zhang et al., 2021).

Domain	Reported outcome	Paper
LibriSpeech ASR	4.01% test-other WER	(Xu et al., 2020)
BEST-RQ ASR	10.1% to 8.8% test-other WER	(Xu et al., 24 Jun 2026)
Code-switching ASR	12.88% / 18.89% MER on devman / devsge	(Yang et al., 6 Jul 2026)
Wheat segmentation	0.7480 / 0.7099 mIoU on Development / Testing	(Jiang et al., 7 Dec 2025)
LiDAR segmentation	60.0 mIoU at 1% labels on nuScenes-lidarseg	(Kwon et al., 8 Apr 2026)
Semi-supervised detection	+12.16 mAP over supervised baseline at 1% labels	(He et al., 2023)
Unsupervised re-ID	+8.8 mAP on MSMT17	(Zhang et al., 2021)

Despite the diversity of tasks, the empirical pattern is relatively consistent. Gains are usually largest when labels are scarce, noisy, weak, or domain-shifted; early and middle training phases often benefit most from refinement; and methods that explicitly denoise or reweight pseudo-labels tend to outperform those that merely regenerate them. ProtoCon’s abstract also emphasizes faster convergence and low-cost scaling by using online clustering and dataset-wide label history without storing image embeddings (Nassar et al., 2023).

Iterative pseudo-label refinement overlaps with, but is not identical to, self-training, temporal ensembling, noisy-label correction, weak-supervision completion, and self-supervised target generation. The overlaps are concrete. RLCC explicitly imports the spirit of temporal ensembling into dynamically changing clustering labels across generations (Zhang et al., 2021). Active refinement for multi-label learning treats pseudo-labels as variables in a bi-level optimization and further uses them to drive a query strategy for interactive annotation (Hsieh et al., 2021). The ID-PLL and 3D WSSS methods frame refinement as supervision completion under partial or sparse labels rather than as classical semi-supervised learning (Qiao et al., 2024, Xu et al., 17 Oct 2025). BEST-RQ refinement operates inside self-supervised speech pretraining, where pseudo-labels are quantized targets rather than class labels (Xu et al., 24 Jun 2026).

These connections clarify the conceptual boundary of the field. Iterative pseudo-label refinement is present whenever a system repeatedly updates automatically generated supervisory targets and uses those updated targets to alter subsequent training. The targets may be transcripts, class labels, segmentation masks, bounding boxes, cluster IDs, codebook indices, or correspondence reliability scores. What unifies the literature is not output type but the recursive dependency between target estimation and model improvement.

A plausible synthesis from the surveyed work is that refinement succeeds when three conditions hold simultaneously: the system can identify which pseudo-labels are most likely wrong, it has an operator capable of producing better alternatives than the current model alone, and the training schedule prevents corrected labels from being immediately drowned by the remaining noisy supervision. The papers surveyed instantiate these conditions through clustering consensus, masked reconstruction, geometry-aware filtering, teacher-student alternation, multimodal correction, active querying, and robust loss design (Kwon et al., 8 Apr 2026, Teh et al., 2021, Liu et al., 21 Feb 2026).