---
title: Curriculum Pseudo Labeling (CPL)
url: https://www.emergentmind.com/topics/curriculum-pseudo-labeling-cpl
type: topic
---

# Curriculum Pseudo Labeling (CPL)

Curriculum Pseudo Labeling (CPL) is a semi-supervised and domain-adaptive learning strategy in which pseudo-labels assigned to unlabeled data are selectively incorporated into training according to a difficulty-ordered schedule—the curriculum. CPL methods modulate which pseudo-labels are accepted for training by dynamically adjusting selection thresholds or weights, based on the learning progress for individual classes, samples, or the model as a whole. This enables reliable and efficient use of unlabeled data, mitigates confirmation bias, and accelerates convergence, particularly in scenarios where labeled data are scarce or data distributions are imbalanced. Originally introduced in the context of image classification with FlexMatch, CPL techniques have since been extended to diverse domains including domain adaptation, regression, multi-label classification, sequence prediction, graph learning, and robotic control.

## 1. Foundational Motivation and Comparison to Fixed-Threshold Approaches

The standard pseudo-labeling paradigm, as exemplified by FixMatch, admits into training only those unlabeled samples whose predicted label confidence exceeds a fixed scalar threshold $\tau$ across all classes and throughout training. This strategy is fundamentally limited: (a) at early training stages, few samples exceed the high threshold, slowing learning; (b) classes with inherently higher difficulty or imbalances are systematically underrepresented among the pseudo-labeled pool, while "easy" classes dominate, reinforcing confirmation bias and class imbalance [2110.08263][2408.07221].

CPL addresses these deficiencies by (i) defining per-class, instance-level, or adaptive thresholds $\varepsilon_c(t)$ that change over training, thereby enabling class-wise or even pixel-wise hard/easy progression; and (ii) establishing a schedule or rationale (the "curriculum") for growing the set of trusted pseudo-labels—starting with those that are easier or more reliable, then incorporating incrementally more difficult examples as the model matures [2110.08263][2302.14013]. This curriculum learning framework enables the model to maximize utilization of the unlabeled pool without sacrificing label quality.

## 2. Core Mathematical Formulations and Algorithms

CPL encompasses a family of formulations. The most canonical, as in FlexMatch, defines for each class $c$ a running statistic of the model’s "learning effect" $\sigma_t(c)$—the count of samples most recently pseudo-labeled as $c$ with confidence exceeding the fixed threshold $\tau$ [2110.08263]. This is normalized to obtain $\beta_t(c) = \sigma_t(c) / \max_{c'} \sigma_t(c')$. A monotonic mapping $M$ (e.g., $M(x) = x/(2-x)$) generates the class-wise dynamic threshold: $\varepsilon_c(t) = \tau \cdot M(\beta_t(c))$. Unlabeled samples $u$ with predicted top-class $c$ and confidence exceeding $\varepsilon_c(t)$ are selected for training.

The pseudo-code for integrating CPL into SSL frameworks is standardized:

```python
for t in range(T):
    # 1. Compute σ_t(c) per class
    # 2. Normalize to get β_t(c)
    # 3. Map via M to get ε_c(t)
    # 4. Use flexible ε_c(t) to filter pseudo-labeled samples for unsupervised loss
    # 5. Proceed with supervised and unsupervised updates
```

Alternative CPL designs for other modalities include percentile-based schedules (monotonically decreasing confidence cutoffs per round) [2001.06001][2302.14013], density-based curricula for domain adaptation [1908.00262], SNR-based filtering for physiological regression [2502.03855], and temporally decaying weights for graph node classification [2504.17641].

A representative summary of key mathematical components is given in the following table:

| Variant                    | Curriculum Variable                    | Selection/Weighting Function          | Update Mechanism         |
|----------------------------|----------------------------------------|---------------------------------------|--------------------------|
| FlexMatch [2110.08263]     | Per-class "effect" $\sigma_t(c)$       | $\varepsilon_c(t) = \tau M(\beta)$    | Class count, mapping     |
| Curriculum Labeling [2001.06001] | Confidence percentile         | Top $r_t$% of confidences             | $\Delta r$ per round     |
| Tabular CPL [2302.14013]   | Confidence & density regularized       | $f(x) = \frac{(\alpha \gamma + 1)c}{\alpha + 1}$ | Density + confidence     |
| ElimPCL [2503.23712]       | Trustworthy set size $|D_{tt}|$        | Entropy/prototype-consistency filter  | Epoch-wise expansion     |
| Semi-rPPG [2502.03855]     | Signal quality (SNR) ranking           | Top $k_e = R_e N_u$ unlabeled samples | Linear ramp $R_e$        |
| PTCL [2504.17641]          | Temporal distance from final label     | $w_u^{t,\tau}$ (exp. decay)           | Step-wise update         |

## 3. Adaptation to Diverse Data Modalities and Learning Problems

CPL was developed in the context of semi-supervised classification but its algorithmic pattern has been transferred to a wide spectrum of settings:

- **Image Classification and SSL Benchmarks**: The FlexMatch framework and its descendants boost both label efficiency and convergence speed on CIFAR-10, CIFAR-100, ImageNet-1K, STL-10, and SVHN, with up to 13.96% relative error reduction versus FixMatch and up to 5× faster convergence in low-label regimes [2110.08263][2309.03469].
- **Unsupervised and Source-Free Domain Adaptation**: CPL methods rank or filter unlabeled target samples by density [1908.00262], prediction entropy/prototype consistency [2503.23712], or other structure, to avoid noise accumulation and mitigate label drift across domains.
- **Regression and Continuous Outputs**: For tasks such as 3D rotation regression (pose), "hardness-aware" curricula use predictive entropy of the output distribution (e.g., matrix-Fisher on SO(3)) to schedule pseudo-label inclusion [2603.21583].
- **Multi-label and Multi-task Learning**: In contextual and label-wise curricula, samples or label-entries are scheduled by intrinsic difficulty, such as label cardinality or confidence [2208.09999][2602.12937].
- **Graph and Temporal Learning**: PTCL weighs pseudo-labels by temporal proximity to final "anchor" labels, using exponentially decaying weights along the temporal axis [2504.17641].
- **Reinforcement and Robotic Learning**: Online grasp learning by CPL exploits per-pixel or per-action confidence schedules (e.g., success probability), ranging from global to spatially contextual [2403.02495].
- **Biomedical Applications**: Semi-rPPG filters unlabeled facial video segments by heart rate signal quality, introducing higher-SNR pseudo-labels first in the curriculum [2502.03855].
- **Tabular Learning**: Regularization by feature density and the cluster assumption complements curriculum selection [2302.14013].

## 4. Theoretical Insights, Convergence, and Empirical Results

CPL methods are grounded in curriculum learning theory, in particular the hypothesis that easy-to-hard progression yields efficient optimization landscapes and mitigates confirmation bias. Though most CPL works do not supply formal convergence proofs, theoretical analyses assert that CPL regularizes pseudo-label selection, yields a balanced empirical risk, and tightens upper bounds on domain adaptation error via progressive trust in pseudo-labels [2112.01948][2110.08263].

Empirical gains are well-quantified:
- FlexMatch reduces error rates by 13.96% and 18.96% over FixMatch on CIFAR-100 and STL-10 (4 labels/class) [2110.08263].
- CPL yields a 5.27% error on CIFAR-10 (4k labels) with ResNet-28-2, rivaling UDA and FixMatch [2001.06001].
- On Office-31 and Office-Home, CPL enhances adaptation accuracy by up to 2.4% [2112.01948][1908.00262].
- SNR-based CPL in Semi-rPPG reduces RMSE by 21.4% relative to supervised-only [2502.03855].
Recent works further confirm that CPL frameworks generalize robustly to long-tail, low-label, and domain-shifted settings, providing marked improvements in convergence time, absolute accuracy metrics, and robustness to out-of-distribution distributions [2309.03469][2503.23712].

## 5. Algorithmic and Practical Implementation Considerations

Most CPL instantiations are computationally efficient and introduce minimal architectural or computational overhead. Standard CPL requires maintaining per-class (or per-label, per-instance) counters or statistics, updating dynamic thresholds, and integrating a schedule (e.g., linear, convex, curriculum residual, or RL-discovered) [2110.08263][2201.08613].

Best practices include:
- Normalizing per-class selection statistics to avoid bias;
- Using rank-based selection (e.g., by percentiles) to bypass sensitive threshold tuning [2001.06001];
- Employing re-initialization or early stopping after pseudo-label integration to avoid confirmation bias and concept drift [2001.06001][2302.14013];
- Combining with strong augmentations (e.g. MixUp, RandAugment, PoseMosaic) for improved regularization [2110.08263][2603.21583][2309.03469];
- Soft pseudo-label interpolation and regularization by density or entropy can further improve pseudo-label quality and downstream performance [2112.01948][2302.14013];
- Adapting curriculum parameters (pace, shape, window size) to dataset scale and class imbalance.

Limitations remain: class-imbalanced or open-set scenarios can distort statistics used for threshold schedules; advanced or meta-learned curricula may be needed for real-world non-uniform distributions; and brute-force threshold-tuning is discouraged in favor of curriculum-driven or percentile-based regularization [2110.08263][2408.07221].

## 6. Variants, Advanced Strategies, and Open Questions

CPL variants have been developed along several axes:
- **Selection metrics**: Confidence, local density, entropy, SNR, temporal proximity, label cardinality, class overlap, contextual or spatial metrics.
- **Weighting schemes**: Binary masking, entropy weighting, exponentially decayed weights, soft label interpolation [2208.09999].
- **Curriculum progression**: Linear, convex, adaptive, RL-discovered, dynamic per-class, or temporal.
- **Integration points**: Global (SSL), per-pixel (robotic grasping), per-timestep (dynamical graphs), across modalities (audio, video, tabular).

Active research topics include:
- Theoretical analysis of convergence and optimal curriculum pacing [2110.08263][2408.07221];
- Performance-based or data-driven discovery of progression schedules [2201.08613];
- Robustness under severe class imbalance or highly non-stationary domains;
- Extension to regression, structured prediction, and other complex output spaces [2603.21583][2502.03855];
- Meta-learning for schedule or selection metric optimization.

## 7. Notable Implementations and Representative Results (Selected Table)

| Domain                | Algorithm (Citation)          | Key CPL Mechanism                        | Relative Gain                |
|-----------------------|------------------------------|------------------------------------------|-----------------------------|
| Image SSL             | FlexMatch [2110.08263]       | Per-class adaptive threshold             | -14% error CIFAR-100        |
| Domain Adaptation     | PCDA [1908.00262]            | Density-based easy→hard curriculum       | +2.7% acc. Office-Home      |
| Multi-label           | PLMCL [2208.09999]           | Momentum, confidence-aware scheduler     | +2–8% F1, various datasets  |
| Regression (SO(3))    | HACMatch [2603.21583]        | Entropy-based, adaptive curriculum       | -2° MeanMed PASCAL3D+       |
| Biomedical (rPPG)     | Semi-rPPG [2502.03855]       | SNR-based linear curriculum              | -21.4% RMSE (VIPL-HR)       |
| Graphs, dynamic node  | PTCL [2504.17641]            | Temporal, exponentially decaying weights | +8–11% ACC/AUC              |
| Tabular Data          | CPL+R-CPL [2302.14013]       | Percentile+likelihood regularization     | +1–2% F1/acc; +29% low-data |

In summary, Curriculum Pseudo Labeling generalizes pseudo-labeling’s reliance on confidence-filtered selection by embedding the sample selection process in a curriculum learning paradigm. By doing so, CPL enables data- and class-adaptive leveraging of unlabeled data, mitigates detrimental error amplification, and drives improved generalization and convergence across domains [2110.08263][2408.07221][2302.14013].

---

### Referenced Works

- FlexMatch: "FlexMatch: Boosting Semi-Supervised Learning with Curriculum Pseudo Labeling" [2110.08263]
- Review: "A Review of Pseudo-Labeling for Computer Vision" [2408.07221]
- Curriculum Labeling: "Curriculum Labeling: Revisiting Pseudo-Labeling for Semi-Supervised Learning" [2001.06001]
- PCDA: "Pseudo-Labeling Curriculum for Unsupervised Domain Adaptation" [1908.00262]
- PLMCL: "PLMCL: Partial-Label Momentum Curriculum Learning for Multi-Label Image Classification" [2208.09999]
- HACMatch: "HACMatch Semi-Supervised Rotation Regression with Hardness-Aware Curriculum Pseudo Labeling" [2603.21583]
- ElimPCL: "ElimPCL: Eliminating Noise Accumulation with Progressive Curriculum Labeling for Source-Free Domain Adaptation" [2503.23712]
- Semi-rPPG: "Semi-rPPG: Semi-Supervised Remote Physiological Measurement with Curriculum Pseudo-Labeling" [2502.03855]
- PTCL: "PTCL: Pseudo-Label Temporal Curriculum Learning for Label-Limited Dynamic Graph" [2504.17641]
- CPL for Tabular: "Revisiting Self-Training with Regularized Pseudo-Labeling for Tabular Data" [2302.14013]

Source: https://www.emergentmind.com/topics/curriculum-pseudo-labeling-cpl