Generalized Pseudo-Label Robust Loss

• GPR Loss is a robust family of loss functions designed for semi-supervised and weakly supervised classification, effectively addressing label noise and class imbalance.
• It unifies generalized, beta, and symmetric cross-entropy losses with adaptive weighting and soft pseudo-label estimation to mitigate false positives and negatives.
• Empirical results demonstrate enhanced performance in tasks such as image classification and medical image segmentation, achieving state-of-the-art results on SPML benchmarks.

Generalized Pseudo-Label Robust (GPR) Loss is a family of loss functions and learning strategies for semi-supervised and weakly supervised classification, particularly Single Positive Multi-Label Learning (SPML), where label noise and class imbalance pose significant challenges. GPR Loss unifies several robust losses—such as the generalized cross-entropy, beta cross-entropy, and symmetric cross-entropy—and generalizes them with adaptive weighting, soft pseudo-label estimation, and instance/class reweighting. This approach systematically mitigates both false positives and false negatives from pseudo-labeled data, producing gains in empirical performance and theoretical robustness across diverse tasks including medical image segmentation, image classification, and large-scale multi-label benchmarks (Cui et al., 2022, Chen et al., 6 May 2024, Tran et al., 28 Aug 2025).

1. Formal Definitions and Loss Structure

GPR Loss is defined over datasets with partially labeled or pseudo-labeled data. It extends simple cross-entropy by combining robust surrogates, soft pseudo-labels, and dynamic weighting. For each sample-label pair, different loss terms are used depending on the status of the label (true, missing, pseudo-positive, pseudo-negative), and these are adaptively weighted.

Multi-class Semi-Supervised Setting

Let:

• $\mathcal{D}_{\ell} = \{(x_i, y_i)\}_{n_\text{lab}}$ be the labeled set.
• $\mathcal{D}_u = \{x_j\}_{n_\text{unlab}}$ be the unlabeled set.
• $\hat{y}_j = \text{Teacher}(x_j)$ is the (potentially noisy) pseudo-label assigned by a teacher network.
• $p_\theta(k|x)$ is the student network output (class probability).

The GPR loss general form (Cui et al., 2022):

$$L_{GPR}(\theta) = \mathbb{E}_{(x,y)\sim\mathcal{D}_\ell}[CE(p_\theta(\cdot|x),y)] + \lambda \mathbb{E}_{x\sim\mathcal{D}_u}[L_r(p_\theta(\cdot|x), \hat{y})]$$

where $L_r$ is a robust surrogate chosen from the following options:

• Generalized Cross-Entropy (GCE):

$$L_{GCE}(p,\hat{y}) = \frac{\left[ 1-p(\hat{y}|x) \right]^q}{q}$$, $0 < q \leq 1$

• Beta Cross-Entropy (BCE):

$$L_{BCE}(p,\hat{y}) = \frac{\beta+1}{\beta}(1-p(\hat{y}|x))^\beta + \sum_k p(k|x)^{\beta+1}$$, $\beta > 0$

• Symmetric Cross-Entropy (SCE):

$$L_{SCE}(p,\hat{y}) = \alpha\,CE(p,\hat{y}) + \gamma\,L_{RCE}(p, \hat{y})$$, $\alpha, \gamma > 0$

Each loss saturates or downweights high-loss outliers as the noise in $\hat{y}$ increases.

Multi-label SPML Setting

For $C$ classes and SPML datasets where only a single positive and many missing labels are provided for each instance (Chen et al., 6 May 2024, Tran et al., 28 Aug 2025):

$$\mathcal{L}_{GPR} = \frac{1}{N} \sum_{n=1}^N \sum_{i=1}^C v(p^n_i; \alpha)\, \mathcal{L}^n_i$$

$$\mathcal{L}^n_i = s^n_i\,\mathcal{L}_1(p^n_i) + (1-s^n_i)\left[\,\hat{k}(p^n_i;\beta)\,\mathcal{L}_2(p^n_i) + (1-\hat{k}(p^n_i;\beta))\,\mathcal{L}_3(p^n_i)\right]$$

where $s^n_i$ is the annotated label indicator; $\hat{k}(p;\beta)$ is a soft pseudo-label estimator (typically a scheduled sigmoid of $p$); $v(\cdot;\alpha)$ is a Gaussian reweighting function centered at $\mu$, with scale $\sigma$; and loss surrogates $\mathcal{L}_1$, $\mathcal{L}_2$, $\mathcal{L}_3$ interpolate between mean absolute error and cross-entropy according to powers $q_1,q_2,q_3$.

2. Robust Loss Components and Mechanisms

The central mechanism of GPR Loss is to improve robustness to noisy or incomplete pseudo-labels by:

• Introducing soft, data-dependent pseudo-label probabilities in place of hard assignments.
• Employing loss surrogates that bound the influence of incorrect pseudo-labels.
• Downweighting ambiguous or boundary predictions through a mixture of Gaussian or clipped weighting functions.
• Smoothly interpolating between mean absolute error and cross-entropy to allow both slow error saturation (for outliers) and sharp discrimination (for clean examples).

Hyperparameters $q_j$ adjust this interpolation: $q_j = 1$ yields MAE-style losses with high noise robustness; $q_j \to 0$ gives BCE-style loss with faster convergence but higher sensitivity to mislabeling.

The pseudo-label estimator $\hat{k}(p;\beta)$ and instance weight $v(p;\alpha)$ can be scheduled over epochs, allowing conservative learning at early stages and more aggressive utilization of pseudo-labels as confidence grows.

3. Training Algorithms and Optimization

GPR Loss is integrated into various training paradigms:

Teacher-Student with Robust Loss (Cui et al., 2022)

1. Pretrain the teacher on labeled data with CE.
2. Generate pseudo-labels for unlabeled data.
3. Initialize student network.
4. Jointly train: In each batch, apply CE on labeled data, GPR loss (any robust surrogate) on pseudo-labeled data. Mix labeled and pseudo-labeled samples in the batch.
5. Optionally, pseudo-labels may be iteratively refreshed as the student improves.

SPML: Adaptive, Multi-Stage Optimization (Chen et al., 6 May 2024, Tran et al., 28 Aug 2025)

• For each epoch, refresh pseudo-labels via external methods (e.g., DAMP).
• Compute GPR Loss per sample/class using current pseudo-labels and adaptive weighting.
• Include a regularization term penalizing deviation of the model's predicted mean number of positives per sample from the dataset prior.
• Optimization typically uses SGD or Adam, with loss surrogates and weighting updated on a linear schedule.

Pseudocode for batch-level loss computation appears in (Tran et al., 28 Aug 2025), exhibiting explicit if-else logic for separating positive, undefined, negative, and pseudo-positive labels, and the corresponding weighting procedures.

4. Applications and Empirical Performance

GPR Loss is deployed in multiple contexts:

• Image classification with limited labeled data. On CIFAR-10 with 10% labeled data, GPR-based student models using BCE, GCE, or SCE achieve up to 13.4% higher accuracy over teacher-only baselines (Cui et al., 2022).
• Medical image segmentation with partial annotations. On BraTS-2018, GPR Loss improves dice scores by 0.01–0.02 (absolute) over vanilla CE in low-label regimes.
• Single Positive Multi-Label Learning (SPML) (Chen et al., 6 May 2024, Tran et al., 28 Aug 2025). On large-scale multi-label datasets (VOC, COCO, NUS-WIDE, CUB), GPR-based models surpass previous state of the art by up to 1.6 mean average precision (mAP), demonstrating both superior precision and separation of true positive and negative prediction distributions.

Dataset	AN Baseline	GR Loss	VLPL	AEVLP (GPR+DAMP)
VOC	85.9	89.8	89.1	90.5
COCO	64.9	73.2	71.5	73.5
NUS-WIDE	42.3	49.1	49.6	50.7
CUB	18.3	21.6	24.0	24.9

This table shows state-of-the-art performance achieved by AEVLP (the framework using GPR Loss with DAMP) compared to prior methods (Tran et al., 28 Aug 2025).

5. Theoretical Properties and Generalization

GPR Loss possesses the following theoretical properties:

• Robustness: The robust surrogates saturate or down-weight the effect of high-loss samples (noisy labels, outliers), which prevents gradient explosions associated with mislabeled data.
• Adaptivity: The loss recovers standard BCE or robust losses in special cases, ensuring backward compatibility and continuity. For reliable pseudo-labels, GPR Loss converges to previously proposed robust designs.
• Consistency: Under the SCAR assumption and with sufficient parameter flexibility, GPR Loss is an unbiased estimator of the true risk and asymptotically converges to the Bayes-optimal classifier as the sample size grows (Chen et al., 6 May 2024).
• Gradient Control: The mixed MAE/BCE loss structure enables large gradients for rare positives (fast learning) and suppressed gradients for confident negatives (noise reduction).

Regularization, such as matching the predicted mean number of positives to the dataset prior, further stabilizes optimization and encourages realistic output distributions (Tran et al., 28 Aug 2025).

6. Extensions: Dynamic Augmented Multi-focus Pseudo-labeling (DAMP) and AEVLP

DAMP augments the GPR Loss regime by periodically regenerating pseudo-labels using CLIP-based global and local views, leveraging spatial context and augmentation diversity. Overlapping grid crops and adaptive similarity aggregation yield both positive and negative pseudo-labels, facilitating continuous label set expansion and pruning, improving the resilience of GPR Loss to early pseudo-labeling errors (Tran et al., 28 Aug 2025).

The Adaptive and Efficient Vision-Language Pseudo-Labeling (AEVLP) framework combines GPR Loss and DAMP, further advancing robustness and empirical performance in SPML. Ablation studies confirm that multi-focus pseudo-labeling and robust reweighting are the most critical contributors to these gains.

7. Comparative Analysis and Ablations

Ablation studies across VOC and COCO demonstrate that the integration of soft pseudo-label estimation, Gaussian reweighting, and robust surrogates collectively yield the best empirical results. Removing any one component degrades mAP by 0.1–0.3. GPR Loss's performance remains stable even as the negative pseudo-label ratio increases, underscoring its resilience to hyperparameter shifts (Tran et al., 28 Aug 2025, Chen et al., 6 May 2024).

This approach generalizes and subsumes a spectrum of existing SPML/MLML loss designs, including traditional assumed-negative BCE, EM Loss, Hill Loss, and focal margin methods, by specific settings of its parameters and weighting functions (Chen et al., 6 May 2024).

---

References:

• (Cui et al., 2022): Semi-supervised Learning using Robust Loss (2022)
• (Chen et al., 6 May 2024): Boosting Single Positive Multi-label Classification with Generalized Robust Loss (2024)
• (Tran et al., 28 Aug 2025): More Reliable Pseudo-labels, Better Performance: A Generalized Approach to Single Positive Multi-label Learning (2025)