Positive-Unlabeled Contrastive Learning (PUCL)

• Positive-Unlabeled Contrastive Learning (PUCL) is a self-supervised and semi-supervised framework that mitigates negative-sampling bias by correcting for unlabeled positive contamination in contrastive loss.
• The approach uses a debiased loss with Monte Carlo estimates to accurately adjust for the mixture of positives and negatives, enabling improved downstream classification and robust representation learning.
• Extensions such as puNCE and UCF introduce prior weighting, pseudo-label clustering, and uncertainty estimation, offering enhanced stability and performance in imbalanced or noisy data settings.

Positive-Unlabeled Contrastive Learning (PUCL) is a framework for self-supervised, semi-supervised, and weakly supervised representation learning in scenarios where only a subset of positive samples are labeled while the remaining dataset consists of unlabeled samples, which may contain both positive and negative instances. PUCL corrects for the negative-sampling bias inherent in standard contrastive learning with unlabeled negatives, enabling improved representation quality and downstream classification—especially under label scarcity, class imbalance, or noise conditions.

1. Motivation and Standard Contrastive Learning Bias

Contrastive learning algorithms such as SimCLR and MoCo typically train an encoder $f(\cdot)$ that projects data $x$ into normalized embeddings $z = f(x) \in \mathbb{S}^{d-1}$. Each anchor sample $x$ is paired with a positive sample $x^+$ (an augmentation), with similarity measured by $h(x, x') = \exp(z^\top z')$. Negatives are sampled from the remainder of the dataset, and the standard objective uses a denominator including these “negative” samples.

However, in realistic settings where negative labels are unavailable, negatives are drawn uniformly from the dataset (excluding $x$ and $x^+$). This introduces “negative-sampling bias” because the negative set is actually a positive-unlabeled (PU) mixture: it includes unlabeled positives. This contamination results in a biased contrastive loss, which can degrade the learned representations and downstream classifier accuracy (Wang et al., 2024).

2. Formalization: PUCL Loss and Estimator

PUCL treats the negative sampling problem as a PU learning problem. Under the SCAR (Selected-Completely-At-Random) assumption with positive prior $\alpha = \Pr(y=1)$ and labeling frequency $c$, the true negative distribution can be written as:

$$p^-_x(x') = \frac{1-\alpha c}{1-\alpha} p^u_x(x') - \frac{\alpha(1-c)}{1-\alpha} p^+_x(x')$$

where $p^u_x$ is the distribution of unlabeled samples (mixture) and $p^+_x$ is the distribution of positives (augmentations).

The debiased loss, denoted $L_{DeCL}$, replaces the noisy negative sum in the denominator with its expectation under the corrected negative distribution:

$$L_{DeCL} = \mathbb{E}_{x, x^+}\left[-\log \frac{h(x, x^+)}{h(x, x^+) + N\mu_x}\right]$$

with the corrected $\mu_x$ defined as

$$\mu_x = \frac{1-\alpha c}{1-\alpha} \mathbb{E}_{x^u \sim p^u_x}[h(x, x^u)] - \frac{\alpha(1-c)}{1-\alpha} \mathbb{E}_{x^+ \sim p^+_x}[h(x, x^+)]$$

In practice, Monte Carlo estimates of the expectations are used:

$$\hat{\mu}_x = \max\Bigg\{ \frac{1-\alpha c}{1-\alpha}\frac{1}{M^u}\sum_{i=1}^{M^u} h(x, x^u_i) - \frac{\alpha(1-c)}{1-\alpha}\frac{1}{M^+}\sum_{j=1}^{M^+} h(x, x^+_j),\, e^{-1} \Bigg\}$$

and the training objective becomes

$$L_{PUCL} = \mathbb{E}_{x, x^+}\left[ -\log \frac{h(x, x^+)}{h(x, x^+) + N \hat{\mu}_x} \right]$$

The lower clamp at $e^{-1}$ ensures numerical stability (Wang et al., 2024).

3. PU-Aware Contrastive Formulations and Extensions

Several variants and extensions of PUCL address different regimes and make varying assumptions about the availability of class prior $\pi$ or leverage explicit pseudo-labeling:

3.1 Unbiased Loss Without Known Priors

When the class prior is unknown, frameworks such as puCL (Acharya et al., 2024) employ minibatch structures such that:

• For labeled positives, anchor views are pulled toward other labeled positives.
• For unlabeled anchors, each is pulled only toward its own augmentation.

The objective achieves unbiasedness and reduces variance compared to standard InfoNCE. Concretely, in a batch of $2b$ views:

$$\mathcal{L}_{puCL} = -\frac{1}{2b} \sum_{i=1}^{2b} \Bigg[ 1_{x_i \in P} \frac{1}{|P \setminus \{i\}|} \sum_{j \in P \setminus \{i\}} \log P_{ij} + 1_{x_i \in U} \log P_{i,a(i)} \Bigg]$$

where $$P_{ij} = \exp(z_i \cdot z_j) / \sum_{k \neq i} \exp(z_i \cdot z_k)$$ and $a(i)$ indexes the augmented pair (Acharya et al., 2024).

3.2 Prior-Aware InfoNCE (puNCE)

If the positive prior $\pi$ is available or estimated, losses such as puNCE softly weight each unlabeled anchor’s positive and negative contributions:

• Each unlabeled anchor is treated as positive with weight $\pi$ and negative with $1-\pi$.
• The loss for unlabeled anchors interpolates between being a positive and a negative anchor by mixing the contributions accordingly (Acharya et al., 2022, Acharya et al., 2024).

3.3 Uncertainty-Weighted PUCL

To address highly noisy and imbalanced PU scenarios (e.g., cybersecurity, biomedical text), the Uncertainty Contrastive Framework (UCF) (Hossain et al., 1 Dec 2025) incorporates:

• A per-sample uncertainty/confidence estimator
• Reweighting of each pair in the contrastive loss by uncertainty
• Positive-only anchor batches to stabilize gradients
• Adaptive, batch-level temperature scaling to further stabilize training

The resultant loss is:

$$L^{PU} = \frac{1}{R} \sum_{i=1}^{R} \bigg[ I(x_i)\frac{1}{|B_1(x_i)|}\sum_{x_p \in B_1(x_i)} \ell(z_i, z_p) + (1 - I(x_i))\frac{1}{|B_0(x_i)|}\sum_{x_n \in B_0(x_i)} \ell(z_i, z_n) \bigg] \cdot u_i$$

where $u_i$ is the per-sample uncertainty, $I$ is an (estimated) positive indicator, and $\ell$ is the standard InfoNCE kernel (Hossain et al., 1 Dec 2025).

4. Algorithmic Implementation and Protocols

The general PUCL procedure can be implemented as a wrapper around standard contrastive pipelines:

• For each anchor, sample a positive view and multiple unlabeled samples.
• Estimate positive prior $\alpha$ (and $c$, if using the SCAR-based loss).
• Compute the correction term $\hat{\mu}_x$ and clamp it for stability.
• Standard augmentations, backbone choices, and big-batch protocols are used, e.g., SimCLR with batch size 512 ($N \approx 510$), MoCo with N=4096 (Wang et al., 2024).
• For UCF, dynamically sample batches containing both positives and unlabeled, use uncertainty-based weighting, positive anchors, and adaptive temperature (Hossain et al., 1 Dec 2025).

Implementation is computationally efficient; the dominant costs are embedding evaluation and a scalar correction per anchor (Wang et al., 2024). Hyperparameter sensitivity is low for the correction, with optimal positive prior typically matching the true class ratio.

5. Theoretical Analysis

PUCL algorithms provide the following guarantees:

• Negligible additional bias compared to the ideal contrastive loss; the deviation is upper bounded by $1/[2\sqrt{N}(e^2-1)]$ for population mean replacement (Wang et al., 2024).
• Unbiasedness for the “fully-labeled” contrastive objective in both puCL and puNCE when their assumptions hold (Acharya et al., 2024, Acharya et al., 2022).
• Variance and gradient-sampling bias are strictly smaller for puCL than for standard self-supervised contrastive losses, especially as labeled positive fraction increases (Acharya et al., 2024).
• For downstream classification, theoretical results link cluster alignment, augmentation concentration, and linear probe error; optimizing the (PU) contrastive objective provably aligns clusters for k-means and enables vanishing classification error under natural conditions (Acharya et al., 2024).
• Use of soft pseudo-labels (via weighting, not hard clustering) in puNCE recovers the expected supervised contrastive gradient in the asymptotic regime (Acharya et al., 2022).

6. Empirical Results and Ablations

Extensive evaluation demonstrates that PUCL yields improvements across vision (CIFAR10/100, STL10, FMNIST), text (SST-2), and graph (InfoGraph) benchmarks:

• On CIFAR10, SimCLR with PUCL improves Top-1 accuracy from 90.44% (baseline) to 92.14%; similar gains hold for CMC and MoCo backbones (Wang et al., 2024).
• On graphs, PUCL consistently yields gains of +3–6 ppt over base InfoGraph, and +2–6 ppt versus strong baselines (Wang et al., 2024).
• UCF achieves over 93% accuracy and near-perfect recall in high-stakes malicious content detection, even with severe class imbalance (Hossain et al., 1 Dec 2025).
• puNCE achieves dramatic downstream PU classification improvements, e.g., on PU-CIFAR10 achieving 97.6% with linear probe (vs. 88–92% for state-of-the-art baselines at $n_p=1k$) (Acharya et al., 2022).
• Label and class prior ablations demonstrate robustness; small deviations have only minor effects on accuracy (Wang et al., 2024), and stable performance persists in the low-supervision regime (Acharya et al., 2022, Acharya et al., 2024).
• Empirical embedding visualizations (e.g., UMAP, t-SNE) confirm tighter clustering of positives and greater separation from negatives in PUCL-trained models (Wang et al., 2024, Hossain et al., 1 Dec 2025).

7. Practical Considerations, Extensions, and Limitations

• PUCL is effective when a significant proportion of negatives are contaminated by unlabeled positives or under class imbalance.
• The main requirement is an estimate of positive class prior; performance is robust to moderate errors in estimation.
• Minimal architectural changes are needed: PUCL can be added as a wrapper to existing contrastive pipelines with low computational or sample overhead.
• Extensions such as UCF address highly noisy PU scenarios by including adaptivity (uncertainty, temperature, positive anchors) (Hossain et al., 1 Dec 2025).
• PUCL methods are currently focused on binary PU settings but show promise for multi-modal, multi-class, and cross-domain applications (Hossain et al., 1 Dec 2025). Further theoretical analysis under higher noise and multimodal distributions is an active area of research.

Method	Main Contribution	arXiv id
PUCL (bias correction)	General bias-corrected loss and population-mean estimator for contrastive learning	(Wang et al., 2024)
puCL, puNCE, puPL	Unbiased/variance-reducing PU contrastive objectives, pseudo-label clustering	(Acharya et al., 2024)
UCF	Uncertainty-weighted, adaptive, stabilized PU contrastive learning	(Hossain et al., 1 Dec 2025)
puNCE	Prior-weighted position for unlabeled views, leading to improved representations	(Acharya et al., 2022)