Positive-Unlabeled GAN (PUGAN)

• PUGAN is a GAN framework that reformulates adversarial learning as a positive-unlabeled classification problem, applying PU risk corrections to model both positive and counterexample distributions.
• The framework leverages techniques such as loss reweighting, multi-generator architectures, and observer networks to stabilize training and mitigate overfitting when positive labels are scarce.
• Empirical evaluations show that PUGAN variants achieve lower FID scores and improved classification accuracy in tasks like image synthesis, recommendation, and collaborative filtering compared to standard GANs.

A Positive-Unlabeled Generative Adversarial Network (PUGAN) is a framework that adapts adversarial generative modeling to problems where observed data is drawn from a positive class and an unlabeled pool, but negative labels are either absent or not directly accessible. PUGAN aims to overcome overfitting in settings with scarce positive labels, to robustly generate samples from both positive and negative (or counterexample) distributions, and to stabilize adversarial training by applying positive-unlabeled (PU) risk corrections to the discriminator objective. Typical applications include image classification, collaborative filtering, and low-label learning scenarios, where the challenge is to accurately model the negative class using only positive and unlabeled data.

1. Formalization and Motivation

Classical GAN frameworks apply a binary discrimination, treating real data as positive and generated (fake) data as negative throughout training. This approach fails to reflect the dynamic nature of sample quality during generator improvement, and ignores the mixture properties of unlabeled pools. PUGAN frameworks reinterpret the adversarial task as a positive-unlabeled classification problem, recasting generated data as unlabeled samples. This pool may include "good" fakes—indistinguishable from real positives—and "bad" fakes, which remain easily discriminable. Letting $p_{\text{data}}(x)$ denote the real data density (positive class), $p_g(x)$ the generator density, and splitting $$p_g(x) = \pi\,p_{\text{gr}}(x) + (1-\pi)p_{\text{gf}}(x)$$ into "good" and "bad" fakes weighted by $\pi$, the discriminator's risk can be written using only observable real and generated batches:

$$R(D) = \pi \, \mathbb{E}_{p_{\text{data}}}[\ell(D, +1)] + \mathbb{E}_{p_g}[\ell(D, -1)] - \pi\,\mathbb{E}_{p_{\text{data}}}[\ell(D, -1)].$$

This approach generalizes to multi-modal, conditional, and collaborative filtering settings by appropriate choices of model architecture, risk estimators, and batch construction (2002.01136, Zhou et al., 2020, Yu et al., 2020, Hou et al., 2017).

2. PUGAN Variants and Loss Functions

Several distinct PUGAN constructions have been proposed:

• Vanilla PU-GAN ("On Positive-Unlabeled Classification in GAN" (2002.01136)): Replaces binary discrimination with PU risk and reweights the loss using a class prior $\pi$ to balance between $R_p^+$ (positive risk), $R_p^-$ (negative surrogate risk on real data), and $R_u^-$ (unlabeled or fake surrogate risk). The discriminator maximizes:

$$V_D(D) = \pi\,\mathbb{E}_{\text{data}}[f_1(D(x))] + \max\{0, \mathbb{E}_{\text{gen}}[f_2(D(G(z)))] - \pi\,\mathbb{E}_{\text{data}}[f_2(D(x))] \}$$

Generator optimization follows the standard adversarial form.

• Multi-Generator/Discriminator Architectures ("Generative Adversarial Positive-Unlabelled Learning" (Hou et al., 2017)): Introduces positive and negative generators $(G_p, G_n)$ and three discriminators $(D_p, D_n, D_u)$. Negative generator loss employs a "reverse-GAN" objective, pushing $G_n$ away from positive data. At equilibrium, $G_p$ and $G_n$ recover the true positive and negative distributions, respectively.
• Observer-conditional PU-GAN ("Learning From Positive and Unlabeled Data Using Observer-GAN" (Zamzam et al., 2022)): Introduces an observer network to actively prevent the generator from drifting toward the positive manifold, addressing mode collapse in stronger baselines. The generator is penalized if its output is classified as positive by the observer, aligning with the negative support.
• Noise-Corrected Conditional PU-GAN ("Classify and Generate Reciprocally..." (Yu et al., 2020)): Couples a multi-class PU classifier, a generator, and a discriminator, correcting for label noise via a learned confusion matrix, and constructing an auxiliary loss to enforce class-label invariance in the generator's outputs.

For all variants, the generator is typically optimized to fool the discriminator on the unlabeled batch or additionally on conditioned class labels.

3. Theoretical Guarantees and Equilibrium Analysis

PUGAN frameworks provide formal analyses of equilibrium properties:

• For vanilla PUGAN with unlimited function capacity and log-loss, the optimal discriminator for fixed $G$ is

$$D^*(x) = \frac{p_{\text{data}}(x)}{p_{\text{data}}(x) + ((1-\pi)/\pi)p_{\text{gf}}(x)}$$

(2002.01136). At equilibrium, the generator converges to the real data distribution, and the discriminator output becomes constant.

• In multi-generator/discriminator configurations (Hou et al., 2017), the system of objectives is minimized precisely when $G_p$ matches the empirical positive distribution and $G_n$ recovers the empirical negative.
• For noise-aware conditional PU-GANs, under assumptions of disjoint support and correct prior estimation, the generator's conditional distributions converge to the clean data conditionals, up to permutation for class-prior ties (Yu et al., 2020).
• In recommendation contexts, theoretical bounds show that the PU-risk estimator can achieve lower estimation error than standard PN, provided the ratio and structure of positive/unlabeled sampling meet specified criteria (Zhou et al., 2020).

4. Canonical Training Algorithms

PUGAN training proceeds via standard GAN-style alternating optimization, but with non-standard batch construction and risk estimation. Below, the core steps for vanilla PUGAN (2002.01136):

• For each discriminator update:
  1. Sample minibatches of real data and latent variables.
  2. Compute $R_p^+$, $R_p^-$ (on data), $R_u^-$ (on generated).
  3. Calculate the corrected negative PU risk $R_\text{pu}^- = \max\{0, R_u^- - \pi R_p^-\}$.
  4. Update discriminator with loss: $\pi R_p^+ + R_\text{pu}^-$.

• Generator is updated to minimize $E_{z}[f_2(D(G(z)))]$.

The class prior $\pi$ can be scheduled, often beginning small and increasing over training to mirror the improving quality of generated examples.

Complex architectures (multi-head, observer, or conditional) require additional components to be trained per update step, including confusion-matrix regularization, observer resets, or auxiliary classifier retraining (Zamzam et al., 2022, Yu et al., 2020).

5. Empirical Results and Benchmarks

Empirical evaluation of PUGANs has demonstrated consistent improvements over baseline GAN variants and PU-learning methods, particularly under label scarcity or difficult training regimes.

• On image synthesis: PUGAN variants achieve lower Fréchet Inception Distance (FID) scores across MNIST, Fashion-MNIST, CIFAR-10, and high-resolution datasets compared to SGAN, LSGAN, WGAN-GP, and relativistic GANs; e.g., CIFAR-10 FID drops from 43.4 (SGAN) to 31.0 (PUSGAN) (2002.01136).
• On robustness: PUGAN remains stable under larger learning rates, removal of batch normalization, and increased generated batch sizes, conditions under which standard GANs exhibit mode collapse.
• In binary and multi-class classification, PUGAN (GenPU, D-GAN, Observer-GAN) attains test accuracy or F1 scores superior to non-negative and unbiased PU-learning, particularly when the number of labeled positives is extremely limited (Hou et al., 2017, Chiaroni et al., 2019, Zamzam et al., 2022).
• For recommendation: On ML-100k, PUGAN achieves P@5 of 0.3901 compared to 0.3776 (BPR) and 0.3697 (PU-GMF), outperforming the best classical and adversarial matrix factorization baselines (Zhou et al., 2020).

Ablation studies confirm that careful scheduling of the class prior, proper normalization in the discriminator, and handling of batch effects are critical for performance and stability.

6. Connections, Extensions, and Limitations

PUGAN frameworks unify adversarial learning and PU classification, extending to several domains:

• Positive-unlabeled matrix factorization ("GAN-based Recommendation with Positive-Unlabeled Sampling" (Zhou et al., 2020)) and collaborative filtering, by modeling the user-item interaction matrix under PU risk.
• Conditional generation under label noise (multi-class PUGANs leveraging classifier-noise-invariant modules (Yu et al., 2020)).
• Weakly supervised and low-label domain adaptation, including settings with out-of-distribution labeled data.

Limitations include:

• Reliance on accurate class prior $\pi$ estimation or scheduling in certain variants (2002.01136).
• Architectural complexity in multi-generator/discriminator or observer configurations.
• Classical PUGANs are primarily analyzed for binary PU tasks; multi-class and structured/unstructured extension remains an active research area.

Improvements are proposed via integration of state-of-the-art GAN variants (Wasserstein, spectral normalization, self-attention), more robust risk estimators, and application to text/sequential domains.

7. Summary Table: PUGAN Variants

Framework	Key Discriminative Structure	Main Theoretical Property
Vanilla PUGAN (2002.01136)	Single D, PU reweighted loss	PU-optimal D, global convergence
GenPU (Hou et al., 2017)	Two G, three D, anti-GAN loss	Joint P/N recovery, JSD optimal
Observer-GAN (Zamzam et al., 2022)	D + observer O, observer penalty	Prevents drift to positives
CNI-CGAN (Yu et al., 2020)	Conditional G/D, confusion matrix	Permutation-invariant optima
D-GAN (Chiaroni et al., 2019)	Standard G/D, PU-biased risk	Counterexample generation, no prior

Further details, including algorithms and empirical results, are available in the cited original research.