Positive-Only Binary Classification

• Binary classification from positive-only samples is a learning paradigm where only positive examples are observed, requiring risk rewriting to infer missing negatives.
• It utilizes techniques like Pconf, PU, and variational methods to reconstruct risk and adjust for confidence distortions and selection biases.
• Methodologies range from boosting and generative models to contrastive learning, demonstrating robust performance even with class imbalance and shift issues.

Binary classification from positive-only samples denotes learning regimes in which training information is restricted to positive examples, while negative information is absent, latent, unlabeled, confidence-weighted, or only indirectly observed. In one formulation, the learner must “learn a binary classifier from only positive data, without any negative data or unlabeled data” by exploiting positive-confidence scores (Ishida et al., 2017). In another, “binary classification from positive-only samples is a variant of PAC learning in which the learner receives i.i.d. samples from the positive region of an unknown target concept, but is evaluated under the original distribution” (Ben-David et al., 26 Jun 2026). A broader applied literature places the problem in positive-unlabeled, exposure-biased, or positive-labeling-source settings, but the common core is the same asymmetry: positives are observed directly, negatives are not.

1. Observation models and problem formulations

The most restrictive observation model is positive-confidence (Pconf) classification. One observes only positive examples $x\sim p(x\mid y=+1)$, each equipped with a confidence score $r(x)=P(y=+1\mid x)$, and learns a real-valued decision function whose sign predicts $y\in\{+1,-1\}$ (Ishida et al., 2017). This differs from ordinary supervised learning because neither negative examples nor unlabeled examples are available at training time.

A second family is positive-unlabeled (PU) learning. Here the input consists of a labeled positive set $P$ and an unlabeled set $U$, where the unlabeled distribution is a mixture of positive and negative classes, for example $P_u=\alpha P_p+(1-\alpha)P_n$ or $p(x)=\pi p_p(x)+(1-\pi)p_n(x)$ depending on notation (Zamzam et al., 2022, Jiang et al., 2020). The task remains binary classification, but the missingness pattern shifts from “no negatives and no unlabeled data” to “no verified negatives.”

Several extensions refine how positives become observed. In positive-unlabeled-exposure (PUE) classification, the observed indicator is $Z=Y\cdot S$, where $S$ is an exposure or selection variable, so observed positives are jointly filtered by exposure and labeling (Kato et al., 2023). In weak supervision with positive labeling sources only, one has labeling functions $\lambda_j:\mathcal X\to\{0,1\}$ that either emit a positive vote or abstain; they never output a negative label (Zhang et al., 2022). In the positive-only PAC model, the learner receives $r(x)=P(y=+1\mid x)$0 i.i.d. positives from the conditional distribution $r(x)=P(y=+1\mid x)$1 and is evaluated under the original distribution $r(x)=P(y=+1\mid x)$2, with error $r(x)=P(y=+1\mid x)$3 (Ben-David et al., 26 Jun 2026).

A common misconception is that these settings collapse to one-class classification. The Pconf formulation explicitly distinguishes them: one-class classification is aimed at “describing” the positive class by clustering-related methods, but it “does not have the ability to tune hyper-parameters and their aim is not on ‘discriminating’ positive and negative classes” (Ishida et al., 2017). This suggests that positive-only binary classification is best understood as a discriminative problem under asymmetric observation, rather than as novelty detection alone.

2. Risk rewriting and estimators from positive information

The central technical device in positive-only learning is a risk identity that replaces unavailable negative expectations by quantities computable from positive observations. In Pconf classification, the ordinary risk

$r(x)=P(y=+1\mid x)$4

can be rewritten as

$r(x)=P(y=+1\mid x)$5

which yields the empirical estimator

$r(x)=P(y=+1\mid x)$6

This estimator is unbiased when the reported confidences are exact; by contrast, the naive alternative

$r(x)=P(y=+1\mid x)$7

is “biased and inconsistent” (Ishida et al., 2017).

PU learning uses a related decomposition. One standard unbiased PU estimator writes

$r(x)=P(y=+1\mid x)$8

and its empirical forms under exponential, logistic, or cross-entropy surrogates underlie methods such as uPU, nnPU, AdaPU, and iFPU (Zhao et al., 2022, Zavitsanos et al., 14 May 2026). The nnPU correction replaces the raw negative-risk estimate by a non-negative version, typically through a term of the form $r(x)=P(y=+1\mid x)$9, to control overfitting in flexible models (Zavitsanos et al., 14 May 2026).

A distinct route avoids explicit prior-dependent negative-risk reconstruction. Variational PU learning introduces

$y\in\{+1,-1\}$0

with empirical version

$y\in\{+1,-1\}$1

Minimizing $y\in\{+1,-1\}$2 is equivalent to minimizing a KL gap between the positive density and the model-induced density $y\in\{+1,-1\}$3, and “no $y\in\{+1,-1\}$4 or negative density $y\in\{+1,-1\}$5 appears” in the objective (Chen et al., 2019).

These estimators establish the main methodological divide in the literature. Some methods reconstruct the supervised risk exactly or unbiasedly under structural assumptions; others replace that objective by a surrogate criterion that is computable from positive-only observations and still targets the Bayesian classifier.

3. Confidence distortion, selection bias, and identifiability

Positive-only formulations are highly sensitive to how the positive signal is produced. In Pconf classification, the reported confidences may be systematically distorted by annotation bias. “Binary Classification from Positive Data with Skewed Confidence” models this by the power-law transformation

$y\in\{+1,-1\}$6

or equivalently $y\in\{+1,-1\}$7, and in the paper’s notation uses adjusted confidences $y\in\{+1,-1\}$8 (Shinoda et al., 2020). The corresponding adjusted empirical risk is

$y\in\{+1,-1\}$9

Since negative examples are unavailable for validation, the hyperparameter is selected by matching the empirical false-negative rate on positives,

$P$0

to known prior knowledge $P$1 through

$P$2

(Shinoda et al., 2020).

Class-prior estimation is another identifiability bottleneck. The mixture-proportion view of PU learning writes the unlabeled density as $P$3, with $P$4 observed and $P$5 unknown. “Nonparametric semi-supervised learning of class proportions” shows that estimation is generally ill-defined unless one adopts the “max-canonical form”

$P$6

equivalently $P$7 in the absolutely continuous case, and proposes the AlphaMax procedure based on an elbow in a constrained likelihood curve $P$8 (Jain et al., 2016). This suggests that identifiability in positive-only learning is often a modeling choice rather than a direct consequence of the data.

Distribution shift further complicates the problem. “Learning from Positive and Unlabeled Data with Arbitrary Positive Shift” allows $P$9 and $U$0 to be arbitrarily different while assuming only

$U$1

and develops statistically consistent estimators based on weighted unlabeled-unlabeled learning and a recursive risk estimator with absolute-value correction (Hammoudeh et al., 2020). In PUE classification, the issue is not only class-prior ambiguity but also selection bias in the observed positives. Under strong ignorability,

$U$2

one has

$U$3

with $U$4, and ADPUE constructs an iterative empirical risk that “automatically” debiases without explicit propensity estimation (Kato et al., 2023).

4. Methodological families beyond classical ERM

A large methodological literature extends positive-only binary classification beyond direct risk minimization. Boosting-based PU learning is represented by AdaPU, which minimizes a PU analogue of the empirical exponential loss

$U$5

Its derived weighted PN dataset includes pseudo-negative examples with negative weight, and the weak learner selection requires both $U$6 and $U$7 to prevent dangerous exploitation of negative weights (Zhao et al., 2022).

Representation-learning and generative approaches replace handcrafted risk decompositions by learned structure. Observer-GAN introduces a generator $U$8, discriminator $U$9, and observer $P_u=\alpha P_p+(1-\alpha)P_n$0; $P_u=\alpha P_p+(1-\alpha)P_n$1 enforces that $P_u=\alpha P_p+(1-\alpha)P_n$2 lie within the support of the unlabeled data, while $P_u=\alpha P_p+(1-\alpha)P_n$3 prevents $P_u=\alpha P_p+(1-\alpha)P_n$4 from falling into the positive distribution and is returned as the final classifier (Zamzam et al., 2022). Positive Unlabeled Contrastive Learning defines a PU-specific InfoNCE-style loss $P_u=\alpha P_p+(1-\alpha)P_n$5 in which unlabeled anchors are treated as a $P_u=\alpha P_p+(1-\alpha)P_n$6-weighted mixture of positive and negative contributions, and then trains a linear head with a cost-sensitive PU risk such as nnPU (Acharya et al., 2022). Tensor-network PU learning constructs positive and negative projectors $P_u=\alpha P_p+(1-\alpha)P_n$7 and $P_u=\alpha P_p+(1-\alpha)P_n$8 in a feature-space tensor embedding and predicts $P_u=\alpha P_p+(1-\alpha)P_n$9 when $p(x)=\pi p_p(x)+(1-\pi)p_n(x)$0; the same model is also generative and can sample new positive and negative instances (Žunkovič, 2022).

Other methods use ranking, graph structure, or repeated pseudo-labeling. “Beyond Myopia” trains on balanced positive-unlabeled minibatches, records full score trajectories $p(x)=\pi p_p(x)+(1-\pi)p_n(x)$1, interprets them as a temporal point process, and computes a robust trend score

$p(x)=\pi p_p(x)+(1-\pi)p_n(x)$2

followed by splitting via Fisher’s natural break (Wang et al., 2023). ProbTagging assigns each unlabeled sample a Bernoulli pseudo-label based on its $p(x)=\pi p_p(x)+(1-\pi)p_n(x)$3-nearest positive-neighbor credibility $p(x)=\pi p_p(x)+(1-\pi)p_n(x)$4, trains multiple PN classifiers, and averages them into an ensemble $p(x)=\pi p_p(x)+(1-\pi)p_n(x)$5 (Jiang et al., 2020). The flow-based 2-HNC method constructs a similarity graph, uses parametric minimum cuts to obtain nested partitions, ranks unlabeled samples by how early they join the negative side, augments the positive set with likely negatives, and selects the final partition whose positive fraction is closest to a prior estimate $p(x)=\pi p_p(x)+(1-\pi)p_n(x)$6 (Hochbaum et al., 13 May 2025). In weak supervision, WEAPO aggregates positive-only labeling functions by a convex model $p(x)=\pi p_p(x)+(1-\pi)p_n(x)$7, partial-order constraints over vote patterns, and a class-prior constraint $p(x)=\pi p_p(x)+(1-\pi)p_n(x)$8 (Zhang et al., 2022).

Taken together, these methods show that positive-only binary classification is no longer confined to one estimator family. The literature spans ERM, variational inference, boosting, adversarial generation, contrastive pretraining, graph cuts, tensor networks, and weak-supervision label models.

5. Statistical and combinatorial theory

Theoretical analysis in positive-only learning splits into estimator-level guarantees and model-class characterizations. For Pconf ERM, under boundedness and Lipschitz assumptions on the loss and a lower bound $p(x)=\pi p_p(x)+(1-\pi)p_n(x)$9, one has

$Z=Y\cdot S$0

and for linear models in an RKHS,

$Z=Y\cdot S$1

so minimizing the Pconf empirical risk is statistically consistent (Ishida et al., 2017). The skewed-confidence extension preserves unbiasedness when the correction exponent matches the true skew parameter, and its tuning rule uses only positives plus a scalar false-negative prior (Shinoda et al., 2020).

The most general structural theory is currently given for the positive-only PAC model. “Surprises in Proper Positive-Only Learning” proves the characterization

$Z=Y\cdot S$2

where, for every finite $Z=Y\cdot S$3, the closure

$Z=Y\cdot S$4

captures the points forced to be positive by consistency with $Z=Y\cdot S$5 (Ben-David et al., 26 Jun 2026). The paper also introduces exact exterior separation, uniform exterior separability (UES), and distributional exterior separability (DES), with

$Z=Y\cdot S$6

and shows that when $Z=Y\cdot S$7, $Z=Y\cdot S$8 (Ben-David et al., 26 Jun 2026).

This characterization yields several separations absent from ordinary PAC learning. Proper and improper learning are separated; randomized and deterministic proper learning are separated; there are classes for which no deterministic proper ERM rule succeeds; and finite VC dimension does not suffice even for non-uniform learning (Ben-David et al., 26 Jun 2026). A common oversimplification is therefore incorrect: finite VC dimension alone is not the full story for positive-only learning.

Complementary asymptotic theory also appears in broader PU settings. The trend score in holistic PUL concentrates around $Z=Y\cdot S$9 and satisfies a high-probability bound of order

$S$0

so $S$1 in probability (Wang et al., 2023). Variational PU learning derives its objective from a KL divergence and states that the minimizer of $S$2 on $S$3-valued $S$4 is, up to a positive scale, the true posterior $S$5 (Chen et al., 2019). These results indicate that positive-only theory now ranges from finite-sample estimator analysis to abstract learnability characterizations.

6. Empirical behavior, applications, and recurrent issues

Empirical work consistently shows that positive-only methods can approach or match supervised baselines when their assumptions are approximately satisfied. On synthetic 2D Gaussians, Pconf classification “achieves accuracy within 1–2% of fully supervised,” and on Fashion-MNIST and CIFAR-10 it “beat the naive weighted-confidence baseline on 70–80% of tasks,” with several cases matching a fully supervised network (Ishida et al., 2017). When confidence is skewed, original Pconf can deteriorate sharply, but adjusted Pconf “recovers performance nearly to that of fully supervised learning” on synthetic problems and “restores >90 % accuracy on Fashion-MNIST” while substantially outperforming original Pconf on CIFAR-10 (Shinoda et al., 2020).

Several papers emphasize robustness under severe imbalance or bias. Holistic PUL reports improvements of “up to $S$6 in key metrics,” including Credit-Card-Fraud Recall from $S$7 to $S$8, $S$9 from $\lambda_j:\mathcal X\to\{0,1\}$0 to $\lambda_j:\mathcal X\to\{0,1\}$1, and AUC from $\lambda_j:\mathcal X\to\{0,1\}$2 to $\lambda_j:\mathcal X\to\{0,1\}$3 (Wang et al., 2023). ProbTagging can “increase the AUC by up to 10%” on industrial and artificial PU data sets and degrades more slowly as the fraction of observed positives decreases (Jiang et al., 2020). Focused PU learning from imbalanced data reports that XGB+iFPU attains $\lambda_j:\mathcal X\to\{0,1\}$4 R-precision on financial misstatement detection, compared with $\lambda_j:\mathcal X\to\{0,1\}$5 for Calibrated-iFPU and $\lambda_j:\mathcal X\to\{0,1\}$6 for PUHRF (Zavitsanos et al., 14 May 2026).

Real-world applications span multiple modalities. Adjusted Pconf was applied to drivers’ drowsiness prediction using “7 heart-rate-variability features” and expert-rated “sleepiness” scores for positive samples only; raw Pconf predicted “all-Alert” and yielded zero F-measure, whereas adjusted Pconf obtained F-measures of $\lambda_j:\mathcal X\to\{0,1\}$7–$\lambda_j:\mathcal X\to\{0,1\}$8 (Shinoda et al., 2020). Observer-GAN reports observer accuracies of $\lambda_j:\mathcal X\to\{0,1\}$9 on CIFAR-10 and $r(x)=P(y=+1\mid x)$00 on AFHQ (Zamzam et al., 2022). Tensor-network PU learning reports average one-vs-one MNIST test accuracy $r(x)=P(y=+1\mid x)$01 for $r(x)=P(y=+1\mid x)$02, and an average $r(x)=P(y=+1\mid x)$03 of $r(x)=P(y=+1\mid x)$04 over $r(x)=P(y=+1\mid x)$05 categorical or mixed-data tasks (Žunkovič, 2022). The 2-HNC method achieves the top accuracy on $r(x)=P(y=+1\mid x)$06 real PU benchmarks and the top balanced accuracy on $r(x)=P(y=+1\mid x)$07 (Hochbaum et al., 13 May 2025).

Recurrent limitations are equally consistent across the literature. Pconf requires reliable estimates of $r(x)=P(y=+1\mid x)$08; when $r(x)=P(y=+1\mid x)$09, the weight $r(x)=P(y=+1\mid x)$10 can inflate and destabilize training (Ishida et al., 2017). Many PU methods assume a known or estimable class prior $r(x)=P(y=+1\mid x)$11, and several papers focus explicitly on prior estimation or robustness to misspecification (Jain et al., 2016, Zavitsanos et al., 14 May 2026). Shift assumptions such as SCAR, representative positives, or exposure ignorability are often essential and often violated, motivating later work on arbitrary positive shift, SAR, and exposure-biased observation (Hammoudeh et al., 2020, Kato et al., 2023, Zavitsanos et al., 14 May 2026). The overall empirical record therefore supports a narrow conclusion: binary classification from positive-only samples is feasible and often competitive, but only through estimators and models that explicitly encode how positive evidence is generated, distorted, or selected.