Proper Positive-Only Learning

• Proper positive-only learning is defined as learning solely from positive samples while requiring the hypothesis to belong to the original concept class, and it uses uniform exterior separability to manage false positives.
• The formal framework utilizes a proper learner model, version spaces, and closure properties alongside a finite VC dimension condition to guarantee controlled extrapolation beyond observed positives.
• Extensions include applications in symbolic DFA learning, interpretable temporal reasoning, federated and active learning scenarios, and reinforcement learning with one-sided feedback.

Proper positive-only learning is the problem of learning from observations that come only from the positive side of a target while still requiring the learner to output a hypothesis in the original target class. In its narrowest formalization, the learner receives i.i.d. samples from the positive region of an unknown concept $c\in\mathcal C$, but is evaluated under the original distribution on the full instance space, so both false negatives and false positives matter; the distinctive issue is therefore not merely recovering positive support, but controlling extrapolation outside it while remaining proper (Ben-David et al., 26 Jun 2026). In a broader literature, the same theme appears in positive-and-unlabeled learning, symbolic inference of automata and temporal formulas, federated multi-class learning with only local positives, one-sided feedback, and positive-only policy optimization, all of which ask when a proper predictor can still be learned despite one-sided supervision (Bekker et al., 2018, Bordais et al., 11 Nov 2025, Roy et al., 2022, Yu et al., 2020, Jiang et al., 2020, Xu et al., 7 May 2026).

1. Formal model and exact characterization

The recent formal treatment of positive-only PAC learning fixes an instance space $X$, a concept class $\mathcal C\subseteq 2^X$, an unknown target concept $c\in\mathcal C$, and an unknown distribution $\mathcal D$ on $X$. The learner observes a multiset

$S_+ \sim \mathcal D_+^n,$

where $\mathcal D_+$ is the conditional distribution on the positive region $c$, and outputs a hypothesis $h\subseteq X$. Error is measured under the full distribution by

$X$0

A proper learner is a map

$X$1

so the output must lie in the same class as the target (Ben-David et al., 26 Jun 2026).

Two induced set-valued objects organize the theory. For a finite realizable set $X$2, the version space is

$X$3

and the closure is

$X$4

Because the true target belongs to $X$5, one always has $X$6. The closure therefore introduces no false positives, but it need not itself belong to $X$7, which is exactly where properness becomes nontrivial (Ben-David et al., 26 Jun 2026).

The central combinatorial condition is uniform exterior separability. The class $X$8 satisfies uniform exterior separability if for every $X$9 there exists $\mathcal C\subseteq 2^X$0 such that for every finite nonempty realizable $\mathcal C\subseteq 2^X$1, there exist hypotheses $\mathcal C\subseteq 2^X$2 with

$\mathcal C\subseteq 2^X$3

This says that although the exact closure may be improper, a bounded randomized list of proper hypotheses can approximate it while spreading false positives thinly over the exterior (Ben-David et al., 26 Jun 2026).

The resulting characterization is exact: $\mathcal C\subseteq 2^X$4 Finite VC dimension controls generalization from observed positives, while uniform exterior separability controls false positives outside the closure. This sharply departs from ordinary realizable PAC learning, where finite VC dimension alone suffices (Ben-David et al., 26 Jun 2026).

2. Consequences: separations, randomness, and non-ERM behavior

The same analysis shows that improper positive-only learning is governed by a different object. Let

$\mathcal C\subseteq 2^X$5

Then

$\mathcal C\subseteq 2^X$6

Proper and improper learning therefore separate. A minimal example is

$\mathcal C\subseteq 2^X$7

for which $\mathcal C\subseteq 2^X$8 has VC dimension $\mathcal C\subseteq 2^X$9, so improper learnability holds, but proper learnability fails because the closure of $c\in\mathcal C$0 is $c\in\mathcal C$1 (Ben-David et al., 26 Jun 2026).

Randomized and deterministic proper learning also separate. The class

$c\in\mathcal C$2

has VC dimension $c\in\mathcal C$3 and satisfies uniform exterior separability, hence is properly positive-only learnable, but no deterministic proper learner exists. The obstruction is singleton closure: $c\in\mathcal C$4 which every deterministic proper positive-only learner must satisfy, and which this class violates (Ben-David et al., 26 Jun 2026).

The model also breaks the usual universality of ERM. For

$c\in\mathcal C$5

the class has VC dimension $c\in\mathcal C$6 and is properly positive-only learnable, but no deterministic proper ERM learner succeeds. The reason is that consistency with observed positives does not determine how to allocate unavoidable false positives across the unseen exterior, and ERM has no mechanism for the anti-concentration strategy that the positive-only model requires (Ben-David et al., 26 Jun 2026).

Several further distinctions follow. Exact exterior separation implies uniform exterior separability, which implies distributional exterior separability, which implies finite exterior separability: $c\in\mathcal C$7 Stable proper positive-only learnability collapses back to the strongest condition: $c\in\mathcal C$8 Finite VC dimension is not enough even for non-uniform proper positive-only learnability, and finite VC dimension does not imply consistency. At the quantitative level, if $c\in\mathcal C$9 and $\mathcal D$0, randomized proper positive-only learning can be achieved with

$\mathcal D$1

random bits and

$\mathcal D$2

positive samples (Ben-David et al., 26 Jun 2026).

3. Positive-only versus positive-and-unlabeled learning

Much of the earlier literature approaches one-sided supervision through positive-and-unlabeled data rather than positive-only samples. In the SAR formulation of PU learning, each example is represented as $\mathcal D$3, where $\mathcal D$4 is the latent class label and $\mathcal D$5 is the observed label indicator with $\mathcal D$6. Under SCAR,

$\mathcal D$7

so

$\mathcal D$8

Under SAR, labeling depends on propensity attributes $\mathcal D$9: $X$0 where

$X$1

The SAR-EM algorithm jointly learns $X$2 and $X$3 via EM, with unlabeled posterior

$X$4

This is proper in the sense that the target remains the true posterior $X$5, but it is not positive-only in the strict sample-only sense because unlabeled data and an explicit labeling model are essential (Bekker et al., 2018).

The same distinction appears in later PU formulations. Selection biases in positive and unlabeled data are common, and an empirical-risk-based method that incorporates the labeling mechanism improves trained classifiers even when the mechanism is unknown; the explicit point is that positive and unlabeled learning becomes brittle when selection bias is ignored (Bekker et al., 2018). Variational PU learning replaces class-prior-weighted supervised-risk decompositions by the criterion

$X$6

with

$X$7

Under SCAR, an anchor condition, and model realizability, the learned posterior satisfies

$X$8

so the method targets the Bayes posterior directly without explicit class-prior estimation (Chen et al., 2019).

A different route is to extract negative information only collectively from unlabeled examples. The collective PU objective uses

$X$9

so unlabeled data contribute through a batch-level prevalence constraint

$S_+ \sim \mathcal D_+^n,$0

This avoids per-instance negative pseudo-labels, but it still depends essentially on unlabeled data and a positive-prevalence parameter (Xie et al., 2020).

4. Proper positive-only learning for symbolic and interpretable hypothesis classes

The proper positive-only paradigm is especially natural for symbolic classes whose outputs are already interpretable objects. In passive DFA learning from positive examples, the input is an alphabet $S_+ \sim \mathcal D_+^n,$1, a finite positive sample set $S_+ \sim \mathcal D_+^n,$2, and a state bound $S_+ \sim \mathcal D_+^n,$3. The proper hypothesis space is

$S_+ \sim \mathcal D_+^n,$4

The learning objective is to find $S_+ \sim \mathcal D_+^n,$5 that is minimal under language inclusion. A finite surrogate exists: if $S_+ \sim \mathcal D_+^n,$6, then minimizing

$S_+ \sim \mathcal D_+^n,$7

is sufficient for language minimality. More precisely, Proposition 1 states that a DFA minimal w.r.t.

$S_+ \sim \mathcal D_+^n,$8

is also minimal w.r.t. strict language inclusion. The associated decision problem—whether there exists $S_+ \sim \mathcal D_+^n,$9 with

$\mathcal D_+$0

—is NP-complete, even over a binary alphabet. The paper also gives an ILP formulation and a heuristic preprocessing algorithm; the exact solver underperforms the symbolic baseline in practice, but the heuristic can improve starting points for symbolic descent (Bordais et al., 11 Nov 2025).

A closely related line studies interpretable temporal explanations from positive traces only. For DFAs, given positive words $\mathcal D_+$1 and size bound $\mathcal D_+$2, the goal is to learn an $\mathcal D_+$3-description $\mathcal D_+$4 with $\mathcal D_+$5 and no strictly smaller-language $\mathcal D_+$6-description. For $\mathcal D_+$7, the goal is an $\mathcal D_+$8-description $\mathcal D_+$9 such that $c$0 is satisfied by every positive word and no strictly stronger $c$1-description exists. The symbolic DFA algorithm uses a SAT encoding

$c$2

iteratively descending in language inclusion until no smaller consistent DFA exists. For $c$3, semi-symbolic and counterexample-guided algorithms combine bounded-size syntax-DAG search with implication checks and generated negatives. The guarantees are proper: Theorem 1 returns a DFA $c$4 such that for every DFA $c$5 that is an $c$6-description,

$c$7

and Theorem 3 returns an $c$8 formula $c$9 such that for every bounded-size positive-consistent $h\subseteq X$0,

$h\subseteq X$1

Here properness is literal class preservation: the learner outputs a DFA or an $h\subseteq X$2 formula, not an improper proxy (Roy et al., 2022).

5. Distributed, active, and sequential variants

Proper positive-only learning also appears in settings where the missing negative information is supplied by geometry, exploration, or active querying rather than by explicit negative labels. In federated multi-class learning with only positive labels, client $h\subseteq X$3 holds

$h\subseteq X$4

can access only its own class embedding $h\subseteq X$5, and cannot access embeddings $h\subseteq X$6 for $h\subseteq X$7. Naively optimizing only the positive term leads to embedding collapse. FedAwS fixes this by a server-side spreadout regularizer

$h\subseteq X$8

combined with local positive updates. Under equal class sizes and $h\subseteq X$9, the resulting objective equals empirical risk under

$X$00

so a proper multi-class classifier can be learned despite strictly local positive-only supervision (Yu et al., 2020).

In one-sided feedback for generalized linear models, the learner observes labels only when it takes the positive action. The proper class is still

$X$01

Naive greedy ERM on observed labels can fail permanently because rejected regions are never labeled. The adaptive solution uses optimistic querying: accept $X$02 whenever

$X$03

The resulting theorem bounds cumulative one-sided loss by

$X$04

which implies vanishing average one-sided loss under the stated GLM assumptions (Jiang et al., 2020).

Active PU learning adds adaptive querying to a one-sided reveal model. A queried example reveals a positive label only if it is positive and an independent coin with bias $X$05 succeeds; otherwise the learner receives $X$06. The output is again proper, $X$07. When the positive class prior $X$08 is known, the paper gives a disagreement-based algorithm whose label complexity is

$X$09

When $X$10 is unknown, there is an additional $X$11-dependent overhead for estimating $X$12, but the final guarantee still returns $X$13 with

$X$14

with probability at least $X$15 (Mansouri et al., 2 Feb 2026).

Positive-only optimization has also been carried into reinforcement learning with verifiable rewards. In POPO, rollouts are partitioned into

$X$16

and the actor objective uses only $X$17: $X$18 with

$X$19

For any incorrect response $X$20,

$X$21

so minimizing the positive-only objective still pushes negative logits downward implicitly. Empirically, on Qwen-Math-7B, POPO achieves $X$22 on AIME 2025 versus GRPO’s $X$23 (Xu et al., 7 May 2026).

6. Limits, boundaries, and open directions

The broad literature repeatedly shows that “positive-only” is not a single statistical regime. A tensor-network PU method can learn from positive and unlabeled data and generate both positive and negative samples, but it is “not a canonical proper PU learner” in the classical statistical sense: it does not derive an unbiased PU risk estimator, estimate the class prior, or prove identifiability of the negative distribution from PU observations alone (Žunkovič, 2022). This marks an important boundary between practical one-stage positive/unlabeled methods and properness as posterior recovery or risk-consistent estimation.

The same distinction becomes sharper under distribution shift. Positive and Imperfect Unlabeled learning assumes access to positive samples from the true feature distribution and to an imperfect unlabeled distribution $X$24 satisfying generalized smoothness

$X$25

This yields strong positive-only corollaries when the learner knows a reference distribution $X$26, but the resulting learners are generally improper: the sample-efficient method outputs an intersection of $X$27 hypotheses, and the efficient method outputs a degree-$X$28 polynomial threshold function. The paper explicitly argues that proper PIU learning is impossible in general, and it also proves that without a reference or unlabeled distribution, positive-only learning is impossible for many natural classes even improperly (Lee et al., 14 Apr 2025).

A complementary general lesson comes from proper PAC theory with access to the full unlabeled marginal. In the distribution-fixed PAC model, there always exists a randomized proper learner governed by distributional regularization, and this learner can be properized by the $X$29-metric projection

$X$30

At the same time, sample complexity can shrink by only a logarithmic factor relative to classic PAC, and proper learnability in the ordinary realizable PAC model can be logically undecidable, non-monotone, and non-local (Asilis et al., 14 Feb 2025). This suggests that side information may restore properness more readily than it changes worst-case information-theoretic complexity.

In the narrow passive positive-only model, the exact characterization leaves several questions open: the exact random-bit complexity of proper positive-only learning, analogous characterizations for non-uniform learnability and consistency, and optimal statistical rates together with computationally efficient algorithms (Ben-David et al., 26 Jun 2026). Taken together, these results indicate that proper positive-only learning is now well understood at the level of information-theoretic structure in its core binary setting, but still fragmented across positive-and-unlabeled, active, sequential, and structured symbolic variants, where properness, identifiability, and tractability do not generally coincide.