Positive Distribution Shift (PDS)

• Positive Distribution Shift (PDS) is a family of beneficial mechanisms where controlled changes in data distributions reveal invariant structures and improve learning outcomes.
• It encompasses methods like invariant prediction, support expansion, mixture design, and tractable learning, each targeting different aspects of training and evaluation.
• PDS applications enhance model performance and sample efficiency by strategically exploiting shifts that extend support or simplify the optimization landscape.

Searching arXiv for the cited papers and closely related work on Positive Distribution Shift. Positive Distribution Shift (PDS) denotes settings in which a distributional change is not treated solely as a source of degradation, but as a condition that can improve learning, test performance, identifiability, or tractability. The recent literature uses the term in several distinct ways. In one line of work, larger shift across training environments helps empirical risk minimization (ERM) recover an invariant predictor (Zheng et al., 18 Jan 2026). In another, PDS refers to support-expansion cases where test support extends beyond training support and standard importance weighting fails unless the out-of-training region is handled explicitly (Fang et al., 2023). Other papers operationalize favorable shift through a negative shift gap, $$\mathrm{Acc}(\mathrm{OOD}) > \mathrm{Acc}(\mathrm{ID})$$, or through deliberate mismatch between training and test mixtures that lowers test loss (Gardner et al., 2023, Medvedev et al., 29 Oct 2025). A more general theoretical formulation treats PDS as learning a target function with respect to a target distribution $D(x)$ while training on a different distribution $D'(x)$ chosen so that learning becomes easier (Medvedev et al., 9 Feb 2026).

1. Conceptual scope and competing definitions

The literature does not impose a single definition of PDS. A compact way to organize the main usages is as follows.

Setting	Operational meaning	Representative paper
Invariant prediction	Larger shift across training domains helps recover a stable predictor	(Zheng et al., 18 Jan 2026)
Support-expansion DS	Test support extends beyond training support, i.e. cases (iii) and (iv)	(Fang et al., 2023)
Benchmark evaluation	Negative shift gap, $$\mathrm{Acc}(\mathrm{OOD}) > \mathrm{Acc}(\mathrm{ID})$$	(Gardner et al., 2023)
Mixture design	Training on $D_q$ with $q \neq p$ improves test loss	(Medvedev et al., 29 Oct 2025)
Tractable learning	Choosing $D'(x)$ makes target learning easier	(Medvedev et al., 9 Feb 2026)
PU learning	The positive class shifts arbitrarily while the negative class stays fixed	(Hammoudeh et al., 2020)

At the most abstract level, these formulations all replace the standard presumption that shift is harmful with a more conditional claim: some shifts expose stable structure, some broaden useful coverage, some reweight task components in a favorable way, and some alter the optimization landscape so that standard learning algorithms succeed where they otherwise would not. This suggests that PDS is better understood as a family of beneficial-shift mechanisms than as a single mathematical object.

A recurrent distinction is whether the beneficial shift occurs in the training distribution, in the test distribution, or in the relation between the two. The invariant-prediction and tractability papers focus on training-side design (Zheng et al., 18 Jan 2026, Medvedev et al., 9 Feb 2026). The support-expansion and benchmark papers focus on test-side deployment or evaluation (Fang et al., 2023, Gardner et al., 2023). Mixture-mismatch results sit between these perspectives by optimizing a training mixture for a fixed test mixture (Medvedev et al., 29 Oct 2025).

2. Positive shift within training domains and invariant prediction

A central formulation of PDS treats diversity across training environments as a helpful signal for invariant learning rather than as nuisance variation. The basic notion of distribution shift across environments is

$$P^e \neq P^{e'}, \quad \forall e \neq e' \in \mathcal E,$$

and the magnitude of this shift is quantified with KL divergence,

$KL(P;Q) = \int \log \frac{p}{q}\, p\, d\mu.$

In the theoretical analysis, pairwise shift across the family of training-domain distributions is bounded by quantities such as

$\sup_{P,P' \in \mathcal P_E} KL(P;P') \le \alpha$

or, in the binary noisy setting,

$D(x)$0

The key claim is that a larger shift parameter $D(x)$1 or $D(x)$2 can help learning because sufficiently separated domains make the stable structure easier to identify (Zheng et al., 18 Jan 2026).

The regression example in that work makes the mechanism explicit. After approximating away weak correlations $D(x)$3, the learned coefficient vector becomes

$D(x)$4

When the spurious or noisy variance $D(x)$5 is much larger than the causal-noise variance $D(x)$6,

$D(x)$7

In the multi-domain extension, the same intuition is expressed by

$D(x)$8

The paper interprets stronger variation in $D(x)$9 across domains as stronger KL separation between domains and therefore as a signal that pushes the learner toward the invariant predictor (Zheng et al., 18 Jan 2026).

The invariant target is stated using the assumption

$D'(x)$0

with the same noise law $D'(x)$1 across environments. Under the stronger condition that the causal relationship between $D'(x)$2 and $D'(x)$3 is linearly invariant across environments and latent shifts, the ERM objective

$D'(x)$4

has an optimum $D'(x)$5 satisfying

$D'(x)$6

In that regime, ERM coincides with invariant prediction rather than failing under shift (Zheng et al., 18 Jan 2026).

The bound-based results make the same point in a more global way. For clean distributions, Theorem 1 uses

$D'(x)$7

under the condition

$D'(x)$8

and the bound tightens at rate

$D'(x)$9

For binary classification under Massart noise, the analogous quantity is

$$\mathrm{Acc}(\mathrm{OOD}) > \mathrm{Acc}(\mathrm{ID})$$0

under

$$\mathrm{Acc}(\mathrm{OOD}) > \mathrm{Acc}(\mathrm{ID})$$1

The corollary states that if $$\mathrm{Acc}(\mathrm{OOD}) > \mathrm{Acc}(\mathrm{ID})$$2 or $$\mathrm{Acc}(\mathrm{OOD}) > \mathrm{Acc}(\mathrm{ID})$$3 is sufficiently large under the stated constraints, then $$\mathrm{Acc}(\mathrm{OOD}) > \mathrm{Acc}(\mathrm{ID})$$4, and in the binary case the deviation can be controlled via

$$\mathrm{Acc}(\mathrm{OOD}) > \mathrm{Acc}(\mathrm{ID})$$5

Empirically, stronger shift in synthetic regression and in CMNIST pushes ERM predictions closer to Oracle or Optimal models and makes them “less sensible for colors” (Zheng et al., 18 Jan 2026).

3. Support expansion, importance weighting, and generalized risk decomposition

A second usage of PDS is specific to support mismatch. Distribution shift is decomposed into a change in the joint distribution $$\mathrm{Acc}(\mathrm{OOD}) > \mathrm{Acc}(\mathrm{ID})$$6 and a change in support. Writing $$\mathrm{Acc}(\mathrm{OOD}) > \mathrm{Acc}(\mathrm{ID})$$7 and $$\mathrm{Acc}(\mathrm{OOD}) > \mathrm{Acc}(\mathrm{ID})$$8 for training and test supports, the four cases are: exact match, training support wider, test support wider, and partial overlap. In that terminology, PDS refers to the support-expansion cases where the test support extends beyond the training support: case (iii), $$\mathrm{Acc}(\mathrm{OOD}) > \mathrm{Acc}(\mathrm{ID})$$9, and case (iv), where $D_q$0 under partial overlap (Fang et al., 2023).

This use of “positive” is paper-specific and does not mean beneficial performance by itself. Instead, it marks the difficult regime in which the model must handle regions that are present at test time but absent from training. Standard importance weighting uses

$D_q$1

and

$D_q$2

When $D_q$3, the usual change-of-measure argument gives $D_q$4, where

$D_q$5

But in cases (iii) and (iv), $D_q$6 is not defined on $D_q$7, and Theorem 2 states that importance weighting is risk-inconsistent, with

$D_q$8

The paper also shows toy case-(iii) examples in which this risk mismatch can become classifier mismatch (Fang et al., 2023).

Generalized Importance Weighting (GIW) repairs this by splitting the test support into an in-training region $D_q$9 and an out-of-training region $q \neq p$0. With $q \neq p$1 on $q \neq p$2 and $q \neq p$3 on $q \neq p$4, and $q \neq p$5, the expected GIW objective is

$q \neq p$6

Theorem 3 proves

$q \neq p$7

The implementation uses a one-class SVM to split validation data into in-training and out-of-training subsets, then applies an IW algorithm to the in-training subset and ordinary supervised learning to the out-of-training subset. In the reported implementation, the IW component is DIW. Across MNIST, Color-MNIST, and CIFAR-20, GIW outperforms DIW, R-DIW, Reweight, MW-Net, CCSA, and DANN in the support-expansion settings, while the paper also notes the practical caveats that GIW requires test-distributed validation data and that the OOT part is used for both training and validation (Fang et al., 2023).

4. Favorable shifted evaluation distributions and shift-gap analysis

A third operational view of PDS measures whether the shifted domain is easier than the in-distribution domain. In TableShift, the key quantity is the shift gap

$q \neq p$8

If $q \neq p$9, then

$D'(x)$0

which is exactly the benchmark’s closest analogue of positive distribution shift (Gardner et al., 2023).

TableShift contains 15 binary classification tasks spanning finance, education, healthcare, public policy, and civic participation, and evaluates 19 methods. Its headline empirical result is not that favorable shifts are dominant, but that ID and OOD accuracy are strongly correlated, with reported correlation $D'(x)$1. Robustness and domain generalization methods can reduce shift gaps, but the reduction usually comes from reduced ID accuracy rather than improved OOD accuracy. No method consistently outperforms XGBoost, LightGBM, or CatBoost on OOD accuracy. The strongest predictor of shift gap is label distribution shift, with reported Pearson correlation $D'(x)$2 or $D'(x)$3, and a linear regression of OOD accuracy on $D'(x)$4 achieving $D'(x)$5 or $D'(x)$6 (Gardner et al., 2023).

The model-based optimization literature provides a different evaluation-time interpretation. In offline MBO, the training distribution $D'(x)$7 and the design distribution $D'(x)$8 differ because optimization explicitly seeks better designs. This is a form of feedback covariate shift: $D'(x)$9 A binary classifier trained to distinguish training from design samples yields a density-ratio proxy

$$P^e \neq P^{e'}, \quad \forall e \neq e' \in \mathcal E,$$0

The resulting OOD score acts as a continuous measure of shift intensity. Empirically, distribution shift increases along optimization trajectories, later designs are more likely to be adversarial or non-functional, and filtering candidates by OOD score reduces regret compared to surrogate prediction alone or deep ensemble uncertainty, in several cases achieving zero regret (Damani et al., 2023).

These results sharpen an important distinction. A shifted distribution can be favorable in intent, as in the search for better designs or in a negative shift gap, without being favorable in realized model behavior. Control+Shift and hybrid-simulation results reinforce that distinction by showing systematic degradation as support-based shift intensity grows and by linking rollout-induced shift to simulation error (Friedman et al., 2024, Zhao et al., 2024).

5. Deliberate training-distribution design, mixture mismatch, and tractability

One of the strongest PDS theses is that a carefully chosen training distribution can improve test performance even when evaluation remains fixed. In the mixture-distribution setting, the test distribution is

$$P^e \neq P^{e'}, \quad \forall e \neq e' \in \mathcal E,$$1

training uses

$$P^e \neq P^{e'}, \quad \forall e \neq e' \in \mathcal E,$$2

and the expected test loss is

$$P^e \neq P^{e'}, \quad \forall e \neq e' \in \mathcal E,$$3

PDS occurs when

$$P^e \neq P^{e'}, \quad \forall e \neq e' \in \mathcal E,$$4

The paper shows that mismatching training and test mixture proportions can improve test performance even when the mixture components are unrelated and there is no transfer between components (Medvedev et al., 29 Oct 2025).

For power-law learning curves

$$P^e \neq P^{e'}, \quad \forall e \neq e' \in \mathcal E,$$5

the optimal mixture $$P^e \neq P^{e'}, \quad \forall e \neq e' \in \mathcal E,$$6 generally differs from $$P^e \neq P^{e'}, \quad \forall e \neq e' \in \mathcal E,$$7, improving the constant in the asymptotic test loss. In the “test-taking” example with $$P^e \neq P^{e'}, \quad \forall e \neq e' \in \mathcal E,$$8, $$P^e \neq P^{e'}, \quad \forall e \neq e' \in \mathcal E,$$9, and $KL(P;Q) = \int \log \frac{p}{q}\, p\, d\mu.$0,

$KL(P;Q) = \int \log \frac{p}{q}\, p\, d\mu.$1

so 20% less data is needed with the shifted mixture. The general theorem states that unless a special conservation condition holds, there is a zero-measure subset $KL(P;Q) = \int \log \frac{p}{q}\, p\, d\mu.$2 such that for all $KL(P;Q) = \int \log \frac{p}{q}\, p\, d\mu.$3,

$KL(P;Q) = \int \log \frac{p}{q}\, p\, d\mu.$4

The same logic extends to a compositional skill-setting, where training on a mixture with 30% uniform-skills data and 70% composition data yields about $KL(P;Q) = \int \log \frac{p}{q}\, p\, d\mu.$5 sample-efficiency speedup (Medvedev et al., 29 Oct 2025).

A broader computational formalization appears in the tractable-learning framework. Here the target function $KL(P;Q) = \int \log \frac{p}{q}\, p\, d\mu.$6 is evaluated under a target distribution $KL(P;Q) = \int \log \frac{p}{q}\, p\, d\mu.$7, but training uses i.i.d. samples from a different distribution $KL(P;Q) = \int \log \frac{p}{q}\, p\, d\mu.$8, still labeled by the true target. Performance is measured by

$KL(P;Q) = \int \log \frac{p}{q}\, p\, d\mu.$9

The paper defines PDS learning, then refines it into $\sup_{P,P' \in \mathcal P_E} KL(P;P') \le \alpha$0-PDS, D-DS-PAC, and R-DS-PAC. Its central claim is that the main benefit is often computational rather than statistical: the shifted distribution can create correlations or staircase structure that standard gradient-based learning can exploit (Medvedev et al., 9 Feb 2026).

The tractability results are concrete. Noisy parities are tractably D-DS-PAC learnable with a specified shifted distribution, sample complexity

$\sup_{P,P' \in \mathcal P_E} KL(P;P') \le \alpha$1

and runtime

$\sup_{P,P' \in \mathcal P_E} KL(P;P') \le \alpha$2

Noisy juntas are D-DS-PAC learnable using $\sup_{P,P' \in \mathcal P_E} KL(P;P') \le \alpha$3 queries with tolerance

$\sup_{P,P' \in \mathcal P_E} KL(P;P') \le \alpha$4

The paper also proves D-DS-PAC $\sup_{P,P' \in \mathcal P_E} KL(P;P') \le \alpha$5 NA-MQ and NA-MQ $\sup_{P,P' \in \mathcal P_E} KL(P;P') \le \alpha$6 R-DS-PAC, making the connection to non-adaptive membership queries explicit (Medvedev et al., 9 Feb 2026).

A specialized semisupervised variant appears in positive-unlabeled learning. There, “arbitrary positive shift” means the positive class-conditional distribution may differ arbitrarily between training and test, while the negative class remains fixed: $\sup_{P,P' \in \mathcal P_E} KL(P;P') \le \alpha$7 Under this assumption, and with unlabeled data from both train and test marginals, the paper gives statistically consistent methods including wUU, a, and PURR, and reports strong robustness under disjoint positive supports and adversarial spam drift (Hammoudeh et al., 2020).

6. Diagnostics, dynamics, and limitations

Several papers do not define PDS directly but provide tools for diagnosing whether a shift is beneficial, harmful, or behaviorally meaningful. DISDE decomposes the performance change between a source distribution $\sup_{P,P' \in \mathcal P_E} KL(P;P') \le \alpha$8 and a target distribution $\sup_{P,P' \in \mathcal P_E} KL(P;P') \le \alpha$9 into three terms: $D(x)$00 The first term corresponds to the target putting more mass on harder but frequently seen examples, the middle term isolates changes in $D(x)$01, and the last term captures poor performance on infrequent or unseen target examples (Cai et al., 2023).

Explanation-based monitoring asks whether the model relies on features differently under shift. With explanation function

$D(x)$02

explanation shift is defined by

$D(x)$03

The paper proves the one-way implication

$D(x)$04

but not the converse, so explanations can shift even when prediction distributions do not. This makes explanation shift a diagnostic for model-behavior change, not a PDS guarantee (Mougan et al., 2023).

Dynamic and strategic settings add another layer. In coupled-gradient-flow models of interacting agents, aligned objectives can produce a shift that improves the performance of the classifier and also improves the performance of the population itself, whereas competitive objectives can induce polarization, bimodality, and disparate impacts. The theory proves asymptotic convergence of retraining dynamics to steady states in both aligned and competitive regimes, with explicit rates in terms of model parameters (Conger et al., 2023).

The broader empirical literature also cautions against overgeneralizing PDS. Control+Shift generates controllable support-based covariate shifts with intensity

$D(x)$05

and reports monotonic degradation as shift intensity increases, even when the shift is almost perceptually unnoticeable and even with RandAugment (Friedman et al., 2024). In machine-learning-augmented hybrid simulation, rollout-induced shift away from the data manifold correlates with long-horizon error, and tangent-space regularization improves performance by reducing that harmful shift rather than by exploiting it (Zhao et al., 2024).

Taken together, these results delimit the scope of PDS. Positive shift is neither a universal property of OOD data nor a synonym for robustness. It is a conditional phenomenon that depends on support relations, mixture structure, causal invariances, optimization dynamics, label-distribution changes, or strategic feedback. When those conditions align, shift can reveal invariant mechanisms, improve sample efficiency, or turn computationally hard problems into tractable ones; when they do not, the same shift language describes regimes in which standard methods fail.