Prediction-Powered Inference (PPI)

• Prediction-Powered Inference (PPI) is a statistical framework that leverages machine learning predictions and bias correction to estimate means accurately under both MCAR and MAR settings.
• The integration of IPW via Horvitz–Thompson or Hájek rectifiers ensures unbiased estimation by adjusting for informative labeling based on estimated inclusion probabilities.
• Empirical studies demonstrate that IPW-PPI reduces variance and bias, yielding narrower and correctly centered confidence intervals compared to classical methods.

Prediction-Powered Inference (PPI) with Inverse Probability Weighting (IPW) is a statistical framework designed for valid estimation and inference when predictions from a ML model supplement a partially labeled dataset, with an explicit mechanism to address informative labeling. PPI achieves credible bias correction and variance reduction by combining predictions on large unlabeled samples with empirical corrections from a smaller labeled subset. The integration of IPW into the bias-correction step guarantees unbiasedness and robust inference even when the probability of labeling varies across units, reflecting both modern semi-supervised learning principles and canonical survey sampling theory (Datta et al., 13 Aug 2025).

1. Foundation and Standard PPI Framework

Prediction-Powered Inference (PPI) is motivated by datasets where only a fraction of observations are labeled, but predictions $f(X)$ are available for the entire sample or population. For finite-sample mean estimation,

$\theta^* = \frac{1}{N} \sum_{i=1}^N Y_i,$

PPI constructs the estimator

$\hat{\theta}_{PPI} = \frac{1}{N} \sum_{i=1}^N f(X_i) - \Deltâ,$

where the plug-in term leverages ML predictions, and the "rectifier" (bias-correction) term

$\Deltâ = \frac{1}{n_{lab}} \sum_{i:R_i=1} [f(X_i) - Y_i],$

uses the labeled set, indexed by indicators $R_i$ ($R_i=1$ if $Y_i$ is observed).

Under Missing Completely at Random (MCAR) labeling, this approach ensures unbiasedness due to independence between $R_i$ and covariates or outcomes, and delivers improved variance properties by exploiting the low-variance nature of the plug-in estimator computed on the entire population.

2. PPI under Informative Labeling: The IPW Approach

In practice, labeled samples might not be MCAR; instead, the probability $\xi_i = P(R_i=1)$ may depend on $X_i$. Under such informative (or Missing At Random, MAR) sampling, an unweighted rectifier introduces bias. The PPI framework generalizes via inverse probability weighting using methods from classical survey sampling:

• Horvitz–Thompson (HT) rectifier:

$\Deltâ_{HT} = \frac{1}{N} \sum_{i=1}^N \frac{R_i}{\xi_i} [f(X_i) - Y_i]$

• Hájek rectifier:

$\Deltâ_{Hájek} = \frac{\sum_{i=1}^N \frac{R_i}{\xi_i} [f(X_i) - Y_i]}{\sum_{i=1}^N \frac{R_i}{\xi_i}}$

The adjusted PPI estimator thus becomes, for HT: $\hat{\theta}_{PPI,HT} = \frac{1}{N} \sum_{i=1}^N f(X_i) - \Deltâ_{HT}$ (and analogously for the Hájek variant). This design-based correction is unbiased for $\theta^*$ whenever $\xi_i$ is known or consistently estimated.

3. Bias Correction, Variance Structure, and Statistical Guarantees

The key merit of PPI is variance reduction versus classical estimators that use only labeled data. The IPW bias correction ensures unbiasedness under MAR by adjusting each labeled case's contribution. Variance is further reduced since the large-scale plug-in term, computed with predictions, has low variance ($O(N^{-1})$ for mean estimation), and the bias-corrector, now reweighted for informativeness, has variance governed by $O(n_{lab}^{-1})$. Importantly, the two terms are independent so variances sum directly when constructing confidence intervals.

The unbiasedness and valid coverage of the estimator depend on correct specification (or consistent estimation) of the inclusion probabilities $\xi_i$. Simulations confirm that even when $\xi_i$ is not known but estimated (e.g., via logistic regression), bias remains minimal and nominal coverage is retained.

4. Empirical Performance and Simulation Studies

Extensive simulations validate the behavior of IPW-PPI under both real and synthetic scenarios (Datta et al., 13 Aug 2025). Major observations include:

• Standard (unweighted) estimators show significant bias when labeling is informative.
• Both HT and Hájek adjusted PPI estimators yield near-zero bias and confidence intervals with correct coverage, even for small labeled fractions ($p_{lab} = 0.01$ or $0.02$).
• As $p_{lab}$ increases, all methods display narrower intervals, but the IPW variants consistently control bias.
• The IPW-adjusted PPI produces confidence intervals that are not only centered correctly but also often narrower than those from classical approaches.

These findings underline the practical utility of IPW-PPI for data with nonrandom selection for labeling, a situation prevalent in many modern applications.

5. Practical Methodology: From MCAR to MAR via Survey Sampling Theory

The formal connection between PPI and survey sampling is established by generalizing the rectifier. Under MCAR, the unweighted rectifier is equivalent to the Horvitz–Thompson estimator. When inclusion probabilities are non-uniform:

• Using the IPW adjusted rectifiers (HT or Hájek) matches the asymptotic and finite-sample unbiasedness properties of classical survey sampling estimators.
• The approach incorporates estimated probabilities $\hat{\xi}_i$ when $\xi_i$ must be fitted from data, yielding estimators nearly as effective as when $\xi_i$ is known.
• The unification of prediction-based bias correction and survey sampling design supports a bridging of historical and modern approaches, aligning PPI with established methodologies in design-based inference.

6. Broader Impact and Theoretical Integration

The advancement of IPW-PPI is significant for high-dimensional, partially-labeled, and semi-supervised settings found in genomics, remote sensing, medical imaging, and other applied fields where labeled data is costly or obtained via complex mechanisms. The formalization and simulation evidence presented demonstrate that combining model-based predictions with IPW bias correction yields:

• Valid inference under informative sampling,
• Robust variance reduction,
• Flexible integration of estimated labeling probabilities,
• Alignment with decades of survey sampling theory (specifically, Horvitz–Thompson and Hájek estimators).

The approach is nonparametric regarding the prediction rule $f(\cdot)$, requiring no strong model assumptions.

7. Summary Table: Estimators in PPI under MCAR and MAR

Scenario	Rectifier	Bias Correction
MCAR	Unweighted	$\Deltâ = \frac{1}{n_{lab}}\sum (f(X_i)-Y_i)$
MAR / Informative	IPW (HT or Hájek)	$\Deltâ_{HT} = \sum \frac{R_i}{\xi_i}(f(X_i)-Y_i)/N$

This table formalizes the connection between standard prediction-powered inference and its IPW extension under informative labeling mechanisms (Datta et al., 13 Aug 2025).

• The comprehensive methodology for IPW-adjusted PPI is developed and systematically validated in "Prediction-Powered Inference with Inverse Probability Weighting" (Datta et al., 13 Aug 2025).