Conformal Prediction via Transported Beta Laws

Abstract: Split conformal prediction provides finite-sample marginal coverage under exchangeability, but this guarantee averages over the random calibration sample. We study instead the law of the calibration-conditional coverage induced by a realized conformal threshold. In the continuous i.i.d. setting this law is exactly $Beta(k,n+1-k)$, so the usual marginal guarantee corresponds to its mean. We take this beta law as a finite-sample reference object and quantify departures from it using Wasserstein distances on $[0,1]$. The framework yields direct bounds on marginal coverage gaps and on bad-calibration probabilities, and separates different sources of non-i.i.d. behavior according to how they deform the beta reference: test-side shift acts through a transport map on the coverage scale, while calibration dependence changes the order-statistic law itself. We instantiate the framework in scale-shift, clustered, and stationary mixing settings, where the induced deformations can be characterized explicitly or through Berry-Esseen approximations. Simulations on dependent processes confirm that the first-order approximation tracks the empirical Wasserstein distance even at moderate sample sizes.

• The paper introduces a transported-beta framework to derive finite-sample bounds on calibration-conditional coverage using beta distributions.
• It leverages Wasserstein distances to measure deviations from the ideal beta law under conditions like distribution shift and dependence.
• Numerical experiments and theoretical analyses illustrate how scale shifts and clustering affect effective calibration sample size and coverage reliability.

Conformal Prediction via Transported Beta Laws: A Technical Overview

Introduction and Motivation

Split conformal prediction is a widely used procedure for constructing prediction sets with finite-sample marginal coverage guarantees under the assumption of exchangeability. The canonical analysis provides coverage assurances averaged over both the calibration and test samples, but does not elucidate the distribution of coverage for a fixed calibration sample—what the authors term the calibration-conditional coverage. In the continuous i.i.d. setting, this conditional coverage is distributed as a beta law, $\operatorname{Beta}(k, n+1-k)$, for the $k$-th order statistic thresholding construction. This observation is leveraged by Ramos, Graziadei, and Cabezas to develop a general transported-beta framework analyzing how the law of realized coverage departs from the i.i.d. beta reference when underlying assumptions are violated, such as in the presence of distribution shift or statistical dependence.

By measuring these departures in Wasserstein distance on $[0, 1]$, the framework yields direct, non-asymptotic bounds on the marginal coverage gap and on the tail probability of bad calibration events. Furthermore, it isolates the effect of test distribution shift from that of calibration dependence, affording a modular diagnostic perspective. This essay presents the main contributions and techniques of "Conformal Prediction via Transported Beta Laws" (2605.19024), highlights the analytical and numerical findings, and discusses implications and future avenues.

Classical Split Conformal Prediction and the Beta Law

The split conformal method divides data into a training set for model estimation and a calibration set for threshold selection. For a nominal level $\gamma\in(0,1)$ and $n$ calibration scores, the conformal threshold is the $k$-th order statistic, $k = \lceil (n+1)\gamma \rceil$. The classical finite-sample guarantee states that the marginal coverage of a new test point is in $[\gamma, \gamma+\frac{1}{n+1})$ under exchangeability.

A finer property is that, conditional on the calibration sample, the coverage probability is itself random; its distribution is exactly $\operatorname{Beta}(k, n+1 - k)$ for continuous i.i.d. scores. This beta law induces non-trivial lower tail probabilities, i.e., the probability that realized coverage dips significantly below nominal, even when the marginal guarantee holds. Specifically, with $n=30$ and $k$0, the probability that realized coverage is below $k$1 is about $k$2, and below $k$3 is about $k$4.

Figure 1: Bad-calibration events under i.i.d. beta reference—simulated draws of $k$5 for $k$6, $k$7, with shaded lower tails showing the frequency of low realized coverage.

Such calibration-conditional characterization refines the usual mean-based guarantee, motivating the need for a more granular comparison framework when moving beyond i.i.d. settings.

Transported Beta Laws and Wasserstein Metrics

The authors propose quantifying the deviation from the i.i.d. beta law using Wasserstein distances, specifically the $k$8 metric, between the actual law $k$9 of the realized coverage and the reference $[0, 1]$0. This geometric comparison leads to transparent and interpretable bounds: the $[0, 1]$1 radius directly upper bounds the marginal coverage gap, while tail deviations control the frequency of bad-calibration events.

For instance, a contaminated law $[0, 1]$2, which replaces a fraction $[0, 1]$3 of the beta mass by a point at $[0, 1]$4, has $[0, 1]$5 distance scaling linearly in $[0, 1]$6, modulated by the position of $[0, 1]$7.

Figure 2: Wasserstein radius $[0, 1]$8 for the contaminated law, showing feasible $[0, 1]$9 regions for various $\gamma\in(0,1)$0.

Two avenues deform the beta reference: (1) test-side distribution shift, which acts via a transport map on the coverage scale, and (2) calibration dependence, which changes the order-statistic mechanism itself.

Distribution Shift: Transport Maps and Explicit Calculations

When the test score is independent but not identically distributed with the calibration scores, the calibration-conditional coverage law is a pushforward (transported) beta law via a monotone map $\gamma\in(0,1)$1. The Wasserstein distance between the transported law and the beta reference quantifies the coverage loss due to distributional shift.

A worked example uses half-normal scores with a scale shift—relevant for regression residuals under heteroscedasticity:

$\gamma\in(0,1)$2

Here, $\gamma\in(0,1)$3 is the scale ratio. The coverage loss is expressed exactly as $\gamma\in(0,1)$4. The direction and magnitude of the shift are explicit: $\gamma\in(0,1)$5 causes undercoverage, and the loss is approximately $\gamma\in(0,1)$6 for small $\gamma\in(0,1)$7.

Figure 3: Transported beta laws under half-normal scale shift for $\gamma\in(0,1)$8, $\gamma\in(0,1)$9, $n$0, illustrating the leftward transport and increased bad-calibration probability as $n$1 increases above $n$2.

Calibration Dependence: Effective Sample Size and Berry–Esseen Approximations

With dependent calibration scores, e.g., due to clustering or time-series dependence, the order statistic law deviates from beta. In perfectly clustered data, $n$3 clusters replace $n$4 calibration points, and the order statistic's law becomes $n$5, directly reflecting the loss in effective calibration sample size.

For stationary mixing or Markov processes, Berry–Esseen-type results provide a Gaussian approximation to the order statistic. The Wasserstein distance to beta is then

$n$6

where $n$7 encodes the long-run variance of the empirical distribution function at $n$8, including dependence. The residual decoupling error for the test sample, e.g., via Markov or $n$9-mixing coefficients, is separated in the analysis.

AR(1) Process Example

The stationary AR(1) process example demonstrates both the effect of finite-lag test-calibration dependence and the impact of calibration dependence on the coverage law.

Figure 4: Realized-coverage laws $k$0 for the Gaussian AR(1) model with $k$1, $k$2, against the beta reference. Increased AR parameter $k$3 induces heavier left tails, especially for small prediction horizons $k$4.

Figures in the appendix quantify how Wasserstein distances and bad-calibration event probabilities decay as the prediction horizon grows or as calibration length increases, supporting the tightness of the proposed theoretical bounds.

Theoretical and Practical Implications

The transported-beta framework provides several key theoretical advances:

• Direct, interpretable, and tight bounds on marginal coverage gaps in terms of $k$5 distances between actual and beta reference laws.
• Explicit separation of non-i.i.d. effects: test-side shift is modeled via transport on $k$6, calibration dependence via perturbation of the order-statistic law.
• Sharp quantitative predictions in concrete settings (scale shift, clustering, mixing), with closed-form or Berry–Esseen-based rates.
• Diagnosis and comparison tool: the beta reference law offers a universal finite-sample benchmark that persists in weakened-exchangeability regimes.

Numerical experiments in the paper show that the Berry–Esseen approximation of the order-statistic law tracks the empirical Wasserstein gap even for moderate calibration sizes, indicating the practical accuracy of the approach.

Future Directions

The framework opens several research directions. Estimation of the Wasserstein radius from observed data could allow adaptive calibration and coverage correction. Extensions to weighted or adaptive conformal strategies under distribution shift, incorporating estimates of the transport map or calibration law, could improve robustness in sequential or online scenarios. Sharper finite-sample comparisons of beta tail probabilities and robustification under more general dependence structures merit further study, with possible implications for quantifying uncertainty in time series, federated, and spatial data regimes.

Conclusion

"Conformal Prediction via Transported Beta Laws" provides a rigorous and flexible geometric framework for understanding and quantifying the validity and variability of split conformal prediction under violations of i.i.d. assumptions. By taking the law of calibration-conditional coverage as the central object and analyzing its transportation from a universal beta reference, the authors unify diagnostics for distribution shift and dependence under a tractable, distribution-free metric. Practical implementation is supported by explicit formulas, tight bounds, and simulations. This approach has significant implications for the reliability and interpretability of statistical inference in dependent and nonstationary environments.