Backward Conformal Prediction (BCP)

• Backward Conformal Prediction (BCP) is a statistical framework that constructs prediction sets with strict size constraints and data-dependent coverage guarantees.
• It employs e-values and a leave-one-out estimator to compute adaptive miscoverage levels, ensuring that the set size remains controlled.
• Extensions like ST-BCP and Bayesian BCP enhance empirical reliability by tightening coverage bounds and adapting to practical applications, such as healthcare and inventory forecasting.

Backward Conformal Prediction (BCP) is a statistical framework for constructing prediction sets that provides rigorous conformal coverage while enforcing explicit constraints on prediction set size. Unlike standard conformal prediction, which prescribes a fixed marginal coverage level but allows the size of conformal sets to vary, BCP inverts this paradigm by stipulating a constraint on set size—either constant or data-dependent—and then calculates a nominal coverage estimate induced by that constraint. Its coverage validity is achieved post hoc via e-values and is made computable through a leave-one-out (LOO) estimator. Extensions such as ST-BCP further tighten the (typically conservative) coverage guarantee, enhancing empirical reliability and reducing conservatism without altering practical output.

1. Formal Structure of Backward Conformal Prediction

Let $f$ be a pre-trained predictor and $S\colon \mathcal X\times \mathcal Y\to \mathbb R_+$ a non-conformity (score) function (with lower scores indicating higher conformity). Suppose $\mathcal D_n = \{ (X_i, Y_i) \}_{i=1}^n$ is a calibration sample and $X_{n+1}$ is a test input; all are exchangeable.

A size-constraint rule

$$T: \left(\mathcal D_n, X_{n+1}\right) \mapsto \{1, \ldots, |\mathcal Y|\}$$

prescribes (deterministically or stochastically) the maximal allowable prediction set size. For each $y \in \mathcal Y$, the e-ratio ("e-value") at the test point is

$$E_{n+1}(y) = \frac{(n+1)S(X_{n+1}, y)}{\sum_{i=1}^n S(X_i, Y_i) + S(X_{n+1}, y)}.$$

Given a size constraint $T_{n+1} := T(\mathcal D_n, X_{n+1})$, define a data-dependent miscoverage $\widetilde\alpha_{n+1}$ as

$$\widetilde\alpha_{n+1} = \inf\{ \alpha \in (0,1) : | \{ y : E_{n+1}(y) < 1/\alpha \} | \leq T_{n+1} \}.$$

The prediction set is then

$S\colon \mathcal X\times \mathcal Y\to \mathbb R_+$0

By construction, $S\colon \mathcal X\times \mathcal Y\to \mathbb R_+$1.

2. Post-hoc Coverage Guarantee and E-value Foundation

BCP leverages a recent e-value result of Gauthier et al. (2025), providing a post-hoc guarantee for prediction sets constructed at any random nominal miscoverage $S\colon \mathcal X\times \mathcal Y\to \mathbb R_+$2 (possibly data-dependent): $S\colon \mathcal X\times \mathcal Y\to \mathbb R_+$3 By Taylor expansion, this yields

$S\colon \mathcal X\times \mathcal Y\to \mathbb R_+$4

hence the marginal coverage satisfies

$S\colon \mathcal X\times \mathcal Y\to \mathbb R_+$5

BCP ensures that, whatever set-size rule is enforced, the empirical coverage will at least match the complement of the estimated expected miscoverage (Gauthier et al., 19 May 2025, Liu et al., 2 Feb 2026).

3. Consistent Estimation via Leave-One-Out

Because the marginal $S\colon \mathcal X\times \mathcal Y\to \mathbb R_+$6 depends on the unknown test label, BCP introduces a leave-one-out (LOO) estimator. For each $S\colon \mathcal X\times \mathcal Y\to \mathbb R_+$7, treat $S\colon \mathcal X\times \mathcal Y\to \mathbb R_+$8 as a pseudo-test point, compute e-values and the induced $S\colon \mathcal X\times \mathcal Y\to \mathbb R_+$9 as above (with the remaining $\mathcal D_n = \{ (X_i, Y_i) \}_{i=1}^n$0 examples as calibration), and define

$\mathcal D_n = \{ (X_i, Y_i) \}_{i=1}^n$1

Under mild assumptions (boundedness, exchangeability, non-degeneracy), $\mathcal D_n = \{ (X_i, Y_i) \}_{i=1}^n$2 consistently estimates $\mathcal D_n = \{ (X_i, Y_i) \}_{i=1}^n$3 with error $\mathcal D_n = \{ (X_i, Y_i) \}_{i=1}^n$4. Hence, the reported coverage $\mathcal D_n = \{ (X_i, Y_i) \}_{i=1}^n$5 is a finite-sample, data-driven lower bound (Gauthier et al., 19 May 2025, Liu et al., 2 Feb 2026).

4. Computational Algorithm and Implementation

The BCP procedure may be implemented as follows:

1. Compute calibration scores $\mathcal D_n = \{ (X_i, Y_i) \}_{i=1}^n$6 for $\mathcal D_n = \{ (X_i, Y_i) \}_{i=1}^n$7.
2. Compute e-values $\mathcal D_n = \{ (X_i, Y_i) \}_{i=1}^n$8 for all candidate $\mathcal D_n = \{ (X_i, Y_i) \}_{i=1}^n$9 at the test point.
3. Identify $X_{n+1}$0 as the minimal $X_{n+1}$1 such that $X_{n+1}$2.
4. Output the prediction set $X_{n+1}$3.
5. For each $X_{n+1}$4, repeat steps 2–3 on $X_{n+1}$5 with the other $X_{n+1}$6 points as calibration to yield $X_{n+1}$7.
6. Report coverage bound $X_{n+1}$8.

Choice of the size-constraint $X_{n+1}$9 enables either fixed-size (e.g., $$T: \left(\mathcal D_n, X_{n+1}\right) \mapsto \{1, \ldots, |\mathcal Y|\}$$0) or feature-adaptive set-size control, and provides a tradeoff: smaller $$T: \left(\mathcal D_n, X_{n+1}\right) \mapsto \{1, \ldots, |\mathcal Y|\}$$1 generally reduces set informativeness but lowers coverage, with the coverage bound adapting accordingly (Gauthier et al., 19 May 2025).

5. Theoretical Properties and Empirical Performance

BCP's finite-sample guarantees include:

• Consistency: Under regularity conditions, $$T: \left(\mathcal D_n, X_{n+1}\right) \mapsto \{1, \ldots, |\mathcal Y|\}$$2, and $$T: \left(\mathcal D_n, X_{n+1}\right) \mapsto \{1, \ldots, |\mathcal Y|\}$$3 (Gauthier et al., 19 May 2025, Liu et al., 2 Feb 2026).
• Robustness: Marginal coverage lower bound holds under any size-constraint rule that is globally Lipschitz with respect to the calibration data.
• Trade-off Control: By “inverting” conformal prediction (controlling set-size, not level), BCP is well-suited to domains where set-size, rather than coverage, is the bottleneck (e.g., diagnostics, inventory control).

Empirical applications demonstrate:

• For the UCI Breast Cancer dataset, standard split-conformal prediction at $$T: \left(\mathcal D_n, X_{n+1}\right) \mapsto \{1, \ldots, |\mathcal Y|\}$$4 sometimes outputs size-2 sets; BCP with $$T: \left(\mathcal D_n, X_{n+1}\right) \mapsto \{1, \ldots, |\mathcal Y|\}$$5 produces forced single-label predictions with adaptive coverage estimates (Gauthier et al., 19 May 2025).
• In real-world classification (e.g., CIFAR-10, Tiny-ImageNet), BCP's predicted miscoverage closely matches empirical miscoverage; BCP reduces set-size variability in Bayesian settings and maintains coverage under misspecification (Wu et al., 3 Feb 2026).

Study	Empirical Coverage Example	Comments
(Gauthier et al., 19 May 2025)	$$T: \left(\mathcal D_n, X_{n+1}\right) \mapsto \{1, \ldots, |\mathcal Y|\}$$6	Consistent LOO estimator
(Liu et al., 2 Feb 2026)	Average coverage gap: 4.20% $$T: \left(\mathcal D_n, X_{n+1}\right) \mapsto \{1, \ldots, |\mathcal Y|\}$$7 1.12% (ST-BCP)	Score transform narrows coverage gap
(Wu et al., 3 Feb 2026)	81% (misspecified regression, target 80%)	Lower set-size variability than split-CP

6. Limitations and Advances: Tightening the BCP Bound

The core BCP coverage bound is limited by the conservatism of Markov's inequality, often yielding a substantial gap relative to the empirical miscoverage, especially for small $$T: \left(\mathcal D_n, X_{n+1}\right) \mapsto \{1, \ldots, |\mathcal Y|\}$$8. ST-BCP (Score-Transformed BCP) addresses this by introducing a computable, data-adaptive score transformation $$T: \left(\mathcal D_n, X_{n+1}\right) \mapsto \{1, \ldots, |\mathcal Y|\}$$9, mapping all scores above a learned threshold to a constant and others to zero. This step-function transformation makes the resulting e-variable nearly two-valued, for which Markov's inequality is tight (Liu et al., 2 Feb 2026).

Key properties of ST-BCP:

• Invariance: Prediction sets remain unchanged; only the estimated coverage becomes sharper (Theorem 3.2 (Liu et al., 2 Feb 2026)).
• Strict Tightening: For all monotone score transformations, the optimal is a jump at the unique threshold $y \in \mathcal Y$0; all other monotone transformations yield weaker bounds (Theorem 3.5 (Liu et al., 2 Feb 2026)).
• Implementation: Requires only a sort or binary search for the threshold; computational cost is $y \in \mathcal Y$1 per test point.
• Empirical Impact: On benchmarks (e.g., ResNet-50/CIFAR-10, $y \in \mathcal Y$2, $y \in \mathcal Y$3), mean coverage gap decreases from $y \in \mathcal Y$4 (baseline) to $y \in \mathcal Y$5 (ST-BCP); similar improvements are observed across datasets and architectures.

Application of ST-BCP is recommended in regimes with small set sizes or observed high coverage gaps. For larger $y \in \mathcal Y$6, the baseline and transformed bounds converge, but ST-BCP incurs negligible additional cost (Liu et al., 2 Feb 2026). In regimes with unreliable Taylor approximation, corrected bounds or robust transformations can be substituted.

7. Extensions: Bayesian BCP and Conformal Risk Control

Recent work extends BCP to a Bayesian setting, combining posterior predictive densities as non-conformity scores with conformal calibration and Bayesian quadrature for expected set size estimation (Wu et al., 3 Feb 2026). The Bayesian non-conformity score is

$y \in \mathcal Y$7

where $y \in \mathcal Y$8 is a leave-one-in posterior predictive mean over sampled models. BCP then formulates the size-coverage trade-off as a PAC-style constrained optimization: $y \in \mathcal Y$9 where $$E_{n+1}(y) = \frac{(n+1)S(X_{n+1}, y)}{\sum_{i=1}^n S(X_i, Y_i) + S(X_{n+1}, y)}.$$0 denotes marginal miscoverage. Coverage is enforced via a Dirichlet-weighted conformal risk statistic ($$E_{n+1}(y) = \frac{(n+1)S(X_{n+1}, y)}{\sum_{i=1}^n S(X_i, Y_i) + S(X_{n+1}, y)}.$$1), and Bayesian quadrature provides low-variance estimates of expected set size. Empirical results demonstrate BCP matches or exceeds the reliability of split-CP and substantially outperforms Bayesian credible intervals under prior misspecification or distribution shift (Wu et al., 3 Feb 2026).

8. Practical Considerations and Use Cases

BCP is particularly effective in domains demanding stringent set-size limits:

• Healthcare: Physicians may enforce maximally interpretable prediction sets (e.g., no more than $$E_{n+1}(y) = \frac{(n+1)S(X_{n+1}, y)}{\sum_{i=1}^n S(X_i, Y_i) + S(X_{n+1}, y)}.$$2 differential diagnoses), then accept the data-dependent coverage guarantee.
• Inventory Forecasting: Set-size constraints adapt to forecast volatility, with BCP providing a post-hoc reliability assessment (Gauthier et al., 19 May 2025).
• Adaptive Size Rules: $$E_{n+1}(y) = \frac{(n+1)S(X_{n+1}, y)}{\sum_{i=1}^n S(X_i, Y_i) + S(X_{n+1}, y)}.$$3 may be made feature-adaptive (e.g., via local neighborhood entropy), yielding larger sets in ambiguous regimes but still retaining a computable coverage bound (Gauthier et al., 19 May 2025, Liu et al., 2 Feb 2026).

Applications requiring margin guarantees under distribution shift (e.g., out-of-distribution classification, adversarial regimes) benefit from the decision-theoretic calibration and the stability of Bayesian BCP (Wu et al., 3 Feb 2026).

---

Backward Conformal Prediction formalizes the logic of fixing set size a priori and letting coverage adapt subject to post-hoc, explicitly estimable guarantees. Its e-variable, Taylor-based foundation, consistency of the LOO estimator, and recent tightening innovations (ST-BCP) enable robust, distribution-free prediction set formation with controlled informativeness. BCP has demonstrated practical competitiveness and reliability across classical and Bayesian workflows, particularly in low-cardinality, high-stakes settings. The method continues to evolve, integrating tighter bounds and risk-minimizing calibration, expanding its theoretical and applied utility (Gauthier et al., 19 May 2025, Liu et al., 2 Feb 2026, Wu et al., 3 Feb 2026).