Backward Conformal Prediction (BCP)

• Backward Conformal Prediction (BCP) is a statistical framework that constructs prediction sets with strict size constraints and data-dependent coverage guarantees.
• It employs e-values and a leave-one-out estimator to compute adaptive miscoverage levels, ensuring that the set size remains controlled.
• Extensions like ST-BCP and Bayesian BCP enhance empirical reliability by tightening coverage bounds and adapting to practical applications, such as healthcare and inventory forecasting.

Backward @@@@1@@@@ (BCP) is a statistical framework for constructing prediction sets that provides rigorous conformal coverage while enforcing explicit constraints on prediction set size. Unlike standard conformal prediction, which prescribes a fixed marginal coverage level but allows the size of conformal sets to vary, BCP inverts this paradigm by stipulating a constraint on set size—either constant or data-dependent—and then calculates a nominal coverage estimate induced by that constraint. Its coverage validity is achieved post hoc via e-values and is made computable through a leave-one-out (LOO) estimator. Extensions such as ST-BCP further tighten the (typically conservative) coverage guarantee, enhancing empirical reliability and reducing conservatism without altering practical output.

1. Formal Structure of Backward Conformal Prediction

Let $f$ be a pre-trained predictor and $S\colon \mathcal X\times \mathcal Y\to \mathbb R_+$ a non-conformity (score) function (with lower scores indicating higher conformity). Suppose $\mathcal D_n = \{ (X_i, Y_i) \}_{i=1}^n$ is a calibration sample and $X_{n+1}$ is a test input; all are exchangeable.

A size-constraint rule

$$T: \left(\mathcal D_n, X_{n+1}\right) \mapsto \{1, \ldots, |\mathcal Y|\}$$

prescribes (deterministically or stochastically) the maximal allowable prediction set size. For each $y \in \mathcal Y$, the e-ratio ("e-value") at the test point is

$$E_{n+1}(y) = \frac{(n+1)S(X_{n+1}, y)}{\sum_{i=1}^n S(X_i, Y_i) + S(X_{n+1}, y)}.$$

Given a size constraint $T_{n+1} := T(\mathcal D_n, X_{n+1})$, define a data-dependent miscoverage $\widetilde\alpha_{n+1}$ as

$$\widetilde\alpha_{n+1} = \inf\{ \alpha \in (0,1) : | \{ y : E_{n+1}(y) < 1/\alpha \} | \leq T_{n+1} \}.$$

The prediction set is then

$$C_{\widetilde\alpha_{n+1}} (X_{n+1}) = \{ y : E_{n+1}(y) < 1/\widetilde\alpha_{n+1} \}.$$

By construction, $|C_{\widetilde\alpha_{n+1}}(X_{n+1})|\leq T_{n+1}$.

2. Post-hoc Coverage Guarantee and E-value Foundation

BCP leverages a recent e-value result of Gauthier et al. (2025), providing a post-hoc guarantee for prediction sets constructed at any random nominal miscoverage $\widetilde\alpha$ (possibly data-dependent): $$\mathbb{E}\left[\frac{\Pr(Y_{n+1} \not\in C_{\widetilde\alpha_{n+1}}(X_{n+1}) | \widetilde\alpha_{n+1})}{\widetilde\alpha_{n+1}}\right] \leq 1.$$ By Taylor expansion, this yields

$$\Pr(Y_{n+1} \notin C_{\widetilde\alpha_{n+1}}) \leq \mathbb{E}[\widetilde\alpha_{n+1}] + O(\operatorname{Var}(\widetilde\alpha_{n+1})),$$

hence the marginal coverage satisfies

$$\Pr(Y_{n+1} \in C_{\widetilde\alpha_{n+1}}) \geq 1 - \mathbb{E}[\widetilde\alpha_{n+1}] + O(\text{error}).$$

BCP ensures that, whatever set-size rule is enforced, the empirical coverage will at least match the complement of the estimated expected miscoverage (Gauthier et al., 19 May 2025, Liu et al., 2 Feb 2026).

3. Consistent Estimation via Leave-One-Out

Because the marginal $\mathbb{E}[\widetilde\alpha_{n+1}]$ depends on the unknown test label, BCP introduces a leave-one-out (LOO) estimator. For each $j=1,\dots,n$, treat $(X_j,Y_j)$ as a pseudo-test point, compute e-values and the induced $\widetilde\alpha_j$ as above (with the remaining $n-1$ examples as calibration), and define

$$\widehat\alpha^{\mathrm{LOO}} = \frac{1}{n} \sum_{j=1}^{n} \widetilde\alpha_j.$$

Under mild assumptions (boundedness, exchangeability, non-degeneracy), $\widehat\alpha^{\mathrm{LOO}}$ consistently estimates $\mathbb{E}[\widetilde\alpha_{n+1}]$ with error $O_P(n^{-1/2})$. Hence, the reported coverage $1-\widehat\alpha^{\mathrm{LOO}}$ is a finite-sample, data-driven lower bound (Gauthier et al., 19 May 2025, Liu et al., 2 Feb 2026).

4. Computational Algorithm and Implementation

The BCP procedure may be implemented as follows:

1. Compute calibration scores $S(X_i,Y_i)$ for $i=1,\dots,n$.
2. Compute e-values $E_{n+1}(y)$ for all candidate $y$ at the test point.
3. Identify $\widetilde\alpha_{n+1}$ as the minimal $\alpha$ such that $|\{y: E_{n+1}(y) < 1/\alpha\}| \leq T_{n+1}$.
4. Output the prediction set $C_{\widetilde\alpha_{n+1}}(X_{n+1})$.
5. For each $j=1,\dots,n$, repeat steps 2–3 on $(X_j,Y_j)$ with the other $n-1$ points as calibration to yield $\widetilde\alpha_j$.
6. Report coverage bound $1-\widehat\alpha^{\mathrm{LOO}}$.

Choice of the size-constraint $T$ enables either fixed-size (e.g., $T\equiv k$) or feature-adaptive set-size control, and provides a tradeoff: smaller $T$ generally reduces set informativeness but lowers coverage, with the coverage bound adapting accordingly (Gauthier et al., 19 May 2025).

5. Theoretical Properties and Empirical Performance

BCP's finite-sample guarantees include:

• Consistency: Under regularity conditions, $$|\widehat\alpha^{\mathrm{LOO}} - \mathbb{E}[\widetilde\alpha_{n+1}]| = O_P(n^{-1/2})$$, and $$\operatorname{Var}(\widehat\alpha^{\mathrm{LOO}}) = O(n^{-1})$$ (Gauthier et al., 19 May 2025, Liu et al., 2 Feb 2026).
• Robustness: Marginal coverage lower bound holds under any size-constraint rule that is globally Lipschitz with respect to the calibration data.
• Trade-off Control: By “inverting” conformal prediction (controlling set-size, not level), BCP is well-suited to domains where set-size, rather than coverage, is the bottleneck (e.g., diagnostics, inventory control).

Empirical applications demonstrate:

• For the UCI Breast Cancer dataset, standard split-conformal prediction at $\alpha=0.02$ sometimes outputs size-2 sets; BCP with $T=1$ produces forced single-label predictions with adaptive coverage estimates (Gauthier et al., 19 May 2025).
• In real-world classification (e.g., CIFAR-10, Tiny-ImageNet), BCP's predicted miscoverage closely matches empirical miscoverage; BCP reduces set-size variability in Bayesian settings and maintains coverage under misspecification (Wu et al., 3 Feb 2026).

Study	Empirical Coverage Example	Comments
(Gauthier et al., 19 May 2025)	$\ge 1-\mathbb{E}[\widetilde\alpha]$	Consistent LOO estimator
(Liu et al., 2 Feb 2026)	Average coverage gap: 4.20% $\rightarrow$ 1.12% (ST-BCP)	Score transform narrows coverage gap
(Wu et al., 3 Feb 2026)	81% (misspecified regression, target 80%)	Lower set-size variability than split-CP

6. Limitations and Advances: Tightening the BCP Bound

The core BCP coverage bound is limited by the conservatism of Markov's inequality, often yielding a substantial gap relative to the empirical miscoverage, especially for small $T$. ST-BCP (Score-Transformed BCP) addresses this by introducing a computable, data-adaptive score transformation $h(s; \mathcal D, X)$, mapping all scores above a learned threshold to a constant and others to zero. This step-function transformation makes the resulting e-variable nearly two-valued, for which Markov's inequality is tight (Liu et al., 2 Feb 2026).

Key properties of ST-BCP:

• Invariance: Prediction sets remain unchanged; only the estimated coverage becomes sharper (Theorem 3.2 (Liu et al., 2 Feb 2026)).
• Strict Tightening: For all monotone score transformations, the optimal is a jump at the unique threshold $w(\mathcal D, X)$; all other monotone transformations yield weaker bounds (Theorem 3.5 (Liu et al., 2 Feb 2026)).
• Implementation: Requires only a sort or binary search for the threshold; computational cost is $O(nK)$ per test point.
• Empirical Impact: On benchmarks (e.g., ResNet-50/CIFAR-10, $n=200$, $T=2$), mean coverage gap decreases from $5.38\%$ (baseline) to $0.72\%$ (ST-BCP); similar improvements are observed across datasets and architectures.

Application of ST-BCP is recommended in regimes with small set sizes or observed high coverage gaps. For larger $T$, the baseline and transformed bounds converge, but ST-BCP incurs negligible additional cost (Liu et al., 2 Feb 2026). In regimes with unreliable Taylor approximation, corrected bounds or robust transformations can be substituted.

7. Extensions: Bayesian BCP and Conformal Risk Control

Recent work extends BCP to a Bayesian setting, combining posterior predictive densities as non-conformity scores with conformal calibration and Bayesian quadrature for expected set size estimation (Wu et al., 3 Feb 2026). The Bayesian non-conformity score is

$s(x, y) = -\log \hat p(y|x, D_{tr}),$

where $\hat p(y|x, D_{tr})$ is a leave-one-in posterior predictive mean over sampled models. BCP then formulates the size-coverage trade-off as a PAC-style constrained optimization: $$\min_\lambda\, \mathbb{E}_X[|C(X; \lambda)|] \qquad \text{s.t.} \qquad P_{D_{cal}} \{ R(\lambda) \leq \alpha \} \geq 1-\beta,$$ where $R(\lambda)$ denotes marginal miscoverage. Coverage is enforced via a Dirichlet-weighted conformal risk statistic ($L^+$), and Bayesian quadrature provides low-variance estimates of expected set size. Empirical results demonstrate BCP matches or exceeds the reliability of split-CP and substantially outperforms Bayesian credible intervals under prior misspecification or distribution shift (Wu et al., 3 Feb 2026).

8. Practical Considerations and Use Cases

BCP is particularly effective in domains demanding stringent set-size limits:

• Healthcare: Physicians may enforce maximally interpretable prediction sets (e.g., no more than $k$ differential diagnoses), then accept the data-dependent coverage guarantee.
• Inventory Forecasting: Set-size constraints adapt to forecast volatility, with BCP providing a post-hoc reliability assessment (Gauthier et al., 19 May 2025).
• Adaptive Size Rules: $T$ may be made feature-adaptive (e.g., via local neighborhood entropy), yielding larger sets in ambiguous regimes but still retaining a computable coverage bound (Gauthier et al., 19 May 2025, Liu et al., 2 Feb 2026).

Applications requiring margin guarantees under distribution shift (e.g., out-of-distribution classification, adversarial regimes) benefit from the decision-theoretic calibration and the stability of Bayesian BCP (Wu et al., 3 Feb 2026).

---

Backward Conformal Prediction formalizes the logic of fixing set size a priori and letting coverage adapt subject to post-hoc, explicitly estimable guarantees. Its e-variable, Taylor-based foundation, consistency of the LOO estimator, and recent tightening innovations (ST-BCP) enable robust, distribution-free prediction set formation with controlled informativeness. BCP has demonstrated practical competitiveness and reliability across classical and Bayesian workflows, particularly in low-cardinality, high-stakes settings. The method continues to evolve, integrating tighter bounds and risk-minimizing calibration, expanding its theoretical and applied utility (Gauthier et al., 19 May 2025, Liu et al., 2 Feb 2026, Wu et al., 3 Feb 2026).