Breaking the curse of horizon in conformal off-policy prediction

Determine how to break the curse of horizon in conformal off-policy prediction for sequential decision making by constructing prediction interval procedures that remain efficient for long decision horizons while retaining valid coverage guarantees for the potential outcome under a given target policy.

Background

The paper proposes COPP (Conformal Off-Policy Prediction) to construct prediction intervals for individual outcomes under a target policy, extending weighted conformal prediction to the off-policy setting via an auxiliary policy and subsampling. In sequential decision making, the method selects subsamples where the auxiliary policy matches the behavior policy along the trajectory to emulate the target outcome distribution, and then uses weighted conformal prediction to account for covariate shift.

A noted limitation is the common curse of horizon in off-policy learning: as the number of stages increases, the probability of selecting trajectories that match the auxiliary and behavior policies at all stages decreases rapidly, leading to dwindling effective sample sizes and reduced efficiency. The authors propose importance-sampling and multi-sampling extensions to partially mitigate this issue, but acknowledge that the fundamental problem persists in long-horizon settings.

References

It can be seen that the proposed method is able to achieve nominal coverage in general. Nonetheless, as commented in our paper, it suffers from the curse of horizon and would be inefficient in long-horizon settings. It remains unclear how to break the curse of horizon and we leave it as future work.

— Conformal Off-policy Prediction  (2206.06711 - Zhang et al., 2022) in Section 5, Synthetic Data Analysis (Results, Example 3)

The frozen anchor is not a coverage-calibrated threshold. It is an empirically-selected stopping rule, a robust (median) estimate of ensemble disagreement at a shallow reference horizon. Its purpose is to shape the training curriculum (which rollout depths receive gradient), not to certify deployment-time rollouts. Reliability is assessed empirically via long-horizon rollout error and policy sim-transfer. A coverage guarantee could be obtained by replacing the anchor with a conformal quantile calibrated on held-out rollouts. We leave this as an extension.

— Adaptive Rollout Truncation Based on Epistemic Uncertainty for Efficient Offline World Model Training  (2609.21482 - Zymla et al., 18 Sep 2026) in Appendix, Section Robustness

Open problems include diagnostics for $\rho$, adaptive cross-fitting under endogenous streams, tighter coverage--regret lower bounds beyond Theorem~\ref{thm:pareto}, adaptive exploration budgets, and anytime counterfactual CP via conformal $e$-processes \citep{vovk2025conformal,vovkwang2026confounding,wang2025stoppedebh,gauthier2025evalues,ramdas2024hypothesis}.

— Counterfactual Online Conformal Prediction Under Adaptive Logging  (2609.30811 - Qiao et al., 25 Sep 2026) in Section 6, Discussion and Limitations, subsection “Limitations and open directions”