---
title: Adaptive Conformal Prediction (ACP)
url: https://www.emergentmind.com/topics/adaptive-conformal-prediction-acp
type: topic
---

# Adaptive Conformal Prediction (ACP)

Adaptive Conformal Prediction (ACP) denotes a family of conformal methods that replace a single static calibration rule with mechanisms that adapt to the input, time index, subpopulation, model state, or downstream objective while retaining a validity notion appropriate to the setting. Across recent work, this adaptivity appears as input-dependent thresholds, online updates of the effective miscoverage level, localized or reweighted calibration, learned coverage policies, score-space transformations, and post-hoc probability calibration constrained by conformal sets [2212.00278][2303.12695][2510.04318][2505.15437]. The unifying motivation is that standard conformal prediction is marginally valid but often rigid: it uses one global correction or one fixed coverage level even when uncertainty is heteroscedastic, nonstationary, subpopulation-dependent, or concentrated on a small set of difficult examples [2303.12695][2510.04318][2605.12917].

## 1. Standard conformal prediction and the motivation for adaptivity

Classical split conformal prediction constructs a set
\[
C_\alpha(x)=\{y:V(x,y)\le q_\alpha\},
\]
where \(q_\alpha\) is a calibration quantile of nonconformity scores, and guarantees
\[
\Pr\bigl(Y_*\in C_\alpha(X_*)\bigr)\ge 1-\alpha
\]
under exchangeability [2505.15437]. In regression, the same logic typically yields intervals by adding a global empirical quantile of residuals to a point predictor; in multiclass classification, it yields a label set with nominal marginal coverage [2303.12695][2505.15437].

The motivation for ACP is that this global correction is often poorly aligned with local uncertainty. One paper states that standard split conformal prediction uses a “single, constant quantile correction” computed from calibration residuals, so every test point receives the same interval width even when the residual distribution depends on \(X\) [2303.12695]. Another emphasizes that once \(\alpha\) is fixed in the classical conformal setup, it generally cannot be chosen after looking at the calibration data without invalidating the guarantee; this is precisely the rigidity that adaptive conformal methods seek to overcome [2510.04318]. In medical image classification, the same issue appears in a different form: standard size-optimized RAPS can achieve good global coverage while becoming nearly deterministic and exhibiting stratified undercoverage on uncertain samples [2605.12917].

ACP therefore addresses several related deficiencies. In nonstationary time series and control, it tracks changing prediction quality online rather than relying on exchangeability across time [2212.00278][2503.17678]. In heteroscedastic regression, it seeks intervals whose lengths vary with the test point rather than using one global residual correction [2303.12695][2306.04648]. Under subpopulation shift, it modifies the calibration threshold so that it reflects the test mixture rather than the pooled calibration mixture [2506.05583]. In multiclass calibration, it may leave the conformal set unchanged but adapt how probability mass is distributed inside the set [2505.15437].

## 2. Mechanisms of adaptivity

A central ACP mechanism is **online adjustment of the effective miscoverage level**. In motion planning among dynamic agents, the update is
\[
\delta_{t+1}:=\delta_t+\gamma(\delta-e_t),
\]
where \(e_t\) indicates whether the current uncertainty set missed the realized trajectory error [2212.00278]. In safe reinforcement learning, an analogous update appears as
\[
\alpha_{k+1} = \alpha_k + \delta(\alpha^c - e_k),
\]
with \(e_k\) determined by whether the current conformal interval covers the barrier-related residual [2503.17678]. In Adaptive Quantum Conformal Prediction, the update is
\[
\alpha_{i+1}=\alpha_i+\gamma(\alpha-\mathrm{err}_i),
\]
and the resulting long-run empirical miscoverage converges almost surely to the target \(\alpha\) under arbitrary hardware noise conditions [2511.18225].

A second mechanism is **input-conditioned calibration**. In “Adaptive Coverage Policies in Conformal Prediction,” the fixed global level is replaced by a learned policy
\[
\pi:\mathbb{R}_+\times \mathcal T \to (0,1),
\]
which outputs
\[
\tilde\alpha = \pi\!\left(\sum_{i=1}^n S(X_i,Y_i),\, t(X_{\rm test})\right)
\]
from the sum of calibration scores and a test statistic [2510.04318]. In localized ACP by reweighting nonconformity scores, the conditional residual distribution is estimated by
\[
\widehat{F}(r\mid x)=\sum_{i=1}^{n} w_n(x,X_i)\,\mathbf{1}_{V_i\le r},
\]
so the interval width depends on the test feature vector through QRF weights [2303.12695]. In “On training locally adaptive CP,” the conformity score itself is transformed by an object-dependent map
\[
A \mapsto \phi_X(A),
\]
which preserves marginal validity when \(\phi_X\) is strictly increasing and all \(\phi_X\) share the same codomain [2306.04648].

A third mechanism is **shift-aware reweighting or grouping**. Under unknown subpopulation shifts, the threshold is chosen by
\[
\hat q_\alpha=\min\left\{q:\sum_{k=1}^K \frac{\hat\lambda_k\,m_k(q)}{n_k+1}\ge 1-\alpha\right\},
\]
where \(\hat\lambda_k\) estimates the test mixture over latent domains [2506.05583]. In Audited Conformal Prediction, a small labeled target sample is used to train an auxiliary audit model \(\hat r\) that predicts where a legacy classifier is likely to fail, and this audit signal is then integrated either into standard split conformal calibration or into a group-conditional conformal construction over audit-defined groups [2606.14909].

A fourth mechanism is **post-hoc probabilistic calibration on top of conformal sets**. In ATS-CP, the conformal set remains the anchor, but an input-specific temperature \(\tau_\alpha(x)\) is chosen so that
\[
\sum_{y\in C_\alpha(x)} \tilde p_V(y\mid x,\tau_\alpha)=1-\alpha.
\]
This makes the threshold or set mass depend on the input in the ACP spirit while preserving the original set-valued conformal guarantee [2505.15437].

## 3. Validity targets and theoretical guarantees

ACP does not have a single universal guarantee; instead, the target validity notion depends on the construction. The most basic guarantee remains the split-conformal marginal coverage statement
\[
\Pr\bigl(Y\in \mathcal{C}(X)\bigr)\ge 1-\alpha
\]
or its multiclass analogue \(\Pr(Y_*\in C_\alpha(X_*))\ge 1-\alpha\) [2505.15437][2605.12917]. Several ACP methods retain this finite-sample marginal guarantee exactly because they alter the score geometry, the local quantile, or the threshold selection without changing the exchangeability argument at calibration time [2303.12695][2306.04648][2603.01719].

A more ambitious target is **training-conditional** or **PAC-style** validity. The QRF-based localized method adds a second calibration stage and states
\[
P^{n_1}\{cov(\mathcal{D}_{n_1})\ge 1-\alpha-\epsilon\}\ge 1-\delta,
\qquad
\delta = K e^{-2n_2\epsilon^2},
\]
thereby strengthening marginal validity to a probably approximately correct training-conditional guarantee [2303.12695]. The same paper also proves asymptotic conditional coverage under continuity and forest regularity assumptions:
\[
\lim_{n\to\infty}P\big(V_{n+1}\in C_V(X)\mid X\big)=1-\alpha
\]
[2303.12695].

In nonexchangeable or sequential settings, ACP often targets **long-run average coverage** rather than fixed-sample exchangeability-based validity. For one-step prediction among dynamic agents, the average coverage theorem is
\[
1-\delta-p_1 \leq \frac{1}{T}\sum_{t=0}^{T-1} \mathrm{Prob}\big(\|Y_{t+1}-\hat y_t^1\|\le C_{t+1}^1\big)\le 1-\delta+p_2,
\]
with \(p_1,p_2\to 0\) as \(T\to\infty\) [2212.00278]. In Adaptive Quantum Conformal Prediction, the long-run miscoverage satisfies
\[
\lim_{N\to\infty}\frac{1}{N}\sum_{j=1}^N \mathrm{err}_j \stackrel{\text{a.s.}}{=}\alpha
\]
under arbitrary hardware noise [2511.18225]. In two-stage sequential models, the adaptive extension preserves the standard long-run ACI guarantee
\[
\lim_{T\to\infty}\frac{1}{T}\sum_{t=1}^T \mathrm{cov}_t \xrightarrow{\text{a.s.}} 1-\alpha
\]
while adapting stage-wise residual contributions [2510.04406]. CP-Traj likewise guarantees long-term coverage for any admissible \(\alpha^{h*}\in I_t^h\) when \(\delta_t=o(t)\) [2508.13362].

Other ACP variants explicitly target **group-conditional** or **audited conditional** performance. Audited Conformal Prediction gives the bound
\[
\mathbb P\!\left[Y_{n+1}\in\hat C(X_{n+1})\mid G_j^*(X_{n+1})=g\right] \ge 1-\alpha -\frac{(1+\alpha)\,\mathbb P[\hat G_j(X_{n+1})\neq G_j^*(X_{n+1})]} {\mathbb P[G_j^*(X_{n+1})=g]},
\]
which approaches the target as the audit model converges to the oracle reliability function [2606.14909]. By contrast, the medical-image ACP paper is explicit that true distribution-free stratum-level guarantees are impossible in general without extra assumptions, so its Adaptive Lambda Criterion is presented as a practical safety heuristic rather than a theorem guaranteeing exact conditional coverage [2605.12917].

## 4. Major methodological families

Several recurring ACP constructions can be distinguished in the recent literature.

| Family | Representative mechanism | Representative papers |
|---|---|---|
| Localized or reweighted ACP | QRF weights, local c.d.f., groupwise conformalization | [2303.12695] |
| Score-space adaptive ACP | Object-dependent transformation \(A\mapsto \phi_X(A)\) | [2306.04648] |
| Policy-based or e-value ACP | Learned \(\tilde\alpha\) from calibration summary and test statistic | [2510.04318] |
| Post-hoc conformal-calibration hybrids | Input-specific temperature constrained by conformal mass | [2505.15437] |
| Shift-aware and audited ACP | Mixture-aware weighting, similarity filtering, audit groups | [2506.05583][2606.14909] |
| Geometry-learning ACP for regression | Jointly optimized center \(m(x)\) and radius \(h(x)\) with split-conformal normalization | [2603.01719] |

In the **localized/reweighted** family, ACP estimates the distribution of nonconformity scores conditional on features and uses local weights to determine a test-point-dependent quantile. The QRF construction is notable because it supports finite-sample marginal validity, training-conditional coverage, and asymptotic conditional coverage, while also exploiting RF-induced partitions for computational savings [2303.12695].

In the **score-space adaptive** family, the calibration sample is not reweighted; instead, the conformity score is changed through a monotone object-dependent transformation. Because the transformed scores remain exchangeable, the method preserves the usual conformal rank argument while producing \(X\)-dependent interval widths [2306.04648].

In the **coverage-policy** family, adaptivity is moved to the coverage level itself. The e-value-based coverage policy paper shows that adaptive \(\alpha\) can be made valid post hoc by replacing classical p-values with e-values, and it trains the policy by leave-one-out pseudo episodes with the objective
\[
\mathcal{L}_\lambda(\theta) = \frac{1}{n}\sum_{j=1}^n \operatorname{Size}\!\left(\hat C_{n-1}^{\tilde\alpha_\theta^j}(X_j)\right) +\lambda\cdot \tilde\alpha_\theta^j
\]
[2510.04318].

In **hybrid classification** methods, the conformal set remains intact but additional adaptive structure is overlaid on top. ATS-CP is explicit that it is not a new set-construction rule that changes the coverage theorem; it is a post-hoc probabilistic reinterpretation of conformal outputs. Its \(\alpha\)-adaptive quantile variants are directly in the ACP family because they use input-dependent thresholds \(q_\alpha(x)\), while the input-specific temperature \(\tau_\alpha(x)\) further adapts the probability calibration inside the set [2505.15437].

In **adaptive interval-geometry** methods for regression, CoCP goes beyond calibrating a fixed center by alternating between learning a radius through folded quantile regression and refining the center with a soft-coverage objective. It then applies split conformal with normalized score
\[
S_i=\frac{|Y_i-\widehat m(X_i)|}{\widehat h(X_i)}
\]
and outputs
\[
\widehat C(x)=\bigl[\widehat m(x)-\widehat q\,\widehat h(x),\ \widehat m(x)+\widehat q\,\widehat h(x)\bigr],
\]
retaining finite-sample marginal validity while asymptotically approaching the length-minimizing conditional interval under the stated regularity conditions [2603.01719].

## 5. Applications and empirical record

ACP has been adopted in safety-critical control, sequential forecasting, medical image classification, distribution-shifted vision and language, and quantum machine learning. In motion planning among dynamic agents, ACP constructs multistep trajectory uncertainty sets from delayed prediction errors and tightens model-predictive-control constraints by a Lipschitz safety margin [2212.00278]. In safe POMDP online planning among multiple pedestrians, ACP-based trajectory regions are turned into unsafe-state sets, winning regions, and a shield for POMCP; the reported safety rate improves from \(0.893\) to \(0.974\) on ETH with \(N=45\), from \(0.944\) to \(0.988\) on Hotel with \(N=35\), and from \(0.91\) to \(0.963\) on GC with \(N=160\), with average planning time per step increasing from \(0.28\) s to \(0.35\) s [2404.15557].

In safe reinforcement learning, ACP is used to quantify uncertainty directly on a control-barrier-function surrogate rather than on the raw dynamics output. The resulting ACP safety margin enforces
\[
B(x_k,u_k)\ge S^{(r)},
\]
so the predicted barrier value must exceed the calibrated uncertainty radius before a control is accepted [2503.17678]. The simulations on a mobile robot and an inverted pendulum report that MPPI and MPPI-CBF can violate safety, while MPPI-CBF-ACP maintains safety across GP, RFF, and QFF surrogate models [2503.17678].

In medical image classification, ACP appears as a stratified, safety-aware adaptation of RAPS. On OrganAMNIST, the proposed method achieves **95.72 percent global coverage** with **average set size 1.09** and **at least 90 percent coverage across all strata**, whereas standard size-optimized RAPS converges to near-deterministic behavior with stratified undercoverage on uncertain samples [2605.12917]. On PathMNIST, the same method reports **95.00%** global coverage, **average size 1.12**, and **worst-case stratum coverage 88.1%** [2605.12917]. The same paper further reports quantitative Grad-CAM evidence that larger prediction sets correspond to more focused attention on ambiguous regions, with Spearman \(\rho=-0.303\) and \(p<10^{-22}\) [2605.12917].

In multiclass probability calibration, ATS-CP evaluates on a synthetic 10-class Gaussian mixture and on CIFAR-10, CIFAR-100, and TinyImageNet using ResNet-18 classifiers. The best reported synthetic result is **ATS-CP-MAQ (MultiQR)** with average **CPSME around 0.0777**, outperforming Platt scaling, isotonic regression, temperature scaling, Dirichlet scaling, and the base model [2505.15437]. On image datasets, the reported mean **ECE \(\times 100\)** values are approximately **4.59** for ATS-CP-AQ (prob), **3.49** for ATS-CP-GQ (prob), and **12.02** for ATS-CP-MAQ (prob, CondNF), compared with **22.42** for the contaminated base model and **22.30** for temperature scaling [2505.15437].

Under unknown subpopulation mixture shift, weighted ACP methods maintain coverage close to the target across many test environments and substantially reduce coverage variance relative to unweighted conformal prediction in vision and language tasks [2506.05583]. In dynamic environments with unknown shifts, multi-model ACP extends adaptation from the conformal threshold to the model itself; SAMOCP is reported to produce smaller prediction sets than baselines, lower regret, coverage close to the target \(1-\alpha\), and a higher fraction of useful singleton prediction sets on CIFAR-10C, CIFAR-100C, TinyImageNet-C, and synthetic image-shift data [2411.03678]. In quantum machine learning, AQCP on an IBM quantum processor achieves target coverage levels and exhibits greater stability than quantum conformal prediction under time-varying hardware noise [2511.18225].

## 6. Misconceptions, boundary cases, and neighboring frameworks

A recurring misconception is that ACP is a single algorithm. The literature instead uses the term for a design space. Some methods adapt the calibration quantile to the input, some update \(\alpha_t\) online, some reweight calibration scores, some learn score transformations, and some use ACP only as one module inside a larger planning or decision system [2303.12695][2212.00278][2510.04318]. A closely related terminology also appears: one 2026 paper uses the initials **ACP** for **Audited Conformal Prediction**, a reliability-aware conformal framework for unknown distribution shift [2606.14909].

A second misconception is that adaptivity automatically yields exact conditional coverage. The papers surveyed here do not make that claim in general. Several methods retain exact finite-sample marginal coverage; others provide training-conditional, PAC-style, asymptotic conditional, long-run average, or group-conditional guarantees under additional assumptions [2303.12695][2511.18225][2606.14909]. The medical-imaging ACP paper is explicit that true distribution-free stratum-level guarantees are impossible in general without extra assumptions, and therefore frames its adaptive criterion as a practical safety heuristic [2605.12917].

A third misconception is that ACP always changes the conformal set itself. ATS-CP is explicit that its probabilities are “calibrated probabilities for a conformal set,” not a replacement for the set. It preserves the conformal coverage guarantee because it does not alter the set construction; it only redistributes probability mass over labels while keeping the conformal set intact [2505.15437]. The same paper also notes that aggregation in the \(\alpha\)-agnostic multi-quantile variants may not preserve a precise coverage interpretation as cleanly as the single-\(\alpha\) procedure [2505.15437].

Finally, ACP intersects but does not coincide with several neighboring areas. CAP, for example, is best viewed as a general framework for online selective conformal prediction with FCR control rather than as a new ACP rule itself; its core novelty is calibration after adaptive selection, with exact selection-conditional coverage and FCR control, while adaptive conformal ideas enter mainly through the distribution-shift module CAP-DtACI [2403.07728]. This suggests that ACP is increasingly functioning as an interface layer between conformal validity and broader requirements such as selective inference, online control, modular diagnostics, and legacy-model auditing.

Adaptive Conformal Prediction is therefore best understood not as a single theorem or algorithm, but as a broad methodological program: retain conformal validity at the level the setting permits, and use adaptive structure—locality, online feedback, shift awareness, or learned interval geometry—to reduce the mismatch between global guarantees and the actual heterogeneity of predictive uncertainty [2303.12695][2510.04318][2603.01719].

Source: https://www.emergentmind.com/topics/adaptive-conformal-prediction-acp