Adaptive Conformal Selection (ACS)

• Adaptive Conformal Selection (ACS) is a methodological family that updates critical elements such as miscoverage levels, thresholds, or model indices, while maintaining formal statistical guarantees.
• ACS encompasses various frameworks including online calibration, interactive screening for FDR control, and adaptive model selection, addressing challenges in distribution shifts and heterogeneous data.
• Key principles of ACS include online correction mechanisms, symmetry preservation, and adaptive operating-point selection to ensure robust performance under changing experimental conditions.

Adaptive Conformal Selection (ACS) denotes a family of conformal-inference and conformal-selection procedures in which some operative element of the method—such as a miscoverage level, a dual variable, a model index, a group definition, a selection set, or a certification threshold—is updated from data while a formal statistical guarantee is retained. In recent arXiv literature, the label does not refer to a single canonical algorithm. Instead, it appears across several distinct but related research programs: online ACI-style threshold adaptation under distribution shift, efficient limited-feedback subset selection, human-in-the-loop false-discovery control, adaptive model and group selection, post-hoc utility-driven conformal selection with e-values, and selective conformal risk certification (Gollapudi et al., 14 May 2026, Gui et al., 21 Jul 2025, Zhou et al., 2024, Zhu et al., 13 Apr 2026, Yu et al., 7 Jun 2026).

1. Terminological scope and problem families

Several contemporary papers use “Adaptive Conformal Selection” or “ACS” for technically different objectives. The shared feature is adaptivity under a validity constraint, but the target quantity being controlled varies substantially.

Variant	Adaptive object	Primary guarantee
Efficient online conformal selection	Dual variable $\lambda_t$	Adversarial validity; $O(\sqrt{T}\log T)$ stochastic efficiency regret
Interactive conformal selection	Screening order $\pi(\ell+1)$	Finite-sample FDR control
Adaptive equalized coverage	Selected attribute $\hat A(X_{n+1})$	Coverage conditional on adaptively chosen groups
Localized conformal model selection	Safe index set $\mathcal K_i(\gamma)$	Exact finite-sample marginal coverage
Post-hoc conformal selection	Operating point along an e-BH path	Post-hoc FDP reliability
Selective conformal risk control	Pair $(\lambda,\tau)$	Joint finite-sample certificate on risk, acceptance, and utility

In “Efficient Online Conformal Selection with Limited Feedback,” ACS is a sequential decision framework in which an agent selects a minimal subset of options so that at least one success is identified with target probability $\phi$, while minimizing cost under bandit or semi-bandit feedback (Gollapudi et al., 14 May 2026). In “ACS: An interactive framework for conformal selection,” ACS generalizes conformal selection to human-in-the-loop adaptive data analysis by controlling what information is revealed during sequential screening and thereby preserving finite-sample FDR control (Gui et al., 21 Jul 2025). In “Conformal Classification with Equalized Coverage for Adaptively Selected Groups,” the adaptive object is a feature-defined subgroup, and the guarantee is coverage conditional on the selected group (Zhou et al., 2024).

Other papers attach ACS-like functionality to model combination, localized model choice, adaptive coverage policies, or selective deployment. “Multi-model Ensemble Conformal Prediction in Dynamic Environments” describes ACS through MOCP and SAMOCP, where models and miss-coverage parameters are updated online (Hajihashemi et al., 2024). “Localized conformal model selection” uses ACS to form a safe index set of candidate regressors and an ensemble interval that retains exact marginal validity (Wang et al., 22 Feb 2026). “Adaptive Coverage Policies in Conformal Prediction” uses a neural coverage policy to choose data-dependent miscoverage levels through e-values and post-hoc conformal inference (Gauthier et al., 5 Oct 2025). “A Joint Finite-Sample Certificate for Adaptive Selective Conformal Risk Control” uses adaptive threshold selection over a finite grid to certify selected risk, acceptance probability, and deployment utility simultaneously (Yu et al., 7 Jun 2026).

This distribution of meanings suggests that ACS is best understood as a methodological family rather than a single standardized estimator.

2. Recurrent design principles

A prominent ACS motif is scalar online correction of a coverage or success error. In Adaptive Conformal Inference (ACI), the control parameter obeys

$$\theta_{t+1}=\theta_t+\eta\cdot(\mathrm{err}_t-\alpha),$$

where $\mathrm{err}_t=1_{Y_t\notin C_t(\theta_t)}$ and $\alpha$ is the target miscoverage (Gibbs et al., 2021). In efficient online conformal selection, the analogous recursion is

$O(\sqrt{T}\log T)$0

with $O(\sqrt{T}\log T)$1 indicating whether the chosen subset succeeded (Gollapudi et al., 14 May 2026). In the modular sequential-model framework, the adaptive extension updates

$O(\sqrt{T}\log T)$2

together with stage-specific quantile adjustments (Zhang et al., 6 Oct 2025). In OnlineSCI, selection-aware adaptation occurs only at selected times,

$O(\sqrt{T}\log T)$3

where $O(\sqrt{T}\log T)$4 counts prior selections (Humbert et al., 14 Aug 2025).

A second recurring principle is symmetry preservation under adaptive decisions. Interactive ACS reveals full information only for screened points and masks whether unscreened points are calibration-null or test, preserving exchangeability between unscreened test points and null calibration points (Gui et al., 21 Jul 2025). Localized conformal model selection restores symmetry by evaluating upper and lower surrogate intervals on leave-one-out calibration problems (Wang et al., 22 Feb 2026). OptCS requires a permutation-equivariant score-generating functional and a permutation-invariant auxiliary size statistic to permit data-driven model optimization without losing FDR control (Bai et al., 2024). Adaptive equalized coverage relies on leave-one-out pseudo-test constructions with placeholder labels to select sensitive attributes without peeking at the true test label (Zhou et al., 2024).

A third principle is adaptive operating-point selection under a certified path or feasible set. PH-CS builds a path of candidate e-BH selection sets, each paired with an FDP estimate, and then maximizes a user-specified utility over that path (Zhu et al., 13 Apr 2026). Adaptive coverage policies learn a neural policy $O(\sqrt{T}\log T)$5 that outputs example-specific miscoverage levels while retaining a post-hoc coverage guarantee through e-values (Gauthier et al., 5 Oct 2025). The selective conformal risk-control certificate searches a finite grid of $O(\sqrt{T}\log T)$6 pairs and selects the point maximizing a lower confidence bound on utility among certified-feasible pairs (Yu et al., 7 Jun 2026).

These mechanisms differ technically, but each addresses the same structural difficulty: adaptivity changes the object being calibrated, so validity must be preserved either through exchangeability-preserving symmetry, through explicit confidence accounting, or through online averaging arguments.

3. Sequential and online ACS

The online ACS line begins from Adaptive Conformal Inference under distribution shift. ACI models non-stationarity as a learning problem in a single parameter whose optimal value varies over time. Without assumptions on the data-generating process, Proposition 3.2 yields

$O(\sqrt{T}\log T)$7

hence the long-run average miscoverage converges almost surely to $O(\sqrt{T}\log T)$8 (Gibbs et al., 2021). Under a hidden-Markov model with spectral gap $O(\sqrt{T}\log T)$9, the same paper gives a large-deviation bound for the empirical error frequency; under a Lipschitz condition on the statewise miscoverage function, Theorem 4.2 controls squared marginal coverage error in terms of the magnitude of the underlying distribution shift and the step size (Gibbs et al., 2021).

“Efficient Online Conformal Selection with Limited Feedback” extends the ACI-style update from prediction-set calibration to resource-constrained subset selection. At round $\pi(\ell+1)$0, the agent sees a candidate set $\pi(\ell+1)$1, chooses $\pi(\ell+1)$2, observes only a binary success indicator $\pi(\ell+1)$3, and pays cost $\pi(\ell+1)$4 or $\pi(\ell+1)$5. The stated desiderata are adversarial validity—time-averaged success tracks the target $\pi(\ell+1)$6 up to $\pi(\ell+1)$7—and stochastic efficiency—under i.i.d. inputs, the total cost has $\pi(\ell+1)$8 regret, up to logarithms, against the best in-expectation policy satisfying $\pi(\ell+1)$9 (Gollapudi et al., 14 May 2026). The benchmark is

$\hat A(X_{n+1})$0

and the efficiency regret is

$\hat A(X_{n+1})$1

Under i.i.d. rewards and costs, the paper states

$\hat A(X_{n+1})$2

and the Lyapunov-function analysis based on $\hat A(X_{n+1})$3 yields the form

$\hat A(X_{n+1})$4

optimized by $\hat A(X_{n+1})$5 (Gollapudi et al., 14 May 2026).

OnlineSCI generalizes ACI further by allowing the algorithm to decide when to make an inference. At time $\hat A(X_{n+1})$6, a selection indicator $\hat A(X_{n+1})$7 is chosen using past information; only when $\hat A(X_{n+1})$8 is a prediction set output and the label observed. The method controls the False Coverage Proportion among selected points,

$\hat A(X_{n+1})$9

with the adversarial bound

$\mathcal K_i(\gamma)$0

and also studies the instantaneous error rate $\mathcal K_i(\gamma)$1 and convergence to an oracle threshold under mild conditions (Humbert et al., 14 Aug 2025).

The modular residual-decomposition framework addresses another sequential setting: two-stage models with upstream and downstream residual components. It decomposes

$\mathcal K_i(\gamma)$2

into a second-stage residual

$\mathcal K_i(\gamma)$3

and a first-stage delta

$\mathcal K_i(\gamma)$4

with $\mathcal K_i(\gamma)$5. Its adaptive extension recalibrates on sliding windows and proves almost-sure convergence of empirical coverage to $\mathcal K_i(\gamma)$6 (Zhang et al., 6 Oct 2025).

4. FDR-controlled and post-hoc selection frameworks

A separate ACS lineage is explicitly about selecting test instances while controlling false discoveries. The interactive ACS framework starts from standard conformal selection and removes the one-shot restriction. After randomly permuting labeled and test indices, it screens points sequentially. At step $\mathcal K_i(\gamma)$7, the method reveals full data only for screened points and only counts for unscreened null calibration and test points. The permitted filtration is

$\mathcal K_i(\gamma)$8

and the next screened index $\mathcal K_i(\gamma)$9 may be any $(\lambda,\tau)$0-measurable function. The estimated FDP is

$(\lambda,\tau)$1

and screening stops at

$(\lambda,\tau)$2

The selected set is the remaining unscreened test set $(\lambda,\tau)$3, and Theorem 1 establishes $(\lambda,\tau)$4 by a supermartingale argument based on

$(\lambda,\tau)$5

Because revealed labels can be used to refit models, update rankings, incorporate diversity, or add newly acquired labels, the framework directly addresses human-in-the-loop adaptive data analysis (Gui et al., 21 Jul 2025).

“Revamping Conformal Selection With Optimal Power: A Neyman--Pearson Perspective” replaces conformal $(\lambda,\tau)$6-value thresholding by a likelihood-ratio rule. Under the null $(\lambda,\tau)$7 and alternative $(\lambda,\tau)$8, the relevant densities are

$(\lambda,\tau)$9

leading to the optimal likelihood ratio

$\phi$0

Equivalently, the procedure works with

$\phi$1

and chooses a threshold $\phi$2 by requiring an empirical FDP estimate

$\phi$3

to stay below $\phi$4. The paper states asymptotic FDR control and asymptotic Neyman--Pearson optimality under correct specification, with weighted extensions for covariate shift (Qin et al., 23 Feb 2025).

PH-CS addresses another limitation of conformal selection: fixed FDR levels chosen before seeing the data. Using a monotone conformity score $\phi$5, it constructs conformal e-variables

$\phi$6

sorts them in decreasing order, defines nested sets $\phi$7, and pairs each with

$\phi$8

A user-specified utility $\phi$9 then selects an operating point on the path. The principal guarantee is post-hoc reliability:

$$\theta_{t+1}=\theta_t+\eta\cdot(\mathrm{err}_t-\alpha),$$0

and the paper gives an analogous general-risk extension PH-RCS (Zhu et al., 13 Apr 2026).

OptCS studies model optimization after substantial data reuse. It introduces a score-generating functional $$\theta_{t+1}=\theta_t+\eta\cdot(\mathrm{err}_t-\alpha),$$1 and auxiliary selection-size vector $$\theta_{t+1}=\theta_t+\eta\cdot(\mathrm{err}_t-\alpha),$$2 under permutation-equivariance and permutation-invariance conditions, then combines valid conformal $$\theta_{t+1}=\theta_t+\eta\cdot(\mathrm{err}_t-\alpha),$$3-values with a pruning-based multiple testing rule to retain finite-sample FDR control after data-driven model optimization (Bai et al., 2024).

5. Adaptive models, groups, and coverage policies

Some ACS formulations adapt not merely a scalar threshold but a structural object such as a model, subgroup, or interval decomposition. In multi-model dynamic environments, MOCP maintains model weights $$\theta_{t+1}=\theta_t+\eta\cdot(\mathrm{err}_t-\alpha),$$4 and time-varying miss-coverage parameters $$\theta_{t+1}=\theta_t+\eta\cdot(\mathrm{err}_t-\alpha),$$5 for $$\theta_{t+1}=\theta_t+\eta\cdot(\mathrm{err}_t-\alpha),$$6 pre-trained classifiers. It updates

$$\theta_{t+1}=\theta_t+\eta\cdot(\mathrm{err}_t-\alpha),$$7

with pinball loss

$$\theta_{t+1}=\theta_t+\eta\cdot(\mathrm{err}_t-\alpha),$$8

and exponentiated-weight model updates

$$\theta_{t+1}=\theta_t+\eta\cdot(\mathrm{err}_t-\alpha),$$9

SAMOCP adds a strongly-adaptive expert ensemble with expert lifetimes

$\mathrm{err}_t=1_{Y_t\notin C_t(\theta_t)}$0

and provides strongly adaptive regret $\mathrm{err}_t=1_{Y_t\notin C_t(\theta_t)}$1 over any interval and dynamic regret $\mathrm{err}_t=1_{Y_t\notin C_t(\theta_t)}$2, while maintaining valid marginal coverage under the stated assumptions (Hajihashemi et al., 2024).

Localized conformal model selection addresses local heterogeneity across the covariate space. Given fixed regressors $\mathrm{err}_t=1_{Y_t\notin C_t(\theta_t)}$3 and a localizer $\mathrm{err}_t=1_{Y_t\notin C_t(\theta_t)}$4, it defines localized conformal intervals

$\mathrm{err}_t=1_{Y_t\notin C_t(\theta_t)}$5

where $\mathrm{err}_t=1_{Y_t\notin C_t(\theta_t)}$6 is a weighted empirical quantile of localized residuals. To perform model selection symmetrically, it builds upper and lower surrogate intervals $\mathrm{err}_t=1_{Y_t\notin C_t(\theta_t)}$7 and $\mathrm{err}_t=1_{Y_t\notin C_t(\theta_t)}$8 at each calibration point and defines the safe index set

$\mathrm{err}_t=1_{Y_t\notin C_t(\theta_t)}$9

A data-dependent admissible band $\alpha$0 is then computed, and the final interval is

$\alpha$1

Theorem-level validity is exact finite-sample marginal coverage under exchangeability (Wang et al., 22 Feb 2026).

Adaptive equalized coverage in classification selects, for the test point, at most one sensitive attribute from $\alpha$2 candidates. Using leave-one-out augmented datasets with placeholder labels, it computes for each candidate label $\alpha$3 the worst-group miscoverage statistic

$\alpha$4

forms $\alpha$5 by testing whether the worst-group rate exceeds $\alpha$6, and fuses these choices via

$\alpha$7

The resulting prediction set satisfies

$\alpha$8

under exchangeability (Zhou et al., 2024).

Adaptive coverage policies formulate the adaptive object as an example-specific miscoverage level. A neural policy $\alpha$9 outputs $O(\sqrt{T}\log T)$00, trained on leave-one-out calibration episodes with a regularized size objective

$O(\sqrt{T}\log T)$01

The post-hoc conformal guarantee gives

$O(\sqrt{T}\log T)$02

and in particular

$O(\sqrt{T}\log T)$03

This permits data-dependent coverage levels without abandoning the stated marginal guarantee (Gauthier et al., 5 Oct 2025).

The modular residual-decomposition framework also belongs in this structural class because it calibrates stage-wise scaling parameters $O(\sqrt{T}\log T)$04 over a grid $O(\sqrt{T}\log T)$05, evaluates empirical miscoverage on a calibration set, computes one-sided binomial $O(\sqrt{T}\log T)$06-values, and uses FWER control to select valid parameter pairs (Zhang et al., 6 Oct 2025).

6. Guarantees, empirical domains, and limitations

ACS papers do not share one universal guarantee. The online ACI-derived line emphasizes long-run calibration under arbitrary sequences, adversarial validity on averages, or sublinear efficiency regret under i.i.d. inputs (Gibbs et al., 2021, Gollapudi et al., 14 May 2026, Humbert et al., 14 Aug 2025). The conformal-selection line emphasizes finite-sample FDR control or post-hoc FDP reliability for selected test points (Gui et al., 21 Jul 2025, Zhu et al., 13 Apr 2026, Bai et al., 2024). Group- and model-adaptive variants emphasize exact marginal validity, adaptive equalized coverage, or risk-controlled certification (Wang et al., 22 Feb 2026, Zhou et al., 2024, Yu et al., 7 Jun 2026). A common misconception is therefore that “ACS” implies a single coverage notion; in the literature, the controlled quantity may instead be average miscoverage, success rate, FDR, selected risk, acceptance probability, or deployment utility.

Empirical evaluations are correspondingly heterogeneous. ACI was tested on stock-market volatility and the 2020 U.S. Presidential election night, where adaptive recalibration maintained coverage near the nominal level while static conformal methods degraded under visible shifts (Gibbs et al., 2021). MOCP and SAMOCP were evaluated on CIFAR-10C, CIFAR-100C, TinyImageNet-C, and synthetic image datasets under sudden and gradual shifts, with reported improvements in average width and adaptive regret while maintaining coverage (Hajihashemi et al., 2024). The residual-decomposition framework reports experiments on synthetic distribution shifts and real-world supply chain and stock market data, emphasizing stage-wise uncertainty attribution (Zhang et al., 6 Oct 2025). Interactive ACS was demonstrated in LLM deployment and drug discovery, including diversity-aware virtual screening and incorporation of newly labeled data (Gui et al., 21 Jul 2025). PH-CS studies synthetic and real datasets such as Recruitment, Musk, and Shuttle, emphasizing user-chosen operating points and reliable FDP estimates (Zhu et al., 13 Apr 2026). The selective risk-control certificate reports ImageNet, COCO val 2017 panoptic, and ADE20K, and explicitly notes that its gains are regime-scoped, not universal, and absent on ADE20K (Yu et al., 7 Jun 2026).

Several limitations recur. Many variants rely on exchangeability or i.i.d. assumptions for finite-sample guarantees, even when the motivating goal is adaptivity (Zhou et al., 2024, Wang et al., 22 Feb 2026, Zhu et al., 13 Apr 2026). Some selection methods provide only marginal rather than group-conditional or instance-conditional guarantees (Gauthier et al., 5 Oct 2025). The Neyman--Pearson conformal-selection line is asymptotically optimal under correct specification but discusses modified procedures for model misspecification and weighted variants for covariate shift, indicating that optimality and robustness need not coincide (Qin et al., 23 Feb 2025). Computational cost can also be substantial: leave-one-out training appears in adaptive coverage policies and adaptive equalized coverage, while expert ensembles or grid-based certification enlarge the state space in dynamic or selective settings (Gauthier et al., 5 Oct 2025, Zhou et al., 2024, Hajihashemi et al., 2024, Yu et al., 7 Jun 2026).

Taken together, the ACS literature shows a consistent methodological agenda: preserve a formal uncertainty or error-control guarantee while allowing the procedure to react to data, distribution shifts, user utilities, or structural heterogeneity. The specific implementation—and the meaning of “selection”—depends on whether the task is online calibration, subset probing, FDR-controlled discovery, model choice, group-conditioned fairness, or selective deployment.