Adversarially Robust Conformal Prediction

• Adversarially robust conformal prediction extends classical CP by ensuring valid coverage under adversarial perturbations through techniques like quantile inflation and randomized smoothing.
• Key methodologies such as two-stage quantile procedures, Lipschitz-based score inflation, and neural network verification balance conservativeness with informativeness.
• Empirical evaluations reveal that these methods improve prediction set efficiency while providing rigorous coverage guarantees amid adversarial threats.

Adversarially robust conformal prediction (ARCP) generalizes the core principle of conformal prediction—distribution-free coverage guarantees for predictive sets—to settings where test inputs are subject to adversarial or otherwise uncertain perturbations that typically break the exchangeability hypothesis. Recent advances span worst-case robustness, probabilistic robustness, randomized smoothing, verification-driven certificates, training-time defenses, federated robustness, group-conditional guarantees, and efficient binarized and quantile-of-quantile procedures. This field addresses both the preservation of marginal coverage and the reduction of set size (efficiency), with trade-offs between conservativeness and informativeness driven by the assumed threat model.

1. Core Principles of Adversarially Robust Conformal Prediction

Classical conformal prediction operates on i.i.d. samples $(X_i, Y_i)$ with a nonconformity score $S(X, y)$, yielding prediction sets $\mathcal{C}(X) = \{y : S(X, y) \leq \tau(\alpha)\}$, where $\tau(\alpha)$ is the empirical $(1-\alpha)$-quantile among the calibration scores. The marginal guarantee is $$\mathbb{P}_{(X,Y)\sim\mathcal{D}}(Y \in \mathcal{C}(X)) \geq 1 - \alpha$$.

Under adversarial perturbations, e.g., $X' = X + \delta, \|\delta\| \le r$, the nominal coverage fails. ARCP aims to recover the guarantee by (1) digitally inflating the quantile threshold (worst-case), (2) certifying robustness under a randomized or structured noise model (probabilistic), or (3) bounding nonconformity score changes via model properties (e.g., Lipschitz constants or neural network verification) (Ghosh et al., 2023, Jeary et al., 2024, Yan et al., 2024, Massena et al., 5 Jun 2025).

Coverage requirements take two canonical forms:

• Worst-case: $$\inf_{\|\delta\|\le r} \mathbb{P}_{(X,Y)}(Y \in \mathcal{C}(X+\delta)) \geq 1-\alpha$$,
• Probabilistic/average-case: $$\mathbb{P}_{(X,Y),\delta \sim \Gamma}(Y \in \mathcal{C}(X+\delta)) \geq 1-\alpha$$.

2. Algorithmic Frameworks and Methodologies

2.1 Quantile Inflation and Two-stage Quantile Procedures

The inflation principle stipulates that, if $S(X+\delta, y) \leq S(X, y) + M_r$ for all $S(X, y)$0, then setting $S(X, y)$1 yields a robust prediction set (Ghosh et al., 2023). The aPRCP (adaptive probabilistically robust conformal prediction) algorithm introduces a two-quantile approach:

• For each calibration point, sample multiple perturbations, compute local quantiles ("perturbation quantile" at $S(X, y)$2),
• Aggregate those local quantiles into a global threshold ("data quantile" at $S(X, y)$3 with slack parameter $S(X, y)$4),
• Form prediction sets that require $S(X, y)$5 to be below the global threshold for most sampled $S(X, y)$6.

This enables a trade-off between worst-case conservativeness and the practical informativeness of the resulting sets, with theoretical guarantees up to finite-sample error $S(X, y)$7 (Ghosh et al., 2023).

2.2 Randomized Smoothing and Single-Certificate Methods

Randomized smoothing leverages the stability of the smoothed score $S(X, y)$8 under perturbation. Classical RSCP (Randomized Smoothed Conformal Prediction) inflates quantiles by $S(X, y)$9 and applies concentration bounds (Hoeffding, Bernstein) to correct for Monte Carlo estimation error (Yan et al., 2024).

Recent single-certificate approaches (RCP1, BinCP) show that robustness can be certified by calibrating on single noise-augmented samples, or by binarizing smoothed samples (i.e., treating $\mathcal{C}(X) = \{y : S(X, y) \leq \tau(\alpha)\}$0 as a Bernoulli variable) and applying one global certificate per run (Zargarbashi et al., 19 Jun 2025, Zargarbashi et al., 7 Mar 2025). These drastically reduce computational overhead and set size compared to per-point, per-label MC certificate calls.

2.3 Verification-based and Lipschitz-bounded Network Methods

VRCP extends ARCP frameworks to arbitrary norm-bounded perturbation sets and regression tasks by utilizing neural network verification (e.g., CROWN, α-CROWN, IBP) to compute or conservatively bound $\mathcal{C}(X) = \{y : S(X, y) \leq \tau(\alpha)\}$1 (Jeary et al., 2024). Prediction sets are then constructed using robustified scores, ensuring coverage under any admissible perturbation.

Lip-RCP leverages models with explicit Lipschitz bounds to estimate score inflation efficiently. If $\mathcal{C}(X) = \{y : S(X, y) \leq \tau(\alpha)\}$2 is $\mathcal{C}(X) = \{y : S(X, y) \leq \tau(\alpha)\}$3-Lipschitz and $\mathcal{C}(X) = \{y : S(X, y) \leq \tau(\alpha)\}$4 is $\mathcal{C}(X) = \{y : S(X, y) \leq \tau(\alpha)\}$5-Lipschitz, then for any $\mathcal{C}(X) = \{y : S(X, y) \leq \tau(\alpha)\}$6,

$\mathcal{C}(X) = \{y : S(X, y) \leq \tau(\alpha)\}$7

and robust sets are constructed by shifting quantiles correspondingly (Massena et al., 5 Jun 2025). This yields essentially vanilla-CP-level complexity but provable robustness for $\mathcal{C}(X) = \{y : S(X, y) \leq \tau(\alpha)\}$8 attacks.

2.4 Adversarial Training and Size-aware Attacks

Robustness also benefits from integrating adversarial training with conformal loss objectives. The OPSA/OPSA-AT framework attacks the set size directly (rather than just accuracy), optimizing for maximum uncertainty (largest conformal set) via a bilevel min-max paradigm (Bao et al., 9 Jun 2025). Beta-weighted and entropy-minimized adversarial training (AT-UR, TRADES-UR, MART-UR) further reduce the population prediction set size while maintaining nominal coverage, with theoretical bounds linking weighted cross-entropy to expected set size (Liu et al., 2024).

3. Robustness under Diverse Adversarial Scenarios

ARCP frameworks have been extended to address several robustness settings:

• Policy-induced distribution shift: In interactive control environments, episodic ARCP quantifies the policy-induced shift in environment distribution, using sensitivity analysis and iterative quantile inflation to maintain safety guarantees under evolving policies (Mirzaeedodangeh et al., 13 Nov 2025).
• Calibration- and label-time poisoning: CAS-type bounds support both evasion (test-time) and poisoning (calibration-time) attacks, including discrete data manipulation, with worst-case meta-quantile optimization via MILP (Zargarbashi et al., 2024).
• Federated and Byzantine-robust CP: Rob-FCP identifies and excludes malicious clients by histogram clustering, securing the quantile estimation and thus coverage against coordinated adversaries in federated calibration (Kang et al., 2024).
• Group-conditional and multivalid coverage: MVP yields threshold-calibrated and multigroup coverage guarantees under adversarial ordering, group-dependent conditional coverage, and non-exchangeable prediction scenarios (Bastani et al., 2022).

4. Empirical Evaluation, Efficiency, and Trade-offs

Empirical comparisons across large benchmarks (CIFAR-10/100, ImageNet, TinyImageNet, MedMNIST, PathMNIST, AwA2) and multiple threat models highlight key findings:

• Worst-case ARCP (e.g., RSCP, VRCP) maintains coverage but often at significant set-size inflation, sometimes up to the full label set (trivial prediction) for strong attacks.
• Probabilistic, two-quantile, and smoothing-based approaches (aPRCP, CAS, RCP1, BinCP) offer set-size reductions of 2x–10x compared to prior robust CP methods, with negligible empirical coverage loss (Ghosh et al., 2023, Yan et al., 2024, Jeary et al., 2024, Zargarbashi et al., 19 Jun 2025, Zargarbashi et al., 7 Mar 2025, Zargarbashi et al., 2024).
• Training-time defenses (OPSA-AT, uncertainty-reducing AT) outperform PGD, TRADES, and vanilla adversarial training on set efficiency under calibration and test-time attacks (Bao et al., 9 Jun 2025, Liu et al., 2024).
• Reasoning-enhanced frameworks (COLEP) further tighten coverage (up to +14% certified coverage improvement) and reduce set size via logical correction and exact marginalization using probabilistic circuits (Kang et al., 2024).
• Federated and adversarial multivalid approaches show near-nominal coverage restoration and competitive efficiency by robust filtering and group-wise threshold calibration (Bastani et al., 2022, Kang et al., 2024).
• Complexity analyses indicate that single-sample, binarized, and Lipschitz-based ARCP methods are tractable at ImageNet scale and compatible with real-time deployment (Massena et al., 5 Jun 2025, Zargarbashi et al., 7 Mar 2025, Zargarbashi et al., 19 Jun 2025).

5. Mathematical Guarantees and Theoretical Insights

Robust coverage theorems for ARCP establish that prediction sets constructed via inflation, two-quantile, or verification methods maintain the canonical $\mathcal{C}(X) = \{y : S(X, y) \leq \tau(\alpha)\}$9 guarantee under adversarial conditions, either worst-case or in expectation over the threat distribution. These guarantees hold by reduction to exchangeability after quantile adjustment, monotonicity of the quantile function, and Lipschitz or verifier-based bounding (Ghosh et al., 2023, Jeary et al., 2024, Massena et al., 5 Jun 2025).

Quantile shift analysis in adversarial calibration reveals monotonic relationships between calibration-, test-time attack strengths and empirical coverage, with theoretical tolerance bands derivable for controlled selection of calibration attack magnitude (Qian et al., 23 Nov 2025). Beta-weighted losses provide upper bounds on expected set size, guiding training toward efficient certifiable uncertainty (Liu et al., 2024).

6. Limitations, Open Problems, and Application Domains

Limitations include worst-case conservativeness (inflated trivial sets), computational cost for verification on deep models, coverage degradation under severe covariate shift, and reliance on a correctly specified threat model. Open directions include extending ARCP to structured prediction (e.g., segmentation), more general perturbation norms (e.g., Wasserstein), hybrid stochastic-certification/verification, automated extraction of knowledge rules for reasoning-based ARCP, and DP-robust federated calibration (Kang et al., 2024, Kang et al., 2024, Jeary et al., 2024, Zargarbashi et al., 2024).

ARCP methods are of direct relevance in:

• Safety-critical perception for autonomous vehicles and medical imaging (Bao et al., 9 Jun 2025, Luo et al., 2024, Mirzaeedodangeh et al., 13 Nov 2025),
• Decentralized/federated learning in privacy-constrained, adversary-exposed environments (Kang et al., 2024),
• Interactive planning and multi-agent/online control (Mirzaeedodangeh et al., 13 Nov 2025),
• Group fairness and adaptive distribution shift settings (Bastani et al., 2022).

7. Summary Table: Key ARCP Approaches

Method	Robustness Model	Efficiency Gains	Coverage Guarantee	Reference
aPRCP	Probabilistic, 2-quantile	20–30% vs RSCP	Probabilistic O(1/√n)	(Ghosh et al., 2023)
OPSA-AT	Adversarial training & attack	Smallest sets under attack	Maintains $\tau(\alpha)$0	(Bao et al., 9 Jun 2025)
RCP1, BinCP	Smoothing, single certificate	10–20x runtime speedup, 2–5x smaller sets	Worst-case, finite-sample	(Zargarbashi et al., 19 Jun 2025, Zargarbashi et al., 7 Mar 2025)
VRCP	NN verification (all norms)	2–4x smaller sets than RSCP	Worst-case, distribution-free	(Jeary et al., 2024)
CAS	CDF-aware smoothing	Tighter/smaller sets than RSCP	Evasion/poisoning robust	(Zargarbashi et al., 2024)
COLEP	Probabilistic circuits + reasoning	Up to +14% coverage	Worst-case certified	(Kang et al., 2024)
MVP	Online, group/multivalid	10–20% narrower intervals	Adversarial, group-conditional	(Bastani et al., 2022)
Rob-FCP	Federated, Byzantine-robust	Recovers benign efficiency	Coverage under arbitrary adversaries	(Kang et al., 2024)
lip-rcp	Lipschitz-bound networks	Vanilla CP-level cost	Robust for all $\tau(\alpha)$1	(Massena et al., 5 Jun 2025)

In summary, adversarially robust conformal prediction is a rapidly developing area that rigorously addresses uncertainty quantification, set efficiency, and robustness under adversarial data manipulations. Diverse algorithmic paradigms—quantile inflation, smoothing, binarization, verification, training-time minimax, learning-reasoning integration, group conditioning, and federated defenses—deliver provable guarantees and practical efficiency. The continuing evolution targets broad perturbation models, efficiency at large scale, and principled coverage in real-world, safety-critical learning systems.