Differentially Private Conformal Prediction

• DPCP is a framework that combines conformal prediction with differential privacy to produce valid prediction sets while safeguarding calibration data.
• It employs methods such as DP quantile mechanisms, randomized response, and noisy binary search to mitigate information leakage in calibration.
• Recent advancements include centralized, local, federated, and full-data strategies that balance privacy noise with coverage fidelity and prediction set efficiency.

Differentially Private Conformal Prediction (DPCP) denotes a family of methods that combine conformal prediction (CP) with differential privacy (DP) so that prediction sets retain formal uncertainty guarantees while limiting leakage about the data used for calibration, training, or both. In standard score-based CP, one computes calibration scores $S_i=S(X_i,Y_i)$, selects a threshold $q$, and outputs $C_q(x)=\{y\in\mathcal Y:S(x,y)\le q\}$; under exchangeability, this yields finite-sample marginal coverage at level $1-\alpha$. DPCP arises because the calibration quantile, the calibration scores, and even the final prediction sets can reveal information about individuals, and a privately trained model by itself does not make a subsequent non-private conformal calibration step private (Angelopoulos et al., 2021). The literature now spans centralized DP calibration, local DP with untrusted aggregators, one-shot and federated variants, full-data non-splitting procedures that exploit DP-induced stability, and e-value-based formulations of private conformal inference (Penso et al., 21 May 2025).

1. Foundations and problem formulation

The canonical CP construction uses a fitted predictor $\hat f$, a score function $S$, and a calibration sample $\{(X_i,Y_i)\}_{i=1}^n$. In regression, scores are often residual magnitudes such as $S(x,y)=|y-\hat f(x)|$; in classification, a standard choice is $S(x,y)=1-\hat f(x)_y$. Split CP calibrates the empirical $(1-\alpha)$-quantile of the calibration scores and then returns the set of candidate labels or responses whose score is below that threshold. Under exchangeability, this gives the usual finite-sample marginal guarantee $q$0, with the familiar $q$1 upper slack when scores are distinct (Angelopoulos et al., 2021).

The privacy problem is concentrated in calibration. Standard conformal thresholds are deterministic or randomized functions of the calibration set, and releasing the threshold or the resulting set-valued predictor can leak information about the points that determine the order statistic around the conformal quantile. One explicit conclusion in the literature is that a DP-trained base learner and a non-private conformal calibration stage do not compose into a private prediction-set pipeline; the calibration step itself must be privatized (Angelopoulos et al., 2021).

DPCP therefore asks for procedures that preserve a DP guarantee with respect to the calibration data, or end-to-end with respect to both training and calibration data, while maintaining coverage as close as possible to the nominal $q$2. Across the literature, the main technical object is a private surrogate for the conformal threshold: a DP quantile of scores, a noise-aware correction of a noisy score distribution, a private exchangeability e-value, or a full-data threshold justified by DP stability rather than sample splitting.

2. Centralized DP calibration and private quantile mechanisms

The central-DP line begins with split conformal calibration in which the only privatized object is the calibration quantile. A representative construction discretizes scores in $q$3, defines a utility $q$4 for each candidate bin edge $q$5, and applies the exponential mechanism to sample a threshold $q$6. To maintain coverage, the method uses an inflated target quantile level

$q$7

where $q$8 is the number of bins and $q$9 is a tuning parameter. This yields an $C_q(x)=\{y\in\mathcal Y:S(x,y)\le q\}$0-DP calibration algorithm and a finite-sample, distribution-free lower bound $C_q(x)=\{y\in\mathcal Y:S(x,y)\le q\}$1; under a continuous bounded density and a suitable bin choice, the excess coverage above nominal is $C_q(x)=\{y\in\mathcal Y:S(x,y)\le q\}$2 (Angelopoulos et al., 2021).

A later central-DP alternative, P-COQS, replaces exponential-mechanism quantiles with a randomized binary search over the score interval using noisy range counts under $C_q(x)=\{y\in\mathcal Y:S(x,y)\le q\}$3-zCDP. If $C_q(x)=\{y\in\mathcal Y:S(x,y)\le q\}$4 binary-search iterations are used on $C_q(x)=\{y\in\mathcal Y:S(x,y)\le q\}$5, each noisy count is Gaussian with variance $C_q(x)=\{y\in\mathcal Y:S(x,y)\le q\}$6, and the resulting private threshold has rank error

$C_q(x)=\{y\in\mathcal Y:S(x,y)\le q\}$7

with probability at least $C_q(x)=\{y\in\mathcal Y:S(x,y)\le q\}$8. The consequent coverage interval is

$C_q(x)=\{y\in\mathcal Y:S(x,y)\le q\}$9

Unlike the inflation-based exponential-mechanism approach, this method accepts a finite-sample possibility of slight under-coverage in exchange for smaller prediction sets; the empirical comparison in the paper reports smaller sets and favorable coverage relative to the existing DP alternative on CIFAR-10, ImageNet, and CoronaHack (Romanus et al., 15 Jul 2025).

These two centralized constructions exemplify a basic division within DPCP. One branch enforces one-sided conservativeness by biasing the private threshold upward. The other branch directly privatizes the target quantile and accepts an explicit, quantifiable finite-sample coverage error. This suggests a recurring design choice in DPCP: whether privacy noise should be absorbed as guaranteed over-coverage or permitted to induce controlled two-sided deviation from $1-\alpha$0.

3. Local differential privacy and untrusted aggregation

Local-DP conformal prediction moves privacy protection from a trusted curator to the data holders themselves. In this model, each user randomizes private information before communicating with an untrusted server, and the aggregator must calibrate conformal sets using only privatized labels, privatized score comparisons, or similarly perturbed messages (Penso et al., 21 May 2025).

One local-DP method, LDP-CP-L, uses $1-\alpha$1-ary randomized response on class labels. If $1-\alpha$2, the privatized label $1-\alpha$3 follows a uniform-flip channel with parameter

$1-\alpha$4

The server then estimates the clean score cdf $1-\alpha$5 from the noisy-label cdf $1-\alpha$6 and the random-label cdf $1-\alpha$7 through

$1-\alpha$8

A binary search over $1-\alpha$9 yields a threshold computed entirely from noisy data, and the resulting coverage deviation is controlled by $\hat f$0 with $\hat f$1. The associated sample complexity is $\hat f$2, showing explicit degradation with stronger privacy and larger class count $\hat f$3 (Penso et al., 21 May 2025).

A complementary local-DP method, LDP-CP-S, assumes users can evaluate the model locally and compute their own conformity scores. The server conducts a binary search over score thresholds, while each user privately responds to exactly one query $\hat f$4 via binary randomized response. Disjoint user subsets eliminate per-user composition across iterations. The resulting quantile estimator is unbiased after linear inversion, and the sample complexity bound is

$\hat f$5

with $\hat f$6. A notable feature of this bound is that it is independent of the number of classes $\hat f$7, unlike the label-randomization method (Penso et al., 21 May 2025).

Within DPCP, the local model is particularly important for settings in which the aggregator is explicitly untrusted, such as sensitive medical imaging or LLM queries. The literature distinguishes two privacy regimes here: one protects labels while still exposing features to the server, and the other hides both features and labels but requires local score computation.

4. Federated, one-shot, and label-shift-aware variants

A distributed branch of DPCP studies calibration when data remain decentralized across agents or clients. In one-shot federated conformal prediction, each agent $\hat f$8 holds a local score sample $\hat f$9, computes a local order statistic $S$0, and sends a single scalar to the server. The server then forms a quantile-of-quantiles threshold

$S$1

The exact coverage of the resulting prediction set is characterized by a closed-form combinatorial function $S$2, allowing the server to choose $S$3 so that coverage is at least $S$4. The locally private extension, FedCP$S$5-QQ, replaces each raw local quantile with an $S$6-LDP approximate quantile computed by the exponential mechanism over score bins and adds a correction

$S$7

to preserve the marginal lower bound $S$8 (Humbert et al., 2023).

A distinct federated line addresses label shift across clients. In this setting, client $S$9 has distribution $\{(X_i,Y_i)\}_{i=1}^n$0, while the target client has a different label prior. The method estimates importance weights

$\{(X_i,Y_i)\}_{i=1}^n$1

from private label counts, defines weighted conformal score measures, and turns quantile estimation into a smooth scalar optimization problem by applying the Moreau envelope to the pinball loss. The resulting quantile is learned by DP-FedAvgQE, where each selected client performs $\{(X_i,Y_i)\}_{i=1}^n$2 noisy local gradient steps and the server aggregates weighted updates. The privacy analysis gives an $\{(X_i,Y_i)\}_{i=1}^n$3-DP guarantee toward a third party, while the coverage analysis combines label-shift approximation error, optimization error, and DP noise into explicit non-asymptotic bounds (Plassier et al., 2023).

These federated constructions show that DPCP is not restricted to centralized calibration. One-shot protocols minimize communication, while federated optimization permits more elaborate shift-corrected calibration objectives. The common constraint is that only compressed or privatized client-side statistics are shared.

5. Full-data and non-splitting private conformal prediction

A major recent development is the attempt to avoid the efficiency loss induced by data splitting. One framework, DP-SCP, trains a single DP model on all $\{(X_i,Y_i)\}_{i=1}^n$4 points, computes in-sample scores $\{(X_i,Y_i)\}_{i=1}^n$5, and then calibrates using a conservative private quantile obtained by a buffered right-endpoint binary search on noisy counts. The paper proves a general limitation first: a generic DP guarantee by itself yields only a universal coverage floor $\{(X_i,Y_i)\}_{i=1}^n$6, and this floor cannot in general recover the nominal $\{(X_i,Y_i)\}_{i=1}^n$7 level. It then shows that mechanism-specific stability changes the picture. Under a stability condition $\{(X_i,Y_i)\}_{i=1}^n$8, Lipschitz scores, and an anti-concentration condition around the oracle quantile, the finite-sample bound becomes

$\{(X_i,Y_i)\}_{i=1}^n$9

and the paper derives $S(x,y)=|y-\hat f(x)|$0, $S(x,y)=|y-\hat f(x)|$1 for DP-SGD, implying asymptotic recovery of nominal coverage (Cho et al., 8 Mar 2026).

A closely related but distinct formulation introduces differential CP as a non-splitting, DP-stability-based procedure and then builds a fully private DPCP pipeline on top of it. Differential CP itself uses the full dataset for both model fitting and calibration with a corrected level $S(x,y)=|y-\hat f(x)|$2, but it is not fully private because the score computation and quantile extraction are not yet privatized. The fully private DPCP construction then composes a DP training algorithm with privacy $S(x,y)=|y-\hat f(x)|$3 and a private quantile mechanism with privacy $S(x,y)=|y-\hat f(x)|$4, yielding end-to-end $S(x,y)=|y-\hat f(x)|$5-DP. Under Assumptions 4.1 and 4.2 in the paper, the resulting prediction sets satisfy exact marginal and training-conditional coverage at level $S(x,y)=|y-\hat f(x)|$6, and the efficiency analysis shows tighter sets than private split conformal approaches under the same privacy budget (Wu et al., 16 Apr 2026).

This full-data line reframes DP as more than a constraint on information release. In these analyses, DP also induces model stability, which becomes a tool for relating in-sample and out-of-sample nonconformity scores. That connection is the core justification for non-splitting private conformal procedures.

6. E-values and private conformal e-prediction

Another formulation of DPCP replaces private quantiles with private e-values. In conformal e-prediction, for calibration scores $S(x,y)=|y-\hat f(x)|$7 and a candidate test score $S(x,y)=|y-\hat f(x)|$8, a standard exchangeability e-value is

$S(x,y)=|y-\hat f(x)|$9

and the conformal e-prediction set is

$S(x,y)=1-\hat f(x)_y$0

To privatize this object, the paper introduces a biased multiplicative-noise mechanism

$S(x,y)=1-\hat f(x)_y$1

where the positive bias in $S(x,y)=1-\hat f(x)_y$2 is necessary to preserve the e-value condition $S(x,y)=1-\hat f(x)_y$3 under the null. Biased Gaussian and biased Laplace versions give RDP guarantees for the released e-values while keeping them statistically valid (Csillag et al., 21 Oct 2025).

For conformal e-prediction, if scores lie in $S(x,y)=1-\hat f(x)_y$4 with $S(x,y)=1-\hat f(x)_y$5, the log-sensitivity obeys

$S(x,y)=1-\hat f(x)_y$6

The paper then privatizes the exchangeability e-values for finitely many quantized score levels and uses them to form DP conformal e-prediction sets. A central theoretical conclusion is that the private e-values are asymptotically as powerful as their non-private counterparts, since the privacy penalty in expected log-growth is $S(x,y)=1-\hat f(x)_y$7 and therefore vanishes with sample size (Csillag et al., 21 Oct 2025).

This strand broadens the scope of DPCP. Rather than calibrating a private score quantile, it calibrates private evidence against non-exchangeability, and it inherits e-value features such as post-hoc validity and optional continuation.

7. Trade-offs, applications, and open problems

Across the literature, the dominant trade-off is between privacy strength, coverage fidelity, and prediction-set size. Stronger privacy increases randomization or noise, which either enlarges conservative prediction sets, as in quantile-inflated central or federated mechanisms, or produces a finite-sample two-sided coverage deviation, as in binary-search-based quantile search. Larger calibration samples reduce these effects, and several methods recover near-nominal or asymptotically nominal coverage as $S(x,y)=1-\hat f(x)_y$8 grows (Angelopoulos et al., 2021, Romanus et al., 15 Jul 2025, Cho et al., 8 Mar 2026).

The structural dependence on the task also varies by mechanism. Under local DP, label-randomization methods degrade with the number of classes $S(x,y)=1-\hat f(x)_y$9, while score-randomization methods have sample complexity independent of $(1-\alpha)$0. In one-shot federated LDP, the correction term for private local quantiles grows with the number of bins and becomes conservative for small $(1-\alpha)$1. In full-data DP methods, the main gain over split-based private CP comes from using all points for both fitting and calibration, thereby reducing both model error and calibration noise (Penso et al., 21 May 2025, Humbert et al., 2023, Wu et al., 16 Apr 2026).

Empirical studies have been reported on CIFAR-10, ImageNet, CoronaHack, BloodMNIST, California Housing, multiple medical imaging datasets including TissueMNIST, OrganMNIST variants, and OCTMNIST, as well as phishing detection and several regression benchmarks. The reported patterns are consistent: central private calibration often adds much less inefficiency than private model training; one-shot and federated methods can match centralized coverage closely when client sample sizes are adequate; local-DP constructions remain viable under substantial randomization; and full-data private methods generally produce sharper sets than split-based private baselines under matched privacy budgets (Angelopoulos et al., 2021, Penso et al., 21 May 2025, Cho et al., 8 Mar 2026, Csillag et al., 21 Oct 2025).

Several limitations recur. Most guarantees are marginal rather than conditional. Exchangeability or i.i.d. assumptions remain standard, with only specialized handling of label shift in the federated literature. Local-DP label perturbation does not protect features in its simpler variant, and small calibration sets can lead to noticeable under-coverage in approximate quantile-search methods. Some papers also identify open directions including regression under local DP, tighter DP corrections in one-shot federated calibration, robustness beyond label shift, and extensions of full-data stability arguments to broader private training pipelines (Penso et al., 21 May 2025, Humbert et al., 2023, Plassier et al., 2023, Romanus et al., 15 Jul 2025).

A plausible implication is that DPCP is best understood not as a single algorithmic template but as a design space organized around where privacy is enforced and how the conformal threshold is represented. Centralized methods privatize a calibration quantile; local methods privatize labels or score comparisons before aggregation; federated methods privatize client summaries or gradients; full-data methods use DP-induced stability to avoid splitting; and e-value methods privatize exchangeability evidence rather than quantiles. Together these approaches define the current technical landscape of differentially private uncertainty sets in conformal inference.