Class-Conditional Coverage: Foundations & Methods

• Class-conditional coverage is a method ensuring that predictive intervals achieve a specified coverage rate within each distinct outcome class for fairness and reliability.
• Methodological approaches like Mondrian CP, clustered CP, and quantile regression tailor calibration to handle class imbalance and deliver near exact conditional coverage.
• Recent advances provide finite-sample guarantees and efficient strategies to overcome challenges in high-dimensional, imbalanced data scenarios while maintaining robust performance.

Class-conditional coverage is a foundational concept in distribution-free uncertainty quantification, denoting the requirement that predictive sets or intervals maintain a prescribed coverage rate not only marginally over the population but also when conditioning on the event that the true outcome belongs to a particular class. In formal terms, for a prediction set-valued function $C(X)$ and a response variable $Y$ taking values in a discrete label set $\mathcal{Y}$, class-conditional coverage at level $1-\alpha$ demands $\mathbb{P}(Y\in C(X)\mid Y=c)\ge 1-\alpha$ for each $c\in\mathcal{Y}$. This property is crucial for fairness, safety, and reliability, as marginal coverage alone can conceal under- or over-coverage for individual classes—an issue that is particularly acute in domains with class imbalance, high stakes, or heterogeneity.

1. Formal Definitions and Theoretical Foundations

Let $(X,Y)\sim P$ with $X\in\mathcal{X}$ and $Y\in\mathcal{Y}$ ($|\mathcal{Y}|=K<\infty$), and let $C:\mathcal{X}\to 2^{\mathcal{Y}}$ denote a set-valued predictor. Standard (marginal) coverage requires

$\mathbb{P}(Y\in C(X))\ge 1-\alpha.$

Class-conditional coverage strengthens this to

$$\forall\,c\in\mathcal{Y}:\quad \mathbb{P}(Y\in C(X)\mid Y=c)\ge 1-\alpha.$$

This condition is equivalent to requiring that the coverage event is valid within each class slice of the population, ensuring fairness across all components of the label space (Ding et al., 2023, Bairaktari et al., 24 Feb 2025, Braun et al., 12 Dec 2025).

For regression and structured prediction, the analogous notion is group-conditional or subgroup coverage, where the group is specified by a (possibly non-exhaustive) attribute function of $X$.

Formal impossibility results (e.g., Vovk, Lei and Wasserman, Barber et al.) show that exact conditional coverage over all $X$ (or over all infinite subgroups) is impossible without further assumptions in finite samples. However, for a finite collection of groups—such as classes—one can attain exact coverage using groupwise split conformal calibration or similar constructions (Gibbs et al., 2023, Bairaktari et al., 24 Feb 2025).

2. Methodological Approaches

Several methodological regimes exist to achieve or approximate class-conditional coverage, each with characteristic trade-offs.

A. Classical Classwise (Mondrian) Conformal Prediction

• Calibrate a separate conformity threshold $q^c_{1-\alpha}$ for each class $c$ using only calibration points with $Y_i=c$.
• The prediction set is $C(x)=\{y: s(x,y)\le q^y_{1-\alpha}\}$.
• Delivers exact class-conditional coverage, i.e., $\mathbb{P}(Y\in C(X)\mid Y=c)\ge 1-\alpha$ for each $c$ (Ding et al., 2023).
• In the low-data regime (small $n_c$), this method yields conservative and inefficiently large sets.

B. Clustered Conformal Prediction

• Cluster classes with similar conformity score distributions, pool calibration data within clusters, and calibrate clusterwise thresholds $q^{(k)}_{1-\alpha}$ (Ding et al., 2023, Jaubert et al., 4 Jun 2025). If the Kolmogorov–Smirnov distance between scores of classes $y,y'$ within a cluster $k$ is $\le\epsilon$, then

$$\mathbb{P}(Y\in C(X)\mid Y=y)\ge 1-\alpha-\epsilon.$$

• Provides a systematic way to "borrow strength" in few-sample regimes, and reduces set sizes while maintaining near class-conditional validity.

C. Regression-based and Quantile Regression Techniques

• Frame the prediction as estimating class-conditional quantiles via quantile regression on calibration data, possibly in a parametric or semiparametric setting (Duchi, 28 Feb 2025, Gibbs et al., 2023, Bairaktari et al., 24 Feb 2025).
• Procedures such as Kandinsky Conformal Prediction (KCP) generalize to a finite-dimensional function class, yielding a minimax-optimal class-conditional coverage gap $O(\sqrt{K/n})$ (Bairaktari et al., 24 Feb 2025).
• Regularization is required for high-dimensional or infinite-dimensional shift classes.

D. Rank Calibrated and Label-wise Screening

• RC3P algorithm: Apply classwise thresholds but restrict to labels $y$ with sufficiently high base classifier rank for $x$, controlling coverage via classwise top-$k$ error rates (Shi et al., 2024).
• Achieves nontrivial reductions in prediction set size while maintaining exact class-conditional coverage.

E. Functional and Diagnostic Approaches

• Excess risk of the target coverage (ERT): Quantifies conditional coverage error via proper-loss classification, supporting rigorous diagnostics at both overall and class-conditional levels (Braun et al., 12 Dec 2025).

3. Theoretical Guarantees and Practical Bounds

Modern works establish both finite-sample and asymptotic guarantees for class-conditional coverage. For pure classwise split conformal, calibration delivers finite-sample exactness under exchangeability: $\Pr(Y\in C(X) \mid Y=c)\ge 1-\alpha \;\forall c.$ However, with $n_c$ calibration points per class, the coverage gap and empirical variance depend on $1/\sqrt{n_c}$. Methods that pool information across classes—such as clustered conformal (Ding et al., 2023), quantile regression (Duchi, 28 Feb 2025), or Kandinsky CP (Bairaktari et al., 24 Feb 2025)—enjoy minimax-optimal convergence $O(\sqrt{K/n})$ for the class-conditional coverage error, where $n$ is the size of the calibration set and $K$ is the number of groups.

Recent analyses of the quantile-regression-based split conformal approach (Duchi, 28 Feb 2025) show that, under mild regularity, for all $j$,

$$\Big|\Pr(Y\in C(X)\mid Y=j)- (1-\alpha)\Big|\le O\Big(\sqrt{\frac{d}{n \Pr(Y=j)}}\Big),$$

with $d$ the VC-dimension of the class of sets.

Extensions such as ERT (Braun et al., 12 Dec 2025) permit estimation of conditional miscoverage even in high-dimensional feature spaces via supervised learning proxies.

4. Empirical Performance and Practical Recommendations

Empirical studies on large-scale benchmarks (e.g., ImageNet, CIFAR-100, CivilComments) demonstrate that:

• Marginal methods (standard CP) systematically undercover rare or challenging classes.
• Mondrian/classwise CP achieves valid class-conditional coverage but with substantial inefficiency at low calibration sample sizes, leading to large or uninformative sets (Ding et al., 2023, Bairaktari et al., 24 Feb 2025).
• Clustered or pooled approaches, including KCP and RC3P, provide sharp class-conditional coverage while reducing average set size and increasing practical utility (Ding et al., 2023, Shi et al., 2024, Bairaktari et al., 24 Feb 2025).
• Calibration based on informative summaries—e.g., classifier confidence and trust scores—further improves class-conditional coverage, as demonstrated on both vision and natural language tasks (Kaur et al., 17 Jan 2025).

Practical recommendations for model selection and calibration include:

• Use standard conformal or global pooling for extremely low sample sizes per class.
• Clustered or regression-based methods are preferable in intermediate data regimes.
• For high group cardinality and data imbalance, regularization and target pooling is critical for stability.
• Diagnostics such as classwise ERT or CovGap should be used to detect residual conditional miscoverage (Braun et al., 12 Dec 2025).

5. Advanced Extensions and Generalizations

Class-conditional coverage forms the basis for further developments in distribution-free inference:

• Adaptive and Fair Coverage: Algorithms such as AFCP adaptively select sensitive attributes or groups to guarantee conditional coverage in the most undercovered strata, providing a principled fairness-efficiency trade-off (Zhou et al., 2024).
• Singleton-set Selective Calibration: Methods such as Venn-ADMIT combine Mondrian partitioning, local trust signals, and inductive Venn predictors to deliver class-conditional singleton-set calibration in selective classification with explicit coverage guarantees (Schmaltz et al., 2022).
• Structured and Regression Problems: In multivariate, structured, or regression settings, class-conditional ideas transfer as conditional or subgroup coverage; PCP extends these ideas via data-driven mixtures (Zhang et al., 2024).
• Score Transformation and Rectification: RCP algorithms achieve approximate conditional validity by learning parametric or nonparametric transformations of conformity scores, aligning conditional score quantiles across groups, with finite-sample error controlled by quantile estimation (Plassier et al., 22 Feb 2025).

6. Limitations, Diagnostics, and Open Challenges

Fundamental barriers prevent exact conditional coverage for arbitrary or infinite group classes in finite samples (Gibbs et al., 2023). Diagnostic tools—such as ERT, CovGap, and partition-based miscoverage estimation—can reveal systematic violations of classwise coverage in both synthetic and real data, with ERT outperforming discrete partitioning in sample efficiency and statistical power (Braun et al., 12 Dec 2025).

A persistent challenge is balancing coverage validity, prediction set efficiency, and computational scalability as the number of classes or group complexity increases. In high-dimensional, imbalanced, or low-data regimes, empirical regularization, cluster pooling, or mapping to lower-dimensional reliability features remain crucial. Extensions to mixed discrete-continuous conditioning and to adversarial group shifts are ongoing research frontiers (Plassier et al., 22 Feb 2025, Gibbs et al., 2023).

7. Algorithmic and Implementation Considerations

The computational cost of achieving class-conditional coverage scales with the number of groups and the complexity of the score calibration. For basic classwise methods, complexity is $O(Kn)$ for $K$ classes. Approaches using quantile regression or dual linear programming scale with the dimension of function classes, while mixture-model methods for conditional score estimation add a further multiplicative cost in mixture components (Bairaktari et al., 24 Feb 2025, Zhang et al., 2024).

A summary of representative algorithms and their class-conditional coverage properties:

Method	Coverage Guarantee	Coverage Gap Decay
Marginal Split-CP	Marginal only	$O(1/\sqrt{n})$
Classwise/Mondrian CP	Exact class-conditional (finite)	$O(1/\sqrt{n_c})$
Clustered CP	Approximate, pooled	$O(\epsilon)$, $\epsilon=$K–S dist.
Kandinsky CP	Minimax-optimal, regression	$O(\sqrt{K/n})$
RC3P	Class-conditional, filtered	$O(1/\sqrt{K/n})$
Quantile-regression split-CP	Approx. CC, efficient	$O(\sqrt{d/n})$

Empirical code, software, and practical implementations are available for many methods, including ERT metrics (covmetrics), arc for CV+/JK+ (Romano et al., 2020, Braun et al., 12 Dec 2025).

---

Class-conditional coverage remains a central metric in the theory and practice of valid uncertainty quantification, supporting robust inference, algorithmic fairness, and trustworthiness in high-stakes prediction tasks. Ongoing research continues to improve efficiency, scalability, and rigorous diagnostics for this critical property.