Rank Conditional Coverage (RCC)

• Rank Conditional Coverage (RCC) is defined as the probability that the true parameter lies within a constructed confidence interval given its empirical rank.
• RCC addresses the under-coverage of conventional marginal intervals by building rank-specific intervals that maintain nominal coverage for extreme estimates.
• Bootstrap methods, both parametric and non-parametric, are used to estimate bias distributions, making RCC robust for high-dimensional inference and predictive applications.

Rank Conditional Coverage (RCC) is a statistical framework for evaluating and constructing confidence intervals in the context of large-scale inference, with a particular focus on the coverage properties conditional on the empirical ranking of parameter estimates or conformity scores. RCC provides an explicit answer to the well-documented failure of marginal confidence intervals to maintain nominal coverage rates at the ranks of most scientific interest—that is, for the most extreme or “significant” estimates. Recent developments extend RCC to predictive set construction via rectified conformal prediction. The RCC concept addresses high-dimensional problems in which multiple parameters, tests, or predictions must be jointly analyzed, and selection or reporting bias poses a major challenge to reliable inference (Morrison et al., 2017, Plassier et al., 22 Feb 2025).

1. Formal Definition of Rank Conditional Coverage

Let $\theta_1, \dots, \theta_p$ be parameters of interest, with point estimates $\hat{\theta}_1, \dots, \hat{\theta}_p$. Estimates are ranked by significance (e.g., by absolute $t$-statistic), with $s(i)$ denoting the index of the $i$-th most significant parameter so that $$|\hat{\theta}_{s(1)}| \ge |\hat{\theta}_{s(2)}| \ge \cdots \ge |\hat{\theta}_{s(p)}|$$. The Rank Conditional Coverage at rank $i$ is

$$\mathrm{RCC}(i) \equiv \mathbb{P}\big[\theta_{s(i)} \in \mathrm{CI}_{s(i)}\big]$$

where $\mathrm{CI}_{s(i)}$ is the confidence set or interval for the $i$-th ranked estimate. RCC can be equivalently expressed as

$$\mathrm{RCC}(i) = \sum_{j=1}^p \mathbb{P}\big[\theta_j \in \mathrm{CI}_j \mid s(i)=j\big]\,\mathbb{P}[s(i)=j].$$

RCC(i) thus gives the expected coverage rate specifically at rank $i$ over repeated sampling (Morrison et al., 2017).

2. Motivations: Marginal Coverage Failure and the Superiority of RCC

Conventional $1-\alpha$ marginal confidence intervals are designed so that

$$\frac{1}{p}\sum_{j=1}^p \mathbb{P}(\theta_j \in \mathrm{CI}_j) = 1-\alpha$$

but this marginal guarantee masks a pronounced under-coverage for the most extreme (top-ranked) estimates and over-coverage for typical or median ones. When scientific or reporting interest is focused on the top $k$ estimates—e.g., in biomarker discovery or variable selection—the realized coverage among those selected ranks may fall well below $1-\alpha$.

Selection-adjusted procedures (e.g., False Coverage-Statement Rate (FCR) control) address average coverage among selected parameters, typically by inflating all intervals, but still produce substantial undercoverage among the most extreme ranks. By shifting the criterion to RCC(i) for all $i$, procedures can guarantee, rank-by-rank, that the observed coverage matches the nominal level (Morrison et al., 2017).

3. Construction of RCC-Controlled Intervals

The central methodological innovation behind RCC is to build intervals that achieve asymptotic $1-\alpha$ coverage at each rank. This is operationalized via the estimation of the rank-specific bias distribution:

$$\delta_{[i]} \equiv \hat{\theta}_{s(i)} - \theta_{s(i)}$$

Denoting its cumulative distribution by $H_{[i]}$, the (oracle) RCC-exact interval is

$$\mathrm{CI}_{s(i)}^{\mathrm{exact}} = \big[\hat{\theta}_{s(i)} - H_{[i]}^{-1}(1-\alpha/2),\ \hat{\theta}_{s(i)} - H_{[i]}^{-1}(\alpha/2)\big]$$

so that $$\mathbb{P}(\theta_{s(i)} \in \mathrm{CI}_{s(i)}^{\mathrm{exact}}) = 1-\alpha$$ for every $i$ in finite samples.

Since $H_{[i]}$ is unknown, it is estimated by bootstrap (parametric or non-parametric):

• Parametric bootstrap: If $\hat{\theta} \sim N_p(\theta, \Sigma)$ or the $\hat{\theta}_j \sim N(\theta_j, \sigma_j^2)$ independently, simulate $\theta^{(k)}$, rerank, and compute $\delta_{[i]}^{(k)}$. Use empirical quantiles for interval endpoints.
• Non-parametric bootstrap: Generate bootstrap datasets, re-estimate $\hat{\theta}^{(k)}$, rerank, and compute $\delta_{[i]}^{(k)}$.

This bootstrap approach yields intervals of the form

$$\mathrm{CI}_{s(i)}^{\text{boot}} = [\hat{\theta}_{s(i)} - \hat{H}_{[i]}^{-1}(1-\alpha/2),\ \hat{\theta}_{s(i)} - \hat{H}_{[i]}^{-1}(\alpha/2)]$$

which, under standard regularity (consistent bootstrap law), asymptotically achieve RCC(i) $= 1-\alpha$ simultaneously for all $i$ (Morrison et al., 2017).

4. Theoretical Properties and Implications

Oracle RCC intervals satisfy $\mathrm{RCC}(i)=1-\alpha$ in finite samples. Boostrap-based intervals achieve this property asymptotically uniformly over $i$, i.e.,

$$\sup_{1 \leq i \leq p} |\mathbb{P}(\theta_{s(i)} \in \mathrm{CI}_{s(i)}^{\text{boot}}) - (1-\alpha)| \to 0$$

as the number of bootstrap samples $K \to \infty$ and $n \to \infty$. An important corollary is that any procedure reporting the top $r$ estimates will have overall FCR $\leq \alpha$ provided RCC(i) $\leq 1-\alpha$ for all $i$, making RCC control a pointwise strengthening of FCR control.

Simulation studies further demonstrate that RCC intervals uniformly maintain target coverage at all ranks and outperform both marginal and FCR-adjusted intervals, especially at extremes where miscoverage is most severe (Morrison et al., 2017).

5. RCC in Predictive Inference and Rectified Conformal Prediction

In predictive inference, the concept of RCC has been adopted to address analogous failures of conditional coverage in conformal prediction frameworks. Classical split conformal prediction guarantees marginal coverage $$\mathbb P\{Y_{n+1} \in \mathcal C_\alpha(X_{n+1})\} \geq 1-\alpha$$, but may provide sub-nominal coverage over subsets defined by the rank of conformity scores.

Recent work introduces an explicit score-rectification mechanism: via regression, estimate the conditional $(1-\alpha)$-quantile of conformity scores $\widehat\tau(x)$, and transform raw scores $u$ as $\tilde V(x,y) = f_{\widehat\tau(x)}^{-1}(V(x,y))$ for a monotonic family $f_t$. Applying ordinary split conformal prediction to these rectified scores ensures coverage that is nearly uniform both marginally and over strata defined by the empirical rank of test conformity scores—i.e., RCC (Plassier et al., 22 Feb 2025).

Theoretical bounds confirm that the resulting coverage conditional on covariates, and thus conditional on rank strata, approaches $1-\alpha$ provided the quantile regression is accurate. Empirical studies in multi-output prediction highlight that RCC-conformal methods reduce the maximal conditional coverage error compared to non-RCC approaches (Plassier et al., 22 Feb 2025).

6. Software Implementations and Illustrative Examples

The R package “rcc” implements both parametric and non-parametric bootstrap methods for RCC interval construction. The package provides utilities for ranking by signed or absolute test statistics and outputs rank-ordered intervals. Basic usage includes:

ci_par <- par_bs_ci(est = theta.hat, se = se.hat, level = 0.90, nboot = 1000) ci_np <- nonpar_bs_ci(data = data, estFUN = estFUN, level = 0.90, nboot = 500)

Comparative simulation studies, including independent normals, block-correlated regression models, and treatment-effect estimation across biomarker cutpoints, demonstrate that RCC intervals track the oracle performance and maintain near-nominal coverage across all ranks, even in the presence of strong correlation and selection (Morrison et al., 2017).

7. Practical Considerations, Extensions, and Current Research

A practical guideline for achieving RCC in predictive inference is to choose a meaningful conformity score $V(x,y)$, split calibration data for quantile regression, estimate the conditional quantile $\widehat\tau(x)$, apply the rectification transformation (additive or multiplicative), and run split conformal prediction on rectified scores. This recipe is model- and score-agnostic, and applies equally to multi-output and structured prediction tasks, provided a scalar conformity score can be evaluated (Plassier et al., 22 Feb 2025).

RCC has been shown to imply stronger guarantees than FCR for selection procedures, aligns coverage properties with scientific usage (publication of top-ranked findings), and is extensible to nonparametric, correlated, and structured inference settings. Ongoing investigations include the statistical and computational tradeoffs associated with complex quantile regression estimators for rectification, and the precise characterization of RCC in dependent or high-dimensional settings.