---
title: Conformalized Quantile Regression
url: https://www.emergentmind.com/topics/conformalized-quantile-regression
type: topic
---

# Conformalized Quantile Regression

Conformalized Quantile Regression (CQR) is a statistical framework that integrates quantile regression—a methodology for estimating conditional quantiles—with conformal prediction, which provides finite-sample, distribution-free marginal coverage guarantees. This fusion yields prediction intervals that adapt to heteroscedasticity and complex data geometry while maintaining rigorous statistical validity. CQR is widely used in modern machine learning and statistics as it balances flexibility, finite-sample control, and computational tractability. Recent research focuses on improving its local adaptivity, conditional coverage, high-dimensional scalability, multivariate extension, and robustness in structured or non-i.i.d. data regimes.

## 1. Problem Setup and Core CQR Methodology

CQR addresses the standard regression setting: observing i.i.d. data $\{(x_i, y_i)\}_{i=1}^N$ from an unknown distribution $P$ over $X \times Y$ (usually $X \subseteq \mathbb{R}^D$, $Y \subseteq \mathbb{R}$), the aim is to produce, for any new test instance $x^* \in X$, an interval $C_{1-\alpha}(x^*)$ such that
\[
\mathbb{P} \{ Y \in C_{1-\alpha}(X) \} \ge 1-\alpha
\]
with the probability being over all randomness in the data, including $x^*$.

The basic CQR algorithm ([1905.03222], [2207.02808], [1909.05433]) consists of:

1. **Data Split**: Divide data into a (proper) training set $D_{\text{train}}$ and a calibration set $D_{\text{cal}}$.

2. **Quantile Regressors**: Train two quantile regression models on $D_{\text{train}}$, targeting the lower and upper conditional quantiles:
   \[
   \hat{q}_{\alpha/2}(x) = \arg \min_\theta \sum_{i \in D_{\text{train}}} \rho_{\alpha/2}(y_i - q(x_i; \theta))
   \]
   \[
   \hat{q}_{1 - \alpha/2}(x) = \arg \min_\theta \sum_{i \in D_{\text{train}}} \rho_{1-\alpha/2}(y_i - q(x_i; \theta))
   \]
   where $\rho_\tau(u) = \max\{ \tau u, (\tau - 1)u \}$ is the pinball (quantile) loss.

3. **Conformal Calibration**: On $D_{\text{cal}}$, compute nonconformity scores:
   \[
   s_i = \max\{ \hat{q}_{\alpha/2}(x_i) - y_i,\, y_i - \hat{q}_{1-\alpha/2}(x_i) \}
   \]
   Let $\hat{Q}_{1-\alpha}$ denote the empirical $(1-\alpha)$-quantile of $\{s_i\}_{i \in D_{\text{cal}}}$.

4. **Interval Construction**: For a test point $x$,
   \[
   C(x) = [\hat{q}_{\alpha/2}(x) - \hat{Q}_{1-\alpha},\; \hat{q}_{1-\alpha/2}(x) + \hat{Q}_{1-\alpha}]
   \]

Exchangeability of calibration and test points guarantees marginal coverage at $1-\alpha$ [1905.03222, 2207.02808].

## 2. Adaptivity, Efficiency, and Marginal Coverage

CQR inherits adaptivity from quantile regression—interval width naturally reflects local conditional spread, enabling the method to handle heteroscedasticity. The conformal calibration step ensures exact finite-sample marginal coverage, regardless of the underlying noise distribution or estimator bias [1905.03222].

Extensive experiments demonstrate that CQR achieves the intended coverage with average interval widths substantially smaller than those of non-adaptive or constant-width conformal methods, especially in heteroscedastic and high-dimensional regimes [1905.03222, 1909.05433].

Non-asymptotic efficiency analyses (see [2510.07093]) quantify the deviation of the CQR interval from oracle width:
\[
\mathcal{O}\left(\frac{1}{\sqrt{n}} + \frac{1}{\alpha^2 n} + \frac{1}{\sqrt{m}} + \exp(-\alpha^2 m)\right)
\]
where $n$ is the training set size, $m$ the calibration set size, and $\alpha$ the target error rate, capturing the tradeoff between sample allocation and coverage/efficiency.

## 3. Extensions: Local and Density-Adaptive Conformalization

Classic CQR applies a single conformal shift across the entire input domain, which limits adaptivity in highly heterogeneous settings. Improved methods stratify the calibration step via clustering, local density calibration, or hybrid approaches.

**Weighted k-Means Stratification** ([2207.02808]). The ICQR procedure clusters $D_{\text{cal}}$ in feature space (with permutation-importance weighting), applies separate conformal calibration in each cluster, and assigns test points to clusters by feature proximity. Each cluster gets its own conformal quantile shift, increasing local adaptivity to variable heteroscedasticity:
- Clustering by feature permutation-importance,
- Cluster-specific conformal quantiles,
- Coverage guarantee holds within each cluster and globally.

**Density-Calibrated CQR (CQR-d)** ([2411.19523]) introduces hybrid local-global conformity scores, using k-NN density estimates:
- Compute local and global conformity quantiles,
- Blend scores via density-based weights,
- Optimize a global scaling hyperparameter for calibration,
- Achieves coverage at $1-\alpha-\epsilon$ (where $\epsilon$ is from optimization) and produces intervals up to 21% narrower in simulated heteroscedastic settings.

## 4. Multivariate, High-Dimensional, and Flexible Model Extensions

CQR has been generalized to several complex regimes:

- **Multivariate Responses** ([2509.25444]): Conditional vector quantile regression with neural optimal transport enables high-dimensional, geometry-adaptive conformal prediction sets, exploiting the OT structure for joint distribution coverage guarantees.

- **Ensembles and Aggregation** ([2103.00083]): Weighted ensembles of base quantile models, post-hoc monotonicity corrections (sorting/isotonic regression), and cross-conformal calibration (QOOB) achieve state-of-the-art risk/efficiency profiles in large benchmark evaluations.

- **Dynamic Quantization for High-Density Regions** ([2411.01266]): Adaptive prototype-based quantile methods provide region-based, non-convex, high-density coverage with scalable dynamic allocation/removal of quantization bins during training. Conformal regions are calculated as unions of high-density Voronoi cells covering $(1-\alpha)$ mass.

## 5. Conditional Coverage, Uncertainty, and Practical Considerations

While CQR provides finite-sample marginal coverage, exact conditional coverage $\mathbb{P}(Y \in C(X) \mid X=x) \ge 1-\alpha$ cannot be achieved in finite samples in a distribution-free way. Nevertheless, numerous approaches improve conditional coverage in practice:

- **Density-Weighted and Conditional Losses** ([2512.24139]): By incorporating local density estimates or penalizing coverage error at the conditional level (via density-weighted pinball losses or auxiliary quantile heads), one can reduce mean squared conditional coverage error and improve worst-slice/partition guarantees while retaining approximate marginal validity.

- **Uncertainty-Aware CQR (UACQR)** ([2306.08693]): Explicitly separates epistemic (estimation) from aleatoric (conditional) uncertainty by ensemble quantile models and inflates intervals adaptively where quantile estimation is less certain. UACQR yields improved conditional coverage in both simulations and real datasets.

- **Support for Non-i.i.d. Data** ([2202.08756]): Ensemble bootstrap mechanisms (EnCQR) extend CQR to time series and non-exchangeable data by forming leave-one-out residuals from ensembles, maintaining approximate marginal validity.

- **Efficiency Tuning** ([1909.05433, 2510.07093]): The data split fraction (training/calibration) and quantile estimation hyperparameters should be tuned empirically, with allocations in the $[0.7, 0.9]$ training fraction range yielding the best compromise between accurate quantile estimation and conformal quantile calibration stability.

## 6. Algorithmic Summary and Theoretical Guarantees

**Standard CQR Algorithm** ([1905.03222], [2207.02808]):
1. Split: $D_{\rm train}/D_{\rm cal}$.
2. Train: Quantile regression at $\tau = \alpha/2, 1-\alpha/2$.
3. Compute: Residual max($\hat q_{\alpha/2}(x_i) - y_i$, $y_i - \hat q_{1-\alpha/2}(x_i)$) on $D_{\rm cal}$.
4. Quantile: Empirical $(1-\alpha)$-quantile $\hat Q_{1-\alpha}$.
5. Predict: For $x^*$, output $[\hat q_{\alpha/2}(x^*) - \hat Q_{1-\alpha},\, \hat q_{1-\alpha/2}(x^*) + \hat Q_{1-\alpha}]$.

**Guarantee:** For any exchangeable sample (calibration + test),
\[
\mathbb{P}\{ Y^* \in C(X^*) \} \ge 1 - \alpha
\]
The guarantee remains exact for split-conformal CQR; cross-conformal and full-conformal variants can achieve near-exact guarantees at higher computational cost.

## 7. Limitations, Extensions, and Current Directions

**Limitations:**
- Provides only marginal, not conditional, coverage.
- Relies critically on the quality of quantile regressors; poorly calibrated or misspecified quantiles yield conservative (wide) intervals to maintain coverage ([1905.03222]).
- The clustering/density-adaptive extensions add algorithmic and computational complexity, especially for high-dimensional feature spaces ([2207.02808], [2411.19523]).

**Extensions/Future Work:**
- Further improving local adaptivity through continuous stratification, dynamic density estimation, or covariate-informed conformalization.
- Online/streaming, multi-fidelity, or structured data settings ([2305.03623]).
- Multivariate and tensor-valued CQR ([2509.25444], [2601.17437]).
- Improved group, subgroup, or conditional coverage through partition learning, density-weighted calibration, and uncertainty estimation strategies ([2512.24139], [2306.08693], [2304.01426]).
- Theoretical analyses of finite-sample and asymptotic properties, especially regarding efficiency, conditional guarantee relaxations, and the tradeoffs inherent in data allocation ([2510.07093], [1909.05433]).

---

Conformalized Quantile Regression constitutes an active research frontier at the intersection of conformal inference, robust regression, uncertainty quantification, and machine learning. Methodological innovations and theoretical refinements continue to extend its utility and optimality across diverse applications such as time-series forecasting, hyperparameter optimization, tensor prediction, and structured probabilistic modeling ([2202.08756], [2305.03623], [2601.17437], [2411.01266], [2411.19523], [2509.25444]).

Source: https://www.emergentmind.com/topics/conformalized-quantile-regression