Weighted Quantiles in Conformal Prediction

• Weighted quantiles are a calibration method that integrates context-dependent weights to achieve locally valid prediction intervals.
• They compute (1-α) thresholds using a weighted empirical CDF, addressing spatial, covariate, and group heterogeneity effectively.
• This approach ensures robust uncertainty quantification under nonstationarity and distribution shifts, with Bayesian extensions enhancing diagnostic insights.

Weighted quantiles in conformal prediction are a rigorous extension of classical quantile-calibration methods, designed to address violations of exchangeability and to enable locally or conditionally valid coverage. They provide a mechanism for incorporating informative, context-dependent weights when computing the empirical distribution of calibration nonconformity scores. This approach is technically central to robust uncertainty quantification in spatial, temporal, distribution-shifted, and group-heterogeneous settings, as well as in domains such as automated valuation models, covariate shift adaptation, and fair or localized inference.

1. Formal Definition of the Weighted Quantile in Conformal Calibration

Let $\{s_i\}_{i=1}^n$ denote nonconformity scores derived from a calibration sample and $\{w_i\}_{i=1}^n$ be associated nonnegative weights, potentially dependent on covariates, location, group, or other structure. The weighted empirical cumulative distribution function (CDF) is defined as

$$F_w(t) = \frac{\sum_{i=1}^n w_i\,\mathbf{1}\{s_i \le t\}}{\sum_{i=1}^n w_i}.$$

The $(1-\alpha)$ weighted quantile is

$$q_{1-\alpha} = \inf\left\{ t \in \mathbb{R} : F_w(t) \ge 1-\alpha \right\},$$

which generalizes the classical empirical quantile (recovered when $w_i \equiv 1$) (Hjort et al., 2023, Tibshirani et al., 2019, Bhattacharyya et al., 2024).

This construction admits several equivalent forms in the literature:

• As the smallest $t$ such that the cumulative sum of sorted weights at scores $s_i \le t$ is at least $(1-\alpha)$ of the total;
• As a solution to $$q_{1-\alpha} = \min\{ s_{(k)} : \sum_{i: s_i \le s_{(k)}} w_i \ge (1-\alpha) \sum_{i=1}^n w_i \}$$, with $\{w_i\}_{i=1}^n$0 the order statistics;
• Including a weight for an "infinite residual" (dummy calibration score), to ensure randomization and exactness at the boundary (Barber et al., 2022).

2. Weighted Quantile Algorithms Across Conformal Prediction Extensions

The weighted quantile is integrated into the CP calibration process in several settings, each motivating a particular choice of weight:

• Spatial Conformal Prediction: Weights are kernelized spatial distances, typically $\{w_i\}_{i=1}^n$1, enhancing coverage in local neighborhoods for spatially nonstationary data (Hjort et al., 2023).
• Covariate Shift: Weights are formed from importance ratios $\{w_i\}_{i=1}^n$2 (often estimated via classification odds), aligning the calibration distribution to the test distribution (Tibshirani et al., 2019, Alijani et al., 26 May 2025, Shin et al., 3 Dec 2025).
• Group-Weighted Conformal Prediction: Each group $\{w_i\}_{i=1}^n$3 has a specified prevalence $\{w_i\}_{i=1}^n$4, and samples in that group are weighted by $\{w_i\}_{i=1}^n$5, where $\{w_i\}_{i=1}^n$6 is the calibration count in group $\{w_i\}_{i=1}^n$7 (Bhattacharyya et al., 2024, Alpay et al., 29 Sep 2025).
• Adaptive/Local Conformal Prediction: Weights are computed by QRF, kernel estimators, or other regression-derived similarity metrics, focusing the quantile on calibration points nearest in feature space to the test input (Amoukou et al., 2023, Lee et al., 2024).
• Model-Uncertainty or Distribution/Shift Diagnostics: Weights reflect model-based uncertainties such as VAE-based reconstruction loss ratios between calibration and test data (Alijani et al., 26 May 2025).

The core algorithmic steps—computation of scores, derivation of weights, sorting and cumulative summation, and weighted threshold selection—remain structurally similar across domains.

3. Theoretical Guarantees and Coverage Properties

Weighted quantile-based conformal prediction achieves a variety of finite-sample and asymptotic coverage guarantees, contingent on the underlying statistical structure:

• Weighted Exchangeability: Under weighted exchangeability (Tibshirani et al., 2019), coverage is $\{w_i\}_{i=1}^n$8, matching the classical result except weights replace permutations in the symmetric group action.
• Conditional and Local Validity: If weights concentrate around regions where the calibration and test distributions match (e.g., under a spatial Gaussian process or in local exchangeability regimes), coverage becomes locally calibrated—exact in the asymptotic neighborhood shrinkage limit (Hjort et al., 2023).
• Robustness to Non-exchangeability: For arbitrary non-exchangeable sequences, coverage loss is upper bounded by a weighted sum of total variation distances between original and "swapped" datasets, scaled by the magnitude of weights (Barber et al., 2022).
• Group-adaptive and Fairness-aware Bounds: In group-weighted settings, under covariate shift and conditional exchangeability within groups, coverage loss per group is $\{w_i\}_{i=1}^n$9 with a sharp explicit bound tied to maximal group weight and calibration count, outperforming infinite-dimensional bounds for arbitrary weights (Bhattacharyya et al., 2024, Alpay et al., 29 Sep 2025).
• Covariate-shift with Distribution Estimation Error: When importance weights are estimated, coverage bounds degrade gracefully with the $$F_w(t) = \frac{\sum_{i=1}^n w_i\,\mathbf{1}\{s_i \le t\}}{\sum_{i=1}^n w_i}.$$0 error of the weight estimator (Alijani et al., 26 May 2025, Shin et al., 3 Dec 2025).

These results extend across conformal methodologies, including survival analysis with censoring, localized regression, and fairness-constrained learning.

4. Bayesian and Empirical Distributions: Uncertainty in Weighted Quantiles

Weighted quantiles, by themselves, provide point estimates for cutoff thresholds. Recent work generalizes the frequentist approach to a Bayesian perspective:

• The Weighted Bayesian Conformal Prediction (WBCP) framework interprets the quantile threshold as a Dirichlet posterior over spacing-probabilities between sorted scores, parameterized by effective sample size $$F_w(t) = \frac{\sum_{i=1}^n w_i\,\mathbf{1}\{s_i \le t\}}{\sum_{i=1}^n w_i}.$$1 and normalized weights $$F_w(t) = \frac{\sum_{i=1}^n w_i\,\mathbf{1}\{s_i \le t\}}{\sum_{i=1}^n w_i}.$$2 (Lou et al., 7 Apr 2026). This yields posterior distributions for the quantile threshold and corresponding meta-uncertainty diagnostics (posterior sd, $$F_w(t) = \frac{\sum_{i=1}^n w_i\,\mathbf{1}\{s_i \le t\}}{\sum_{i=1}^n w_i}.$$3), and affords sharper, data-conditional coverage guarantees.
• The resulting stochastic dominance and conditional coverage theorems provide $$F_w(t) = \frac{\sum_{i=1}^n w_i\,\mathbf{1}\{s_i \le t\}}{\sum_{i=1}^n w_i}.$$4 improvement in the miscoverage rate via highest posterior density (HPD) thresholding, directly reflecting information-theoretic content in the weight profile.

5. Empirical and Practical Impact

Empirical studies consistently demonstrate key benefits from weighted quantile calibration in conformal prediction:

Scenario	Weighted Quantile Effects	Coverage / Efficiency Tradeoff
Spatial heterogeneity	Flattens local coverage discrepancy, minor width increase	±1% local gap, 1–3% width rise
Covariate shift	Maintains coverage under shift, reduces set inflation	Robust to moderate misweighting
Group/fairness settings	Achieves uniform group-wise coverage with small excess error	O(1/n) excess for finite groups
Adaptive/local calibration	Interval width adapts to uncertainty, improved conditional fit	Achieves local PAC coverage

Weighted quantiles are crucial in environments with systematic variation (spatial, temporal, covariate-dependent), enabling predictive intervals to adapt accurately to underlying nonuniformity without sacrificing finite-sample guarantees (Hjort et al., 2023, Tibshirani et al., 2019, Bhattacharyya et al., 2024, Amoukou et al., 2023, Chen et al., 30 Dec 2025).

6. Variants and Extensions: Differential Weight Construction

The literature proposes multiple functionals for weight construction, matched to problem structure:

• Spatial kernels, covariate similarity metrics, or leverage scores (Hjort et al., 2023, Amoukou et al., 2023, Fadnavis, 13 Feb 2026).
• Density ratio or likelihood-based importance functions under known or estimated covariate shift (Tibshirani et al., 2019, Shin et al., 3 Dec 2025).
• Group-level proportions for stratified or fairness-driven inference (Bhattacharyya et al., 2024, Alpay et al., 29 Sep 2025).
• Model-based uncertainty as in VAE reconstruction losses or residual variance proxies (Alijani et al., 26 May 2025).
• Data-adaptive weights from nonparametric regression or random forest partitions (Amoukou et al., 2023, Lee et al., 2024).

Critically, the calibration weighting must be independent of the test label, and effective sample size diagnostics or regularization (e.g., weight clipping, normalization) are recommended for numerical and statistical stability.

7. Limitations, Open Questions, and Extensions

Weighted quantiles introduce several practical and theoretical considerations:

• Weight estimation error and variance inflation degrade coverage guarantees; group-weighted schemes offer improved rates when the group structure captures the dominant heterogeneity (Bhattacharyya et al., 2024).
• Strong weighting (e.g., one dominant weight) reduces effective sample size and can yield wide, conservative intervals; Bayesian posterior diagnostics, such as $$F_w(t) = \frac{\sum_{i=1}^n w_i\,\mathbf{1}\{s_i \le t\}}{\sum_{i=1}^n w_i}.$$5, allow explicit quantification of this reliability loss (Lou et al., 7 Apr 2026).
• Adaptivity vs. fixed weighting: Many effective adaptive methods (e.g., QRF or RNW kernels) depart from fixed, a priori weighting but retain similar coverage properties via local exchangeability or bagging arguments (Amoukou et al., 2023, Lee et al., 2024).
• Conditional coverage impossibility: Finite-sample exact conditional coverage is unattainable; however, weighted quantile approaches minimize mean squared conditional error and worst-slice undercoverage, as seen in density-weighted pinball methods (Chen et al., 30 Dec 2025).
• Algorithmic complexity: While weighted quantile computation requires additional sorting and prefix sums, it is computationally negligible in most high-dimensional or large-scale settings, with groupwise or local partitioning offering further improvements (Amoukou et al., 2023).

Weighted quantiles operationalize the extension of conformal prediction to nonstationary, complex, and high-stakes domains by providing a principled mechanism for localizing coverage and quantifying model reliability. These techniques underpin modern robust predictive inference under realistic data-generating assumptions.