---
title: Quantile Regression Forests (QRF)
url: https://www.emergentmind.com/topics/quantile-regression-forests-qrf
type: topic
---

# Quantile Regression Forests (QRF)

Quantile Regression Forests (QRF) are ensemble methods that extend random forests to construct nonparametric estimates of the entire conditional distribution of a response variable, not just the conditional mean. QRF provide a unified framework to estimate all conditional quantile levels with a single model for regression, probabilistic forecasting, uncertainty quantification, anomaly detection, and survival analysis, while inheriting robustness and scalability properties of classical regression forests [2408.02355][2302.11239][2311.07511][1711.10937][2410.12209]. The QRF methodology offers both empirical CDF outputs and high-resolution prediction intervals, underpinned by formal statistical theory, extensions for censored and time series data, targeted tuning for coverage, and proximity-based generalizations for computational efficiency.

## 1. Methodological Foundation

At the core of Quantile Regression Forests is the replacement of the point-estimate aggregation in regression forests by a weighted, nonparametric estimate of the conditional cumulative distribution function (CDF). For a training set $\mathcal{D} = \{ (X_i, Y_i) \}_{i=1}^n$ with $Y_i\in\mathbb{R}$ and $X_i \in \mathbb{R}^p$, and a new query point $x$, the QRF computes weights $\{ w_i(x) \}$, where
\[
w_i(x) = \frac{1}{B} \sum_{b=1}^B \frac{1_{i \in L_b(x)}}{|L_b(x)|}
\]
with $L_b(x)$ denoting the set of training indices whose predictors fall in the same terminal node as $x$ in tree $b$ [2408.02355][2302.11239][1711.10937].

The conditional CDF estimator is
\[
\hat F(y \mid x) = \sum_{i=1}^n w_i(x) \cdot 1_{\{ Y_i \leq y \}}
\]
and the conditional quantile at level $\tau\in(0,1)$ is obtained by inversion:
\[
\hat Q(\tau \mid x) = \inf \{ y : \hat F(y \mid x) \geq \tau \}.
\]
This approach enables nonparametric estimation of the full conditional distribution with a single ensemble model.

## 2. Algorithmic Structure and Variants

### Tree Construction and Weighting

QRF grows $B$ randomized regression trees on bootstrap replicates of training data. At each node, splitting proceeds by minimizing the post-split squared error, as in standard CART, or by more tailored criteria (e.g., gradient-quantile splits in gradient forests) [1711.10937]. Each tree assigns weights inversely proportional to leaf size for observations in the same terminal node as $x$ [2408.02355].

### Proximity-based Quantile Estimation

A recent generalization replaces the aggregation of per-tree leaf weights with a direct computation of sample-wise proximities:
\[
\operatorname{prox}(x, X_i) = \frac{1}{B} \sum_{b=1}^B 1_{\{ x, X_i \text{ in same leaf in } b\}}
\]
These proximities serve as learned local similarity kernels that can be normalized and used analogously to standard QRF weights for CDF estimation and quantile extraction. The computational structure makes this approach advantageous for batch quantile queries [2408.02355].

### Extensions for Censoring, Multivariate, and Time Series Data

Censored QRF variants replace the outcome $Y_i$ by observed pairs $(Y_i, \delta_i)$ (with $\delta_i$ indicating event/censoring) and modify the estimation equation using a local Kaplan–Meier or Beran estimator for the conditional survival function [2001.03458][2410.12209].

In the multivariate setting, QRF-derived weights are combined with center-outward optimal transport to recover (conditional) vector quantile contours [2303.11054].

Generalized and time-series QRF (GRF, tsQRF) adapt tree-splitting to preserve "honesty" and other regularity conditions required for consistency under dependent data [2211.02273].

## 3. Theoretical Guarantees and Statistical Properties

Quantile Regression Forests are consistent estimators of the conditional CDF under suitable regularity (trees grown deeply enough, honesty, randomization, Lipschitz regression function, etc.) [2302.11239][2211.02273][2511.23212]. Uniform consistency is established for tsQRF under $\alpha$-mixing sequences and GRF splitting [2211.02273].

In high dimensions and large sample settings, bias-variance decomposition reveals a phase transition in statistical inference for variable importance via pinball loss risks, controlled by the forest subsampling rate $\beta$ (with $s \asymp n^\beta$) [2511.23212]. For $\beta < 1/2$, the QRF quantile estimator is asymptotically normal at $O((n/s)^{-1/2})$ rate; for $\beta \ge 1/2$, bias dominates and inference without analytic correction is invalid.

Censored QRF and GCQRF provide consistency for quantile estimation under right-censoring, using weighted quantile loss minimization adjusted for censoring via survival probabilities [2001.03458][2410.12209]. Weak convergence is described via incomplete infinite-degree U-processes for GCQRF [2410.12209].

## 4. Tuning, Efficiency, and Computational Complexity

Default QRF parameter settings often yield competitive performance, but targeted tuning is important for prediction interval coverage and efficiency [2507.01430]. The quantile coverage loss (QCL) objective directly optimizes the (out-of-bag) difference between empirical quantile coverage and the nominal level, substantially reducing both bias and MSE of coverage in repeated experiments compared to MSE or c-index tuning [2507.01430].

Computationally, QRF inference cost is $O(B \log n + \ell)$ per test point, where $\ell$ is the number of quantile levels evaluated. Proximity-based QRF incurs $O(n B + n)$ upfront per query but enables efficient extraction of arbitrarily many quantile levels and reuse across queries [2408.02355].

For extreme quantile levels, standard QRF cannot extrapolate beyond observed outcomes; extensions using local likelihood estimation of generalized Pareto parameters (ERF) address this limitation by fitting weighted EVT models using QRF weights [2201.12865][1711.10937].

## 5. Applications

QRF have been adopted in multiple domains:
- **Probabilistic forecasting** of precipitation using satellite and sensor data, with rigorous assessment via quantile scoring rules and continuous ranked probability score (CRPS). QRF yields calibrated quantiles, competitive with and sometimes superior to ensemble model output statistics (EMOS) and boosting methods in spatial prediction, but lags LightGBM in speed and marginally in aggregate score [2311.07511][1711.10937].
- **Anomaly detection** via context-specific QRF for explainable contextual anomaly scoring, leveraging locally estimated conditional quantiles and interval widths [2302.11239].
- **Survival analysis** with right-censored event times and high-dimensional predictors, using GCQRF and crf for robust, nonparametric estimation of quantile processes and variable importance [2410.12209][2001.03458].
- **Extreme value prediction**, combining QRF proximity weights with local GPD likelihood for tail risk estimation [2201.12865].
- **Multivariate quantiles** via optimal transport mapped from QRF-derived empirical measures [2303.11054].
- **Time series quantile regression** in stationary and nonlinear autoregressive models, with tsQRF providing robust, volatility-sensitive estimates [2211.02273].
- **Financial prediction intervals** and risk estimation, including daily bond volume and asset volatility prediction, where proximity-based QRF yields tighter intervals and more efficient coverage [2408.02355].

## 6. Limitations and Practical Considerations

QRFs are fundamentally restricted to the range of observed training responses and do not extrapolate for extreme quantiles unless combined with EVT or parametric tail models [1711.10937][2201.12865]. Complexity–interpretability trade-offs arise: QRFs provide rich, flexible distributional estimates, but the ensemble structure is "black-box" and variable-importance inference becomes subtle in high-dimensional or highly correlated data [2511.23212][2311.07511]. Variable importance estimation requires attention to bias regimes and potential analytic correction [2511.23212].

Hyperparameter tuning aligned with quantile estimation objectives (coverage, loss) is essential for predictive validity and interval width minimization [2507.01430]. For high-resolution or repeated quantile extraction, proximity-based QRF or batch strategies are preferable for computational efficiency [2408.02355].

## 7. Table: Core QRF Formulas and Use-cases

| Context               | Key Formula                                                                 | Papers          |
|-----------------------|------------------------------------------------------------------------------|-----------------|
| Conditional quantile  | $\hat Q(\tau|x) = \inf \{y: \sum_i w_i(x) 1_{Y_i \le y} \ge \tau \}$        | [2408.02355]    |
| Censored QRF          | $\hat Q^T_\tau(x): \min_q | (1-\tau)\hat G(q|x) - \sum_i w_i(x) 1_{Y_i>q} |$ | [2001.03458]    |
| Proximity-based QRF   | $\operatorname{prox}(x, X_i)$ as similarity weights for all quantile queries | [2408.02355]    |
| Variable importance   | $\hat V_\tau(S) = \frac{1}{n}\sum_{i=1}^n [\rho_\tau(Y_i-\hat q_{\tau,-S}) - \rho_\tau(Y_i - \hat q_\tau)]$ | [2511.23212] |
| EVT extension (ERF)   | $L_n(\theta;x)=\sum_i w_i(x)\, \ell_\theta(Z_i)$ and tail quantile extrapolation | [2201.12865] |
| Targeted tuning (QCL) | $\mathcal{L}_\tau = |\tilde\tau - \tau|,\ \tilde{\tau}=\frac{1}{n}\sum_i 1_{t_i\le \hat{q}_\tau(x_i)}$ | [2507.01430] |

## References

- "Quantile Regression using Random Forest Proximities" [2408.02355]
- "Asymptotic Theory and Phase Transitions for Variable Importance in Quantile Regression Forests" [2511.23212]
- "Explainable Contextual Anomaly Detection using Quantile Regression Forests" [2302.11239]
- "Uncertainty estimation of machine learning spatial precipitation predictions from satellite data" [2311.07511]
- "Global Censored Quantile Random Forest" [2410.12209]
- "Extremal Random Forests" [2201.12865]
- "Censored Quantile Regression Forest" [2001.03458]
- "Targeted tuning of random forests for quantile estimation and prediction intervals" [2507.01430]
- "Forest-based methods and ensemble model output statistics for rainfall ensemble forecasting" [1711.10937]
- "Some novel aspects of quantile regression: local stationarity, random forests and optimal transportation" [2303.11054]
- "Time series quantile regression using random forests" [2211.02273]

Source: https://www.emergentmind.com/topics/quantile-regression-forests-qrf