Quantile Random Forest Model

Updated 23 October 2025

Quantile random forest models are ensemble methods that nonparametrically estimate conditional quantiles by leveraging tree-based similarity weights.
They enable estimation of full outcome distributions, support robust prediction intervals, and facilitate feature importance analysis across diverse applications.
Despite their flexibility, careful tuning is required to balance bias and variance, especially in high-dimensional or censored data settings.

A quantile random forest (QRF) model is an ensemble learning method that extends the classical random forest to the estimation of conditional quantiles, thereby capturing the full conditional distribution of the response variable rather than restricting inference to the conditional mean. QRFs are fundamentally nonparametric and utilize tree-based, data-driven similarity kernels to nonlinearly model heterogeneity and approximate the distributional behavior of outcomes. The method is widely applicable and supports flexible, robust quantile estimation in regression tasks, prediction interval construction, feature importance analysis, and sensitivity analysis. Below, key methodological concepts, estimation principles, practical inference procedures, theoretical guarantees, and applied advantages of quantile random forests are reviewed in detail.

1. Foundational Principles and Forest Weights

In quantile random forests, the original Breiman random forest (Hothorn et al., 2017) is adapted so that each tree in the forest is used to induce a set of local neighborhoods in the covariate space. Given a regression sample $\{(X_i, Y_i)\}_{i=1}^n$ , each tree partitions the feature space into terminal nodes; for a given query point $x$ , the set of training instances in the same leaf as $x$ constitute its neighborhood. The forest aggregates these neighborhood assignments over all $T$ trees to derive non-negative weights $w_i^N(x)$ for each training point $i$ with:

$w^N_i(x) = \frac{1}{T} \sum_{t=1}^{T} \frac{\mathbb{1}\{x, X_i \text{ in same leaf of tree } t\}}{\#\text{ training points in that leaf}}$

These weights satisfy $\sum_{i=1}^n w_i^N(x) = 1$ and reflect the degree of similarity between $x$ and the training points under the forest-induced metric (Hothorn et al., 2017, Elie-Dit-Cosaque et al., 2020). The mechanism underlies the nonparametric estimation of conditional functionals, including quantiles.

2. Quantile Estimation via Weighted Empirical Distributions

To estimate the conditional cumulative distribution function (CDF) at $x$ , the QRF computes:

$\hat{F}(y \mid x) = \sum_{i=1}^n w^N_i(x) \, \mathbb{1}\{Y_i \leq y\}$

The conditional $\tau$ -quantile at $x$ is then obtained by “inverting” this CDF:

$\hat{Q}_\tau(x) = \inf\{y: \hat{F}(y \mid x) \geq \tau\}$

This approach enables coherent, non-crossing estimation of all quantiles from a single ensemble without fitting separate models for different quantile levels (Hothorn et al., 2017, Gyamerah et al., 2019, Elie-Dit-Cosaque et al., 2020). QRF thus generalizes standard random forests by replacing conditional means with arbitrary functionals of the estimated CDF.

3. Adaptive Local Likelihood and Transformation Forests

While classical QRFs aggregate the empirical distribution, “transformation forests” (Hothorn et al., 2017) propose a parametric extension where the conditional distribution of $Y$ given $X = x$ is modeled as

$F(y \mid x) = F_0(h(y; \theta(x)))$

where $F_0$ is a known CDF (e.g., Gaussian, logistic), and $h(y; \theta(x))$ is a monotonic transformation with parameter vector $\theta(x)$ , which varies with $x$ . The transformation function $h$ is typically parameterized (e.g., via Bernstein polynomials), and $\theta(x)$ is estimated adaptively through localized likelihood maximization:

$\hat{\theta}(x) = \arg\max_{\theta \in \Theta} \sum_{i=1}^n w^N_i(x) \, \ell_i(\theta)$

with log-likelihood contribution $\ell_i(\theta) = \log \Big(f_0(h(Y_i; \theta)) \cdot h'(Y_i; \theta)\Big)$ .

This parametric approach yields smoothly varying inference for the entire conditional distribution and allows likelihood-based inference such as the model-based bootstrap or likelihood-ratio tests (Hothorn et al., 2017). In contrast to empirical QRFs, transformation forests offer direct modeling of higher moments (variance, skewness) and facilitate statistical testing and prediction interval construction.

4. Handling Censoring and Extensions to Survival Analysis

Observed responses are often subject to right-censoring, particularly in survival analysis. Standard QRFs are not natively robust in this context; naive application yields biased quantile estimates. Censored Quantile Regression Forests (CQRFS) (Li et al., 2019, Li et al., 2020, Zhou et al., 16 Oct 2024) address this by modifying the estimating equation. The censored quantile score equation, for quantile $\tau$ , solves:

$S_n(q; \tau) = (1-\tau) \hat{G}(q|x) - \sum_{i=1}^n w^N_i(x) \mathbb{1}\{Y_i > q\} \approx 0$

where $\hat{G}(q|x)$ estimates the conditional survival function of the censoring variable. This adjusting mechanism ensures that the estimator converges to the true quantile of the unobserved failure time and is consistent under mild independence and regularity conditions.

For “global” censored quantile forests (Zhou et al., 16 Oct 2024), the focus is on estimating the entire quantile process $\tau \mapsto Q_T(\tau|x)$ , using inverse-probability weighting and integrated quantile loss as splitting and evaluation criteria. This class of models provides nonparametric quantile process estimation under censoring, without linearity assumptions, and admits a U-process-based asymptotic theory for prediction interval uncertainty quantification.

5. Consistency and Theoretical Properties

Theoretical analysis has established that QRF estimators of the conditional CDF and quantiles are uniformly consistent almost surely under general regularity conditions (Elie-Dit-Cosaque et al., 2020). Formally, for both the bootstrap-based and original-sample-based variants,

$\sup_{y,x} |\hat{F}(y|x) - F(y|x)| \to 0\text{ a.s.}$

provided, e.g., the diameter of each leaf shrinks with sample size, nodesizes diverge appropriately, and $F(y|x)$ is continuous in $y$ and Lipschitz in $x$ . Similar consistency results extend to censored settings for quantile estimators defined via survival-adjusted estimating equations, again under regularity and independence assumptions (Li et al., 2019, Li et al., 2020).

For transformation forests, the parametric structure enables formal likelihood-based inference, including estimation of standard errors and distributional approximations via model-based bootstrapping (Hothorn et al., 2017).

6. Practical Applications and Performance

Quantile random forests and their extensions have found use in:

Survival analysis, reliability engineering, and heterogeneous treatment effect evaluation, supporting flexible, nonparametric estimation of conditional quantiles in the presence of censoring (Li et al., 2019, Li et al., 2020, Zhu et al., 2022, Zhou et al., 16 Oct 2024).
Probabilistic crop yield forecasting, where QRFs combined with kernel density estimation (e.g., with Epanechnikov kernel and Sheather-Jones bandwidth selection) provide full probability distributions, accurate prediction intervals (e.g., 100% PICP), and feature importance rankings for climatological risk analysis (Gyamerah et al., 2019).
Predictive uncertainty quantification in traffic engineering, using QRF with dimension reduction via PCA to generate interpretable prediction intervals for annual average daily traffic, with interval coverage approaching 88% and informative interval widths on high-dimensional spatial data (Yao et al., 21 Oct 2025).
Model interpretability; forward variable selection based on Continuous Ranked Probability Score (CRPS) enables identification of parsimonious predictor sets that retain full conditional distributional predictive accuracy (Velthoen et al., 2020).

Performance metrics commonly employed include empirical coverage probability, prediction interval width, mean squared error, the quantile loss, CRPS, and Winkler score. Empirical and simulation studies consistently demonstrate that QRF and its extensions yield calibration and coverage exceeding that of parametric or mean-based models, exhibit robustness to outliers, and perform comparably to “oracle” methods that have access to uncensored or complete data. For instance, coverage probability in agricultural yield prediction using QRF intervals can reach 100% even when normalized average widths remain narrow (12–16%) (Gyamerah et al., 2019).

7. Advantages, Limitations, and Ongoing Developments

Quantile random forests offer several methodological advantages:

Nonparametric, fully conditional quantile estimation: No assumption of linear or parametric structure; estimation leverages local, data-adaptive similarity metrics.
Intrinsic prediction interval and quantile process estimation: The model natively supports multiple quantile levels, avoiding cross-quantile incoherence.
Flexibility for high-dimensional data: QRFs scale favorably in settings with many covariates and nonlinear dependencies.
Integration with other statistical tools: QRF can be coupled with kernel smoothing, MIDAS filters for mixed-frequency data, random effects for longitudinal data, and more (Andreani, 24 Feb 2025).
Formal inference: Likelihood-based methods are accessible, especially within transformation forest formulations, enabling model-based bootstrap and hypothesis testing (Hothorn et al., 2017).

Limitations include sensitivity to the tuning of hyperparameters (e.g., leaf size selection), which impacts the bias–variance tradeoff and the precision of quantile estimation—careful cross-validation or out-of-bag error analysis is required (Elie-Dit-Cosaque et al., 2021). QRF methods address only marginal, not conditional, coverage unless specifically adapted; conditional quantile interval validity may not be guaranteed universally (Berkowitz et al., 2 Jul 2025). In censored data settings, robust estimation requires consistent censoring distribution estimation and the validity of conditional independence assumptions (Li et al., 2019, Li et al., 2020).

Ongoing research directions involve improved hyperparameter tuning (targeted at quantile loss rather than mean squared error), extensions to mixed-frequency and longitudinal data, tailored procedures for high-dimensional and structured data, and integration with kernel density estimation, beyond forest-based similarity weights.

Summary Table: QRF Core Properties and Capabilities

Aspect	QRF Implementation	Reference
Quantile Estimation	Weighted inversion of empirical CDF	(Hothorn et al., 2017, Elie-Dit-Cosaque et al., 2020)
Censoring Handling	Survival-adjusted estimating equations	(Li et al., 2019, Li et al., 2020, Zhou et al., 16 Oct 2024)
Likelihood-Based Inference	Parametric modeling via transformation forests	(Hothorn et al., 2017)
Performance Metrics	Coverage, PINAW, CRPS, MSE, Winkler Score	(Gyamerah et al., 2019, Yao et al., 21 Oct 2025, Velthoen et al., 2020)
Theoretical Consistency	Uniform a.s. consistency, regularity assumptions	(Elie-Dit-Cosaque et al., 2020, Li et al., 2019)
Dimensionality	High-dimensional and structured data supported	(Yao et al., 21 Oct 2025, Andreani, 24 Feb 2025)

In conclusion, quantile random forests constitute a robust, theoretically justified, and highly adaptive class of ensemble methods for conditional quantile estimation, supporting both distributional inference and prediction interval construction in a variety of data regimes, including censored and high-dimensional settings. The methodology is under continuous development to address challenges in tuning, inference under censoring, high-dimensional predictor sets, and integration with advanced statistical tools.