Quantile Regression Neural Networks

• QRNNs are neural networks that estimate conditional quantiles instead of means by minimizing the pinball loss.
• They are employed in diverse fields such as weather forecasting, survival analysis, and energy demand modeling, providing full predictive distributions.
• Extensions include multi-quantile estimation, censoring adjustments, and robust loss formulations that address quantile crossing and tail behavior.

A Quantile Regression Neural Network (QRNN) is a neural network trained to estimate conditional quantiles $Q_Y(\tau \mid X=x)$ rather than a conditional mean, typically by minimizing the quantile, pinball, or check loss. In the literature, QRNN denotes both a general paradigm—neural quantile estimation for probabilistic prediction—and a family of concrete models ranging from feed-forward multilayer perceptrons to recurrent, censored, semiparametric, Bayesian, and extreme-value extensions. QRNNs are used to approximate predictive distributions, construct prediction intervals, and quantify aleatoric uncertainty in settings including precipitation correction, multi-step time-series forecasting, survival analysis under censoring, mobility-demand modeling, electricity forecasting, and conditional neural processes (Papacharalampous et al., 2024, Hatalis et al., 2017, Jia et al., 2020, Hüttel et al., 2021, Zhou et al., 2021, Mohseni et al., 2023).

1. Definition and loss-based formulation

For a quantile level $\tau \in (0,1)$, the conditional quantile function is defined by

$$Q_Y(\tau \mid X=x) \quad \text{such that} \quad \mathbb{P}(Y \le Q_Y(\tau \mid X=x)\mid X=x)=\tau.$$

A QRNN replaces a linear quantile predictor with a neural network $f_\theta(x)$, or in some formulations $f_\theta(x,\tau)$, and estimates parameters by minimizing the pinball loss

$$\rho_\tau(u)=u\big(\tau-\mathbf{1}_{\{u<0\}}\big) = \begin{cases} \tau u, & u \ge 0,\ (\tau-1)u, & u<0. \end{cases}$$

This is the standard objective in deep quantile regression, and it appears in feed-forward, recurrent, and multi-output QRNN formulations (Padilla et al., 2020).

Several strands of the literature emphasize the asymmetric-Laplace interpretation of this objective. In Bayesian QRNNs, the response is modeled with an asymmetric Laplace distribution whose location parameter is the network output, so maximizing the likelihood is equivalent to minimizing the check loss (Jantre et al., 2020). Conditional Quantile Neural Processes extend the same connection further by representing the predictive likelihood through mixtures of asymmetric Laplace components indexed by quantile levels, so that $\mu_\tau(x)$ functions as a conditional quantile while $\sigma_\tau(x)$ and $\alpha_\tau(x)$ parameterize scale and mixture weight (Mohseni et al., 2023).

For numerical optimization, several papers replace the nonsmooth pinball loss by smooth surrogates. One line uses a Huber-type check loss for censored survival data, which preserves the quantile objective while making the loss differentiable for backpropagation (Jia et al., 2020). Another uses Zheng’s smooth approximation in composite quantile forecasting for time series, with the smooth loss converging to the pinball loss as the smoothing parameter tends to zero (Hatalis et al., 2017).

2. Neural architectures and model families

The most common QRNN architecture is a feed-forward multilayer perceptron. In the precipitation-correction setting, QRNN is a standard feed-forward neural network trained to predict conditional precipitation quantiles from nine predictors: four distance-weighted IMERG variables, four distance-weighted PERSIANN variables, and station elevation; implementation uses the qrnn R package, and QRNN serves both as an individual predictor and as a stacking combiner (Papacharalampous et al., 2024). In DeepQuantreg, the QRNN is a fully connected feed-forward network with one to three hidden layers, dropout, and a single scalar output interpreted as the log conditional survival-time quantile (Jia et al., 2020).

QRNNs also appear in specialized structural variants. The panel semiparametric QRNN models

$$Q_\tau(Y_{it}\mid X_{it},Z_{it},\alpha_i)=Z_{it}^\top\beta_\tau+ANN(X_{it};\theta_\tau)+\alpha_i,$$

combining an interpretable linear component, province-specific effects, and a neural nonlinear component for electricity forecasting (Zhou et al., 2021). The partially linear QRNN uses the same basic decomposition,

$\tau \in (0,1)$0

with $\tau \in (0,1)$1 represented by a sparse deep ReLU network, so that variables of inferential interest remain parametric while the nuisance component is learned nonparametrically (Zhong et al., 2021).

Time-series work broadens the architectural picture further. The benchmark QRNN in the Quantile Fourier Neural Network study is a one-hidden-layer ReLU MLP trained with a composite smooth quantile loss, while the proposed Fourier variant replaces generic hidden units by cosine activations with learned frequencies and phases (Hatalis et al., 2017). Extreme Quantile Regression Neural Networks use recurrent architectures, including LSTM-based models, in a two-stage design where a recurrent QRNN estimates an intermediate conditional quantile and a second recurrent network estimates conditional tail parameters (Pasche et al., 2022).

A broader interpretation appears in neural-process models. Conditional Quantile Neural Processes retain the encoder–aggregator–decoder structure of neural processes but replace Gaussian predictive heads with quantile-conditioned asymmetric-Laplace mixtures, so quantile regression is performed in function space rather than in a single-task supervised setting (Mohseni et al., 2023).

3. Multi-quantile estimation and the non-crossing problem

A central distinction in the QRNN literature is between estimating one quantile at a time and estimating several quantiles jointly. Early and application-driven work often trains separate models for each $\tau \in (0,1)$2, whereas later multi-output formulations share hidden layers and output multiple quantiles simultaneously. In the student-growth-percentile setting, this joint problem is written with a composite pinball loss over a grid $\tau \in (0,1)$3, and the resulting NNQR is described as a multi-output neural network with shared representation across quantiles (Chang, 25 Oct 2025).

Joint modeling is attractive because it shares statistical strength and reduces computational overhead, but it raises the problem of quantile crossing. Independent estimation does not guarantee

$\tau \in (0,1)$4

and the student-growth-percentile paper argues that post-hoc isotonization creates an “interpolation paradox”: interpolation requires monotonicity, but the corrected quantiles may no longer satisfy the quantile property $\tau \in (0,1)$5 (Chang, 25 Oct 2025). That paper contrasts three approaches: independent QR, constrained joint quantile regression with explicit non-crossing constraints, and shared-network NNQR. Constrained joint QR is methodologically coherent but computationally expensive, with worst-case complexity scaling like $\tau \in (0,1)$6, whereas NNQR is trained with composite pinball loss and has per-epoch cost $\tau \in (0,1)$7; in the reported examples, NNQR reduces or eliminates crossing in practice, but no formal guarantee is claimed (Chang, 25 Oct 2025).

Several QRNN variants instead impose non-crossing by construction. Incremental (Spline) Quantile Functions define the output layer itself as a monotone quantile function: knot values are generated by cumulative non-negative increments, interpolation is monotone, and tails are handled by monotone parametric extrapolation. This yields quantile functions that do not cross and can be queried at arbitrary levels that were not used in training; the same framework also admits analytical CRPS evaluation (Park et al., 2021). The Sorting Composite Quantile Regression Neural Network prevents quantile crossing by inserting a differentiable sorting operator into a composite quantile network, so the outputs are sorted during training. In that formulation, SCQRNN has forward complexity $\tau \in (0,1)$8, simplifying to $\tau \in (0,1)$9 under the paper’s regime, compared with $$Q_Y(\tau \mid X=x) \quad \text{such that} \quad \mathbb{P}(Y \le Q_Y(\tau \mid X=x)\mid X=x)=\tau.$$0, simplifying to $$Q_Y(\tau \mid X=x) \quad \text{such that} \quad \mathbb{P}(Y \le Q_Y(\tau \mid X=x)\mid X=x)=\tau.$$1, for MCQRNN; on the U-bend dataset the reported mean epochs to convergence are $$Q_Y(\tau \mid X=x) \quad \text{such that} \quad \mathbb{P}(Y \le Q_Y(\tau \mid X=x)\mid X=x)=\tau.$$2 for SCQRNN and $$Q_Y(\tau \mid X=x) \quad \text{such that} \quad \mathbb{P}(Y \le Q_Y(\tau \mid X=x)\mid X=x)=\tau.$$3 for CQRNN (Decke et al., 2024).

A simpler compromise appears in applied studies that do not alter the network architecture. In the precipitation application, negative outputs are truncated at zero and crossed quantiles are corrected after prediction by replacing any quantile estimate smaller than the immediate lower quantile with the lower value, applied uniformly across algorithms including QRNN (Papacharalampous et al., 2024). This suggests that post-processing remains common in practice even when its statistical interpretation is weaker than joint monotone estimation.

4. Extensions for censoring, extremes, and robustness

One major branch of QRNN research adapts quantile regression to censored outcomes. DeepQuantreg models the conditional quantile of log survival time with a feed-forward neural network and handles right censoring through inverse-probability-of-censoring weights,

$$Q_Y(\tau \mid X=x) \quad \text{such that} \quad \mathbb{P}(Y \le Q_Y(\tau \mid X=x)\mid X=x)=\tau.$$4

inside a Huber-smoothed quantile loss. In this construction, censored observations have zero direct contribution and uncensored events are weighted by the inverse probability of remaining uncensored (Jia et al., 2020). In mobility-demand modeling, censored QRNNs are extended to multi-output form. The resulting Multi-CQNN estimates several quantiles jointly and uses a censored loss of the form

$$Q_Y(\tau \mid X=x) \quad \text{such that} \quad \mathbb{P}(Y \le Q_Y(\tau \mid X=x)\mid X=x)=\tau.$$5

which accommodates known left- or right-censoring thresholds and empirically yields fewer quantile crossings and lower computational overhead than fitting one censored model per quantile (Hüttel et al., 2021).

Another major extension targets the far tail of the conditional distribution. Extreme Quantile Regression Neural Networks first use a QRNN to estimate an intermediate quantile $$Q_Y(\tau \mid X=x) \quad \text{such that} \quad \mathbb{P}(Y \le Q_Y(\tau \mid X=x)\mid X=x)=\tau.$$6, then fit a conditional generalized Pareto tail with neural-network-parameterized $$Q_Y(\tau \mid X=x) \quad \text{such that} \quad \mathbb{P}(Y \le Q_Y(\tau \mid X=x)\mid X=x)=\tau.$$7 and $$Q_Y(\tau \mid X=x) \quad \text{such that} \quad \mathbb{P}(Y \le Q_Y(\tau \mid X=x)\mid X=x)=\tau.$$8, and finally extrapolate to more extreme quantiles via EVT. This is explicitly motivated by the failure of direct quantile-loss methods when $$Q_Y(\tau \mid X=x) \quad \text{such that} \quad \mathbb{P}(Y \le Q_Y(\tau \mid X=x)\mid X=x)=\tau.$$9 is small or finite, since empirical quantiles cannot extrapolate beyond the observed range (Pasche et al., 2022).

Robustness to anomalous inputs is a separate concern. Standard quantile regression is robust to response outliers but can be sensitive to outlier covariates. The $f_\theta(x)$0-quantile regression paper introduces a robust QRNN objective

$f_\theta(x)$1

derived from a robust divergence perspective. Because large pinball losses are exponentiated and then saturated, extreme observations receive small effective weight; in the reported MRI translation task, $f_\theta(x)$2-QR yields prediction error $f_\theta(x)$3 and quantile error $f_\theta(x)$4, compared with $f_\theta(x)$5 and $f_\theta(x)$6 for the non-robust baseline under contamination (Akrami et al., 2023).

A different generalization abandons the pinball loss entirely. “Generalized Quantile Loss for Deep Neural Networks” imposes the quantile property through an explicit count constraint on the number of predictions above or below the observations, while allowing any base regression loss, and optimizes the resulting problem through an alternating scheme because the count constraint has zero gradients almost everywhere (Or et al., 2020). This suggests that QRNNs can be defined either by their loss function or by the empirical quantile property they enforce.

5. Applications and reported empirical behavior

QRNNs are used wherever full predictive distributions or interval forecasts are operationally important. In precipitation correction, QRNN appears both as a base learner and as a stacking combiner alongside QR, QRF, GRF, GBM, and LightGBM. On 15 years of monthly gauge-measured and satellite precipitation in the contiguous United States, stacking with QR and stacking with QRNN are the two best-performing methods, improving over the QR reference by $f_\theta(x)$7 to $f_\theta(x)$8 across quantile levels from $f_\theta(x)$9 to $f_\theta(x,\tau)$0; sample coverages are close to nominal for all methods, including QRNN-based stacking (Papacharalampous et al., 2024).

In censored survival analysis, DeepQuantreg is evaluated on simulated nonlinear censored survival data and on the NKI70 and METABRIC breast-cancer datasets. The paper reports that traditional linear censored quantile regression performs worst, while DeepQuantreg competes strongly with and often outperforms nonparametric censored quantile smoothing splines, especially for higher quantiles and more complex group effects (Jia et al., 2020). In mobility-demand forecasting, Multi-CQNN is applied to synthetic censored data and to two shared-mobility datasets from the Copenhagen metropolitan area; the reported outcome is fewer quantile crossings and less computational overhead without compromising model performance (Hüttel et al., 2021).

In electricity forecasting, the panel semiparametric QRNN is applied to annual provincial electricity consumption in China. For the 2014–2018 test period, the reported total MAPE is $f_\theta(x,\tau)$1 for PSQRNN, compared with $f_\theta(x,\tau)$2 for SVM, $f_\theta(x,\tau)$3 for QRNN, and $f_\theta(x,\tau)$4 for BP, with the authors attributing the improvement to the combination of panel structure, semiparametric decomposition, and neural flexibility (Zhou et al., 2021). In multi-step time-series forecasting, the benchmark QRNN in the Quantile Fourier Neural Network study serves as a generic extrapolation-based quantile forecaster using time as the only input, but it is consistently weaker than the Fourier-structured alternative on datasets with strong periodic or cyclic structure (Hatalis et al., 2017).

QRNNs also appear in meta-learning and high-dimensional image translation. Conditional Quantile Neural Processes and Adaptive Conditional Quantile Neural Processes improve target log-likelihood over Gaussian neural-process baselines on multimodal synthetic processes, traffic speed-flow data, and image-completion benchmarks, with ACQNP reported as best overall in the synthetic multimodal setting (Mohseni et al., 2023). In medical imaging, robust $f_\theta(x,\tau)$5-QR is used inside a diffusion-model-based T1-to-T2 MRI translation pipeline to produce median and interval estimates that remain close to an outlier-free baseline even when lesion images contaminate training data (Akrami et al., 2023).

6. Statistical theory, calibration, and limitations

Several papers provide formal statistical guarantees for QRNNs. The Bayesian QRNN paper establishes posterior consistency for feed-forward neural-network quantile regression under a misspecified asymmetric-Laplace model, with the number of hidden nodes allowed to grow with the sample size and the posterior concentrating on the true conditional quantile function under stated entropy and sieve conditions (Jantre et al., 2020). A separate deep-ReLU analysis derives general risk bounds for quantile regression with neural networks and then proves minimax-optimal rates, up to logarithmic factors, on compositional Hölder classes and Besov spaces under minimal assumptions that allow heavy-tailed errors (Padilla et al., 2020). The partially linear quantile-regression-neural-network paper shows root-$f_\theta(x,\tau)$6 consistency and asymptotic normality for the parametric coefficient estimator together with the minimax optimal convergence rate for the neural nuisance estimator (Zhong et al., 2021).

Calibration is a recurrent empirical criterion, but the literature distinguishes it from proper scoring. In the precipitation study, sample coverage is close to nominal for all methods, including QRNN and QRNN-based stacking, yet the authors stress that coverage alone is not a strictly consistent ranking criterion and therefore use quantile skill scores for model comparison (Papacharalampous et al., 2024). In the SCQRNN study, overall reliability is defined by the discrepancy between nominal quantile levels and empirical coverages across all predicted quantiles, and SCQRNN is evaluated jointly on quantile RMSE, reliability, and convergence behavior (Decke et al., 2024). In I(S)QF, analytical CRPS is used precisely because it evaluates the entire learned quantile function rather than a small collection of nominal levels (Park et al., 2021).

The main limitations are also consistent across papers. Unconstrained joint multi-quantile QRNNs do not generally provide a formal guarantee of non-crossing, and shared hidden representation only reduces crossing empirically unless monotonicity is built into the architecture or optimization (Chang, 25 Oct 2025). Optimization can be difficult because composite pinball losses are nonsmooth and sometimes ill-conditioned, motivating smooth surrogates, full-batch quasi-Newton methods, or long training schedules (Chang, 25 Oct 2025, Jia et al., 2020). Performance gains from nonlinear combiners can be small when the second-stage predictor space is tiny; in the precipitation stacking study, QRNN and linear QR combiners are nearly identical because the stacking stage has only six predictors (Papacharalampous et al., 2024). This suggests that QRNN capacity is most useful when the covariate–quantile relationship is genuinely nonlinear, sufficiently sampled, or structurally complex.

In aggregate, the literature treats QRNN not as a single fixed model but as a broad class of neural quantile estimators: single-quantile or multi-quantile, direct or likelihood-based, feed-forward or recurrent, unconstrained or monotone by construction, and extendable to censoring, tail extrapolation, panel structure, robustness, and Bayesian inference. Across these variants, the defining feature remains unchanged: the network is trained to represent conditional quantiles and therefore to deliver distributional predictions rather than only point forecasts (Papacharalampous et al., 2024, Padilla et al., 2020).