Convolution-Smoothed Quantile Regression

• Convolution-smoothed quantile regression replaces the discontinuous check loss with a kernel-convoluted surrogate, yielding smooth gradients and Hessians for efficient optimization.
• The method enables derivative-based quantile density estimation and supports scalable penalized algorithms in high-dimensional, panel, and functional regression contexts.
• Kernel smoothing reduces estimation variance and bias while ensuring uniform asymptotic properties, facilitating robust inference in complex regression models.

Convolution-smoothed quantile regression is a family of methods that replaces the non-differentiable quantile check loss, or an equivalent discontinuous quantile score or moment condition, by a kernel-convolution surrogate. In the linear quantile model $Q(\tau\mid x)=x^\top\beta(\tau)$, a canonical construction writes the standard empirical objective as $$\widehat R(b;\tau)=n^{-1}\sum_{i=1}^n \rho_\tau(Y_i-X_i^\top b)$$ and then replaces it by a smoothed criterion $$\widehat R_h(b;\tau)=\int \rho_\tau(t)\,d\widehat F_h(t;b)=n^{-1}\sum_{i=1}^n (\rho_\tau*k_h)(e_i(b))$$, where $e_i(b)=Y_i-X_i^\top b$, $k_h(u)=k(u/h)/h$, and $*$ denotes convolution. This construction appears as “smoothing the objective function, rather than only the indicator on the check function,” and it has been used to obtain differentiable quantile paths, bandwidth-uniform asymptotics, scalable penalized algorithms, quantile-density estimation, and a range of extensions in panel, functional, transfer-learning, and neural-network settings (Fernandes et al., 2019, Man et al., 2022, Luo et al., 7 May 2026).

1. Formal construction and core variants

In standard linear quantile regression, the empirical criterion is

$$\widehat R(b;\tau)=\frac1n\sum_{i=1}^n \rho_\tau(Y_i-X_i^\top b), \qquad \rho_\tau(u)=u\bigl[\tau-\mathbf 1(u<0)\bigr].$$

The central difficulty is that $\rho_\tau$ is not differentiable at zero, so the map $b\mapsto \widehat R(b;\tau)$ is non-smooth, and the estimator path in $\tau$ can have jumps (Fernandes et al., 2019, Finn et al., 2024).

A defining convolution-type construction smooths the residual distribution before integrating the check loss. With residual cdf $$\widehat R(b;\tau)=n^{-1}\sum_{i=1}^n \rho_\tau(Y_i-X_i^\top b)$$0, smoothed density

$$\widehat R(b;\tau)=n^{-1}\sum_{i=1}^n \rho_\tau(Y_i-X_i^\top b)$$1

and corresponding smoothed cdf $$\widehat R(b;\tau)=n^{-1}\sum_{i=1}^n \rho_\tau(Y_i-X_i^\top b)$$2, the objective becomes

$$\widehat R(b;\tau)=n^{-1}\sum_{i=1}^n \rho_\tau(Y_i-X_i^\top b)$$3

with estimator

$$\widehat R(b;\tau)=n^{-1}\sum_{i=1}^n \rho_\tau(Y_i-X_i^\top b)$$4

This is the formulation studied in “Smoothing quantile regressions” and used as the quantile-regression stage in “Convolution Mode Regression” (Fernandes et al., 2019, Finn et al., 2024).

The resulting gradient and Hessian take explicit forms. Writing $$\widehat R(b;\tau)=n^{-1}\sum_{i=1}^n \rho_\tau(Y_i-X_i^\top b)$$5, the smoothed QR derivatives are

$$\widehat R(b;\tau)=n^{-1}\sum_{i=1}^n \rho_\tau(Y_i-X_i^\top b)$$6

and

$$\widehat R(b;\tau)=n^{-1}\sum_{i=1}^n \rho_\tau(Y_i-X_i^\top b)$$7

Thus the discontinuous indicator in the ordinary QR score is replaced by the smooth cdf-like term $$\widehat R(b;\tau)=n^{-1}\sum_{i=1}^n \rho_\tau(Y_i-X_i^\top b)$$8, and the Hessian becomes a kernel-weighted second-moment matrix (Finn et al., 2024).

A distinct but closely related line smooths the score or moment condition directly rather than starting from the full objective. In smoothed GMM for quantile models, the unsmoothed moment

$$\widehat R(b;\tau)=n^{-1}\sum_{i=1}^n \rho_\tau(Y_i-X_i^\top b)$$9

is replaced by

$$\widehat R_h(b;\tau)=\int \rho_\tau(t)\,d\widehat F_h(t;b)=n^{-1}\sum_{i=1}^n (\rho_\tau*k_h)(e_i(b))$$0

with $$\widehat R_h(b;\tau)=\int \rho_\tau(t)\,d\widehat F_h(t;b)=n^{-1}\sum_{i=1}^n (\rho_\tau*k_h)(e_i(b))$$1 a kernel of order $$\widehat R_h(b;\tau)=\int \rho_\tau(t)\,d\widehat F_h(t;b)=n^{-1}\sum_{i=1}^n (\rho_\tau*k_h)(e_i(b))$$2 (Castro et al., 2017). In panel quantile regression with fixed effects, the smoothed second-step loss replaces $$\widehat R_h(b;\tau)=\int \rho_\tau(t)\,d\widehat F_h(t;b)=n^{-1}\sum_{i=1}^n (\rho_\tau*k_h)(e_i(b))$$3 by $$\widehat R_h(b;\tau)=\int \rho_\tau(t)\,d\widehat F_h(t;b)=n^{-1}\sum_{i=1}^n (\rho_\tau*k_h)(e_i(b))$$4 and uses

$$\widehat R_h(b;\tau)=\int \rho_\tau(t)\,d\widehat F_h(t;b)=n^{-1}\sum_{i=1}^n (\rho_\tau*k_h)(e_i(b))$$5

as a smooth approximation to the check loss (Chen et al., 2019). In limited dependent variable models, the same logic appears as the substitution

$$\widehat R_h(b;\tau)=\int \rho_\tau(t)\,d\widehat F_h(t;b)=n^{-1}\sum_{i=1}^n (\rho_\tau*k_h)(e_i(b))$$6

with the cumulative distribution of a Gaussian kernel used in implementation (Alejo et al., 2021).

These constructions are not treated as interchangeable in the literature. One recurring distinction is between smoothing the entire objective via a smoothed residual cdf and smoothing only the indicator inside the check loss. The former is explicitly presented as different from Horowitz-style smoothing and as asymptotically preferable in several second-order respects (Fernandes et al., 2019).

2. Differentiability, quantile-density estimation, and uniform asymptotics

A central technical consequence of convolution smoothing is differentiability of the fitted quantile path in the quantile index. Because $$\widehat R_h(b;\tau)=\int \rho_\tau(t)\,d\widehat F_h(t;b)=n^{-1}\sum_{i=1}^n (\rho_\tau*k_h)(e_i(b))$$7 and $$\widehat R_h(b;\tau)=\int \rho_\tau(t)\,d\widehat F_h(t;b)=n^{-1}\sum_{i=1}^n (\rho_\tau*k_h)(e_i(b))$$8 is smooth, the Implicit Function Theorem yields

$$\widehat R_h(b;\tau)=\int \rho_\tau(t)\,d\widehat F_h(t;b)=n^{-1}\sum_{i=1}^n (\rho_\tau*k_h)(e_i(b))$$9

with $e_i(b)=Y_i-X_i^\top b$0 (Fernandes et al., 2019, Finn et al., 2024). This contrasts with ordinary QR, whose estimator path is described as piecewise linear, non-differentiable, or step-like in $e_i(b)=Y_i-X_i^\top b$1.

This asymptotic differentiability is exploited to estimate the conditional quantile density

$e_i(b)=Y_i-X_i^\top b$2

The smoothed estimator is

$e_i(b)=Y_i-X_i^\top b$3

which, in the linear QR model, avoids the curse of dimensionality because it leverages the parametric structure rather than smoothing in the full covariate space (Fernandes et al., 2019). This derivative-based construction is the mechanism behind the mode-regression estimator in “Convolution Mode Regression,” where the mode is recovered by minimizing the quantile density, or equivalently maximizing the negative quantile density (“sparsity”) (Finn et al., 2024).

The asymptotic theory developed for smoothed QR is explicitly uniform in both the quantile index and the bandwidth interval. In “Smoothing quantile regressions,” the smoothing bias satisfies

$e_i(b)=Y_i-X_i^\top b$4

uniformly over $e_i(b)=Y_i-X_i^\top b$5, and the paper states that the asymptotic theory holds uniformly with respect to the bandwidth and quantile level (Fernandes et al., 2019). The same paper states that the smoothed estimator has a smaller Bahadur–Kiefer remainder and lower asymptotic mean squared error than the standard estimator, and that smoothing reduces variance by an $e_i(b)=Y_i-X_i^\top b$6 correction: $e_i(b)=Y_i-X_i^\top b$7 (Fernandes et al., 2019).

In the mode-regression application, the smoothed Hessian estimator obeys the uniform bound

$e_i(b)=Y_i-X_i^\top b$8

uniformly in $e_i(b)=Y_i-X_i^\top b$9 and $k_h(u)=k(u/h)/h$0. This yields

$k_h(u)=k(u/h)/h$1

uniformly over $k_h(u)=k(u/h)/h$2, $k_h(u)=k(u/h)/h$3, and $k_h(u)=k(u/h)/h$4, and then

$k_h(u)=k(u/h)/h$5

$k_h(u)=k(u/h)/h$6

uniformly for $k_h(u)=k(u/h)/h$7 and $k_h(u)=k(u/h)/h$8 (Finn et al., 2024). The same paper emphasizes as an advantage that the estimator is uniform not only over design points $k_h(u)=k(u/h)/h$9 but also over the bandwidth interval.

3. Penalization, convex surrogates, and large-scale optimization

Convolution smoothing is especially prominent in high-dimensional penalized quantile regression because it turns the non-differentiable check loss into a smooth convex surrogate. In the generic penalized formulation,

$*$0

the smoothed loss

$*$1

is convex, twice differentiable, and locally strongly convex (Man et al., 2022). The resulting objective supports first-order smooth optimization rather than linear programming or subgradient methods.

A unified algorithmic template for penalized convolution-smoothed QR is Local Adaptive Majorize-Minimization (LAMM). Given current iterate $*$2, LAMM constructs the quadratic majorizer

$*$3

and updates

$*$4

The paper gives closed-form proximal updates for weighted lasso, elastic net, group lasso, and sparse group lasso, and states that each LAMM iteration is dominated by matrix-vector multiplication with per-iteration cost $*$5 (Man et al., 2022).

The same smooth-plus-penalty structure underlies concave-regularized high-dimensional QR. “High-Dimensional Quantile Regression: Convolution Smoothing and Concave Regularization” studies

$*$6

with gradient and Hessian

$*$7

$*$8

This paper proves that the smoothed empirical loss is twice continuously differentiable and locally strongly convex with high probability, and that iteratively reweighted $*$9-penalized smoothed QR achieves the optimal rate of convergence, the oracle rate, and the strong oracle property under an almost necessary and sufficient minimum signal strength condition (Tan et al., 2021).

Composite quantile regression admits the same treatment. In high-dimensional CQR, the empirical composite loss

$$\widehat R(b;\tau)=\frac1n\sum_{i=1}^n \rho_\tau(Y_i-X_i^\top b), \qquad \rho_\tau(u)=u\bigl[\tau-\mathbf 1(u<0)\bigr].$$0

is replaced by

$$\widehat R(b;\tau)=\frac1n\sum_{i=1}^n \rho_\tau(Y_i-X_i^\top b), \qquad \rho_\tau(u)=u\bigl[\tau-\mathbf 1(u<0)\bigr].$$1

The smoothed composite loss is convex, twice continuously differentiable, and locally strongly convex with high probability, and a gradient-based majorize-minimization algorithm substantially improves computational efficiency over ADMM without compromising statistical performance (Moon et al., 2022).

For practical implementation, the package conquer is repeatedly used as the software realization of convolution-smoothed quantile regression. In penalized settings it is implemented for several convex penalties (Man et al., 2022), and in the mode-regression application the estimation of $$\widehat R(b;\tau)=\frac1n\sum_{i=1}^n \rho_\tau(Y_i-X_i^\top b), \qquad \rho_\tau(u)=u\bigl[\tau-\mathbf 1(u<0)\bigr].$$2 “was done via the package \textsf{conquer}” (Finn et al., 2024).

4. Functional, panel, and limited-dependent-variable extensions

Convolution smoothing has been extended beyond finite-dimensional exogenous linear QR to settings where the unsmoothed problem is especially difficult.

In panel quantile regression with fixed effects, a two-step estimator replaces Canay’s unsmoothed second step by the smoothed objective

$$\widehat R(b;\tau)=\frac1n\sum_{i=1}^n \rho_\tau(Y_i-X_i^\top b), \qquad \rho_\tau(u)=u\bigl[\tau-\mathbf 1(u<0)\bigr].$$3

The paper derives explicit derivatives

$$\widehat R(b;\tau)=\frac1n\sum_{i=1}^n \rho_\tau(Y_i-X_i^\top b), \qquad \rho_\tau(u)=u\bigl[\tau-\mathbf 1(u<0)\bigr].$$4

$$\widehat R(b;\tau)=\frac1n\sum_{i=1}^n \rho_\tau(Y_i-X_i^\top b), \qquad \rho_\tau(u)=u\bigl[\tau-\mathbf 1(u<0)\bigr].$$5

and establishes

$$\widehat R(b;\tau)=\frac1n\sum_{i=1}^n \rho_\tau(Y_i-X_i^\top b), \qquad \rho_\tau(u)=u\bigl[\tau-\mathbf 1(u<0)\bigr].$$6

with an explicit incidental-parameter bias formula and both analytical and split-panel jackknife corrections (Chen et al., 2019). Here smoothing is not merely computational; it is used to derive the higher-order expansion exposing the fixed-effect bias.

In functional linear quantile regression, kernel convolution is used to smooth the loss under an RKHS formulation. The smoothed empirical risk is

$$\widehat R(b;\tau)=\frac1n\sum_{i=1}^n \rho_\tau(Y_i-X_i^\top b), \qquad \rho_\tau(u)=u\bigl[\tau-\mathbf 1(u<0)\bigr].$$7

with representer-theorem reduction to finite dimensions and Fréchet derivatives

$$\widehat R(b;\tau)=\frac1n\sum_{i=1}^n \rho_\tau(Y_i-X_i^\top b), \qquad \rho_\tau(u)=u\bigl[\tau-\mathbf 1(u<0)\bigr].$$8

The paper proves existence and uniqueness via the Banach fixed-point theorem, establishes a convergence rate, a functional Bahadur representation, pointwise asymptotic normality, and weak convergence of the estimated coefficient function (Sang et al., 2022).

A more elaborate functional extension is CLoSE, “Convolution-smoothing based Locally Sparse Estimation,” for scalar-on-function quantile regression with scalar and functional covariates. It uses

$$\widehat R(b;\tau)=\frac1n\sum_{i=1}^n \rho_\tau(Y_i-X_i^\top b), \qquad \rho_\tau(u)=u\bigl[\tau-\mathbf 1(u<0)\bigr].$$9

with gradient and Hessian driven by

$\rho_\tau$0

respectively. The method is designed to select significant functional covariates, identify nonzero regions of functional coefficients, and estimate those coefficients in one step; the paper establishes functional oracle properties, asymptotic normality, simultaneous confidence bands, and consistency of a split wild bootstrap (Liu et al., 1 Dec 2025).

In limited dependent variable models, smoothing is used to regularize threshold indicators arising in censored and binary QR. The Stata command ldvqreg replaces

$\rho_\tau$1

uses the cumulative distribution of a Gaussian kernel as the smoothing function, and adopts the rule

$\rho_\tau$2

for bandwidth selection (Alejo et al., 2021). The same paper reports that in binary examples the smoothed estimator “runs smoothly” where standard QR alternatives exhibit convergence failures.

5. Related smoothing paradigms and non-crossing quantile processes

The convolution-smoothed literature is explicit that different smoothing strategies target different objects and are not equivalent. One major contrast is between “smooth then estimate” and “estimate then smooth.” The former smooths the criterion before optimization, as in

$\rho_\tau$3

while the latter first computes ordinary QR at a grid of $\rho_\tau$4-values and only then smooths the estimated quantile curve. “Convolution Mode Regression” states this contrast directly and presents the “smooth then estimate” route as technically advantageous for obtaining uniformity in bandwidth (Finn et al., 2024).

A second contrast is between smoothing the entire objective and smoothing only the indicator in the check function. “Smoothing quantile regressions” writes Horowitz’s smoothed objective as

$\rho_\tau$5

whose derivative differs from the full-objective smoothing score by the extra term

$\rho_\tau$6

The paper states that this difference leads asymptotically to larger bias and larger variance for Horowitz’s estimator, and more generally that not all smoothing schemes are equivalent (Fernandes et al., 2019).

Borrowing strength across nearby quantiles can also be imposed without kernel convolution. “Spline Quantile Regression” formulates

$\rho_\tau$7

so smoothness is imposed directly on the coefficient process over $\rho_\tau$8 rather than by convolution of the quantile objective (Li et al., 7 Jan 2025). That paper explicitly presents spline QR as an alternative to ad hoc post-processing by kernel smoothing across quantiles. It also notes that smoothing across quantiles does not by itself guarantee noncrossing.

A different non-crossing mechanism appears in smoothed SGD for unconditional quantiles. There the indicator in the SGD score is replaced by the piecewise linear function $\rho_\tau$9, yielding the recursion

$b\mapsto \widehat R(b;\tau)$0

with $b\mapsto \widehat R(b;\tau)$1. The paper proves that for $b\mapsto \widehat R(b;\tau)$2,

$b\mapsto \widehat R(b;\tau)$3

for all $b\mapsto \widehat R(b;\tau)$4, so the estimated quantile curves do not cross (Chen et al., 19 May 2025). The same work explicitly links its recursive scheme to convolution-type smoothed quantile regression by writing

$b\mapsto \widehat R(b;\tau)$5

with a uniform kernel and iteration-dependent bandwidth.

6. Modern uses, bandwidth practice, and open limitations

The contemporary literature applies convolution-smoothed quantile regression to settings where standard QR is either computationally prohibitive or inferentially unstable.

In transfer learning for high-dimensional QR, the $b\mapsto \widehat R(b;\tau)$6-penalized smoothed estimator is

$b\mapsto \widehat R(b;\tau)$7

and the paper proposes a two-step smoothed transfer algorithm with adaptive source selection. It establishes $b\mapsto \widehat R(b;\tau)$8 error bounds and selection consistency under regular conditions (Zhang et al., 2022).

In prediction-powered inference, convolution smoothing is used to regularize both a score-debiasing estimator and a predict-then-debias estimator. The smoothed score takes the form

$b\mapsto \widehat R(b;\tau)$9

and the smoothed loss is

$\tau$0

The paper states that the proposed estimators are computationally tractable and that numerical studies show they mitigate the overcoverage observed with unsmoothed prediction-powered quantile regression (Takeishi et al., 2 Jun 2026).

In deep nonparametric QR, ConquerNet trains ReLU networks with the convolution-smoothed loss

$\tau$1

and proves that, under Besov smoothness assumptions and suitable network scaling, the estimator achieves

$\tau$2

with bandwidth restriction

$\tau$3

The numerical studies reported there show improved estimation accuracy and training efficiency across multiple quantile levels, with especially pronounced gains at high and low quantiles (Luo et al., 7 May 2026).

Bandwidth choice is a recurrent practical issue. The literature repeatedly balances a smoother objective against smoothing bias. In standard smoothed QR, a Silverman-style rule of thumb

$\tau$4

is recommended in simulations, with $\tau$5 computed from standard QR residuals (Fernandes et al., 2019). In mode regression, a $\tau$6-dependent rule

$\tau$7

is used, with $\tau$8 (Finn et al., 2024). In high-dimensional penalized SQR, the practical default is

$\tau$9

(Man et al., 2022). In prediction-powered quantile regression, simulations use

$$\widehat R(b;\tau)=n^{-1}\sum_{i=1}^n \rho_\tau(Y_i-X_i^\top b)$$00

(Takeishi et al., 2 Jun 2026).

The limitations emphasized in the literature are also consistent. Several foundational results depend on correctly specified linear quantile regression, bounded or compactly supported covariates, smooth conditional densities, and nontrivial kernel moment conditions (Fernandes et al., 2019, Finn et al., 2024). Some extensions remain specialized: the transfer-learning theory is high-dimensional and linear (Zhang et al., 2022); the deep-network minimax theory is developed for single-quantile estimation rather than full quantile processes (Luo et al., 7 May 2026); and the fixed-effects panel paper explicitly leaves optimal data-driven bandwidth choice as future work (Chen et al., 2019). These restrictions do not diminish the central role of convolution smoothing in the current literature, but they delimit where its strongest theory presently applies.