Distributionally Robust Quantile Regression

• The paper introduces a framework where quantile regression is robustified by minimizing the worst-case check loss over Wasserstein ambiguity sets.
• It employs a convex reformulation with an additive penalty and radius-dependent intercept correction for p>1, ensuring tractability.
• Finite-sample guarantees and dual norm geometry are used to provide robust out-of-sample risk bounds and enhanced stability against data shifts.

Searching arXiv for the specified papers and closely related work on distributionally robust quantile regression. Distributionally robust quantile regression (DR-QR) studies quantile regression under ambiguity about the data-generating law. In the Wasserstein formulation, one minimizes the worst-case expected check loss over all distributions in a type-$p$ Wasserstein ball around the empirical law, and this yields a tractable convex reformulation with an additive penalty $c_{\tau,p}\epsilon \|\overline\beta\|_*$ and, for $p>1$, a radius-dependent intercept correction (Zhang et al., 16 Mar 2026). A related robust line based on $\beta$-divergence motivates a distributionally robust interpretation through a $\beta$-divergence ball around the empirical distribution, but implements a tractable surrogate loss that down-weights high-loss samples and is used for robust uncertainty estimation in deep neural networks in the presence of outliers (Akrami et al., 2023).

1. Problem setup and core objects

In DR-QR with type-$p$ Wasserstein ambiguity sets, one observes i.i.d. samples $\{(X_i,Y_i)\}_{i=1}^N$ of a random pair $(X,Y)\in \mathbb{R}^d\times \mathbb{R}$ with unknown law $P$. For $\tau\in(0,1)$, quantile regression uses the check loss

$c_{\tau,p}\epsilon \|\overline\beta\|_*$0

where $c_{\tau,p}\epsilon \|\overline\beta\|_*$1 and $c_{\tau,p}\epsilon \|\overline\beta\|_*$2.

Let $c_{\tau,p}\epsilon \|\overline\beta\|_*$3 be a norm on $c_{\tau,p}\epsilon \|\overline\beta\|_*$4 and $c_{\tau,p}\epsilon \|\overline\beta\|_*$5. For distributions $c_{\tau,p}\epsilon \|\overline\beta\|_*$6 on $c_{\tau,p}\epsilon \|\overline\beta\|_*$7, the type-$c_{\tau,p}\epsilon \|\overline\beta\|_*$8 Wasserstein distance is

$c_{\tau,p}\epsilon \|\overline\beta\|_*$9

and

$p>1$0

Here $p>1$1 is the set of couplings. With the empirical distribution

$p>1$2

the ambiguity set is

$p>1$3

The dual norm associated with $p>1$4 is

$p>1$5

A central quantity is

$p>1$6

with dual norm $p>1$7. The DR-QR objective is

$p>1$8

This setup makes the geometry of the ambiguity set, the order $p>1$9, and the transport norm jointly determine the robustification.

2. Wasserstein DR-QR and the tractable reformulation

A key reduction projects the Wasserstein ball onto the one-dimensional residual variable. For any $\beta$0 and $\beta$1,

$\beta$2

where $\beta$3 is the law of $\beta$4 under $\beta$5 and $\beta$6 denotes the type-$\beta$7 ball with the same transport norm (Zhang et al., 16 Mar 2026). This one-dimensional reduction is the basis for the closed-form analysis of the worst-case loss.

Let $\beta$8 be the Hölder conjugate of $\beta$9, so that $\beta$0. Define

$\beta$1

Then for any $\beta$2 on $\beta$3 and $\beta$4,

$\beta$5

$\beta$6

and

$\beta$7

where

$\beta$8

At fixed $\beta$9, the inner supremum does not yield an additive Lipschitz correction when $p$0. However, after optimizing over the intercept, an exact identity holds:

$p$1

This leads to the tractable reformulation

$p$2

Moreover, if $p$3 solves the right-hand program, then an optimizer $p$4 for the left-hand DRO problem is obtained by

$p$5

In particular, the slope vector $p$6 is the minimizer of the regularized empirical risk with penalty $p$7, and the intercept is adjusted by a radius- and $p$8-dependent correction when $p$9.

3. Regimes of Wasserstein order and transport geometry

The distinction between $\{(X_i,Y_i)\}_{i=1}^N$0 and $\{(X_i,Y_i)\}_{i=1}^N$1 is qualitative rather than merely notational. For $\{(X_i,Y_i)\}_{i=1}^N$2, the optimizer $\{(X_i,Y_i)\}_{i=1}^N$3 from DR-QR coincides with the optimizer of the regularized problem

$\{(X_i,Y_i)\}_{i=1}^N$4

Conditional on $\{(X_i,Y_i)\}_{i=1}^N$5, DR-QR and the regularized formulation yield the same slopes $\{(X_i,Y_i)\}_{i=1}^N$6. The intercept differs:

$\{(X_i,Y_i)\}_{i=1}^N$7

Thus, the Wasserstein order $\{(X_i,Y_i)\}_{i=1}^N$8 affects only this correction through $\{(X_i,Y_i)\}_{i=1}^N$9 and $(X,Y)\in \mathbb{R}^d\times \mathbb{R}$0, while the slopes already solve the regularized objective (Zhang et al., 16 Mar 2026).

For $(X,Y)\in \mathbb{R}^d\times \mathbb{R}$1, the inner worst-case expectation equals $(X,Y)\in \mathbb{R}^d\times \mathbb{R}$2 plus the Lipschitz coefficient times $(X,Y)\in \mathbb{R}^d\times \mathbb{R}$3, so the DRO and regularized problems coincide without any intercept correction:

$(X,Y)\in \mathbb{R}^d\times \mathbb{R}$4

In certain edge cases, such as no mass on one side of the residual, the worst-case distribution may not be attained, reflecting the extremal geometry of the $(X,Y)\in \mathbb{R}^d\times \mathbb{R}$5 ball.

The transport norm determines the dual norm $(X,Y)\in \mathbb{R}^d\times \mathbb{R}$6 that shapes both the slope regularizer $(X,Y)\in \mathbb{R}^d\times \mathbb{R}$7 and the intercept correction magnitude via $(X,Y)\in \mathbb{R}^d\times \mathbb{R}$8. The following examples summarize the norm geometry.

Transport norm on $(X,Y)\in \mathbb{R}^d\times \mathbb{R}$9	Dual norm	$P$0
$P$1	$P$2	$P$3
$P$4	$P$5	$P$6
$P$7	$P$8	$P$9

This geometry also appears in the characterization of worst-case distributions. For optimal $\tau\in(0,1)$0, adversarial perturbations align with the dual norm direction of $\tau\in(0,1)$1. The constructions differ with $\tau\in(0,1)$2: extreme-tail concentration for $\tau\in(0,1)$3, uniform shifts for $\tau\in(0,1)$4, and quantile-weighted shifts for $\tau\in(0,1)$5.

The intercept correction has a direct out-of-sample interpretation. If $\tau\in(0,1)$6 minimizes the regularized problem with $\tau\in(0,1)$7, and

$\tau\in(0,1)$8

then

$\tau\in(0,1)$9

where $c_{\tau,p}\epsilon \|\overline\beta\|_*$00. If the regularized estimator undercovers, for example $c_{\tau,p}\epsilon \|\overline\beta\|_*$01 and $c_{\tau,p}\epsilon \|\overline\beta\|_*$02 on $c_{\tau,p}\epsilon \|\overline\beta\|_*$03, then DR-QR yields strictly lower out-of-sample loss.

4. Uniqueness of the check loss and finite-sample guarantees

For $c_{\tau,p}\epsilon \|\overline\beta\|_*$04, the check loss has a special status. Let $c_{\tau,p}\epsilon \|\overline\beta\|_*$05 be convex and define, for any distribution $c_{\tau,p}\epsilon \|\overline\beta\|_*$06,

$c_{\tau,p}\epsilon \|\overline\beta\|_*$07

There exists $c_{\tau,p}\epsilon \|\overline\beta\|_*$08 such that, for all $c_{\tau,p}\epsilon \|\overline\beta\|_*$09 on $c_{\tau,p}\epsilon \|\overline\beta\|_*$10 and $c_{\tau,p}\epsilon \|\overline\beta\|_*$11,

$c_{\tau,p}\epsilon \|\overline\beta\|_*$12

holds if and only if $c_{\tau,p}\epsilon \|\overline\beta\|_*$13 is an affine transform of the check loss:

$c_{\tau,p}\epsilon \|\overline\beta\|_*$14

for some $c_{\tau,p}\epsilon \|\overline\beta\|_*$15, $c_{\tau,p}\epsilon \|\overline\beta\|_*$16, and $c_{\tau,p}\epsilon \|\overline\beta\|_*$17 (Zhang et al., 16 Mar 2026). Thus, for $c_{\tau,p}\epsilon \|\overline\beta\|_*$18, quantile loss is the unique convex loss yielding an exact additive Wasserstein regularization once the intercept is optimized out.

The finite-sample out-of-sample guarantee is dimension-free. Fix $c_{\tau,p}\epsilon \|\overline\beta\|_*$19 and $c_{\tau,p}\epsilon \|\overline\beta\|_*$20. Let $c_{\tau,p}\epsilon \|\overline\beta\|_*$21 be the data-generating law of $c_{\tau,p}\epsilon \|\overline\beta\|_*$22 with finite $c_{\tau,p}\epsilon \|\overline\beta\|_*$23-th moment

$c_{\tau,p}\epsilon \|\overline\beta\|_*$24

for some $c_{\tau,p}\epsilon \|\overline\beta\|_*$25. For any $c_{\tau,p}\epsilon \|\overline\beta\|_*$26, with probability at least $c_{\tau,p}\epsilon \|\overline\beta\|_*$27 over the sample,

$c_{\tau,p}\epsilon \|\overline\beta\|_*$28

simultaneously for all $c_{\tau,p}\epsilon \|\overline\beta\|_*$29, where

$c_{\tau,p}\epsilon \|\overline\beta\|_*$30

with

$c_{\tau,p}\epsilon \|\overline\beta\|_*$31

Consequently, the DRO in-sample risk upper-bounds the true out-of-sample risk with $c_{\tau,p}\epsilon \|\overline\beta\|_*$32 rates, without a curse of dimensionality.

The same result yields a finite-sample, model-agnostic guarantee for regularized quantile regression. If $c_{\tau,p}\epsilon \|\overline\beta\|_*$33 minimizes

$c_{\tau,p}\epsilon \|\overline\beta\|_*$34

then for $c_{\tau,p}\epsilon \|\overline\beta\|_*$35 the in-sample DRO risk equals the regularized in-sample risk. Under the same moment conditions,

$c_{\tau,p}\epsilon \|\overline\beta\|_*$36

where

$c_{\tau,p}\epsilon \|\overline\beta\|_*$37

5. Optimization, computation, and practical use

In the empirical version, DR-QR reduces to the convex program

$c_{\tau,p}\epsilon \|\overline\beta\|_*$38

The optimality conditions reflect the standard quantile structure together with norm-based regularization. At optimum, $c_{\tau,p}\epsilon \|\overline\beta\|_*$39 is any empirical $c_{\tau,p}\epsilon \|\overline\beta\|_*$40-quantile of the residuals $c_{\tau,p}\epsilon \|\overline\beta\|_*$41, corresponding to the subgradient condition

$c_{\tau,p}\epsilon \|\overline\beta\|_*$42

For the slope,

$c_{\tau,p}\epsilon \|\overline\beta\|_*$43

Here $c_{\tau,p}\epsilon \|\overline\beta\|_*$44 is the subdifferential of the dual norm with respect to $c_{\tau,p}\epsilon \|\overline\beta\|_*$45 (Zhang et al., 16 Mar 2026).

The computational options stated for this program are proximal subgradient or proximal coordinate descent, conic or LP formulations for common norms via epigraph tricks, and a second-order cone formulation for $c_{\tau,p}\epsilon \|\overline\beta\|_*$46. Each proximal iteration is $c_{\tau,p}\epsilon \|\overline\beta\|_*$47, and convergence follows standard rates for nonsmooth convex optimization; strong convexity may arise with appropriate smoothing or additional ridge terms.

Practical calibration follows the same structure. Theory suggests $c_{\tau,p}\epsilon \|\overline\beta\|_*$48 with explicit calibration $c_{\tau,p}\epsilon \|\overline\beta\|_*$49, while in practice cross-validation over a grid, often logarithmic, works well; empirical studies show that the cross-validated $c_{\tau,p}\epsilon \|\overline\beta\|_*$50 tracks $c_{\tau,p}\epsilon \|\overline\beta\|_*$51. For $c_{\tau,p}\epsilon \|\overline\beta\|_*$52, DR-QR performs an intercept correction governed by $c_{\tau,p}\epsilon \|\overline\beta\|_*$53 through $c_{\tau,p}\epsilon \|\overline\beta\|_*$54 and $c_{\tau,p}\epsilon \|\overline\beta\|_*$55. Empirically, $c_{\tau,p}\epsilon \|\overline\beta\|_*$56 often yields stable performance; $c_{\tau,p}\epsilon \|\overline\beta\|_*$57 can be overly conservative and may suffer from non-attainment in edge cases. The transport norm sets the regularizer geometry through $c_{\tau,p}\epsilon \|\overline\beta\|_*$58 and the magnitude of the intercept correction, so feature and response scaling are important because $c_{\tau,p}\epsilon \|\overline\beta\|_*$59 includes the intercept coordinate.

Reporting fitted quantiles requires distinguishing the regularized intercept from the DRO-adjusted intercept. One first solves the regularized program to obtain $c_{\tau,p}\epsilon \|\overline\beta\|_*$60 with $c_{\tau,p}\epsilon \|\overline\beta\|_*$61. One then reports $c_{\tau,p}\epsilon \|\overline\beta\|_*$62 for $c_{\tau,p}\epsilon \|\overline\beta\|_*$63, and

$c_{\tau,p}\epsilon \|\overline\beta\|_*$64

for $c_{\tau,p}\epsilon \|\overline\beta\|_*$65. This single parameter $c_{\tau,p}\epsilon \|\overline\beta\|_*$66 jointly calibrates slope regularization and intercept robustness.

Relative to standard QR, DR-QR shrinks slopes and adjusts intercepts toward conservative quantiles, for example upward for upper quantiles, reducing under-coverage bias common in small $c_{\tau,p}\epsilon \|\overline\beta\|_*$67 and extreme $c_{\tau,p}\epsilon \|\overline\beta\|_*$68. By guarding against Wasserstein perturbations, it improves out-of-sample stability relative to standard QR and plain regularization that lacks intercept correction.

6. Related robust formulations, empirical behavior, and limitations

A distinct robust approach is "Beta quantile regression for robust estimation of uncertainty in the presence of outliers" (Akrami et al., 2023). Standard quantile regression minimizes

$c_{\tau,p}\epsilon \|\overline\beta\|_*$69

with the pinball loss written equivalently as

$c_{\tau,p}\epsilon \|\overline\beta\|_*$70

The paper adopts a form of $c_{\tau,p}\epsilon \|\overline\beta\|_*$71-divergence inspired by density power divergences and motivates a DRO interpretation

$c_{\tau,p}\epsilon \|\overline\beta\|_*$72

However, it does not explicitly optimize this constrained DRO objective. Instead, it derives and implements the tractable surrogate

$c_{\tau,p}\epsilon \|\overline\beta\|_*$73

with $c_{\tau,p}\epsilon \|\overline\beta\|_*$74 in the experiments. For a single quantile this becomes

$c_{\tau,p}\epsilon \|\overline\beta\|_*$75

The associated per-sample weight is

$c_{\tau,p}\epsilon \|\overline\beta\|_*$76

and

$c_{\tau,p}\epsilon \|\overline\beta\|_*$77

Robustness is therefore achieved via continuous down-weighting of high-loss samples rather than by discarding samples or introducing case-specific parameters.

The comparison baselines are least trimmed quantile regression,

$c_{\tau,p}\epsilon \|\overline\beta\|_*$78

where $c_{\tau,p}\epsilon \|\overline\beta\|_*$79 is a subset of samples with smallest errors, and robust regression via case-specific parameters,

$c_{\tau,p}\epsilon \|\overline\beta\|_*$80

The paper emphasizes that BQR keeps all samples but down-weights them, whereas TQR discards high-loss samples and RCP adds $c_{\tau,p}\epsilon \|\overline\beta\|_*$81 auxiliary parameters.

The empirical settings include the CYB OB1 star cluster dataset with $c_{\tau,p}\epsilon \|\overline\beta\|_*$82 observations and $c_{\tau,p}\epsilon \|\overline\beta\|_*$83 high-leverage points, a toy synthetic dataset with $c_{\tau,p}\epsilon \|\overline\beta\|_*$84 outliers, and a medical imaging translation task using diffusion models. On CYB OB1, the Frobenius norm between estimated quantiles and the inlier-only reference is reported as follows: TQR gives $c_{\tau,p}\epsilon \|\overline\beta\|_*$85, $c_{\tau,p}\epsilon \|\overline\beta\|_*$86, and $c_{\tau,p}\epsilon \|\overline\beta\|_*$87 for $c_{\tau,p}\epsilon \|\overline\beta\|_*$88, $c_{\tau,p}\epsilon \|\overline\beta\|_*$89, and $c_{\tau,p}\epsilon \|\overline\beta\|_*$90; RCP gives $c_{\tau,p}\epsilon \|\overline\beta\|_*$91, $c_{\tau,p}\epsilon \|\overline\beta\|_*$92, and $c_{\tau,p}\epsilon \|\overline\beta\|_*$93; and $c_{\tau,p}\epsilon \|\overline\beta\|_*$94-QR gives $c_{\tau,p}\epsilon \|\overline\beta\|_*$95, $c_{\tau,p}\epsilon \|\overline\beta\|_*$96, and $c_{\tau,p}\epsilon \|\overline\beta\|_*$97. In the diffusion-model experiment, the outlier-free model yields prediction error $c_{\tau,p}\epsilon \|\overline\beta\|_*$98; baseline QR with outliers yields prediction error $c_{\tau,p}\epsilon \|\overline\beta\|_*$99 and quantile error $p>1$00; $p>1$01-QR yields prediction error $p>1$02 and quantile error $p>1$03; and TQR yields prediction error $p>1$04 and quantile error $p>1$05. These results are reported as evidence that the robust surrogate substantially restores performance toward the outlier-free baseline.

The limitations of the two lines are different. For Wasserstein DR-QR, the risk bounds assume finite $p>1$06-moments, the penalty depends on $p>1$07 so scaling is important, and extending $p>1$08 formulations to high-dimensional sparsity-inducing penalties is described as a natural direction. Heteroscedasticity, covariate shift, and fixed-design settings are identified as structured cases in which targeted ambiguity sets or companion fixed-design analyses may be useful. For $p>1$09-QR, the paper states that formal guarantees such as finite-sample risk bounds or breakdown-point analysis are not provided, that $p>1$10 is fixed to $p>1$11, and that very large $p>1$12 can over-suppress informative hard examples while very small $p>1$13 approaches standard QR. The paper also notes that, in severe distribution shift beyond outliers, pure $p>1$14-divergence balls might be less effective than Wasserstein-based uncertainty sets.

Taken together, these results delineate two technically different notions of robust quantile regression. Wasserstein DR-QR admits an exact convex reformulation, a radius-dependent intercept correction for $p>1$15, and finite-sample out-of-sample guarantees of order $p>1$16 under mild moments. The $p>1$17-divergence approach provides a tractable robust surrogate with exponential down-weighting that is straightforward to integrate into SGD- and Adam-based deep learning workflows.