---
title: Wasserstein Distributionally Robust Regret Optimization
url: https://www.emergentmind.com/topics/wasserstein-distributionally-robust-regret-optimization-drro
type: topic
---

# Wasserstein Distributionally Robust Regret Optimization

Searching arXiv for recent papers on Wasserstein distributionally robust regret optimization and closely related formulations.
arxiv_search(query="Wasserstein distributionally robust regret optimization", max_results=10)
arxiv_search(query="Distributionally Robust Regret Minimization Wasserstein", max_results=10)
Wasserstein Distributionally Robust Regret Optimization (DRRO) denotes a family of minimax decision models in which ambiguity about a probability law is represented by a Wasserstein ball, while performance is measured by regret rather than raw loss or cost. The literature now contains several non-equivalent variants: single-stage ex post regret for linear objective uncertainty over type-1 Wasserstein balls; ex-ante regret over type-\(p\) Wasserstein balls in static stochastic optimization; finite- and infinite-horizon control formulations over type-2 Wasserstein disturbance laws; and an RLHF formulation in which ambiguity is imposed on reward functions and reduced to promptwise \(\ell_1\) perturbations [2412.15406, 2504.10796, 2304.06783, 2307.04966, 2406.07248, 2508.10150, 2605.00155]. Across these variants, the defining feature is that the adversarial distribution is evaluated relative to the best action or policy for that same distribution, which separates DRRO from standard Wasserstein DRO.

## 1. Scope, terminology, and model classes

The expression “Wasserstein DRRO” does not identify a single canonical optimization problem. It covers a collection of models that share two ingredients: a Wasserstein ambiguity set and a regret benchmark. What changes across papers is the timing of the benchmark, the uncertainty object, and the decision class.

| Setting | Regret benchmark | Ambiguity geometry |
|---|---|---|
| Single-stage linear decision problem | realized cost vs perfect-information minimum | type-1 Wasserstein ball around \(P_0\) |
| Static ex-ante stochastic optimization | expected loss vs distribution-specific Bayes optimum | \(B_\delta^p(\mathbb{P}_0)\) |
| Finite- or infinite-horizon control | causal policy vs optimal noncausal controller | type-2 Wasserstein ball on disturbance laws |
| RLHF promptwise model | policy value vs best policy under same perturbed reward | 1-Wasserstein / promptwise \(\ell_1\) ambiguity |

This variation matters because ex post and ex ante regret are distinct objects. In the single-stage linear formulation, regret is realized after the uncertain coefficients are revealed; in the ex-ante static formulation, regret is defined as the gap in expected loss relative to the best decision for the same adversarial distribution; in control, regret is usually measured against a clairvoyant noncausal controller; and in RLHF, regret is measured against the best policy under the same plausible reward perturbation [2412.15406, 2504.10796, 2304.06783, 2605.00155].

A recurring misconception is that replacing worst-case risk by worst-case regret is only a cosmetic change. A conceptually related non-Wasserstein learning-theoretic result shows that worst-case risk need not imply uniformly small regret under distribution shift: in the paper’s counterexample, \(\sup_{P\in\mathcal Q}\mathrm{Reg}_P(f_{\mathrm{DRO}})=0.21\) whereas \(\sup_{P\in\mathcal Q}\mathrm{Reg}_P(f_{\mathrm{MRO}})=0.03\) [2202.05436]. This supports the central DRRO viewpoint that robustification should be applied to relative performance, not merely to absolute loss.

## 2. Canonical mathematical formulations

A basic ex post Wasserstein DRRO model is the single-stage linear decision problem with feasible set \(\mathcal X\subseteq\mathbb R^n\), uncertain objective coefficients \(w\in\mathbb R^n\), and realized cost \(w^\top x\). The regret of choosing \(x\) after \(w\) is realized is
\[
R(x,w):=w^\top x-\inf_{y\in\mathcal X} w^\top y
      =\sup_{y\in\mathcal X} w^\top(x-y).
\]
With a type-1 Wasserstein ball
\[
\mathcal P:=\{P\in\mathcal M(\mathbb R^n)\mid W_1(P,P_0)\le r\},
\]
the distributionally robust regret problem is
\[
\inf_{x\in\mathcal X}\sup_{P\in\mathcal P}\mathbb E_P[R(x,w)].
\]
Here \(P_0\) may be any probability measure with finite first moment, including an empirical distribution obtained from data [2412.15406].

A distinct ex-ante static formulation defines regret at the distribution level. For loss \(\ell:\Theta\times\Xi\to\mathbb R\), decision \(\theta\in\Theta\subseteq\mathbb R^d\), and type-\(p\) Wasserstein ball
\[
B_\delta^p(\mathbb P_0)=\{\mathbb P\in\mathcal P(\Xi):\mathcal W_p(\mathbb P,\mathbb P_0)\le \delta\},
\]
the worst-case ex-ante regret is
\[
R(\theta)=\sup_{\mathbb P\in B_\delta^p(\mathbb P_0)}
\left\{\mathbb E_{\mathbb P}[\ell(\theta,X)]-\inf_{\beta\in\Theta}\mathbb E_{\mathbb P}[\ell(\beta,X)]\right\},
\]
and Wasserstein DRRO is \(\inf_{\theta\in\Theta}R(\theta)\). The same paper distinguishes this from ex-post regret,
\[
R^{\mathrm{post}}(\theta)=\sup_{\mathbb P\in B_\delta^p(\mathbb P_0)}
\mathbb E_{\mathbb P}\!\left[\ell(\theta,X)-\inf_{\beta\in\Theta}\ell(\beta,X)\right],
\]
and notes that ex-post regret does not recover ERM when \(\delta=0\), whereas ex-ante regret does [2504.10796].

These formulations are related but not interchangeable. In the ex post linear model, the comparator depends on the realized scenario \(w\). In ex-ante DRRO, the comparator is the distribution-specific optimizer \(\beta^*(\mathbb P)\). In RLHF, the ex-ante comparator is a policy \(\beta\in\Pi\) maximizing value under the same perturbed reward distribution, and the problem becomes
\[
\min_{\pi\in\Pi}\max_{P\in\mathcal U}\left[\max_{\beta\in\Pi}V(\beta,P)-V(\pi,P)\right].
\]
The formal similarity hides different tractability mechanisms and different geometric effects [2605.00155].

## 3. Regularization, geometry, and limiting behavior

A central single-stage result states that if \(\mathcal X\subset\mathbb R^n\) is nonempty and compact, then for any \(x\in\mathcal X\),
\[
\sup_{P\in\mathcal P}\mathbb E_P[R(x,w)]
=
\mathbb E_{P_0}[R(x,w)]
+
r\sup_{v\in\mathcal X}\|x-v\|_*.
\]
Since \(\mathbb E_{P_0}[R(x,w)]\) differs from \(\mathbb E_{P_0}[w^\top x]\) only by an \(x\)-independent constant, the problem is equivalent to
\[
\inf_{x\in\mathcal X}
\left\{\mathbb E_{P_0}[w^\top x]+r\sup_{v\in\mathcal X}\|x-v\|_*\right\}.
\]
The induced regularizer is the radius of the smallest dual-norm ball centered at \(x\) that contains \(\mathcal X\), so increasing the Wasserstein radius pulls solutions toward a feasible center rather than toward the origin. The same geometric term appears in worst-case CVaR of regret:
\[
\sup_{P\in\mathcal P}\operatorname{CVaR}_\alpha^P(R(x,w))
=
\operatorname{CVaR}_\alpha^{P_0}(R(x,w))
+
\frac{r}{1-\alpha}\sup_{v\in\mathcal X}\|x-v\|_*.
\]
The paper also notes that this term coincides with the Lipschitz modulus of \(R(x,\cdot)\) with respect to the ground norm. By contrast, standard Wasserstein DRO for raw linear cost yields \(\mathbb E_{P_0}[w^\top x]+r\|x\|_*\), which pulls toward the origin [2412.15406].

Recent ex-ante analysis shows a different but related limiting picture. Under smoothness and regularity assumptions, the DRRO optimal value \(V(\delta)\) satisfies
\[
V'(0)=
\inf_{\theta^*\in\Theta^*}\sup_{\beta^*\in\Theta^*}
\left(
\mathbb E_{\mathbb P_0}
\left[
\|\nabla_x\ell(\theta^*,X)-\nabla_x\ell(\beta^*,X)\|_*^q
\right]
\right)^{1/q},
\qquad q=\frac{p}{p-1}.
\]
If the ERM solution set \(\Theta^*\) is a singleton, then \(V'(0)=0\), so DRRO coincides with ERM up to first-order terms. In the convex quadratic case with Euclidean transport, the equivalence is exact for all radii: if
\[
\ell(\theta,x)=
\begin{pmatrix}\theta\\x\end{pmatrix}^\top
\begin{pmatrix}Q&S^\top\\S&R\end{pmatrix}
\begin{pmatrix}\theta\\x\end{pmatrix}
+
2\begin{pmatrix}\theta\\x\end{pmatrix}^\top
\begin{pmatrix}q\\r\end{pmatrix},
\qquad Q\succ0,
\]
then the DRRO-optimal policy is
\[
\theta^*=-Q^{-1}(S^\top\hat\mu+q)
\]
for every \(\delta\ge0\), exactly as in ERM, and the regret is
\[
R(\delta,\theta^*)=\delta^2\lambda_{\max}(SQ^{-1}S^\top).
\]
This establishes an exact ERM/DRRO coincidence in that class [2504.10796].

These two strands are complementary. The ex post linear theory makes the regularization term explicit and geometric. The ex-ante static theory identifies when Wasserstein DRRO does not move the optimizer at first order, and when it never moves it at all.

## 4. Tractability, exact solvability, and hardness

Single-stage ex post DRRO is not uniformly easy. In general, evaluating
\[
\sup_{v\in\mathcal X}\|x-v\|_*
\]
is a norm maximization problem, and the paper remarks that such problems are NP-hard in general when \(\|\cdot\|_*\) is a \(p\)-norm with \(p\in[1,\infty)\) and \(\mathcal X\) is a compact convex polytope in halfspace form. Two tractable cases are identified. If \(\mathcal X=\operatorname{conv}\{v_1,\dots,v_m\}\), then
\[
\sup_{v\in\mathcal X}\|x-v\|_*=\max_{i=1,\dots,m}\|x-v_i\|_*,
\]
and DRRO becomes a finite-dimensional convex program; with polyhedral norms it is an LP, with Euclidean norm an SOCP. If \(\|\cdot\|_*=\|\cdot\|_\infty\), the regularizer admits a support-function reformulation valid for any nonempty compact \(\mathcal X\), yielding another finite-dimensional convex program when the support function is computable [2412.15406].

For ex-ante Wasserstein DRRO, the computational picture is sharper. In the scalar newsvendor with loss \(\ell(\theta,x)=b\theta-s\min(\theta,x)\), the paper proves that the regret function \(h(\beta;\theta)\) is concave on \([0,\theta]\) and on \([\theta,\infty)\). Hence
\[
R(\theta)=\max\!\left\{\sup_{\beta\in[0,\theta]}h(\beta;\theta),\;
\sup_{\beta\in[\theta,\infty)}h(\beta;\theta)\right\},
\]
so exact regret evaluation reduces to maximizing two one-dimensional concave functions. This extends Wasserstein newsvendor tractability to the regret setting [2504.10796].

Outside special structures, exact ex-ante computation is hard. For empirical reference distributions and max-affine losses, the same paper proves that evaluating \(R(\theta)\) is NP-hard in general, even when \(p=1\), the ground norm is \(\ell_1\), and the loss contains no bilinear \(x\)-\(\theta\) cross terms. The source of hardness is the supremum over the benchmark decision \(\beta\), not the Wasserstein worst-case expectation alone. To address this, the paper proposes a convex relaxation obtained by replacing bilinear products \(\gamma_{ik}\beta\) with auxiliary variables \(z_{ik}\) and perspective constraints. The relaxation is upper bounded by ex-post regret, and the paper reports that it improves over recent alternatives while remaining convex and broadly computable [2504.10796].

A plausible implication is that Wasserstein DRRO inherits the classical tractability of Wasserstein DRO only when the regret benchmark preserves convex structure. Once the comparator is itself optimized against the same adversarial law, exact solvability becomes substantially more delicate.

## 5. Control formulations

Control-theoretic Wasserstein DRRO is organized around linear-quadratic systems and regret relative to a noncausal benchmark. In the finite-horizon full-information formulation, the system is
\[
x_{t+1}=A_t x_t+B_t u_t+w_t,
\]
the cost is quadratic, and the regret of a strictly causal linear disturbance-feedback controller \(K\) is
\[
R(K,w)=J(Kw,w)-J(u^\star(w),w)
      =w^\top (K-K^\star)^\top D (K-K^\star) w,
\]
where \(u^\star(w)=K^\star w\) is the optimal noncausal controller with perfect knowledge of the disturbance trajectory. With a type-2 Wasserstein ball around a nominal law \(P_0\) on the stacked disturbance trajectory, the minimax expected-regret problem admits an exact semidefinite reformulation, and the worst-case distribution is the push-forward of \(P_0\) through the linear map \(z\mapsto \gamma^\star(\gamma^\star I-C)^{-1}z\) [2304.06783].

Under partial observability, the benchmark cannot be a universally pointwise-optimal noncausal controller, and the measurement-feedback formulation instead uses the noncausal controller minimizing the Frobenius norm of the closed-loop transfer operator. The resulting finite-horizon Wasserstein DRRO-MF problem combines Wasserstein quadratic duality with a Youla-type parameterization and a suboptimal Nehari approximation, yielding an SDP for fixed \(\gamma\) whose size is proportional to the time horizon [2307.04966].

The infinite-horizon formulation replaces finite stacked disturbances by a stationary disturbance process and studies
\[
\inf_{K\in\mathcal K}\overline R_\infty(K,r),
\qquad
\overline R_\infty(K,r)=
\limsup_{T\to\infty}\frac1T
\sup_{\mu_T\in\mathcal W_T(\mu_{\circ,T},r_T)}
\mathbb E_{\mu_T}[\mathrm{regret}_T(Kw,w)].
\]
Here the ambiguity set allows time correlation. In the stationary limit, the problem becomes a saddle-point problem over covariance operators constrained by a Bures-Wasserstein ball,
\[
\sup_{M\succ0}\inf_{K\in\mathcal K}\operatorname{tr}(R_KM)
\quad\text{s.t.}\quad
d_{BW}(M,M_\circ)\le r.
\]
The optimal controller is causal, LTI, and stabilizing, but generally non-rational; the paper nevertheless shows that it is characterized by a finite-dimensional parameter and computes it through a frequency-domain Frank–Wolfe method with an \(O(1/k)\) rate up to discretization error [2406.07248].

A later output-feedback formulation uses purified outputs \(u=K\eta+g\) and places a type-2 Wasserstein ball on the joint law of initial state, disturbances, and measurement noise. The paper derives strong duality results for general quadratic objectives, then eliminates first \(g\) and \(\beta\), and later the structured feedback variable \(K\), yielding a lower-dimensional SDP and an equivalent distributed optimization reformulation [2508.10150].

A closely related multistage ex-ante DRRO model studies finite-horizon LQR under a Gelbrich ball on the common stage-law moments. That paper states that the Gelbrich distance is a lower bound on squared \(2\)-Wasserstein distance and coincides with it for elliptical distributions, in particular Gaussian laws. This suggests a moment-based Wasserstein-type relaxation rather than a full Wasserstein ball, but the resulting SDP and the strictly causal empirical-mean correction it identifies are directly relevant to multistage regret robustness [2604.06158].

## 6. RLHF and reward misspecification

Wasserstein DRRO has also been formulated for reinforcement learning from human feedback, where the reward model is treated as a proxy for an unobserved true human-preference reward. In that setting, a policy \(\pi\) is evaluated under a distribution \(P\) over reward functions through
\[
V(\pi,P)=\mathbb E_{R\sim P}[J_R(\pi)],
\]
and ex-ante DRRO is
\[
\min_{\pi\in\Pi}\max_{P\in\mathcal U}
\left[\max_{\beta\in\Pi}V(\beta,P)-V(\pi,P)\right].
\]
A key simplification is that \(V(\pi,P)\) depends on \(P\) only through the mean reward \(\bar r(P)\). Starting from a 1-Wasserstein ambiguity set over reward-function distributions,
\[
\mathcal U=\{P:W(P,P_0)\le \rho\},
\]
the paper shows that \(\|\bar r(P)-\bar r(P_0)\|\le \rho\), and then specializes to a promptwise ambiguity model
\[
\mathcal U_\delta(r)=
\left\{
s:\|s(x,\cdot)-r(x,\cdot)\|_1\le \delta(x)\ \text{for every }x\in\mathcal X
\right\}.
\]
For a fixed prompt with response simplex variable \(p\in\Delta_n\), proxy reward vector \(r\), and perturbation \(s=r+\Delta\), the promptwise problem becomes
\[
\min_{p\in\Delta_n}\max_{\|\Delta\|_1\le \delta}\Reg(p,r+\Delta),
\qquad
\Reg(p,s)=\max_{\beta\in\Delta_n}(\beta-p)^\top s.
\]
The paper proves the exact inner solution
\[
\max_{\|\Delta\|_1\le \delta}\Reg(p,r+\Delta)
=
\delta+\max_i(r_i-\delta p_i)-p^\top r,
\]
with an adversary that can place all budget on a single coordinate. It then derives a water-filling optimal policy:
\[
p_i^\star=\frac{(r_i-t^\star)_+}{\delta},\quad i=2,\dots,n,
\qquad
p_1^\star=1-\frac1\delta\sum_{i=2}^n(r_i-t^\star)_+,
\]
where \(t^\star\) is defined through the threshold equations in the paper. As \(\delta\downarrow0\), the solution approaches the greedy policy \(e_1\); as \(\delta\to\infty\), it approaches the uniform distribution [2605.00155].

The same work translates the exact promptwise theory into a PPO/GRPO-style algorithm. In the hard version, the “dangerous” sampled response receives a bonus \(\delta(x)\); in the soft version, the bonus is weighted by self-normalized importance weights. A dynamic ambiguity budget,
\[
\delta_\theta(x)=\delta_0+\alpha\,\mathrm{KL}\!\big(\pi_\theta(\cdot\mid x)\,\|\,\pi_0(\cdot\mid x)\big),
\]
is motivated through a Donsker–Varadhan bound on reward misspecification. In the reported experiment, the main methods achieved the following peak held-out gold rewards: PPO \(1.20\), GRPO \(1.35\), DRO-RLHF \(0.54\), DRRO-RLHF (hard) \(1.29\), and DRRO-RLHF (soft + dynamic) \(1.59\); the corresponding peak proxy rewards were \(2.43\), \(3.06\), \(1.72\), \(2.87\), and \(3.03\), with peak KL values \(18.25\), \(44.83\), \(16.54\), \(32.33\), and \(35.74\) [2605.00155].

## 7. Statistical calibration, related frameworks, and boundaries

The direct statistical theory for Wasserstein DRRO is still fragmented, but adjacent Wasserstein-DRO results provide reusable tools. A general asymptotic analysis of Wasserstein DRO shows that if regret can be treated as a loss function \(\ell(\xi,x)=\mathrm{Regret}(x,\xi)\), then the robust objective
\[
\sup_{P:D_c(P_n,P)\le \delta} E_P[\mathrm{Regret}(x,\xi)]
\]
admits the standard dual reformulation
\[
\inf_{\lambda\ge0}
\left[
\lambda\delta+\frac1n\sum_{i=1}^n
\sup_{\zeta}\{\mathrm{Regret}(x,\zeta)-\lambda c(\xi_i,\zeta)\}
\right].
\]
The same work proposes a radius-calibration rule \(\delta\asymp n^{-1}\) for squared transport cost, more precisely \(\delta=n^{-1}\eta_{1-\alpha}\), based on an optimal-transport projection argument. It also interprets Wasserstein robustness as adaptive variation regularization, a perspective that transfers naturally whenever regret is sufficiently smooth [2108.02120].

A residuals-based conditional Wasserstein DRO framework with covariates shows how a Wasserstein ball can be centered at an \(x\)-specific residual-generated empirical distribution,
\[
\hat P_n^{ER}(x)=\frac1n\sum_{i=1}^n
\delta_{\operatorname{proj}_{\mathcal Y}(f_n(x)+\hat\varepsilon_n^i)},
\]
with radius \(\zeta_n(\alpha,x)=\kappa^{(1)}_{p,n}(\alpha,x)+\kappa^{(2)}_{p,n}(\alpha)\). This is not a regret paper, but it provides an explicit template for contextual Wasserstein DRRO by replacing cost with a benchmark-relative regret loss [2012.01088].

Distributed Wasserstein DRO supplies another reusable template. In that setting, a samplewise loss \(f(x,\xi)\) leads to the finite-dimensional robust objective
\[
\inf_{\lambda\ge0,x}
\left\{
\lambda\epsilon^2+\frac1N\sum_{k=1}^N
\max_{\xi\in\Xi}\bigl(f(x,\xi)-\lambda\|\xi-\hat\xi^k\|^2\bigr)
\right\},
\]
together with per-sample adversarial variables and a saddle-point dynamics. This is not specifically regret-based, but it becomes directly relevant when regret is expressible as a samplewise loss [1711.04839].

By contrast, online DRO over time with shrinking ambiguity sets develops dynamic regret bounds of order \(\mathcal O(\log T/\sqrt T)\), but its main theory is not Wasserstein-based. It is best viewed as a template for online DRRO with evolving ambiguity rather than as a direct Wasserstein DRRO result [2304.05377].

The current boundaries of Wasserstein DRRO are therefore sharp. Exact structure is available for single-stage linear ex-post regret, scalar newsvendor, quadratic control, and promptwise RLHF with \(\ell_1\) ambiguity. General max-affine ex-ante Wasserstein DRRO is NP-hard [2504.10796]. Several control results rely critically on linear-quadratic structure and quadratic Wasserstein duality [2304.06783, 2307.04966, 2406.07248, 2508.10150]. The RLHF formulation depends critically on promptwise \(\ell_1\) geometry for its coordinate-local adversary and sampled-bonus interpretation [2605.00155]. A reasonable summary is that Wasserstein DRRO is now a well-defined research area, but not yet a uniform methodology: its most explicit results remain highly structure-dependent.

Source: https://www.emergentmind.com/topics/wasserstein-distributionally-robust-regret-optimization-drro