---
title: Minimax Weighted Expected Regret (MWER)
url: https://www.emergentmind.com/topics/minimax-weighted-expected-regret-mwer
type: topic
---

# Minimax Weighted Expected Regret (MWER)

Minimax Weighted Expected Regret (MWER) is a decision-theoretic criterion that generalizes classical minimax expected regret to settings where uncertainty is represented not by a single probability measure, nor by an unweighted set, but by a weighted set of probability measures. This framework provides a rigorous approach for robust decision-making under ambiguity, interpolating smoothly between Bayesian expected utility and traditional minimax regret, and supports a fully axiomatic characterization in both static and dynamic (updating) settings [1302.5681][1210.4853]. MWER has been developed across decision theory, reinforcement learning, and online learning, delivering tight theoretical bounds and tractable algorithms for robust yet adaptive choice.

## 1. Foundations: Weighted Sets of Probabilities and Regret-Based Decision Rules

MWER is defined in terms of a *weighted set of probabilities* on a finite state space $S$. Let $P^+ = \{ (Pr, \alpha_{Pr}) : Pr \text{ is a probability on } S,\, 0 \le \alpha_{Pr} \le 1 \}$, where each $\alpha_{Pr}$ quantifies the significance or credibility of $Pr$. The normalization constraint $\sup_{Pr} \alpha_{Pr} = 1$ ensures comparability across measures.

Given a set $X$ of possible prizes and a utility function $u:X\to\mathbb{R}$, a Savage act is $f:S\to X$. For every feasible act $f$ and menu $M$ of available acts, the ex post optimal utility in state $s$ is $u^*_M(s) = \max_{g \in M} u(g(s))$. The regret of act $f$ in $s$ is $reg_M(f, s) = u^*_M(s) - u(f(s))$, with expected regret under $Pr$ given by $EReg_{M,Pr}(f) = \sum_{s} Pr(s)\, reg_M(f,s)$. The **weighted expected regret** (WER) for $f$ is
$$
WER_{M,P^+}(f) = \sup_{Pr}\left[ \alpha_{Pr}\, EReg_{M,Pr}(f) \right].
$$
The MWER decision rule selects any act in $M$ minimizing this quantity.

When all weights are unity (i.e., $\alpha_{Pr} = 1$ for all $Pr$), MWER reduces to standard minimax regret. When $P$ is a singleton, MWER becomes subjective expected utility maximization, making it a natural generalization capable of interpolating between fully robust and fully Bayesian behaviors [1302.5681][1210.4853].

## 2. Weight Assignment and Likelihood-Based Updating

Initial weights $\alpha_{Pr}$ can arise from subjective confidence, expert judgment, or second-order priors. The only requirement is proper normalization ($\sup_{Pr} \alpha_{Pr} = 1$).

Upon observing new information $E \subseteq S$ (with $\sup_{Pr}\alpha_{Pr}\,Pr(E) > 0$), MWER employs a **likelihood updating** rule. Each original $(Pr, \alpha_{Pr})$ with $Pr(E)>0$ is replaced by $(Pr(\cdot | E), \alpha_{Pr|E})$, where
$$
\alpha_{Pr|E} = \frac{\sup\{ \alpha_{Pr'} Pr'(E) : Pr'(\cdot|E) = Pr(\cdot|E)\}}{\sup_{Pr''}\alpha_{Pr''}Pr''(E)}
$$
and all measures yielding the same conditional are merged by taking the supremum of possible weights. This approach ensures that the updated set is again normalized.

This updating preserves **consistency**, in that sequential updates commute: $(P^+|E)|F = (P^+|F)|E = P^+|(E\cap F)$. Additionally, under repeated observations generated by some $Pr^*\in P$, the weights concentrate: $\alpha_{Pr^*|E_1\ldots E_n}\to 1$ almost surely, so MWER converges to expected utility under the true $Pr^*$ [1302.5681][1210.4853].

## 3. Axiomatic Characterization: Static and Dynamic

MWER is fully characterized by an axiomatic system within the Anscombe–Aumann framework. For static (single-stage) choice, the following conditions must be satisfied (for every menu $M$):

1. **Transitivity**: If $f\succeq_M g$ and $g\succeq_M h$, then $f\succeq_M h$.
2. **Completeness**: For any $f,g$, either $f\succeq_M g$ or $g\succeq_M f$.
3. **Non-triviality**: There exist $M,f,g$ with $f\succ_M g$.
4. **Monotonicity**: If $f$ state-wise dominates $g$, then $f\succeq_M g$.
5. **Mixture continuity**: Preferences are continuous under convex mixtures.
6. **Ambiguity aversion**: If $f\sim_M g$, then $p f\oplus(1-p)g \succeq_{M\cup\{p f\oplus(1-p)g\}} g$.
7. **Independence**: Preference over $f,g$ is stable under independent mixing with any $h$.
8. **Menu-independence for constants**: For constant acts, preferences do not depend on the menu.
9. **INA**: Adding acts never strictly optimal in any state does not change relative preferences among the rest.
10. **Boundedness**: Every menu admits a dominating constant act.

There is a representation theorem: preferences satisfying these axioms correspond precisely to MWER, with unique (up to affine transformation) utility and maximal normalized $P^+$ [1302.5681][1210.4853].

In dynamic settings, with sequential observations, an additional axiom applies:

- **Menu-Dependent Dynamic Consistency (MDC)**: If, after learning $E$, $f$ is preferred to $g$, then before learning $E$, the conditional act "play $f$ on $E$, $h$ otherwise" is preferred to the analogous $g$-plan.

This extension ensures that, after likelihood updating, preferences continue to admit a MWER representation with the appropriately updated $P^+$.

## 4. MWER in Robust Sequential Learning and Reinforcement Learning

MWER admits a natural formalization for sequential decision-making problems, notably in reinforcement learning (RL) and online learning [2410.16013][1301.6058]. Given an unknown Markov Decision Process (MDP) parameterized by $\theta$, the regret of a policy $\pi$ is
$$
R(\pi, \theta) = U^*(\theta) - U(\pi, \theta)
$$
where $U(\pi, \theta)$ is the expected cumulative reward and $U^*(\theta)$ the value of the optimal policy for $\theta$. Defining a weighted prior $w \in \Delta(\Theta)$, the weighted expected regret is $R_w(\pi) = \mathbb{E}_{\theta \sim w} [R(\pi, \theta)]$. The minimax weighted expected regret is then
$$
\mathrm{MWER} = \sup_{w \in \Delta(\Theta)} \inf_{\pi} R_w(\pi) = \sup_{w}\inf_{\pi} \mathbb{E}_{\theta\sim w}[ R(\pi,\theta) ].
$$

A minimax duality theorem shows that, under standard regularity (convexity, compactness, continuity), MWER coincides with classical minimax regret:
$$
\inf_{\pi} \sup_{\theta} R(\pi, \theta) = \sup_{w} \inf_{\pi} \mathbb{E}_{\theta\sim w} R(\pi, \theta)
$$
[2410.16013].

MWER enables the direct use of information-theoretic Bayesian regret bounds to obtain robust minimax rates, including for finite-horizon MDPs, linear and contextual bandits. For example, in multi-armed bandits with $K$ arms and $T$ rounds, MWER achieves $O(\sqrt{K \ln K\, T})$ regret. This framework reduces robust sequential learning to the optimization of weighted expected regret, facilitating tractable approximation and computation via duality and game-theoretic techniques [2410.16013].

## 5. Algorithmic Realization: Weighted Minimax in Online Learning

In online linear regression with adversarial labels, MWER is instantiated by the Weighted Last-Step Min-Max (WEMM) algorithm [1301.6058]. At each round $t$, the algorithm predicts using the weighted least-squares solution formed from the history, with weights $a_t$ selected so as to ensure feasibility of the min-max saddle point. The weighted cumulative loss for a comparator $u$ is
$$
L_T^{\alpha}(u) = \sum_{t=1}^T a_t (y_t - u^\top x_t)^2
$$
and the algorithm guarantees
$$
L_T(\mathrm{WEMM}) \leq \inf_{u} [b\|u\|^2 + L_T^{\alpha}(u)]
$$
for any feasible weight sequence, delivering **zero weighted minimax regret**.

By careful design, including recursive updates and data-driven choice of $a_t$, the difference between $L_T^{\alpha}(u)$ and the standard unweighted loss $L_T(u)$ can be controlled, yielding logarithmic or sub-logarithmic regret in $T$ when the data or labels are favorable. The approach extends to weakly non-stationary environments, where regret is measured relative to slowly drifting comparators.

Compared to prior last-step min-max forecasters that required known bounds and uniform weights, WEMM achieves improved constants, relaxes the need for a-priori adversarial bounds, and is competitive in environments with mild non-stationarity [1301.6058].

## 6. Relation to Classical Decision Criteria and Properties

MWER rigorously interpolates between minimax expected regret (MER) and subjective expected utility (SEU):

- When all $\alpha_{Pr}=1$, MWER coincides with MER, fully robust to ambiguity.
- When $|P|=1$ and $\alpha_{Pr^*}=1$, MWER coincides with SEU, fully Bayesian.
- As learning progresses and likelihood-based updating concentrates, MWER transitions smoothly from MER to SEU, capturing learning from data [1302.5681][1210.4853].

Distinctive features of MWER include:

- **Ambiguity sensitivity:** MWER handles ambiguity aversion through its axiomatic basis.
- **Menu dependence:** Preferences can depend on the set of available acts, reflecting the regret criterion's sensitivity to alternative actions, in contrast to maximin expected utility (MMEU), which is menu-independent.
- **Dynamic consistency:** Through its updating rule and dynamic axioms, MWER ensures that plans made before and after new information are revealed are mutually consistent in behavior.
- **Overcoming set-model limitations:** MWER remedies the inability of pure set-based probability models to learn relative likelihoods through data and avoids the collapse to SEU of second-order probability approaches.

## 7. Illustrative Example and Implications

A representative example is the delivery robot problem, with two possible states (“1 broken cake”, “10 broken cakes”) and three acts (“continue”, “back”, “check”). Initial symmetric weights $\alpha_{Pr_1} = \alpha_{Pr_{10}} = 1$ lead MWER to behave identically to MER. With repeated favorable observations (e.g., "first $N$ cakes are unbroken"), likelihood updating increases the weight on the more plausible $Pr_1$, and MWER shifts toward SEU-optimal actions for $Pr_1$. This demonstrates MWER’s ability to interpolate between robust and data-driven decision-making, adapting preferences as weights evolve through learning [1302.5681][1210.4853].

In summary, MWER unifies and extends foundational approaches to robust choice under ambiguity, admits explicit and tractable updating, supports strong theoretical guarantees across decision theory, online learning, and reinforcement learning, and is characterized through natural and interpretable axioms. The framework enables both principled robust decision-making and smooth adaptation as information accrues.

Source: https://www.emergentmind.com/topics/minimax-weighted-expected-regret-mwer