Minimax Risk: Definition, Bounds, and Applications

• Minimax risk is the smallest worst-case expected loss over a specified parameter class, observation model, loss function, and estimator class, making it a benchmark for statistical difficulty.
• Researchers derive minimax lower bounds by reducing estimation to testing and using tools such as packings, covering numbers, divergences, Fano inequalities, and Le Cam methods, while matching procedures establish optimal rates or exact constants.
• Applications span sparse regression, covariance estimation, predictive density estimation, classification, fairness-constrained learning, quantile guarantees, bandits, and local asymptotic analysis, with results depending strongly on model structure and performance criteria.

Minimax risk is the smallest worst-case statistical loss attainable over a prescribed class of parameters, probability laws, environments, or distributions. In a decision problem with parameter $\theta\in\Theta$, observation $X\sim P_\theta$, estimator $\widehat\theta$, metric $\rho$, and nondecreasing loss transform $\ell$, it is defined by

$$R^*=\inf_{\widehat\theta}\sup_{\theta\in\Theta} \mathbb E_\theta\!\left[\ell\bigl(\rho(\theta,\widehat\theta(X))\bigr)\right].$$

The estimator is selected before the worst-case parameter is chosen. Minimax analysis therefore characterizes irreducible difficulty under a specified loss, parameter space, observation model, and admissible estimator class. Its extensions include local asymptotic minimax risk, minimax regret, minimax quantile risk, distributionally robust minimax risk, and minimax procedures in interactive decision problems.

1. Decision-theoretic formulation and reductions

The minimax criterion depends jointly on the parameter space, statistical experiment, loss, and estimator class. Estimators may take values in a decision space $\mathcal D$ larger than $\Theta$, and the infimum ranges over measurable estimators. Different losses can produce different minimax estimators and rates; for example, quantum tomography has distinct conclusions under relative entropy, fidelity, or trace distance (Ferrie et al., 2015).

A fundamental lower-bound reduction replaces estimation by testing on a finite separated subset. Let $F\subseteq\Theta$ satisfy

$$\rho(\theta,\theta')\geq\eta \qquad(\theta\neq\theta',\ \theta,\theta'\in F).$$

Any estimator attaining distance less than $X\sim P_\theta$0 can be converted into a correct test over $X\sim P_\theta$1. Consequently,

$X\sim P_\theta$2

where

$X\sim P_\theta$3

is the minimax testing error. Introducing a prior $X\sim P_\theta$4 on $X\sim P_\theta$5 gives the Bayes testing risk

$X\sim P_\theta$6

and $X\sim P_\theta$7. Lower bounds on the Bayes testing risk therefore yield minimax lower bounds.

For a finite family with densities $X\sim P_\theta$8 relative to a common measure $X\sim P_\theta$9,

$\widehat\theta$0

For a uniform prior on $\widehat\theta$1, with $\widehat\theta$2, the Bayes risk is denoted $\widehat\theta$3 and satisfies

$\widehat\theta$4

The most general divergence-based formulation in the supplied literature uses an $\widehat\theta$5-divergence,

$\widehat\theta$6

where $\widehat\theta$7 is convex and $\widehat\theta$8. Important instances include Kullback–Leibler divergence, $\widehat\theta$9 divergence, Hellinger distance, total variation, reverse KL, and power divergences. For a finite family and prior $\rho$0, the testing theorem gives

$\rho$1

where $\rho$2 is arbitrary and $\rho$3 is determined by the Bayes rule. Under a uniform prior,

$\rho$4

The proof uses convexity and Jensen’s inequality rather than model-specific likelihood calculations (Guntuboyina, 2010).

For two distributions $\rho$5, total variation is the testing distance. If $\rho$6, then the uniform-prior Bayes error satisfies $\rho$7. Consequently,

$\rho$8

This single inequality yields two-point consequences for KL, reverse KL, $\rho$9, Hellinger, and total variation. Taking $\ell$0 produces Fano-type bounds, while the two-point case implies the sharp Pinsker inequality

$\ell$1

2. Divergence, entropy, and information-theoretic lower bounds

For a finite parameter packing, the relevant approximation quantity is

$\ell$2

It measures how tightly the corresponding distributions are packed in $\ell$3-divergence. Basic comparisons include

$\ell$4

A stronger construction approximates the entire family by a smaller family $\ell$5, with $\ell$6 and an assignment $\ell$7. A mixture argument gives

$\ell$8

For KL divergence this becomes

$\ell$9

and hence

$$R^*=\inf_{\widehat\theta}\sup_{\theta\in\Theta} \mathbb E_\theta\!\left[\ell\bigl(\rho(\theta,\widehat\theta(X))\bigr)\right].$$0

The construction is a generalized Yang–Barron mixture bound. It permits a large parameter-separated packing whose distributions need not be pairwise close; it suffices that they are approximable by a smaller mixture family (Guntuboyina, 2010).

Global entropy bounds combine parameter-space packings and model-space coverings. If $$R^*=\inf_{\widehat\theta}\sup_{\theta\in\Theta} \mathbb E_\theta\!\left[\ell\bigl(\rho(\theta,\widehat\theta(X))\bigr)\right].$$1 is a lower bound on the $$R^*=\inf_{\widehat\theta}\sup_{\theta\in\Theta} \mathbb E_\theta\!\left[\ell\bigl(\rho(\theta,\widehat\theta(X))\bigr)\right].$$2-packing number of $$R^*=\inf_{\widehat\theta}\sup_{\theta\in\Theta} \mathbb E_\theta\!\left[\ell\bigl(\rho(\theta,\widehat\theta(X))\bigr)\right].$$3 and $$R^*=\inf_{\widehat\theta}\sup_{\theta\in\Theta} \mathbb E_\theta\!\left[\ell\bigl(\rho(\theta,\widehat\theta(X))\bigr)\right].$$4 is an upper bound on the covering number of $$R^*=\inf_{\widehat\theta}\sup_{\theta\in\Theta} \mathbb E_\theta\!\left[\ell\bigl(\rho(\theta,\widehat\theta(X))\bigr)\right].$$5 in $$R^*=\inf_{\widehat\theta}\sup_{\theta\in\Theta} \mathbb E_\theta\!\left[\ell\bigl(\rho(\theta,\widehat\theta(X))\bigr)\right].$$6, then

$$R^*=\inf_{\widehat\theta}\sup_{\theta\in\Theta} \mathbb E_\theta\!\left[\ell\bigl(\rho(\theta,\widehat\theta(X))\bigr)\right].$$7

where, for KL divergence,

$$R^*=\inf_{\widehat\theta}\sup_{\theta\in\Theta} \mathbb E_\theta\!\left[\ell\bigl(\rho(\theta,\widehat\theta(X))\bigr)\right].$$8

and for $$R^*=\inf_{\widehat\theta}\sup_{\theta\in\Theta} \mathbb E_\theta\!\left[\ell\bigl(\rho(\theta,\widehat\theta(X))\bigr)\right].$$9 divergence,

$\mathcal D$0

Power-divergence versions are also available. The KL form can be suboptimal in finite-dimensional settings because it depends logarithmically on packing and covering numbers, whereas the $\mathcal D$1 form can preserve polynomial relationships.

These principles yield lower bounds in geometric and covariance problems. For convex-body reconstruction from noisy support-function measurements, an $\mathcal D$2 packing satisfies

$\mathcal D$3

and a $\mathcal D$4-based global entropy argument gives

$\mathcal D$5

For banded-type covariance classes under spectral loss, a Hamming-separated packing and a reduced mixture family yield

$\mathcal D$6

when $\mathcal D$7 (Guntuboyina, 2010).

3. Rates, exact constants, and structural complexity

Minimax risk can be characterized at different levels of precision. A rate statement identifies the order in sample size, whereas an asymptotically sharp result identifies the leading constant. In isotropic Gaussian random-design sparse regression, with

$\mathcal D$8

the squared-$\mathcal D$9 minimax risk satisfies, under $\Theta$0 and $\Theta$1,

$\Theta$2

The constant $\Theta$3 reflects support-selection complexity and the extreme-value behavior of Gaussian noise over irrelevant coordinates. The upper-bound construction combines a Lasso with a constrained maximum-likelihood estimator to control rare nearly singular design events (Guo et al., 2024).

For full random-design linear prediction under squared loss, the exact well-specified minimax risk is

$\Theta$4

where

$\Theta$5

Ordinary least squares attains this value for every nondegenerate covariate distribution when $\Theta$6. An equivalent leverage-score representation is

$\Theta$7

The universal lower bound

$\Theta$8

follows from exchangeability and Jensen’s inequality. Under Gaussian covariates, the exact value is

$\Theta$9

Under small-ball and moment conditions, the risk is

$F\subseteq\Theta$0

when $F\subseteq\Theta$1 (Mourtada, 2019).

Minimax rates also depend on empirical entropy. In random-design regression with squared loss, define the well-specified minimax risk

$F\subseteq\Theta$2

and the misspecified minimax regret

$F\subseteq\Theta$3

If the empirical entropy obeys

$F\subseteq\Theta$4

then the well-specified rate is

$F\subseteq\Theta$5

For misspecified regret, the aggregation-of-leaders estimator obtains

$F\subseteq\Theta$6

Thus minimax risk and minimax regret have the same rate for $F\subseteq\Theta$7, while for $F\subseteq\Theta$8 misspecification can make regret strictly slower. For VC-type classes, the optimal rate is

$F\subseteq\Theta$9

without requiring convexity of the function class (Rakhlin et al., 2013).

In inverse estimation, structural constraints need not improve the minimax exponent. For invertible bi-Lipschitz maps on a two-dimensional square under integrated squared loss, the direct, predictive inverse, and inverse-only risks have lower rate

$$\rho(\theta,\theta')\geq\eta \qquad(\theta\neq\theta',\ \theta,\theta'\in F).$$0

A geometric estimator based on boundary alignment, partitioning, and locally bijective interpolation attains the same rate up to logarithmic factors. Invertibility therefore does not reduce the polynomial minimax complexity in this setting (Okuno et al., 2021).

For operator learning between infinite-dimensional Hilbert spaces, the covariance spectrum of the input measure controls the minimax risk. For generic bounded Lipschitz operators,

$$\rho(\theta,\theta')\geq\eta \qquad(\theta\neq\theta',\ \theta,\theta'\in F).$$1

so no algebraic rate is possible. With exponentially decaying covariance eigenvalues $$\rho(\theta,\theta')\geq\eta \qquad(\theta\neq\theta',\ \theta,\theta'\in F).$$2, $$\rho(\theta,\theta')\geq\eta \qquad(\theta\neq\theta',\ \theta,\theta'\in F).$$3,

$$\rho(\theta,\theta')\geq\eta \qquad(\theta\neq\theta',\ \theta,\theta'\in F).$$4

The spectrum determines how close the subalgebraic rate can approach an algebraic one (Adcock et al., 19 Dec 2025).

4. Minimax procedures in prediction, classification, and structured decision problems

Minimaxity can concern not only parameter estimation but also predictive distributions, classifiers, fairness-constrained predictors, and distribution-specific performance.

In predictive density estimation for nonparametric regression, the loss is average KL divergence per future observation,

$$\rho(\theta,\theta')\geq\eta \qquad(\theta\neq\theta',\ \theta,\theta'\in F).$$5

For Sobolev ellipsoids with smoothness $$\rho(\theta,\theta')\geq\eta \qquad(\theta\neq\theta',\ \theta,\theta'\in F).$$6, the minimax predictive KL risk satisfies

$$\rho(\theta,\theta')\geq\eta \qquad(\theta\neq\theta',\ \theta,\theta'\in F).$$7

For an $$\rho(\theta,\theta')\geq\eta \qquad(\theta\neq\theta',\ \theta,\theta'\in F).$$8-ball with $$\rho(\theta,\theta')\geq\eta \qquad(\theta\neq\theta',\ \theta,\theta'\in F).$$9, it converges to

$X\sim P_\theta$00

A carefully selected diagonal Gaussian-prior predictive density is asymptotically minimax among all predictive-density estimators, not only among Gaussian-prior rules (Xu et al., 2010).

Minimax risk classification replaces the unknown distribution by an uncertainty set based on feature expectations:

$X\sim P_\theta$01

An MRC solves

$X\sim P_\theta$02

For $X\sim P_\theta$03-$X\sim P_\theta$04 loss, Fenchel duality reduces the infinite-dimensional problem to a convex optimization over feature coefficients $X\sim P_\theta$05:

$X\sim P_\theta$06

If the uncertainty set contains the true distribution with probability at least $X\sim P_\theta$07, the minimax value is a high-probability upper bound on the true classification risk. Characteristic-kernel feature maps yield strong universal consistency when the number of features increases and the confidence radii decrease appropriately (Mazuelas et al., 2022).

MRCpy implements this framework using convex optimization, accelerated subgradient methods, stochastic optimization, and constraint generation. Its variants include MRC, CMRC, AMRC for concept drift, and DWGCS for general covariate shift (Bondugula et al., 2021).

Fairness-constrained regression introduces an approximation component that persists even with infinite data. Let $X\sim P_\theta$08 be the weighted squared Wasserstein distance of group-wise prediction distributions to a common barycenter. An $X\sim P_\theta$09-relative improvement constraint requires

$X\sim P_\theta$10

Under atomlessness, the population oracle is

$X\sim P_\theta$11

and its risk is exactly

$X\sim P_\theta$12

The general minimax lower bound is the maximum of ordinary statistical difficulty and fairness-induced approximation cost:

$X\sim P_\theta$13

For Gaussian-design linear regression with group-dependent bias, the minimax scale is

$X\sim P_\theta$14

up to universal constants and confidence terms (Chzhen et al., 2020).

Minimax excess risk optimization, or MERO, compares distributions after subtracting their individual optimal risks:

$X\sim P_\theta$15

This differs from distributionally robust optimization, which minimizes $X\sim P_\theta$16 and may prioritize distributions with intrinsically high noise. Stochastic mirror-descent procedures estimate each $X\sim P_\theta$17 while solving the resulting biased convex-concave problem. The fixed-budget method achieves optimization error

$X\sim P_\theta$18

up to logarithmic factors, while an anytime method retains the $X\sim P_\theta$19 polynomial dependence with additional logarithms (Zhang et al., 2023).

5. Quantile minimax risk and local asymptotic minimax risk

Expected minimax risk averages the random loss and may conceal rare but severe failures. Minimax quantile risk instead evaluates the $X\sim P_\theta$20-quantile:

$X\sim P_\theta$21

The lower minimax quantile is

$X\sim P_\theta$22

They satisfy

$X\sim P_\theta$23

for $X\sim P_\theta$24, and quantile lower bounds imply expected-risk lower bounds through

$X\sim P_\theta$25

High-probability Le Cam and Fano inequalities replace expected loss by the indicator of a bad event. If two models have total variation less than $X\sim P_\theta$26, then a separation of $X\sim P_\theta$27 yields

$X\sim P_\theta$28

A KL sufficient condition is

$X\sim P_\theta$29

High-probability Fano bounds similarly preserve explicit dependence on $X\sim P_\theta$30. Applications include robust mean estimation, stochastic convex optimization, covariance estimation, sparse regression, nonparametric density estimation, and isotonic regression (Ma et al., 2024).

For random-design linear regression under realizable Gaussian noise, quantile minimax risk can be exact even for nonquadratic symmetric strictly convex error functions. Ordinary least squares is exactly minimax, with the minimax quantile determined by the conditional Gaussian law involving $X\sim P_\theta$31. For square loss,

$X\sim P_\theta$32

where, conditionally on the design,

$X\sim P_\theta$33

For sufficiently regular input distributions, this has scale

$X\sim P_\theta$34

Over a larger heteroscedastic class, a trimmed min-max regression estimator is minimax up to an absolute constant under fourth-moment conditions (Hanchi et al., 2024).

In interactive statistical decision problems, the same quantile framework applies to adaptive algorithms. For a two-armed Gaussian bandit with horizon $X\sim P_\theta$35,

$X\sim P_\theta$36

Thus the minimax quantile of cumulative regret is of order

$X\sim P_\theta$37

The result follows from an interactive Le Cam argument: two environments have pointwise regret sum $X\sim P_\theta$38, while their transcript KL divergence is $X\sim P_\theta$39 (Bongole et al., 7 Oct 2025).

Local asymptotic minimax risk concerns $X\sim P_\theta$40-local alternatives in locally asymptotically normal experiments. For a local risk sequence $X\sim P_\theta$41, the classical criterion uses

$X\sim P_\theta$42

whereas attainment arguments use

$X\sim P_\theta$43

The former can be satisfied by an estimator that performs well only along a subsequence; the latter requires uniform asymptotic performance. For regular estimator sequences the two criteria often coincide, but they can rank irregular sequences differently (Li, 17 Sep 2026).

For nonregular parameters of the form

$X\sim P_\theta$44

ordinary semiparametric efficiency theory may fail because $X\sim P_\theta$45 has no influence function. Under LAN, translation-scale equivariance of $X\sim P_\theta$46, and a Lipschitz transformation $X\sim P_\theta$47 with finitely many nondifferentiability points, the local asymptotic minimax risk reduces to

$X\sim P_\theta$48

The scalar $X\sim P_\theta$49 is a minimax bias-adjustment constant. At kinks, the maximal local slope convention can make the minimax risk discontinuous in the true probability. Local robustification over nearby probabilities does not remove this discontinuity (Song, 2014).

6. Applications, limitations, and interpretation

Minimax risk is not a universal scalar independent of modeling choices. It changes with the loss, parameter class, distributional uncertainty, confidence level, and whether the criterion is expected, local asymptotic, regret-based, or quantile-based. Several distinctions are therefore essential.

Worst-case versus pointwise performance. A minimax estimator equalizes or controls the largest risk, and may sacrifice performance at easy parameters. In quantum tomography under relative entropy, exact minimax estimators are strongly biased toward difficult near-pure states, while hedged maximum likelihood retains $X\sim P_\theta$50 risk in full-rank interiors and is nearly minimax in the worst case (Ferrie et al., 2015).

Statistical error versus approximation cost. In fairness-constrained regression, ordinary estimation error decreases with sample size, whereas the cost of modifying the Bayes rule to satisfy a fairness constraint persists even with infinite data. In misspecified learning, minimax regret can likewise be slower than well-specified minimax risk when empirical entropy is sufficiently large (Rakhlin et al., 2013).

Expected risk versus high-probability risk. Expected minimaxity does not imply minimaxity at a specified confidence level. Quantile minimax analysis exposes confidence penalties such as $X\sim P_\theta$51 and can remain informative when expected risk is infinite under contamination or heavy tails (Ma et al., 2024).

Exact versus rate-optimal results. Exact minimaxity is available in selected settings: OLS for well-specified random-design linear prediction, OLS for realizable Gaussian regression under broad symmetric error evaluations, and a nonlinear shrinkage procedure for finite-population randomized experiments. In the latter problem, the exact minimax risk for binary potential outcomes has the expansion

$X\sim P_\theta$52

where $X\sim P_\theta$53 is characterized through the Airy function. Balanced Bernoulli randomization combined with the optimal nonlinear estimator attains this finite-sample minimax risk, while standard procedures are only first-order minimax (Sudijono et al., 13 Aug 2026).

Infinite-dimensional complexity. For generic Lipschitz operators between infinite-dimensional spaces, no algebraic minimax decay in sample size is possible. Covariance-spectrum decay improves the subalgebraic rate but does not eliminate the curse of sample complexity for the generic operator class (Adcock et al., 19 Dec 2025).

Computational formulations. Minimax problems that appear infinite-dimensional can admit finite convex duals. MRCs reduce distributional uncertainty over probability measures to optimization in feature coefficients; MERO reduces multiple distribution-specific excess risks to a stochastic saddle-point problem; and divergence-based testing converts estimation lower bounds into finite-hypothesis testing problems.

The principal limitations are correspondingly structural. Lower bounds may depend on carefully constructed packings, covering numbers, or moment conditions. Exact constants can require Gaussian designs or highly specific asymptotic regimes. Robust procedures may achieve minimaxity only up to absolute constants. Quantile risk can be infinite when the sample covariance is rank deficient or when contamination permits catastrophic events. Local asymptotic criteria need not approximate finite-sample risk uniformly near nondifferentiability points. Finally, minimax optimality is a worst-case property and does not, by itself, imply favorable average-case, pointwise, computational, or inferential behavior.