Minimax-Optimal Estimators: Theory & Practice

• Minimax-optimal estimators are statistical procedures that achieve nearly the lowest worst-case risk over a set of signals using a prescribed loss function.
• Truncated series estimators use the Kolmogorov width to balance geometric approximation errors with noise, often achieving performance within an O(log m) factor of the minimax risk.
• The approximation radius, linked to projection metrics through duality, provides tight volume-based lower bounds, guiding the design of optimal high-dimensional estimators.

A minimax-optimal estimator is a statistical procedure that attains, up to a universal constant or prescribed factor, the lowest possible maximum (worst-case) risk over a prescribed class of signals or models and with respect to a chosen loss function. The minimax framework provides a benchmark for evaluating estimator design under ambiguity or structural constraints, balancing inherent geometric complexity with noise, and is central to modern high-dimensional and shape-constrained statistics.

1. Foundations of Minimax Risk and Estimation

The minimax risk for an estimation problem quantifies the smallest achievable worst-case risk over all possible estimators, given that the unknown parameter or function belongs to a set X. For normal means problems with observations $y = x + w$ and $w \sim N(0, \sigma^2 I)$, the risk of estimator $M$ is: $$R(M, X, \sigma) = \sup_{x \in X} \mathbb{E}_{y|x} \|x - M(y)\|^2,$$ and the minimax risk is

$R(X, \sigma) = \inf_{M} R(M, X, \sigma).$

A minimax-optimal estimator is one achieving this infimum exactly, or up to a factor dictated by geometry or structural specifics of X.

The critical value of minimax risk arises in regularizing estimator design, benchmarking estimator performance, and formalizing notions of "difficulty" in statistical recovery under specified constraints or adversarial models.

2. Truncated Series Estimators and Kolmogorov Width

In linear inverse problems and high-dimensional estimation over convex constraint sets, truncated series estimators (TSEs) are prominent minimax-optimal strategies, especially over symmetric convex polytopes. A TSE projects the observation onto a carefully chosen k-dimensional subspace: $M(y) = P y,$ where $P = P_k$ is an orthogonal projection. The performance of TSEs is governed by the Kolmogorov width $d_k(X)$ of $X$, defined as

$d_k(X) = \min_{P_k} \sup_{x \in X} \|x - P_k x\|,$

where the minimum is over all k-codimensional subspaces. The TSE's risk is controlled via

$$R_T(X, \sigma) = \min_k \left\{ d_k(X)^2 + k \sigma^2 \right\}.$$

The estimator seeks a truncation dimension k that mediates the trade-off between geometric approximation error in the subspace and the accumulation of noise in the retained coefficients.

A fundamental result (Javanmard et al., 2012) is that for symmetric convex polytopes defined by m hyperplanes, the TSE is guaranteed to be within a factor $O(\log m)$ of the true minimax risk: $R_T(X, \sigma) \leq O(\log m) R(X, \sigma).$ This result generalizes Pinsker's theorem (optimal for ellipsoids, with $O(1)$ factor) to complex polytopal geometries.

3. Approximation Radius: Volume-Based Lower Bounds

The paper introduces the approximation radius $z_{c, k}(X)$, a geometric measure defining the largest k-dimensional Euclidean ball for which a centrally symmetric convex subset of $X$ occupies at least a constant c fraction of the volume: $z_{c, k}(X) = \sup\{ r : \vr_k(X, r) \geq c \},$ with $vr_k(X, r)$ the maximal $n$-th root volume ratio over k-dimensional subspaces.

This construct provides a lower bound on the minimax risk: $$R(X, \sigma) \geq C \cdot c^2 \cdot \max_k \min\left\{ z_{c, k}(X)^2, \, k \sigma^2 \right\},$$ for universal constants $C$ and $c$ [(Javanmard et al., 2012), Eq. (2)].

Significantly, the approximation radius captures the effective local Euclidean structure present in $X$, regardless of global irregularity, enabling tight lower bounds for a broad range of convex polytopes. This leads to improved minimax lower bounds, particularly in scenarios where traditional geometric quantities such as the largest inscribed hyperrectangle yield suboptimal results.

4. Kolmogorov Width—Approximation Radius Duality

A central theoretical advance is the explicit duality between the Kolmogorov width and the approximation radius, mediated by results from convex geometry. Specifically, for a symmetric convex body $X$ and its polar dual $X^\circ$, the following duality holds: $$d_k(X) \cdot d_{n-(1-\varepsilon)k}(X^\circ) \leq c_1 \sqrt{\frac{k}{\varepsilon}},$$ for a universal constant $c_1$ and $0 < \varepsilon < 1$. This enables a direct comparison between volume-based and projection-based metrics.

Through this duality, the approximation radius is bounded below in terms of the Kolmogorov width of the dual body: $$z_{c, k}(X) \geq c_2 \sqrt{\frac{k}{\ln m}} \cdot \frac{1}{d_{n-k}(X^\circ)},$$ for some $c_2>0$. Consequently, this shows that the projection-based approximability of $X$ is tightly linked to its volumetric core structure, and that even in extreme cases—such as high-dimensional polytopes defined by a large number of hyperplanes—the correspondent TSE remains nearly minimax-optimal.

5. Geometric Insights and Implications for Estimation

The formulation of the approximation radius as a flexible, volume-based measure permits more refined analysis of minimax risk. Unlike simpler geometric measures (such as inscribed ellipsoids or hyperrectangles), the approximation radius can exploit the local structure of complex convex bodies, thus facilitating tight minimax lower bounds even when $X$ is defined by a large or irregular set of constraints.

The established duality links approximation-theoretic (Kolmogorov width) and volumetric (approximation radius) characteristics, deepening the interaction between convex geometry and statistical estimation. This, in turn, opens up new directions in both geometric functional analysis and high-dimensional statistics.

6. Practical Consequences and Applications

One notable application is in the estimation of Lipschitz functions under additive Gaussian noise. When the Lipschitz constraint is encoded as linear inequalities on the signal vector (yielding a symmetric polytope), the results from (Javanmard et al., 2012) guarantee that a truncated series estimator will achieve worst-case risk within $O(\log n)$ of the minimax optimal rate, even with irregularly spaced sample points.

Broader practical implications encompass any estimation problem where the constraint set $X$ is the solution space of linear inequalities—arising in nonparametric regression, signal processing (constrained denoising), and high-dimensional inference with shape (e.g., monotonicity, convexity) or smoothness constraints.

Furthermore, the techniques developed—especially the approximation radius and the Kolmogorov width duality—offer technical frameworks potentially useful in designing new estimators for settings where computational and minimax optimality must be simultaneously addressed.

7. Summary Table: Core Quantities and Bounds

Quantity/Method	Definition or Characterization	Minimax Connection
Minimax Risk $R(X, \sigma)$	$\inf_M \sup_{x\in X} \mathbb{E}_{y|x}\|x-M(y)\|^2$	Benchmark for estimator quality
TSE risk $R_T(X, \sigma)$	$\min_k \{ d_k(X)^2 + k \sigma^2 \}$	Achieves $O(\log m)$ of minimax
Approx. radius $z_{c,k}(X)$	Max radius so $X$ occupies $\geq c$ vol. fraction	Lower bound for minimax risk
Duality: $d_k(X)\cdot d_{n-k}(X^\circ)$	$\leq c_1\sqrt{k/\varepsilon}$	Links TSE and approx. radius

The use of TSEs, with dimension k guided by the Kolmogorov width, and lower bounds informed by the approximation radius, together deliver minimax-optimal (up to logarithmic factors) performance for high-dimensional estimation over symmetric convex polytopes. The formal geometric framework and duality not only yield efficient statistical estimators but also, via their generality and tightness, establish fundamental limits and design principles for estimators over complex convex bodies (Javanmard et al., 2012).