Minimax Risk of Score Estimation

• Minimax risk of score estimation is defined as the fundamental limit in recovering the score function using expected squared L2 loss under various distributional assumptions.
• Key results establish sharp rates that vary with constraints like subgaussianity, Lipschitz/Hölder smoothness, and log-concavity, highlighting the curse of dimensionality.
• Methodologies such as regularized kernel estimation and multiscale aggregation are used, with implications for score-based generative modeling and high-dimensional statistical analysis.

The minimax risk of score estimation interrogates the fundamental limits of recovering the score function—the gradient of the log-density—for an unknown probability law, typically from i.i.d. samples. The principal loss criterion is the expected squared $L^2$ risk with respect to the data-generating density. This statistical problem is essential in multiple domains, notably in nonparametric statistics and score-based generative modeling, and its complexity is determined by the interplay between underlying distributional assumptions (such as tail-decay, smoothness, or shape constraints) and dimensionality.

1. Formal Framework and Loss Metric

Given a dataset $X_1, \dots, X_n$ sampled i.i.d. from a continuous distribution $\rho^*$ on $\mathbb{R}^d$ with density $\rho^*$, the target of estimation is the score function $s^* = \nabla\log\rho^*$. For an estimator $\hat{s}$ constructed from the sample, performance is quantified via the weighted squared loss

$$\ell(\hat{s}, s^*) = \|\hat{s} - s^*\|^2_{L^2(\rho^*)} = \int \|\hat{s}(x) - s^*(x)\|^2 \rho^*(x)\, dx.$$

The $n$-sample minimax risk over a class $\mathcal{P}$ of candidate densities is

$$R_n(\mathcal{P}) = \inf_{\hat{s}} \sup_{\rho^* \in \mathcal{P}} E_{\rho^*}\left[\|\hat{s} - s^*\|^2_{L^2(\rho^*)}\right].$$

This general structure applies across 1D (with additional shape constraints such as log-concavity) and $d$-dimensional models with regularity or tail-decay constraints (Wibisono et al., 12 Feb 2024, Lewis et al., 16 Dec 2025).

2. Canonical Distributional Assumptions

The minimax risk for score estimation is sharply influenced by the function class $\mathcal{P}$ imposed on the underlying density. Canonical examples include:

• Subgaussianity: $\rho^*$ exhibits exponential tail decay, formally $$E_{X \sim \rho^*} \exp(\theta^\top(X-\mathbb{E}X)) \leq \exp(\alpha^2 \|\theta\|^2/2)$$, with $\alpha$ the subgaussian parameter (Wibisono et al., 12 Feb 2024, Zhang et al., 23 Feb 2024).
• Score Regularity:
  • Lipschitz Condition: $s^*$ is $L$-Lipschitz, i.e., $\|s^*(x) - s^*(y)\| \leq L \|x - y\|$, corresponding to $\beta=1$ Hölder regularity.
  • Hölder Smoothness: $s^*$ is $\beta$-Hölder for $0 < \beta \leq 1$: $\|s^*(x) - s^*(y)\| \leq L \|x-y\|^\beta$.
  • Sobolev Regularity (for deconvolution): Densities with $\beta$ derivatives in $L^2$.
• Shape Constraints: In 1D, log-concavity is a nonparametric shape constraint, with the score function $s$ being nonincreasing (antitonic) (Lewis et al., 16 Dec 2025).
• Tail-Growth and Fisher Information Constraints: Control over the growth of the score in the distribution's tails, often formalized in subclasses such as $\mathcal{J}_{\gamma,L}$ and Fisher information-bounded classes (Lewis et al., 16 Dec 2025).

The intersection of regularity, tail, and shape constraints determines both the attainability of finite minimax risk and its rate.

3. Sharp Minimax Rates: Upper and Lower Bounds

3.1 Multivariate Subgaussian Models

For $\alpha$-subgaussian densities with $L$-Lipschitz score ($P_{\alpha,L}$), the minimax risk attains (up to logarithmic factors): $R_n(P_{\alpha,L}) = \tilde{\Theta}(n^{-2/(d+4)})$ The rate deteriorates exponentially with dimension $d$, exhibiting the curse of dimensionality. The upper bound is achieved by a regularized score estimator of a Gaussian-kernel density estimator, with optimal bandwidth balancing bias and variance: $$E\|\hat{s}_h - s^*\|^2_{L^2(\rho^*)} \leq C d \alpha^2 L^2 (\log n)^{2d/(d+4)} n^{-2/(d+4)}$$ The matching lower bound is obtained via a Fano- or Assouad-type construction, perturbing a Gaussian base density so that the scores remain $L$-Lipschitz and subgaussian (Wibisono et al., 12 Feb 2024). For general $\beta$-Hölder scores ($0<\beta\leq 1$), the optimal rate becomes $$R_n(P_{\alpha,L}^{(\beta)}) = \tilde{\Theta}(n^{-2\beta/(d+2\beta+2)})$$, in accordance with classical minimax theory for derivative estimation.

3.2 Univariate Log-Concave and Shape-Constrained Classes

In the 1D log-concave setting, score estimation minimax risk is governed by both tail-behavior and local regularity:

• Tail-Growth Class $\mathcal{J}_{\gamma,L,r}$: For $f \in \mathcal{J}_{\gamma,L,r}$, the minimax rate is $\tilde{\Theta}(n^{-(\gamma\wedge1/3)})$ (Lewis et al., 16 Dec 2025). For the unrestricted log-concave class, only $n^{-1/3}$ is attainable in risk, even as density estimation achieves $n^{-4/5}$ in Hellinger loss.
• Log-Concave Hölder Class $\mathcal{F}_{\beta,L,r}$: Over log-concave densities with $\beta$-Hölder score, the minimax risk improves to the classical $n^{-\beta/(2\beta+1)}$ (Lewis et al., 16 Dec 2025). Notably, when $1<\beta<2$, this rate is strictly faster than attainable by either the shape constraint or by smoothness alone.

Main upper bounds are achieved by a multiscale aggregation estimator that adapts locally via kernel smoothing, constructing simultaneous confidence bands for the score. Lower bounds are produced by local perturbations of Laplace-type densities (for tails) and by blockwise perturbations with amplitude tuned to $\beta$ (for Hölder smoothness).

3.3 Deconvolution and Score Estimation of Noisy Densities

In score-based diffusion models, estimation of the smoothed score $s_t=\nabla\log(p_0 * \mathcal{N}(0,tI_d))$ yields minimax risk

$$R_n(t; \mathcal{P}_{\rm subG}) \lesssim \operatorname{polylog}(n) n^{-1} t^{-(d+2)/2}$$

If $p_0$ is $\beta$-Sobolev with $\beta \leq 2$, the integrated risk over $[t_0,T]$ becomes $\operatorname{polylog}(n) n^{-2\beta/(2\beta+d)}$ (Zhang et al., 23 Feb 2024). Under mild regularity, these rates match minimax lower bounds for density and gradient estimation.

Table: Representative Minimax Risk Rates

Model/Class	Minimax Risk Rate	Reference
Multivariate subgaussian, $\beta=1$	$\tilde{\Theta}(n^{-2/(d+4)})$	(Wibisono et al., 12 Feb 2024)
Multivariate subgaussian, $\beta$-Hölder	$\tilde{\Theta}(n^{-2\beta/(d+2\beta+2)})$	(Wibisono et al., 12 Feb 2024)
1D log-concave, tail-growth $\gamma$	$\tilde{\Theta}(n^{-(\gamma\wedge1/3)})$	(Lewis et al., 16 Dec 2025)
1D log-concave, $\beta$-Hölder	$n^{-\beta/(2\beta+1)}$	(Lewis et al., 16 Dec 2025)
Deconvolution, $s_t$ subgaussian	$\tilde{O}(n^{-1} t^{-(d+2)/2})$	(Zhang et al., 23 Feb 2024)
Diffusion, Sobolev $\beta$	$\tilde{O}(n^{-2\beta/(2\beta+d)})$	(Zhang et al., 23 Feb 2024)

4. Estimator Design: Regularization, Aggregation, and Adaptivity

Minimax-optimal score estimators typically utilize regularized kernel methods or multiscale adaptation:

• Regularized Kernel Estimation (Wibisono et al., 12 Feb 2024, Zhang et al., 23 Feb 2024):

$$\hat{\rho}_h(x) = \frac{1}{n} \sum_{i=1}^n N(x; X_i, hI_d), \quad \hat{s}_h(x) = \frac{\nabla \hat{\rho}_h(x)}{\max\{\hat{\rho}_h(x), \epsilon\}}$$

with bandwidth $h$ and regularization parameter $\epsilon$ optimally balanced to minimize risk.

• Multiscale Band Aggregation (Lewis et al., 16 Dec 2025): Construction of uniform confidence bands for the score across multiple bandwidths, followed by adaptive aggregation into a final estimator that minimizes local bias-variance tradeoff.
• Truncation and Density-adaptive Estimators (Zhang et al., 23 Feb 2024): For deconvolution settings, thresholding low-density regions to control estimator instability, using subgaussian tail bounds to control the estimator risk even when the target density is not bounded away from zero.

These methodologies are fully rigorous, with theoretical guarantees matching lower bounds up to logarithmic corrections.

5. Implications, Curse of Dimensionality, and Comparison to Density Estimation

The minimax risk of score estimation exposes stark contrasts with classical density estimation:

• Curse of Dimensionality: For $d$-dimensional densities with only subgaussian and Lipschitz/Hölder conditions, sample complexity exhibits exponential (or near-exponential) scaling in $d$, both in kernel regularization and in score-based generative modeling sample complexity (Wibisono et al., 12 Feb 2024).
• Smoothness Benefit: Unlike density estimation under log-concavity, where smoothness does not improve minimax rates, score estimation is strictly accelerated by higher smoothness, especially for $\beta<2$ (Lewis et al., 16 Dec 2025).
• Tail Behavior: Score estimation over log-concave families is substantially more sensitive to tail growth than density estimation; extremely light tails can degrade rates to $n^{-\gamma}$, a phenomenon not seen in $L^2$ density estimation (Lewis et al., 16 Dec 2025).
• Diffusion Models: Score estimation in the context of SGMs plays a critical role in the sample complexity of synthesis: accurate estimation of time-varying smoothed scores drives the convergence rates of modern generative models, with minimax rates dictated by the underlying class of data densities (Zhang et al., 23 Feb 2024, Wibisono et al., 12 Feb 2024).

6. Proof Techniques and Statistical Limits

Proofs of sharp minimax rates for score estimation employ:

• Assouad and Fano Reductions: Information-theoretic constructions of parameter separation and concentration of measure for lower bounds in both isotropic Gaussian and log-concave settings (Wibisono et al., 12 Feb 2024, Lewis et al., 16 Dec 2025).
• Bias-Variance Decomposition: Fine-grained, measure-aware balancing of density estimation risk, score regularity, and the impact of smoothing. This includes detailed analysis of Hellinger distances, tail-truncation, and local confidence over quantile bins.
• Adaptivity via Multiscale Confidence Bands and Truncation: Exploitation of scale-varying bias-variance profiles for aggregation; superiority of locally adaptive over globally tuned estimators under heterogeneous conditions (tail growth, local smoothness).

A plausible implication is that while density estimation attains minimax rates under even mild regularity, score estimation places higher demands on both smoothness and tail behavior for statistical tractability; shape and tail constraints can become risk-limiting. In high-dimensional or long-tailed settings, this suggests an intrinsic hardness barrier for advanced score-based modeling without stronger structural priors.

7. Future Research and Open Questions

• Dimension-Reduction and Structure Utilization: Mitigating the curse of dimensionality necessitates either exploiting additional structure (e.g., sparsity, independence, manifold constraints) or tightening regularity/smoothness assumptions.
• Rate Optimality Beyond Log-Concavity: The difference in achievable minimax rates under shape versus smoothness constraints for score estimation, and the absence of rate improvement for density estimation, prompts further study of intermediate constraint classes and multi-scale adaptive estimation.
• Minimax Theory for Score-Based SGMs: Understanding the tight correspondence between statistical lower bounds for score estimation and convergence rates in generative models remains a primary avenue, with practical importance in the training and evaluation of high-dimensional generative architectures.
• Density-Lower-Bound-Free Optimality: Recent advances demonstrate minimax optimality for SGMs without density lower bound assumptions (Zhang et al., 23 Feb 2024), replacing this prerequisite with subgaussian tail and high-order kernel truncation techniques, suggesting potential for even broader applicability.

These ongoing lines of inquiry continue to inform both theoretical statistics and the practical design of flexible, scalable, and sample-efficient score estimators and generative models.