Asymptotic & Finite-Sample Schemes

• Asymptotic and finite-sample schemes are unified methodologies that integrate traditional limit theory with explicit risk bounds and exponential deviation guarantees.
• The framework uses local quadratic bracketing of the log-likelihood to derive nonasymptotic confidence and error estimates, ensuring robustness in high-dimensional and finite-sample settings.
• It bridges classical asymptotic results with modern inference challenges, providing practical sample size guidelines and robust performance even under model misspecification.

Asymptotic and finite-sample schemes form a spectrum of methodologies unifying limiting (asymptotic) statistical theory with explicit, quantitative, nonasymptotic results at fixed sample sizes. Modern research recasts classical parametric estimation so that finite-sample guarantees (expressed via exponential deviation bounds, local quadratic bracketing, and explicit risk/control of the behavior of estimators) yield a rigorous framework that incorporates model misspecification, high-dimensionality, and complex real-world data structures. This synthesis bridges the gap between traditional asymptotic parametric results and nonparametric, or even adversarial, data scenarios.

1. Nonasymptotic Framework and Finite-Sample Guarantees

Finite-sample theory, as articulated in (Spokoiny, 2011), provides a rigorous framework for parametric estimation where the sample size $n$ is fixed and does not tend to infinity. The approach departs fundamentally from limit-based arguments by offering uniform, explicit exponential deviation inequalities valid for any finite sample. The central object of interest is the quasi-maximum likelihood estimator (qMLE), whose deviation from the target parameter $\theta^*$ is controlled not by asymptotic vanishing terms but by quantifiable and optimized error bounds. For a set (ellipsoidal in the metric induced by an information-like matrix $V_0$), explicit bounds of the form

$$\mathbb{P}\{\|\hat{\theta} - \theta^*\|_{V_0} > r\} \leq e^{-x}$$

are established, where $x$ scales with $r^2$. Such results are robust to moderate sample sizes and remain valid when the parameter space dimension $p$ grows with $n$, provided $n \gtrsim Cp$ for a model-dependent constant $C$. This quantification enables practitioners to calibrate required sample sizes for a prescribed accuracy, a feature absent in classical asymptotic analysis.

2. Quadratic Bracketing and Local Approximation of Log-Likelihood

A central technical advance is the local quadratic bracketing of the log-likelihood process:

$$\ell_{-}(\theta,\theta^*) - \Delta_{-}(r) \leq L(\theta) - L(\theta^*) \leq \ell_{+}(\theta,\theta^*) + \Delta_{+}(r),$$

for all $\theta$ in a local set $\Theta_0(r)$. Here, $\ell_{-}$ and $\ell_{+}$ are (randomized) quadratic functions in $(\theta - \theta^*)$, with “shrinking” and “stretching” constants that explicitly account for the finite-sample regime. The bracketing errors $\Delta_{-}(r)$ and $\Delta_{+}(r)$ remain controlled even on neighborhoods that grow with $p$, in contrast to classical LAN theory, which is restricted to root-$n$ neighborhoods.

This device yields direct finite-sample analogues of results such as Wilks’ theorem:

$$2[L(\hat{\theta}) - L(\theta^*)] \approx \|\xi\|^2,$$

where $\xi$ is a localized, normalized score vector. The bracketing further controls confidence region coverage, excess risk, and the expansion of the MLE, with all statements expressed in nonasymptotic probabilistic and risk terms.

3. Model Misspecification and Robustness

A distinctive feature is the relaxation of the global parametric assumption. The data distribution $P$ is not assumed to lie in the parametric model $\{P_\theta\}$; instead, inference targets the parameter

$$\theta^* = \arg\max_{\theta \in \Theta} \mathbb{E}[L(\theta)],$$

which minimizes the KL divergence from $P$ to the parametric family. All concentration, risk, and expansion bounds reference the excess $L(\hat{\theta}) - L(\theta^*)$, providing robust guarantees even under systematic model misspecification. Confidence sets constructed under this framework retain valid coverage for this “best parametric fit” parameter regardless of the truth of the parametric model.

4. Asymptotic Results as Corollaries of Finite-Sample Theory

The finite-sample constructs are not limited in scope—by sending $n \to \infty$, the framework recovers the full range of traditional efficiency and limit theorems:

• As the bracketing errors vanish, classical Fisher expansions and efficiency bounds (e.g., Cramér–Rao and Local Asymptotic Minimax) re-emerge as precise corollaries.
• Under standard conditions, the qMLE obeys the asymptotic normality

$$\sqrt{n}(\hat{\theta} - \theta^*) \approx I^{-1/2} Z,$$

with $Z$ standard normal and $I$ the Fisher information.

• The likelihood ratio statistic converges to $\chi_p^2$, uniform in high-dimensional regimes provided the sample size is suitably large relative to $p$.

Thus, the finite-sample perspective offers a true unification, in which classical results are special limit cases with explicit quantitative error control provided at every finite $n$.

5. High-Dimensional and Non-Classical Regimes

The quadratic bracketing approach generalizes to scenarios with growing or large parameter dimension $p$. Classic LAN theory requires localization within root-$n$ neighborhoods, which can be too restrictive (or invalid) in modern statistical or machine learning applications. The bracketing error bound $n \gtrsim Cp$ is explicit, allowing practitioners to determine when their problem size remains tractable. Uniform exponential deviation inequalities and risk bounds apply in this regime, which is especially relevant for generalized linear models, median regression, and settings with sparsity or robust loss functions.

6. Applications: i.i.d., GLMs, and Robust Estimation

The unified finite-sample theory accommodates a range of standard models, with explicit formulas and sharp probabilistic statements:

• i.i.d. models: For $L(\theta) = \sum_{i} \log p(Y_i;\theta)$, concentration bounds and finite-sample analogues of likelihood ratio and score tests are available, uniformly over all $p$.
• Generalized Linear Models: The quadratic approximation applies to $$L(\theta) = \sum_{i} [Y_i(x_i^\top\theta) - d(x_i^\top\theta)]$$, with finite-sample expansions and concentration bounds for the qMLE established for any $n, p$.
• Median (LAD) regression: Despite nondifferentiable loss $$L(\theta) = -\frac{1}{2} \sum_i |Y_i - x_i^\top\theta|$$, localized versions of the theory provide explicit expansions and finite-sample inequalities using tools for bounded differences.

Probability bounds for likelihood excess of the form

$$\mathbb{P}\{L(\hat{\theta}) - L(\theta^*) \geq z\} \leq 2 \exp(-z + \text{error terms})$$

are derived directly and hold even in misspecified, high-dimensional, and moderate-sample regimes.

7. Quantitative Sample Size Bounds and Root-n Accuracy

The framework offers explicit, quantitative lower bounds on $n$ required to attain root-$n$ estimator accuracy, directly in terms of $p$ and constants from exponential moment (tail) conditions. For instance,

$n \geq C \cdot p$

guarantees that, with high probability,

$\sqrt{n} (\hat{\theta} - \theta^*) = O_p(1).$

This condition is both necessary and sufficient within the finite-sample framework. It elucidates when classical rates are achievable and provides design guidance in high-dimensional inference or resource-limited applications.

---

In summary, the finite-sample theory for parametric estimation (Spokoiny, 2011) constructs a comprehensive, nonasymptotic framework. It delivers uniform exponential deviation and risk bounds for the qMLE under possible model misspecification, high-dimensionality, and fixed $n$, employs novel local quadratic bracketing to control the full log-likelihood process, and encompasses classical asymptotic results as precise limiting corollaries. Model misspecification, robust and high-dimensional estimation, and explicit sample-size-to-accuracy tradeoffs are seamlessly handled, yielding a unified and quantitatively transparent statistical theory that bridges traditional parametrics with modern high-dimensional and adversarial regimes.