Non-Asymptotic Sub-Gaussian Bound

• Non-asymptotic sub-Gaussian concentration bounds are probabilistic tools that provide explicit exponential tail decay for sums, norms, and quadratic forms of sub-Gaussian random variables.
• They utilize techniques such as AMGF, epsilon-net arguments, and PAC-Bayes variational methods to deliver sharp, parameter-controlled bounds that capture dimension-dependent or dimension-free rates.
• These bounds are pivotal in high-dimensional statistics, randomized algorithms, and optimization, offering reliable performance guarantees even for moderate sample sizes.

A non-asymptotic sub-Gaussian concentration bound characterizes the deviation probabilities for sums, norms, or quadratic forms of sub-Gaussian random variables and vectors in finite samples, with explicit, parameter-controlled exponential tail decay. These bounds quantify, for arbitrary sample size and confidence level, the probability that an observable deviates from its expectation by a prescribed amount, with constants and rates reflecting the sub-Gaussian nature of underlying distributions. Non-asymptotic analysis is critical for modern high-dimensional statistics, randomized algorithms, and theoretical computer science, and has seen technical innovations allowing sharper, dimension-dependent or dimension-free formulations.

1. Sub-Gaussian Random Variables, Vectors, and Norms

A real random variable $X$ is sub-Gaussian with variance proxy $\sigma^2$ if its moment generating function (MGF) satisfies

$$E[e^{\lambda X}] \leq \exp\left(\frac{\lambda^2 \sigma^2}{2}\right), \quad \forall \lambda\in\mathbb{R}.$$

For a vector $X\in\mathbb{R}^n$, sub-Gaussianity requires that for every unit vector $\ell \in S^{n-1}$, the projection $\langle \ell, X \rangle$ is sub-Gaussian with the same proxy. Equivalently,

$$E[e^{\lambda\langle \ell, X\rangle}] \leq \exp\left(\frac{\lambda^2\sigma^2}{2}\right), \quad \forall\lambda\in\mathbb{R},\,\ell\in S^{n-1}.$$

Sub-Gaussianity implies sub-Gaussian tails: $$P(|X| \geq t) \leq 2\exp\left(-\frac{t^2}{2\sigma^2}\right), \quad t \geq 0,$$ and similarly for sums of independent centered sub-Gaussians $S = \sum_{i=1}^n X_i$ with variance proxy $\Sigma^2 = \sum_i \sigma_i^2$: $$P\left(S \geq t\right) \leq \exp\left(-\frac{t^2}{2\Sigma^2}\right).$$ For vectors, the Euclidean norm $\|X\|_2$ is typically considered; its concentration is nontrivial due to geometric structure of $\mathbb{R}^n$.

2. Non-Asymptotic Vector Norm Concentration via AMGF

"A New Proof of Sub-Gaussian Norm Concentration Inequality" (Liu et al., 18 Mar 2025) introduces the averaged moment generating function (AMGF)

$$\Phi_{n,\lambda}(x) := E_{\ell \sim \mathrm{Unif}(S^{n-1})}[e^{\lambda\langle \ell,x\rangle}],$$

and its expectation, $$M_\mathrm{avg}(\lambda) := E_X[\Phi_{n,\lambda}(X)]$$. By rotational invariance and convexity arguments, for every $\epsilon \in (0,1)$ and $x\in\mathbb{R}^n$,

$$\Phi_{n,\lambda}(x) \geq (1-\epsilon^2)^{n/2} \, e^{\epsilon\lambda \|x\|}.$$

Using Markov's inequality, the bound

$$P(\|X\|_2 > r) \leq \frac{E[\Phi_{n,\lambda}(X)]}{\Phi_{n,\lambda}(r\eta)} \leq \frac{\exp(\lambda^2\sigma^2/2)}{(1-\epsilon^2)^{n/2}\,e^{\epsilon\lambda r}}$$

is optimized at $\lambda^* = \epsilon r / \sigma^2$, yielding

$$P(\|X\|_2 > r) \leq (1-\epsilon^2)^{-n/2} \exp\left( -\frac{\epsilon^2 r^2}{2\sigma^2} \right).$$

Solving for $r$ to achieve confidence $1-\delta$, with any $\delta\in(0,1)$, one obtains the explicit non-asymptotic bound: $$\|X\|_2 \leq \sigma \sqrt{ \frac{ \log(1/(1-\epsilon^2)) }{ \epsilon^2 } n + \frac{2}{\epsilon^2} \log(1/\delta) },$$ with leading constant $C_1 = \log(1/(1-\epsilon^2))/\epsilon^2$, which is strictly smaller than traditional $\epsilon$-net approaches. For instance, for $\epsilon=1/2$, $C_1\approx5.54$ versus $\approx16$ for the union-bound-based proof (Liu et al., 18 Mar 2025).

3. Classical Covering Arguments and Matrix Extensions

Traditional non-asymptotic vector/matrix concentration bounds employ $\epsilon$-net techniques over the unit sphere, resulting in union bounds with cardinality $(1+2/\epsilon)^n$ and dimension-dependent rates \cite{Vershynin2010}. For i.i.d. sub-Gaussian vectors or random matrices, operator norm and singular value concentration typically take the form: $$P(\|\sum_i X_i\| > t) \leq (m+n)\exp\left( -\frac{t^2}{2b^2 m} \right),$$ where $b$ bounds the sub-Gaussian $\psi_2$-norm of entries, and $m,n$ are matrix dimensions (Gao et al., 2019). Recent matrix concentration works refine these to tighter two-regime bounds, offering exponentially small tails even for moderate deviations, and dimension-free rates for large deviations in suitable regimes (Gao et al., 2019, Vershynin, 2010).

Operator norm concentration can also be established for heteroskedastic Wishart-type matrices, with tight tail bounds in terms of maximal column, row, and element variance proxies ($\Sigma_C$, $\Sigma_R$, $\Sigma_*$), matching minimax lower bounds (Cai et al., 2020).

4. Quadratic Forms and Hanson–Wright Inequality

The non-asymptotic Hanson–Wright inequality (Rudelson et al., 2013) provides tail bounds for quadratic forms $Q(X) = X^\top A X$ where $X$ has independent sub-Gaussian coordinates: $$P\left( |Q(X) - E Q(X)| > t \right) \leq 2 \exp\left( -c\min\left( \frac{t^2}{K^4\|A\|^2_{HS}},\frac{t}{K^2\|A\|} \right) \right),$$ where $K = \max_i \|X_i\|_{\psi_2}$ and $\|A\|_{HS}$ is the Hilbert–Schmidt norm of $A$. For the squared norm $\|AX\|_2^2$, tail bounds of the same form hold, and deviations concentrate sharply around their expectation uniformly over finite $n$ (Rudelson et al., 2013).

5. Optimality, Lower Bounds, and Dimension-Free Variational Bounds

Recent advances guarantee sharp, non-asymptotic lower bounds matching upper exponential tails up to constants (Zhang et al., 2018), e.g.,

$$c \exp\left(-C\frac{t^2}{\Sigma^2} \right) \leq P(S\ge t) \leq \exp\left(-\frac{t^2}{2\Sigma^2}\right),$$

with universal $c,C$. These bounds remain valid for weighted sums and heavy-tailed regimes interpolating sub-Gaussian and sub-Weibull cases (Zhang et al., 2021, Zhang et al., 2023).

Dimension-free forms are established by PAC-Bayes variational techniques, yielding self-normalized confidence ellipsoids for vector-valued stochastic processes, such as

$$\|S_\tau\|^2_{(V_\tau+U_0)^{-1}} \leq \log\frac{ \det(V_\tau+U_0) }{ \det(U_0) } + 2\log(1/\delta),$$

for any stopping time $\tau$, regularizer $U_0$, and confidence level $1-\delta$ (Chugg et al., 8 Aug 2025). This structure avoids explicit $\sqrt{d}$ or $d$ factors typical in union-bound-based concentration and enables efficient high-dimensional inferential procedures.

6. Applications and Implications

Non-asymptotic sub-Gaussian concentration bounds underpin sharp sample complexity and threshold computations in high-dimensional inference, randomized numerical linear algebra, compressive sensing, bandit algorithms, and empirical mean estimation. In optimization, gradient-based algorithms under sub-Gaussian noise (e.g., Stochastic Mirror Descent) inherit explicit concentration rates for function value gaps and iterates, with dependence on accuracy, confidence, and sample size spelled out quantitatively (Paul et al., 2024). Statistical procedures, such as robust covariance matrix estimation (Tyler's and Maronna’s M-estimators), admit non-asymptotic guarantees with stronger-than-classical exponential rates for the sup-norm deviation of weights and operator norm (Romanov et al., 2022).

Operator and singular value bounds for random matrices, critical for high-dimensional statistics and signal processing, are non-asymptotically valid and yield dimension-dependent rates only through log-determinant or structural constants, not as explicit leading factors (Vershynin, 2010, Gao et al., 2019, Cai et al., 2020).

7. Comparative Table: Methods and Constants

Approach	Dimensional Constant $C_1$	Structure
AMGF (spherical avg. MGF) (Liu et al., 18 Mar 2025)	$C_1= \log(1/(1-\epsilon^2))/\epsilon^2$ (e.g., $\approx5.5$ for $\epsilon=1/2$)	Rotation-invariant, union-free
$\epsilon$-net & Union Bound	$C_1=2\log(1+2/\epsilon)/\epsilon^2$ ($\approx16$ for $\epsilon=1/2$)	Covering number, union bound
Hanson–Wright (Rudelson et al., 2013)	N/A (dimension enters via $\|A\|_{HS}$, $\|A\|$)	Quadratic forms, operator norm
Variational PAC-Bayes (Chugg et al., 8 Aug 2025)	Implied via $\log\det$ only	Dimension-free, log determinant
Matrix Bernstein (Vershynin, 2010, Gao et al., 2019)	Prefactor $(m+n)$, exponent $t^2/(2b^2m)$	Operator norm, structure-dependent

The AMGF method (Liu et al., 18 Mar 2025) yields the smallest known leading constant for the dimension term among valid methods for sub-Gaussian vectors, and the dimension-free variational approach (Chugg et al., 8 Aug 2025) gives width only via the log-determinant term, not via explicit dimension factors.

• "A New Proof of Sub-Gaussian Norm Concentration Inequality" (Liu et al., 18 Mar 2025)
• "On the Non-Asymptotic Concentration of Heteroskedastic Wishart-type Matrix" (Cai et al., 2020)
• "Hanson-Wright Inequality and Sub-Gaussian Concentration" (Rudelson et al., 2013)
• "On the Non-asymptotic and Sharp Lower Tail Bounds of Random Variables" (Zhang et al., 2018)
• "Tyler's and Maronna's M-estimators: Non-Asymptotic Concentration Results" (Romanov et al., 2022)
• "A variational approach to dimension-free self-normalized concentration" (Chugg et al., 8 Aug 2025)
• "Introduction to the Non-Asymptotic Analysis of Random Matrices" (Vershynin, 2010)
• "A Refined Non-asymptotic Tail Bound of Sub-Gaussian Matrix" (Gao et al., 2019)
• "Almost Sure Convergence and Non-asymptotic Concentration Bounds for Stochastic Mirror Descent Algorithm" (Paul et al., 2024)
• "Tight Non-asymptotic Inference via Sub-Gaussian Intrinsic Moment Norm" (Zhang et al., 2023)

Non-asymptotic sub-Gaussian concentration bounds continue to evolve, with refinements in constants, structure, and dimensional dependence matching both worst-case and typical behaviors in high-dimensional random systems.