Sub-Weibull Random Variables

• Sub-Weibull random variables are defined by Weibull-type tail decay, satisfying bounds like P(|X| ≥ x) ≤ 2 exp(−(x/K)^(1/θ)) and corresponding moment conditions that bridge sub-Gaussian and sub-exponential behaviors.
• They exhibit algebraic stability under operations such as scaling, addition, and products, which underpins their powerful role in concentration theory and random matrix analysis.
• Their modeling advantages support applications in stochastic optimization and distributed learning, allowing precise high-probability convergence estimates even under heavy-tailed noise.

Sub-Weibull random variables form a tail class characterized by Weibull-type decay and used to interpolate between sub-Gaussian, sub-Exponential, and heavier-tailed stochastic models. In a standard convention, they satisfy bounds of the form

$$\mathbb P(|X|\ge x)\le 2\exp\!\left(-\left(\frac{x}{K}\right)^{1/\theta}\right),$$

with equivalent moment and Orlicz-type formulations; this framework now appears in concentration theory, stochastic optimization, distributed learning, random matrix theory, and stretched-exponential large deviations (Vladimirova et al., 2019, Bastianello et al., 2021, Kim et al., 2021).

1. Definitions and parameter conventions

A widely used definition declares a real random variable $X$ to be sub-Weibull with tail parameter $\theta>0$ if there exists $K_1>0$ such that

$$\mathbb{P}(|X| \ge x) \le 2\exp\left( - (x/ K_1)^{1/\theta} \right), \quad x\ge 0.$$

This is equivalent to the moment-growth condition

$\|X\|_k \le K_2 k^\theta,\qquad \forall k\ge 1,$

to MGF control for $|X|^{1/\theta}$,

$$\mathbb{E}\!\left[\exp\!\left((\lambda |X|)^{1/\theta}\right)\right] \le \exp\!\left((\lambda K_3)^{1/\theta}\right),\qquad 0<\lambda\le \frac1{K_3},$$

and to the one-point Orlicz bound

$$\mathbb{E}\!\left[\exp\!\left((|X|/K_4)^{1/\theta}\right)\right]\le 2.$$

This convention is used directly in the shorthand $X\sim \mathrm{subW}(\theta,K)$, and it identifies $X$0 with sub-Gaussian behavior and $X$1 with sub-Exponential behavior (Vladimirova et al., 2019, Kim et al., 2021, Bastianello et al., 2021, Yu et al., 15 Jun 2025).

The same class also appears in conditional and norm-based forms. In distributed optimization and SGD analyses, the assumption is often that the norm of a random vector, conditionally on the past filtration, is sub-Weibull: $X$2 This formulation is tailored to adapted stochastic processes rather than iid scalar variables (Yu et al., 15 Jun 2025, Madden et al., 2020).

Another strand of the literature uses a reversed exponent convention. In that notation, $X$3 is sub-Weibull when $X$4 and

$X$5

so that $X$6 is sub-exponential and $X$7 is sub-Gaussian. This suggests that comparisons of “tail index” across papers require checking whether the exponent is written as $X$8 or $X$9 (Zhang et al., 2021).

2. Equivalent formulations and structural properties

The sub-Weibull class is useful partly because it is stable under common algebraic operations. Under the moment-growth convention, if $\theta>0$0, then scalar multiplication preserves the class,

$\theta>0$1

addition yields

$\theta>0$2

and, when independence is available, products satisfy

$\theta>0$3

A power rule also holds: $\theta>0$4 with the frequently used special case

$\theta>0$5

These rules underpin many optimization proofs, because gradient-noise terms often enter through sums, products, or squares (Bastianello et al., 2021, Kim et al., 2021, Vladimirova et al., 2019).

The class is nested in the sense that

$\theta>0$6

under the $\theta>0$7-exponent convention, so larger $\theta>0$8 corresponds to heavier tails. Translation and scaling change constants but not the tail order, and bounded or finitely supported random variables are included as special cases, either directly or via their sub-Gaussian membership (Vladimirova et al., 2019, Bastianello et al., 2021, Ospina et al., 2021).

Several works also emphasize that centering is not intrinsic to the definition. A sub-Weibull variable need not be mean-zero, a constant random variable is still sub-Weibull in the scale-based definition, and centered versions remain sub-Weibull with modified constants. In stochastic-process settings, one may impose mean-zero only on the noise increment, while the sub-Weibull assumption is placed on its norm or on each coordinate (Yu et al., 15 Jun 2025, Madden et al., 2020, Ospina et al., 2021).

3. Concentration theory and tail inequalities

A basic high-probability consequence of the tail bound is that if $\theta>0$9, then

$K_1>0$0

Equivalent forms appear throughout the optimization literature and are a main reason sub-Weibull modeling yields polylogarithmic, rather than polynomial, confidence penalties (Bastianello et al., 2021, Kim et al., 2021, Ospina et al., 2021).

For sums of independent centered variables, recent work develops a mixed-tail theory with a sub-Gaussian regime for smaller deviations and a sub-Weibull regime for larger deviations. One formulation uses the generalized Bernstein-Orlicz function

$K_1>0$1

and proves for weighted sums $K_1>0$2 that

$K_1>0$3

with matching lower bounds in the sharpness analysis. A related constant-specified result controls the GBO norm of $K_1>0$4 and yields deviation bounds of the form

$K_1>0$5

These results recover the classical sub-Gaussian and sub-exponential regimes as special cases while retaining explicit constants (Bong et al., 2023, Zhang et al., 2021).

For dependent data, a major development is the self-normalized Freedman-type inequality for martingale difference sequences with conditional sub-Weibull tails. This extends concentration beyond the sub-exponential threshold. In particular, when $K_1>0$6, the ordinary MGF of $K_1>0$7 may be infinite for all $K_1>0$8, so classical MGF-based martingale arguments fail; the proof instead uses truncation plus self-normalization (Madden et al., 2020).

These concentration results feed directly into statistical procedures. Tight mixed-tail inequalities improve graphical-model inference by changing the sample-size condition for a leading error term from

$K_1>0$9

and a log-log quantile regression estimator for the tail index is proposed via the slope in

$$\mathbb{P}(|X| \ge x) \le 2\exp\left( - (x/ K_1)^{1/\theta} \right), \quad x\ge 0.$$0

The same 2019 study cites a theorem of Vladimirova et al. stating that, for Bayesian neural networks with standard Gaussian priors, units in layer $$\mathbb{P}(|X| \ge x) \le 2\exp\left( - (x/ K_1)^{1/\theta} \right), \quad x\ge 0.$$1 are sub-Weibull with tail parameter

$$\mathbb{P}(|X| \ge x) \le 2\exp\left( - (x/ K_1)^{1/\theta} \right), \quad x\ge 0.$$2

showing that depth can induce progressively heavier tails at finite width (Bong et al., 2023, Vladimirova et al., 2019).

4. Sub-Weibull noise in optimization and learning

In online stochastic optimization, sub-Weibull assumptions allow nonasymptotic analyses under heavier-tailed gradient errors. For online gradient descent with

$$\mathbb{P}(|X| \ge x) \le 2\exp\left( - (x/ K_1)^{1/\theta} \right), \quad x\ge 0.$$3

and step size $$\mathbb{P}(|X| \ge x) \le 2\exp\left( - (x/ K_1)^{1/\theta} \right), \quad x\ge 0.$$4, the regret recursion under the Polyak–Łojasiewicz condition takes the form

$$\mathbb{P}(|X| \ge x) \le 2\exp\left( - (x/ K_1)^{1/\theta} \right), \quad x\ge 0.$$5

This yields expectation bounds, iteration-wise high-probability bounds with

$$\mathbb{P}(|X| \ge x) \le 2\exp\left( - (x/ K_1)^{1/\theta} \right), \quad x\ge 0.$$6

and the almost sure asymptotic estimate

$$\mathbb{P}(|X| \ge x) \le 2\exp\left( - (x/ K_1)^{1/\theta} \right), \quad x\ge 0.$$7

The corresponding proximal-gradient recursion under the proximal-PL condition replaces the quadratic noise term by $$\mathbb{P}(|X| \ge x) \le 2\exp\left( - (x/ K_1)^{1/\theta} \right), \quad x\ge 0.$$8, again giving linear convergence up to a noise- and variability-dependent floor (Kim et al., 2021).

A broader operator-theoretic framework models stochastic algorithms as iterated stochastic operators with additive sub-Weibull errors and random coordinate updates. In this setting the noise is persistent rather than vanishing, and the analysis gives mean and high-probability convergence to a neighborhood determined by the contraction factor, coordinate-update probabilities, temporal drift, and the sub-Weibull scale parameter. The same framework is motivated explicitly by federated learning, where stochastic gradients, compression, and asynchrony all produce non-Gaussian additive errors (Bastianello et al., 2021).

Sub-Weibull noise also appears in nonconvex SGD, feedback-based control optimization, and distributed mirror descent. For nonconvex SGD, conditional norm sub-Weibull gradient noise supports high-probability convergence with logarithmic dependence on failure probability, and the martingale part of the proof uses the sub-Weibull Freedman inequality. In feedback-based projected gradient with intermittent measurements, the combined gradient error norm satisfies a sub-Weibull bound and the tracking error obeys both expectation and high-probability estimates under Bernoulli update availability. In distributed composite stochastic mirror descent over time-varying multi-agent networks, conditional norm sub-Weibull gradient noise yields high-probability convergence rates of order $$\mathbb{P}(|X| \ge x) \le 2\exp\left( - (x/ K_1)^{1/\theta} \right), \quad x\ge 0.$$9 up to polylogarithmic factors, without smoothness or strong convexity assumptions and without gradient clipping or truncation (Madden et al., 2020, Ospina et al., 2021, Yu et al., 15 Jun 2025).

5. Related Weibull-type regimes and analytical models

One important sub-Weibull regime is the stretched-exponential case

$\|X\|_k \le K_2 k^\theta,\qquad \forall k\ge 1,$0

studied under conditioning on a large sum. These variables have finite moments of all orders but no finite positive exponential moment,

$\|X\|_k \le K_2 k^\theta,\qquad \forall k\ge 1,$1

Under the conditioning

$\|X\|_k \le K_2 k^\theta,\qquad \forall k\ge 1,$2

the maximum and the reversed order statistics satisfy large deviation principles at speed

$\|X\|_k \le K_2 k^\theta,\qquad \forall k\ge 1,$3

The resulting rate function for the maximum is non-convex and obeys a Bellman-type recursion, reflecting a competition between a big-jump mechanism and a Gaussian bulk mechanism (Jansen, 2024).

A distinct but related regime arises for light-tailed Weibull-like sums with tails asymptotic to

$\|X\|_k \le K_2 k^\theta,\qquad \forall k\ge 1,$4

Here the tail decays faster than exponential, and large values of $\|X\|_k \le K_2 k^\theta,\qquad \forall k\ge 1,$5 are typically produced by all summands being of order $\|X\|_k \le K_2 k^\theta,\qquad \forall k\ge 1,$6. The asymptotics are governed by a Laplace or saddlepoint principle rather than the “single big jump” principle. This analysis is presented as a tail-based reformulation and extension of earlier density-based results of Rootzén (1987) and Balkema, Klüppelberg, and Resnick (1993) (Asmussen et al., 2017).

Weibull variables also serve as canonical models in geometric functional analysis. For iid symmetric Weibull variables with

$\|X\|_k \le K_2 k^\theta,\qquad \forall k\ge 1,$7

one has moment growth $\|X\|_k \le K_2 k^\theta,\qquad \forall k\ge 1,$8, and two-sided Chevet-type inequalities lead to sharp operator-norm estimates for random matrices and their submatrices. The parameter $\|X\|_k \le K_2 k^\theta,\qquad \forall k\ge 1,$9 interpolates continuously between the exponential and Gaussian regimes. A different line of work constructs asymmetric generalized Weibull laws as scale-location or variance-mean mixtures of normal laws, producing two-sided Weibull-type tails that are motivated by stopped random walks and financial modeling (Latała et al., 2023, Korolev et al., 2015).

6. Scope, special cases, and recurrent points of confusion

A recurrent source of confusion is that “sub-Weibull” is a family of parameterizations rather than a single universal notation. In the $|X|^{1/\theta}$0-exponent convention, $|X|^{1/\theta}$1 is sub-Gaussian and larger $|X|^{1/\theta}$2 means heavier tails; in the $|X|^{1/\theta}$3 convention, $|X|^{1/\theta}$4 is sub-Gaussian and larger $|X|^{1/\theta}$5 means lighter tails. Bounded or finitely supported variables are included in both viewpoints, either directly or through their sub-Gaussian membership (Vladimirova et al., 2019, Bastianello et al., 2021, Ospina et al., 2021).

A second point is that sub-Weibull control concerns transformed exponential moments or moment growth, not necessarily the ordinary MGF of $|X|^{1/\theta}$6. The condition

$|X|^{1/\theta}$7

may hold even when

$|X|^{1/\theta}$8

as happens in stretched-exponential regimes and in the martingale analysis beyond the sub-exponential threshold. This distinction is central to why sub-Weibull concentration requires different tools from classical Hoeffding-, Bernstein-, or Freedman-type arguments (Madden et al., 2020, Jansen, 2024).

A third point is terminological. Some papers use “Weibull-like” for stretched-exponential heavy tails with exponent in $|X|^{1/\theta}$9, others for faster-than-exponential light tails with exponent $$\mathbb{E}\!\left[\exp\!\left((\lambda |X|)^{1/\theta}\right)\right] \le \exp\!\left((\lambda K_3)^{1/\theta}\right),\qquad 0<\lambda\le \frac1{K_3},$$0, and still others for canonical $$\mathbb{E}\!\left[\exp\!\left((\lambda |X|)^{1/\theta}\right)\right] \le \exp\!\left((\lambda K_3)^{1/\theta}\right),\qquad 0<\lambda\le \frac1{K_3},$$1 variables with $$\mathbb{E}\!\left[\exp\!\left((\lambda |X|)^{1/\theta}\right)\right] \le \exp\!\left((\lambda K_3)^{1/\theta}\right),\qquad 0<\lambda\le \frac1{K_3},$$2. This suggests that the common feature is the functional form of the tail envelope rather than a single probabilistic regime. Across these settings, the main analytical benefit remains the same: sub-Weibull models preserve explicit, often sharp, dependence on tail parameters while covering phenomena that lie outside the strictly sub-Gaussian and sub-exponential classes (Zhang et al., 2021, Asmussen et al., 2017, Latała et al., 2023).