Heavy-Tailed Diffusion Models (HTDMs)

Updated 5 July 2026

Heavy-Tailed Diffusion Models (HTDMs) are diffusion constructions that incorporate polynomial (heavy-tailed) decay in dynamics, targets, or training gradients.
They encompass diverse approaches including fractional diffusion in mutation kernels, Student’s t generative models, and conditional diffusion with heavy-tailed gradients for privacy.
HTDMs provide both theoretical insights and empirical advantages in capturing extreme events, while highlighting trade-offs in statistical estimation and model stability.

Heavy-Tailed Diffusion Models (HTDMs) denote a family of diffusion-based constructions in which polynomial-tail behavior is built into the dynamics, the asymptotic target, or the training process. In the current literature, the term covers at least four technically distinct uses: nonlocal selection–mutation equations with fat-tailed mutation kernels and fractional diffusion; generative diffusion and flow models that replace Gaussian laws by multivariate Student’s $t$ or $\alpha$ -stable laws; Gaussian score-based diffusion applied to heavy-tailed targets; and conditional diffusion systems whose training dynamics exhibit heavy-tailed per-example gradients under heterogeneous conditioning (Mirrahimi, 2018, Pandey et al., 2024, Yu et al., 10 Jan 2026, Huang et al., 26 Feb 2026). This diversity suggests that HTDMs are not a single canonical model class, but a broader heavy-tail paradigm for diffusion-like systems.

1. Terminological scope and principal constructions

Across the cited literature, “heavy-tailed” refers to polynomial rather than exponential tail decay, but the location of the heavy-tail mechanism varies. In some works it is the forward perturbation law; in others it is the target distribution, the mutation kernel, the waiting-time distribution, or the gradient statistics induced during training. That distinction is technically decisive because it changes the relevant limiting equation, the statistical estimation problem, and the sampling algorithm (Pandey et al., 2024, Yu et al., 10 Jan 2026, Cherkaoui et al., 13 May 2026).

Setting	Heavy-tail locus	Representative works
Selection–mutation PDEs	Mutation kernel $K(z)\asymp c\|z\|^{-(d+2\alpha)}$ and fractional Laplacian	(Mirrahimi, 2018)
Generative diffusion and flow	Student’s $t$ or $\alpha$ -stable prior/forward corruption	(Pandey et al., 2024, Wakatsuki et al., 1 Jun 2026, Cherkaoui et al., 13 May 2026)
Gaussian diffusion on heavy-tailed data	Heavy-tailed target $p_0$ with Gaussian noising	(Yu et al., 10 Jan 2026)
Private conditional diffusion training	Heavy-tailed per-example gradients on the conditioning pathway	(Huang et al., 26 Feb 2026)
Anomalous diffusion and kinetic limits	Heavy-tailed equilibria or waiting times	(Crouseilles et al., 2015, Guo et al., 2024, Garbaczewski et al., 2010, Borovkov et al., 2011)

A central structural divide separates HTDMs that alter the diffusion law itself from those that keep Gaussian diffusion but analyze heavy-tailed targets. The former aim to improve tail fidelity by changing the prior, forward corruption, or reverse kernel; the latter derive guarantees for conventional score-based diffusion under heavy-tailed data. A further divide separates output-tail models from training-tail models: the private conditional-diffusion literature is concerned with heavy-tailed gradients rather than heavy-tailed samples (Huang et al., 26 Feb 2026).

2. Selection–mutation HTDMs and Hamilton–Jacobi singular limits

In evolutionary dynamics, HTDMs arise from a nonlocal reaction–diffusion equation for the phenotypic density $n(t,x)$ on trait space $x\in\mathbb R^d$ ,

$\partial_t n(t,x) + (-\Delta)^\alpha n(t,x) = n(t,x)R(x,I(t)), \qquad I(t)=\int_{\mathbb R^d} n(t,x)\,dx,$

with $\alpha\in(0,1)$ , a fractional diffusion operator modeling mutations, and a nonlocal reaction term encoding selection through the total population size $\alpha$ 0 (Mirrahimi, 2018). The mutation operator can equivalently be written as

$\alpha$ 1

with $\alpha$ 2 as $\alpha$ 3, so the heavy-tail mechanism is an algebraically decaying jump kernel corresponding to Lévy flights with index $\alpha$ 4.

The defining asymptotic contribution of this framework is a rescaling that captures large-time evolution, shrinks typical mutation steps, and still preserves the algebraic tails of the mutation law. Writing $\alpha$ 5 with $\alpha$ 6 and $\alpha$ 7, and then applying the time rescaling $\alpha$ 8 and jump rescaling $\alpha$ 9, one obtains a rescaled mutation operator whose covariance is $K(z)\asymp c|z|^{-(d+2\alpha)}$ 0 but whose tail signature remains heavy-tailed in the original displacement variable (Mirrahimi, 2018). Under WKB-prepared initial data, mass bounds, and tail control at scale $K(z)\asymp c|z|^{-(d+2\alpha)}$ 1, the Hopf–Cole transform $K(z)\asymp c|z|^{-(d+2\alpha)}$ 2 yields a constrained Hamilton–Jacobi limit with Hamiltonian

$K(z)\asymp c|z|^{-(d+2\alpha)}$ 3

$K(z)\asymp c|z|^{-(d+2\alpha)}$ 4

The heavy-tailed character appears analytically through the finite domain of $K(z)\asymp c|z|^{-(d+2\alpha)}$ 5: $K(z)\asymp c|z|^{-(d+2\alpha)}$ 6 only if $K(z)\asymp c|z|^{-(d+2\alpha)}$ 7. This yields the sharp gradient bound

$K(z)\asymp c|z|^{-(d+2\alpha)}$ 8

and the stronger global logarithmic modulus

$K(z)\asymp c|z|^{-(d+2\alpha)}$ 9

Unlike thin-tailed Hamilton–Jacobi limits, the WKB transform does not converge to a full viscosity solution but to a viscosity supersolution that is minimal in an admissible class. The obstruction is twofold: $t$ 0 for $t$ 1, which prevents testing with arbitrary $t$ 2 functions, and the total mass $t$ 3 is only $t$ 4 in time and may be discontinuous (Mirrahimi, 2018).

The macroscopic effect is concentration of $t$ 5 onto the zero set of $t$ 6. The limiting measures are supported on $t$ 7, and at continuity points of $t$ 8 this set is contained in $t$ 9. In one dimension, if $\alpha$ 0 is monotone in $\alpha$ 1, the support collapses to a single point and the limit is a moving Dirac mass,

$\alpha$ 2

at almost every $\alpha$ 3 (Mirrahimi, 2018). The paper does not derive a canonical ODE for $\alpha$ 4, in contrast with light-tailed adaptive-dynamics limits, precisely because the limiting object is only a minimal viscosity supersolution with a weak subsolution property.

3. Student’s $\alpha$ 5 generative HTDMs and state-dependent reverse diffusion

In deep generative modeling, HTDMs replace Gaussian priors and forward corruption kernels with multivariate Student’s $\alpha$ 6 laws. A basic construction uses

$\alpha$ 7

with reparameterization

$\alpha$ 8

The conditional Student’s $\alpha$ 9 posterior gives a denoising kernel for the reverse process, and a $p_0$ 0-divergence with $p_0$ 1 yields a training objective that reduces to mean-squared error in $p_0$ 2-prediction form (Pandey et al., 2024). On this basis, the literature introduces $p_0$ 3-EDM and $p_0$ 4-Flow as heavy-tailed counterparts of existing diffusion and flow models with minimal code changes relative to Gaussian baselines.

Empirically, these Student’s $p_0$ 5-based HTDMs were evaluated on the HRRR weather dataset, where rare and extreme events are central. For unconditional generation on the VIL channel, KS-tail on test is reduced from $p_0$ 6 for EDM+PCP or $p_0$ 7 for plain EDM down to $p_0$ 8 for $p_0$ 9-EDM with $n(t,x)$ 0. On the $n(t,x)$ 1 channel, KS-tail improves from $n(t,x)$ 2 for EDM+PCP or $n(t,x)$ 3 for EDM to $n(t,x)$ 4 for $n(t,x)$ 5-EDM with $n(t,x)$ 6. For conditional next-hour forecasting, $n(t,x)$ 7-EDM improves CRPS and calibration while reducing KS-tail; for $n(t,x)$ 8, CRPS changes from $n(t,x)$ 9 to $x\in\mathbb R^d$ 0 and KS from $x\in\mathbb R^d$ 1 to $x\in\mathbb R^d$ 2 (Pandey et al., 2024).

A subsequent development derives an SDE-based sampler for Student’s $x\in\mathbb R^d$ 3 HTDMs with a state-dependent diffusion coefficient. In the variance-exploding specialization $x\in\mathbb R^d$ 4 and $x\in\mathbb R^d$ 5, the reverse SDE becomes

$x\in\mathbb R^d$ 6

with

$x\in\mathbb R^d$ 7

This produces a “self-regulating annealing” mechanism: when $x\in\mathbb R^d$ 8 is far from the denoiser target, $x\in\mathbb R^d$ 9 grows and the effective noise scale increases; when $\partial_t n(t,x) + (-\Delta)^\alpha n(t,x) = n(t,x)R(x,I(t)), \qquad I(t)=\int_{\mathbb R^d} n(t,x)\,dx,$ 0 is close, the diffusion amplitude decreases (Wakatsuki et al., 1 Jun 2026). In a synthetic $\partial_t n(t,x) + (-\Delta)^\alpha n(t,x) = n(t,x)R(x,I(t)), \qquad I(t)=\int_{\mathbb R^d} n(t,x)\,dx,$ 1D Student’s $\partial_t n(t,x) + (-\Delta)^\alpha n(t,x) = n(t,x)R(x,I(t)), \qquad I(t)=\int_{\mathbb R^d} n(t,x)\,dx,$ 2 target with $\partial_t n(t,x) + (-\Delta)^\alpha n(t,x) = n(t,x)R(x,I(t)), \qquad I(t)=\int_{\mathbb R^d} n(t,x)\,dx,$ 3, the proposed $\partial_t n(t,x) + (-\Delta)^\alpha n(t,x) = n(t,x)R(x,I(t)), \qquad I(t)=\int_{\mathbb R^d} n(t,x)\,dx,$ 4-SDE attains $\partial_t n(t,x) + (-\Delta)^\alpha n(t,x) = n(t,x)R(x,I(t)), \qquad I(t)=\int_{\mathbb R^d} n(t,x)\,dx,$ 5 and $\partial_t n(t,x) + (-\Delta)^\alpha n(t,x) = n(t,x)R(x,I(t)), \qquad I(t)=\int_{\mathbb R^d} n(t,x)\,dx,$ 6, while the ablated $\partial_t n(t,x) + (-\Delta)^\alpha n(t,x) = n(t,x)R(x,I(t)), \qquad I(t)=\int_{\mathbb R^d} n(t,x)\,dx,$ 7-SDE with $\partial_t n(t,x) + (-\Delta)^\alpha n(t,x) = n(t,x)R(x,I(t)), \qquad I(t)=\int_{\mathbb R^d} n(t,x)\,dx,$ 8 yields $\partial_t n(t,x) + (-\Delta)^\alpha n(t,x) = n(t,x)R(x,I(t)), \qquad I(t)=\int_{\mathbb R^d} n(t,x)\,dx,$ 9 and $\alpha\in(0,1)$ 0. The paper therefore identifies state-dependent diffusion as necessary for reproducing heavy tails (Wakatsuki et al., 1 Jun 2026).

4. Statistical guarantees, Gaussian alternatives, and the heavy-tail trade-off

A distinct theoretical line studies heavy-tailed targets under conventional Gaussian score-based diffusion rather than modifying the noise law. In the variance-exploding setting $\alpha\in(0,1)$ 1, with $\alpha\in(0,1)$ 2, the score $\alpha\in(0,1)$ 3 is estimated by a thresholded Gaussian KDE score estimator. For $\alpha\in(0,1)$ 4 in a Sobolev class $\alpha\in(0,1)$ 5, the analysis proves a sharp dichotomy between exponential and polynomial tails (Yu et al., 10 Jan 2026).

Under exponential tails, the fixed-time score error satisfies

$\alpha\in(0,1)$ 6

and the generated distribution converges in total variation at the rate

$\alpha\in(0,1)$ 7

up to logarithmic factors, with $\alpha\in(0,1)$ 8 and $\alpha\in(0,1)$ 9. Under polynomial tails with index $\alpha$ 00,

$\alpha$ 01

and the total-variation sampling rate becomes

$\alpha$ 02

The paper leaves open whether the polynomial-tail sampling rate is minimax optimal (Yu et al., 10 Jan 2026). The main implication is that Gaussian diffusion remains statistically well behaved for heavy-tailed data with exponential decay, but polynomial tails impose an intrinsic $\alpha$ 03-dependent penalty.

This conclusion interacts directly with the recent debate on whether heavy-tailed noise helps generative diffusion. A comparative analysis of Gaussian DDPM-style diffusion and heavy-tailed DLPM with $\alpha$ 04-stable increments shows a subtle trade-off between initialization and training. Heavy-tailed noise can reduce terminal mismatch, giving an initialization term that decays as $\alpha$ 05, but the associated training error accumulates linearly in the reverse horizon $\alpha$ 06,

$\alpha$ 07

whereas the Gaussian alternative carries a weaker initialization term $\alpha$ 08 but avoids the same $\alpha$ 09-multiplicative training accumulation (Cherkaoui et al., 13 May 2026). The paper’s conclusion is that heavy-tailed noise makes the statistical estimation problem harder, often enough to dominate the initialization gain in finite-sample regimes.

The empirical evidence in that study reflects the theoretical bound structure. On a $\alpha$ 10D isotropic $\alpha$ 11-stable target with $\alpha$ 12, GF-Linear and DDPM achieve MMD-RBF $\alpha$ 13, whereas DLPM attains $\alpha$ 14 to $\alpha$ 15; TCE(99\%) is often $\alpha$ 16– $\alpha$ 17 for DLPM versus $\alpha$ 18 for DDPM and GF-Linear. On Wildfires, DLPM( $\alpha$ 19) has the best MMD-RBF ( $\alpha$ 20), yet GF-Linear achieves the lowest upper-tail TCE at $\alpha$ 21 and $\alpha$ 22 (Cherkaoui et al., 13 May 2026). The resulting controversy is not whether heavy-tailed priors can improve tail fidelity in some regimes, but whether they do so after accounting for the harder reverse estimation problem.

5. Heavy-tailed gradients in conditional diffusion and differential privacy

Another usage of HTDM concerns training dynamics rather than output distributions. In conditional diffusion transformers for time series, heavy tails can appear in per-example gradients induced by heterogeneous conditioning contexts such as observed history, missingness patterns, and outlier covariates. The parameter space is partitioned into conditioning-path parameters $\alpha$ 23 and the remainder $\alpha$ 24, with per-example gradient decomposition $\alpha$ 25. Heavy tails are defined empirically by higher tail probability at clipping-relevant thresholds,

$\alpha$ 26

for sufficiently large $\alpha$ 27 (Huang et al., 26 Feb 2026).

The mechanism identified is AdaLN-Zero conditioning. For hidden state $\alpha$ 28 and conditioning $\alpha$ 29,

$\alpha$ 30

with $\alpha$ 31 produced from $\alpha$ 32 by linear projection. Large conditioning realizations can induce extreme $\alpha$ 33, amplify local Jacobian gain, and create rare spikes in $\alpha$ 34. Under DP-SGD, these rare spikes trigger global clipping,

$\alpha$ 35

so conditioning outliers shrink the entire update and increase clipping bias (Huang et al., 26 Feb 2026).

The proposed mitigation is DP-aware AdaLN-Zero, which deterministically bounds both the conditioning vector and the modulation parameters: $\alpha$ 36

$\alpha$ 37

Under bounded-Jacobian assumptions, this yields a per-example gradient bound

$\alpha$ 38

and a sensitivity ratio

$\alpha$ 39

The paper emphasizes that this is a forward-pass sensitivity-control mechanism rather than a modification of DP-SGD itself (Huang et al., 26 Feb 2026).

The empirical effect is targeted tail suppression on the conditioning pathway. On PrivatePower forecasting at $\alpha$ 40, DP-vanilla RMSE is $\alpha$ 41 versus $\alpha$ 42 for DP-aware, and dist_JS changes from $\alpha$ 43 to $\alpha$ 44. On PrivatePower interpolation/imputation at $\alpha$ 45, RMSE changes from $\alpha$ 46 to $\alpha$ 47, and MAE from $\alpha$ 48 to $\alpha$ 49. At $\alpha$ 50, gradient diagnostics show $\alpha$ 51 $\alpha$ 52 of $\alpha$ 53 for DP-vanilla versus $\alpha$ 54 for DP-aware, with $\alpha$ 55 moving from $\alpha$ 56 to $\alpha$ 57, indicating milder clipping (Huang et al., 26 Feb 2026). The work explicitly distinguishes this gradient-tail problem from heavy-tailed outputs: the actionable locus is gradient norm control for conditional diffusion training.

The broader mathematical background of HTDMs includes kinetic, stochastic-process, and quantum formulations in which heavy-tail structure changes macroscopic diffusion behavior. In a linear BGK kinetic equation with heavy-tailed equilibrium $\alpha$ 58 for $\alpha$ 59, the correct scaling is $\alpha$ 60 with $\alpha$ 61, and the macroscopic limit is the fractional anomalous diffusion equation

$\alpha$ 62

rather than classical diffusion. The corresponding numerical analysis develops Asymptotic Preserving schemes and a Duhamel-based scheme with uniform accuracy; a central warning is that naive velocity truncation destroys the fractional limit because it removes the contribution of large velocities (Crouseilles et al., 2015).

In a quantum setting, heavy-tailed waiting times $\alpha$ 63 with $\alpha$ 64 generate anomalous transport and ergodicity breaking. If the state is frozen during waiting, the ensemble width scales as $\alpha$ 65, giving subdiffusion, normal diffusion, or superdiffusion depending on whether $\alpha$ 66, $\alpha$ 67, or $\alpha$ 68. If the system evolves during waiting, the scaling becomes $\alpha$ 69, so only subdiffusion remains. The time-averaged squared width follows different exponents, which the paper interprets as ergodicity breaking (Guo et al., 2024).

Other adjacent constructions show that heavy tails can be produced without explicit jump noise. A transformed Ornstein–Uhlenbeck model with observable process $\alpha$ 70, where $\alpha$ 71 is Gaussian OU and $\alpha$ 72 in the tails, yields regularly varying stationary tails with exponents

$\alpha$ 73

while preserving a Gaussian copula and exponential mean reversion (Borovkov et al., 2011). Ordinary Fokker–Planck dynamics with logarithmic confinement,

$\alpha$ 74

produces Gibbs stationary densities $\alpha$ 75, and the transient variance may display $\alpha$ 76 with $\alpha$ 77 over substantial time windows; the paper explicitly rejects any universal time-rate hierarchy under confinement (Garbaczewski et al., 2010).

Open problems remain different in each subfield but are structurally related. In the evolutionary Hamilton–Jacobi theory, the limit is identified as a minimal viscosity supersolution rather than a full viscosity solution, and deriving uniqueness, rates, broader heavy-tailed kernels, and a canonical ODE for the Dirac location remains open (Mirrahimi, 2018). In Student’s $\alpha$ 78 generative HTDMs, higher-dimensional validation beyond the synthetic $\alpha$ 79D case and principled estimation of $\alpha$ 80 are unresolved (Wakatsuki et al., 1 Jun 2026). In Gaussian diffusion with heavy-tailed targets, the minimax optimality of the polynomial-tail sampling rate is open (Yu et al., 10 Jan 2026). In the heavy-tailed-noise debate, a central unresolved question is whether one can obtain the initialization benefit of heavy-tailed priors without the training penalty induced by weaker concentration and error accumulation (Cherkaoui et al., 13 May 2026). In private conditional diffusion, adaptive and DP-safe calibration of conditioning bounds is future work (Huang et al., 26 Feb 2026).

Taken together, these literatures show that HTDMs are unified less by a single architecture than by a recurring analytical theme: polynomial tails alter the effective Hamiltonian, the admissible score field, the stability of reverse dynamics, the scaling limit of transport, and the statistics of training. The resulting models are therefore best classified by where the heavy-tail mechanism enters the diffusion system and by which quantity—state, target, mutation, waiting time, or gradient—is allowed to retain rare extreme events at leading order.