Recursively Trained Diffusion Models: Limiting Collapse Distribution and Spectral Characterization

Abstract: Recursive training of generative models on their own outputs can lead to model collapse, a compounding drift away from the true data distribution. Existing theoretical works bound finite-round error accumulation in the context of diffusion models, but two questions remain open:~what distribution does the recursion converge to, and how fast? We answer both, isolating a mechanism distinct from imperfect learning: even with perfect score estimation and exact sampling, the early stopping of the reverse diffusion (required for numerical stability) drives a progressive drift away from the data distribution. We prove that this recursion converges geometrically to a unique limiting distribution, which admits a closed-form characterization as an infinite mixture of increasingly Gaussian-smoothed versions of the data distribution. A Hermite spectral decomposition of this limit reveals that recursive training acts as a low-pass filter: higher-order modes, which encode fine non-Gaussian structure, are attenuated much more strongly than coarse modes. This spectral picture motivates annealed truncation schedules that progressively shrink truncation times across retraining rounds; we prove that any schedule converging to $0$ asymptotically eliminates recursive compounding. Finally, we show our idealized characterization is robust: in the presence of discretization and score estimation errors, the learned distribution remains in a Wasserstein-2 ball around the ideal limit, with mode-dependent contraction rates that contract high-order errors faster than low-order ones. We validate the theory on synthetic Gaussian mixtures and CIFAR-10.

• The paper introduces a rigorous operator-theoretic analysis of recursively trained diffusion models, revealing that truncation inherently drives asymptotic collapse.
• It derives a closed-form collapse distribution via an infinite Neumann series and quantifies geometric convergence rates based on model parameters.
• The study employs spectral Hermite decomposition to demonstrate progressive high-frequency loss and proposes annealed truncation schedules to mitigate collapse.

Asymptotic Collapse and Spectral Attenuation in Recursively Trained Diffusion Models

Introduction and Motivation

This paper provides a rigorous operator-theoretic analysis of recursive training in diffusion models, focusing on the phenomenon of model collapse—progressive drift caused by models being retrained on mixtures of their own synthetic outputs and small fractions of true data. Unlike prior studies which bound finite-round error, this work addresses the asymptotic distributional outcome of recursive retraining, explicitly characterizing both the limiting collapse distribution and the rate of convergence. The key insight is that model collapse occurs even in the ideal case of perfect training and sampling, solely due to the necessity of truncating the reverse diffusion process for numerical stability. This truncation, and its spectral effect, are formally analyzed, providing mode-by-mode degradation rates and suggesting principled strategies to counteract compounding collapse.

Model and Mechanism of Collapse

The recursive training scheme is constructed as follows: at generation $i$, a mixture distribution $q_i = \alpha p_{\text{data}} + (1-\alpha)p^i$ is formed, with $p^i$ the current model, $p_{\text{data}}$ the true data distribution, and $\alpha \in (0,1)$ the fraction of real samples. Synthetic samples are generated by integrating the reverse-time SDE from $T$ to a truncation time $t_0>0$ to avoid instability near $t=0$. The resulting distribution, denoted $U_{t_0}q_i$, forms the next generation's model, $p^{i+1}$.

Remarkably, even if both score estimation and numerical integration are performed exactly, the act of truncating the reverse process results in nonzero residual smoothing. Repeated recursion systematically compounds this smoothing, yielding an operator recursion whose fixed-point is the collapse distribution.

Limiting Distribution: Closed-Form and Rates

The collapse distribution $q_i = \alpha p_{\text{data}} + (1-\alpha)p^i$0 is shown to be the unique fixed point of the operator

$q_i = \alpha p_{\text{data}} + (1-\alpha)p^i$1

where $q_i = \alpha p_{\text{data}} + (1-\alpha)p^i$2 is the Ornstein-Uhlenbeck propagation operator at time $q_i = \alpha p_{\text{data}} + (1-\alpha)p^i$3. This fixed point is explicitly given as an infinite Neumann series: $q_i = \alpha p_{\text{data}} + (1-\alpha)p^i$4 This representation clarifies that the limit is a geometrically weighted average of increasingly smoothed versions of the original data, with weights and smoothing increments determined by $q_i = \alpha p_{\text{data}} + (1-\alpha)p^i$5 and $q_i = \alpha p_{\text{data}} + (1-\alpha)p^i$6.

Convergence of the recursion is geometric, with rate $q_i = \alpha p_{\text{data}} + (1-\alpha)p^i$7. Various limiting regimes are established: as $q_i = \alpha p_{\text{data}} + (1-\alpha)p^i$8, $q_i = \alpha p_{\text{data}} + (1-\alpha)p^i$9 converges to the once-smoothed data $p^i$0; as $p^i$1, it collapses to the standard Gaussian; as $p^i$2, collapse vanishes and the process recovers $p^i$3.

Figure 1: Limiting behaviors of $p^i$4 on a 2D Gaussian mixture, including the closed-form Neumann-series representation and the transitions between different limiting regimes.

Spectral Decomposition and Low-Pass Filtering

The limiting collapse distribution is further decomposed in the Hermite polynomial basis, the natural coordinates of the Ornstein-Uhlenbeck semigroup: $p^i$5 where $p^i$6 are multivariate Hermite polynomials and $p^i$7 are explicit attenuation factors: $p^i$8 High-degree (high-frequency, non-Gaussian) Hermite modes are exponentially suppressed compared to low-degree modes encoding mean and covariance. This shows recursive diffusion training acts as a low-pass filter, confirming and quantifying prior qualitative claims regarding tail and multimodality loss under recursive retraining.

Figure 2: Hermite-mode attenuation of $p^i$9 for varying $p_{\text{data}}$0, demonstrating progressive loss of high-frequency components and the corresponding transition to more Gaussian-like limiting distributions.

Mitigation: Annealed Truncation Schedules

Given that model collapse in diffusion settings is fundamentally due to truncation, the paper next considers dynamic truncation times $p_{\text{data}}$1 (annealed schedules). The main result is that if $p_{\text{data}}$2 as $p_{\text{data}}$3, recursive compounding is asymptotically eliminated: the limiting distribution converges to $p_{\text{data}}$4. Schedules that merely decrease $p_{\text{data}}$5 to a positive limit only partially mitigate collapse; the residual smoothing floor persists.

Figure 3: Annealed truncation schedules eliminate recursive compounding; KL divergence decays to zero under schedules with $p_{\text{data}}$6, but plateaus if $p_{\text{data}}$7 tends to a positive value.

This annealing directly targets the diffusion-specific collapse mechanism, complementing mitigation strategies based on increasing fresh-data proportion ($p_{\text{data}}$8).

Numerical and Spectral Validation

Experiments, both on controlled Gaussian mixtures and high-dimensional datasets such as CIFAR-10, empirically validate the predicted contraction rates and the structure of collapse. Notably:

• For fixed $p_{\text{data}}$9 and $\alpha \in (0,1)$0, the Fréchet Inception Distance (FID) between $\alpha \in (0,1)$1 and the limiting $\alpha \in (0,1)$2 decays at the predicted geometric rate, with stronger suppression for smaller $\alpha \in (0,1)$3.
• Measured high-frequency (Fourier/Hermite) energy of $\alpha \in (0,1)$4 rapidly contracts, with the largest deficit accruing in low-$\alpha \in (0,1)$5 regimes. The deficit is monotonic in $\alpha \in (0,1)$6 and $\alpha \in (0,1)$7, consistent with theory.

Figure 4: $\alpha \in (0,1)$8 trajectories and the suppression of high-frequency spectral energy during recursive training on CIFAR-10, demonstrating both the predicted contraction to the asymptotic collapse distribution and the preferential loss of fine-scale sample diversity.

Robustness to Imperfect Score Estimation and Discretization

The paper extends its analysis to accommodate practical sources of error: non-ideal score estimation and numerical integration. Under bounded per-generation errors, the empirical distribution $\alpha \in (0,1)$9 is shown to remain within a Wasserstein ball of the ideal collapse distribution, with radius determined by the error and a spectral bias: higher-order (finer) modes contract faster, so errors are more persistent in low-frequency components.

Practical and Theoretical Implications

This analysis challenges the dominant view that collapse in generative self-training is exclusively a consequence of poor learning or insufficient data refreshment. Instead, it reveals a diffusion-specific mechanism rooted in dynamics and numerical constraints, independent of estimator quality. This insight prompts diffusion model practitioners to consider truncation-aware mitigation, such as truncation annealing, and to be cautious about spectral bias introduced by repeated retraining—even when score estimation appears accurate.

Future research is suggested in several technical directions: integrating this spectral analysis with non-asymptotic, finite-sample bounds; designing adaptive, data-driven truncation schedules; and generalizing the framework to alternative noising/sampling processes and non-Gaussian reference measures.

Conclusion

This work provides the first full asymptotic and spectral characterization of model collapse in recursively trained diffusion models, rigorously identifying truncation-induced smoothing as the principal driver in the ideal setting, quantifying the geometric convergence rate, and revealing the precise loss of high-frequency content via Hermite decomposition. Annealed truncation schedules are theoretically validated as effective mitigation, providing practical guidance for design of robust recursive generative pipelines. These advances build a foundation for quantitatively principled, dynamically stable generative modeling in data-limited or self-consuming regimes.