Lazy Diffusion: Mitigating spectral collapse in generative diffusion-based stable autoregressive emulation of turbulent flows (2512.09572v1)

Abstract: Turbulent flows posses broadband, power-law spectra in which multiscale interactions couple high-wavenumber fluctuations to large-scale dynamics. Although diffusion-based generative models offer a principled probabilistic forecasting framework, we show that standard DDPMs induce a fundamental \emph{spectral collapse}: a Fourier-space analysis of the forward SDE reveals a closed-form, mode-wise signal-to-noise ratio (SNR) that decays monotonically in wavenumber, $|k|$ for spectra $S(k)!\propto!|k|^{-λ}$, rendering high-wavenumber modes indistinguishable from noise and producing an intrinsic spectral bias. We reinterpret the noise schedule as a spectral regularizer and introduce power-law schedules $β(τ)!\propto!τ^γ$ that preserve fine-scale structure deeper into diffusion time, along with \emph{Lazy Diffusion}, a one-step distillation method that leverages the learned score geometry to bypass long reverse-time trajectories and prevent high-$k$ degradation. Applied to high-Reynolds-number 2D Kolmogorov turbulence and $1/12^\circ$ Gulf of Mexico ocean reanalysis, these methods resolve spectral collapse, stabilize long-horizon autoregression, and restore physically realistic inertial-range scaling. Together, they show that naïve Gaussian scheduling is structurally incompatible with power-law physics and that physics-aware diffusion processes can yield accurate, efficient, and fully probabilistic surrogates for multiscale dynamical systems.

• The paper demonstrates that power-law noise scheduling mitigates spectral collapse and preserves high-wavenumber fidelity in turbulent flow emulation.
• It presents lazy diffusion, a one-step denoising framework that leverages pretrained score models to maintain multiscale physical accuracy with lower computational cost.
• Empirical results on Kolmogorov turbulence and oceanic flows confirm significant gains in spectral accuracy and long-horizon forecast stability.

Authoritative Summary of "Lazy Diffusion: Mitigating Spectral Collapse in Generative Diffusion-Based Stable Autoregressive Emulation of Turbulent Flows" (2512.09572)

Introduction and Motivation

The paper addresses the challenge of stable, long-horizon emulation of high-dimensional turbulent flows using generative diffusion models. Traditional physics-based simulation is computationally prohibitive for such systems, and existing deep learning surrogates show poor generalization due to spectral bias. In turbulent flows characterized by broadband, power-law spectra, multiscale interactions propagate errors from high-wavenumber to large-scale modes, destabilizing autoregressive emulation and causing out-of-distribution (OOD) drift.

The work situates diffusion-based generative models as promising fully probabilistic surrogates for physical systems but identifies a critical incompatibility: standard Denoising Diffusion Probabilistic Models (DDPMs) with Gaussian noise scheduling induce "spectral collapse." In the Fourier domain, this manifests as early, wavenumber-dependent signal depletion, especially at high-k, leading to intrinsic spectral bias and instability in long-term forecasts.

Theoretical Analysis of Spectral Collapse in Diffusion Models

By rigorously analyzing the forward SDE of the DDPM framework for data with power-law spectra, the authors provide closed-form expressions showing that the signal-to-noise ratio (SNR) for each mode decays monotonically in wavenumber. The SNR at a diffusion time $T$ and wavenumber $k$ is:

$$\text{SNR}(k, T) \sim \frac{\alpha_T |k|^{-\beta}}{(1-\alpha_T)\sigma^2}$$

where $\alpha_T$ denotes the time-dependent signal amplitude and $|k|^{-\beta}$ is the turbulent power spectrum ($\beta > 0$). This reveals that high-k structure is destroyed much earlier in the diffusion process, depriving the network of learnable gradients in those bands and causing the model to focus on irreducible noise, not physical fine-scale structure. The result is a systematic underfitting of high-wavenumber content, which is highly consequential for systems governed by turbulent cascades.

Moreover, the spectral form of the score-matching loss amplifies this effect: the contribution of high-k modes to the loss increases linearly with wavenumber, forcing the model to expend modeling capacity on noisy, information-depleted modes. These insights provide a formal foundation for observed empirical and numerical pathologies in diffusion-based turbulent emulation.

Power-Law Noise Schedules as Spectral Regularization

To address the wavenumber-stratified collapse, the paper proposes power-law noise schedules for the forward process, parameterized as:

$$\beta(T) = \beta_{\min} + (\beta_{\max} - \beta_{\min}) T^\gamma$$

By tuning the exponent $\gamma$, the scheduling delays the bulk of noise injection toward the end of diffusion time. This design delays the corruption of high-k modes, preserving their structure deeper into the diffusion trajectory and providing the score network with sufficient signal for effective high-wavenumber gradient learning.

Empirical sweeps show that values of $\gamma \approx 5$ optimize spectral fidelity and long-horizon stability, significantly outperforming the standard linear scheduling ($\gamma=1$). However, excessive concentration of noise near $T=1$ ($\gamma > 5.5$) triggers numerical instability due to discrete-time approximation limitations.

Lazy Diffusion: Distilled Single-Step Emulation

Recognizing that conditional forecast distributions encountered in autoregressive emulation are often unimodal (rather than multimodal as in e.g., generative image modeling), the authors propose Lazy Diffusion—a distilled one-step denoising framework. A pretrained score-based model is distilled into a single-step network by learning to reconstruct clean data from moderately noised samples at a fixed intermediate diffusion time $T^*$.

The lazy distillation leverages the geometry already encoded in the pretrained score network, allowing for high-fidelity forecast generation with orders-of-magnitude lower computational cost compared to reverse SDE integration. Critically, lazy diffusion maintains improved high-k fidelity and long-horizon stability, inherited from the physics-aware representation learned by the power-scheduled base model.

Experimental Results

Validation is performed on two canonical systems:

• System 1: 2D forced Kolmogorov turbulence at $Re=10,000$, simulated via a pseudo-spectral solver.
• System 2: Surface ocean reanalysis data (GLORYS) for the Gulf of Mexico, evaluated for both zonal and meridional velocity fields.

Quantitative metrics include RMSE, Earth Mover's Distance, KL divergence of physical state distributions, and spectral error measures (relative error in power-law scaling and high-wavenumber fidelity).

Key findings include:

• Power-law scheduling ($\gamma = 5$) reduces relative spectral error from 96.3% (standard DDPM) to 10.2% in Kolmogorov turbulence.
• Earth Mover's Distance and KL divergence are minimized under the power schedule compared to linear DDPM.
• Lazy diffusion retains essentially all the spectral and statistical advantages of the base model while requiring a single forward pass.
• Excessive power scheduling ($\gamma > 5.5$) produces catastrophic errors, affirming the need for careful schedule resolution.
• Oceanic flow emulation matches physical spectra in both zonal and meridional components for power-scheduled/lazy models, whereas standard DDPMs show rapid drift toward unphysical, overly dissipative states.

Notably, RMSE fails to capture structural and spectral fidelity, illustrating the necessity for physically meaningful evaluation protocols.

Practical and Theoretical Implications

Practical implications include the feasibility of stable, computationally efficient, physically accurate, long-horizon generative forecasting in high-dimensional turbulent systems—a crucial capability for geosciences, climate science, and engineering. Lazy diffusion presents a clear path to real-time uncertainty propagation in high-dimensional autoregressive simulation.

Theoretical implications are more foundational: the work demonstrates that naive application of diffusion models with out-of-the-box Gaussian noise schedules is fundamentally incompatible with turbulent or power-law-governed nonlinear physics. Instead, noise injection must be interpreted and designed as a spectral regularizer to ensure the preservation of multi-scale physical information throughout the generative process.

Additionally, the framework highlights the inherent limitations of one-step deterministic denoisers, the necessity for distributionally aware conditioning (to counter distribution shift in autoregression), and unifies spectral analysis with diffusion-based uncertainty quantification in dynamical systems.

Limitations and Future Directions

The power-law scheduling approach introduces sensitivity to the schedule and discretization scheme; aggressive schedules conflict with discrete-time sampling and can be destabilizing. Lazy diffusion inherits the limitations of its base score model and cannot transcend biases present there. The study is confined to two-dimensional dynamics and moderate resolution; the extension to fully three-dimensional turbulence, anisotropic flows, and coupled multi-physics phenomena remains an open question.

Potential future research directions include:

• Spectrally adaptive, data-driven noise scheduling responsive to local or nonstationary energy content,
• Unification of diffusion-based generative modeling with operator learning frameworks for broader generalization,
• Distributed, high-performance implementations for operational-scale forecasting,
• Application to joint state-parameter estimation, uncertainty propagation, and assimilation in hybrid physics-ML systems.

Conclusion

The paper establishes a rigorous connection between the mathematical structure of diffusion models and the physics of turbulent, power-law-dominated systems. By redesigning the forward process with power-law-aware noise scheduling and distilling the score geometry into lazy diffusion, the intrinsic spectral bias and instability of standard diffusion-based forecasting are mitigated. This work provides both a theoretical and algorithmic blueprint for stable, uncertainty-preserving, physically consistent generative modeling in complex multiscale dynamical systems, suggesting a path forward for data-driven simulation frameworks in the geosciences and beyond.