Spectral Diffusion Models (GSDM)

• Spectral Diffusion Models (GSDM) are generative models that operate over spectral representations such as Fourier, spherical harmonics, and graph eigen-decompositions to capture data geometry.
• They leverage stochastic differential equations and specialized score matching techniques to incorporate inductive biases, improving performance in applications like image processing and turbulence modeling.
• GSDMs reduce computational cost and improve error scaling by working in low-dimensional spectral spaces, while addressing challenges like basis selection and anisotropic noise.

Spectral diffusion models (GSDM) constitute a class of generative models that extend the theoretical and algorithmic principles of diffusion processes to operate in spectral domains. This approach leverages frequency, eigen, or harmonic representations to enhance modeling fidelity, computational efficiency, and inductive bias relative to grid- or spatial-domain diffusion. GSDM has seen rapid development across domains such as high-dimensional functionals, spherical data, graphs, image and signal processing, and physics-informed learning, building on the foundations of score-based generative modeling and stochastic differential equations (SDEs).

1. Mathematical Framework and Foundations

Spectral diffusion models are constructed by mapping data into a spectral basis—such as Fourier, spherical harmonics, eigenvectors of a graph Laplacian, or a Karhunen–Loève basis. A forward stochastic process is defined on the spectral coefficients, perturbing them through a sequence of Gaussian or structured noises. The reverse-time process is parameterized (usually with neural networks) to generate samples by denoising from noise back to data, guided by the learned spectral score.

For functional data on compact domains, consider a random process $Y(x)$ with covariance kernel $K_Y(x,x')$. The Karhunen–Loève expansion gives an orthonormal basis $\{e_m\}$ and uncorrelated coefficients $\{Z_m\}$:

$$Y(x) = \sum_{m=0}^\infty \sqrt{\lambda_m} Z_m e_m(x), \qquad Z_m \sim \mathcal{N}(0,1), \quad \mathbb{E}[Z_mZ_{m'}]=\delta_{m,m'}.$$

A spectral SDE is defined on $Z_t$:

$$dZ_t = -\frac{1}{2}Z_t dt + \beta(t) dB_t,\quad Z_0\sim p_0(Z),$$

with time reversal incorporating the spectral score $\nabla_{z}\log p_t(z)$ (Phillips et al., 2022).

On the 2-sphere $\mathbb S^2$, the spectral diffusion process is defined on spherical harmonic coefficients $\hat a$ via the spherical discrete Fourier transform with geometry-dependent covariance $\Sigma$:

$$d\hat{\mathbf{a}} = \widehat{\mathbf{f}}(\hat{\mathbf{a}}, t)\, dt + g(t) d\tilde{\mathbf{v}}(t),$$

where $\tilde{\mathbf{v}}$ is spherical mirrored Brownian motion, yielding non-isotropic, geometry-induced inductive bias in the spectral domain (Brutti et al., 28 Jan 2026).

For graphs, one exploits the eigen-decomposition of adjacency or Laplacian matrices, restricting diffusion to the eigenvalues (spectrum) and, in advanced models, also treating eigenvectors as SDE-driven variables for full permutation invariance (Schwarz et al., 9 Oct 2025, Luo et al., 2022).

2. Score Matching, Inductive Bias, and Loss Functionals

Spectral diffusion entails specialized score matching objectives that reflect the domain geometry and statistical structure. In $\mathbb R^d$, score matching in spectral space can be rendered isotropic; but on $\mathbb S^2$, the score objective must be $\Sigma$-weighted due to anisotropic noise:

$$\mathbb{E}_{t,\hat a(0),\hat a(t)}\left\|\hat s_\theta(\hat a(t),t) - \Sigma \nabla_{\hat a} \log \hat p_{t|0}(\hat a(t)|\hat a(0)) \right\|^2_2$$

and is not equivalent to spatial score matching, reflecting essential geometric coupling between modes (Brutti et al., 28 Jan 2026).

On graphs, diffusion in the spectrum directly provides permutation invariance "in the dynamics," eliminating the need for specific architectural inductive biases such as GNNs. For the Dyson Diffusion Model (DyDM), spectral scores are learned over eigenvalues via pathwise denoising-score matching, enforcing the inherent Coulomb-repulsion of eigenvalues and reflecting the Weyl chamber structure (Schwarz et al., 9 Oct 2025).

For image or signal modeling, frequency grouping and hierarchical spectral noising can further impose a multi-scale inductive prior, enabling disentanglement of semantic abstraction from fine detail, as in Groupwise Spectral Diffusion (Lee et al., 2023).

3. Algorithmic Construction, Sampling, and Optimization

Spectral diffusion models admit efficient sampling and inference procedures due to the reduced dimension and diagonal structure in the spectral domain. For models on $\mathbb S^2$, the SDFT is implemented via explicit construction of the $Y$ and $Q$ matrices. Sampling proceeds by:

1. Forwardly diffusing the spectral coefficients under the prescribed SDE.
2. At each reverse step, denoising using learned spectral scores (often a simple multi-layer perceptron for low-dimensional spectra).
3. Optionally applying domain-specific constraints or physics-informed losses in spectral space (Gallon et al., 10 Feb 2026, Liu et al., 2023).

For physics-informed learning, spectral diffusion is coupled with posterior-guided sampling in the latent spectral space, with functional constraints and data-fit losses enforced by differentiable solvers such as Adam at each diffusion step. This maintains controllable regularity and suppresses high-frequency artifacts (Gallon et al., 10 Feb 2026).

In graph GSDMs, the forward and reverse SDEs are defined directly over the eigenvalues $\Lambda$ of the adjacency or Laplacian, with eigendecomposition and reconstruction mapping back to the spatial domain. Empirically, this leads to exponentially improved error bounds and much reduced sampling costs compared to full-matrix diffusion (Luo et al., 2022).

Spectral feature-preserving diffusion (e.g., InSPECT) further modifies the forward chain to converge selectively toward a Gaussian with empirically estimated means and variances for each Fourier mode, thereby preserving invariant class-specific information throughout the generative process (Yan et al., 19 Dec 2025).

4. Geometric Phases, Spectral Bias, and Manifold Structure

Analysis of the spectral dynamics of the score Jacobian reveals distinct geometric phases in the generative process:

• Trivial phase: The score field is isotropic; sample paths exhibit Gaussian-like behavior with no clear separation between modes.
• Manifold coverage: Intermediate spectral gaps open; diffusion explores tangent modes along the data manifold, learning the density of submanifolds.
• Manifold consolidation: Spectral gaps drive strong contraction onto the data support; the score projects noise orthogonally to the manifold (Ventura et al., 2024).

This phase separation explains why spectral diffusion models avoid "manifold overfitting" typical of plain likelihood-based generative models. Spectral diffusion respects the intrinsic data geometry, as shown by closed-form random-matrix-theory-derived gap formulas:

$$\Delta(t) = \frac{\gamma_+}{t + \gamma_+}, \qquad t_\text{max} = \sqrt{\gamma_- \gamma_+}$$

where $\gamma_+$ and $\gamma_-$ denote maximal and minimal signal variances in the data covariance spectrum (Ventura et al., 2024).

Spectral bias is ubiquitous in diffusion learning: convergence time for a given mode scales inversely with its variance, i.e., $\tau_k^* \sim \lambda_k^{-1}$. High-variance modes are learned rapidly, necessitating spectral-aware scheduling, loss reweighting, or data prewhitening to avoid over-smoothing or loss of detail (Wang, 5 Mar 2025).

5. Applications Across Domains

Image and Hyperspectral Modeling: Spectral diffusion priors have yielded improvements in hyperspectral image super-resolution by enforcing per-pixel spectral fidelity, permitting optimization over spectra with low-dimensional denoisers and tractable maximum a posteriori fusion (Liu et al., 2023).

Turbulence and Scientific Surrogate Modeling: Conditioning diffusion models on neural operator predictions recovers lost high-frequency content in turbulent flows, yielding state-of-the-art modal energy spectrum accuracy, long-term stability, and substantial reduction in spectral $L^2$ error across several PDE-based benchmarks (Oommen et al., 2024).

Physics-informed Diffusion: Spectral diffusion in latent variable space allows for explicit encoding of Sobolev regularity, significantly reducing training and inference cost for inverse and forward PDE problems, while constraining generated solutions to physically-meaningful subspaces (Gallon et al., 10 Feb 2026).

Graph Generation: GSDM has proven effective for graph generation, outperforming previous models in degree, clustering, and orbit metric benchmarks for both generic and molecular graphs, while achieving 10-70x speedup due to low-rank spectral inference (Luo et al., 2022).

Quantum Dot Spectroscopy: Spectral-diffusion-based models have been rigorously analyzed for quantum dot blinking, providing mechanistic explanations (e.g., the Frantsuzov–Marcus fluctuating-rate model) and matching experimental power-law behavior in on/off times and $1/f$-type noise, distinguishing them from less realistic alternative models (Busov et al., 2018).

6. Design Principles, Extensions, and Limitations

Spectral diffusion models enable:

• Reduced sampling dimension and computational cost (for $K\ll N$ spectral coefficients).
• Natural incorporation of geometry-aware and scale-dependent inductive biases.
• Plug-and-play priors: spectral diffusion objectives can be analytically embedded in fusion and inverse problems via KL or denoising loss terms, avoiding explicit generation steps for the prior (Liu et al., 2023).
• Hierarchical and groupwise latent codes in the frequency domain, supporting interpretable disentanglement and controlled manipulation of semantic abstraction and detail (Lee et al., 2023).

Limitations and open directions include:

• Necessity of domain-adapted basis selection (e.g., band-limited bases, learned eigenmodes).
• Non-equivalence of spatial and spectral score objectives on manifolds with curvature or non-trivial topology (Brutti et al., 28 Jan 2026).
• Potential mismatches between spectral and spatial training metrics, especially under band limitation and anisotropic noise.
• Invariant mode estimation in feature-preserving GSDM requires either class labels or clustering; unsupervised extensions remain an open research area (Yan et al., 19 Dec 2025).

7. Theoretical Guarantees and Empirical Performance

Spectral GSDMs provide quantifiable theoretical improvements over full-rank, spatial diffusion models. For graph generation, spectral diffusion reduces error scaling from $O(n^2 \exp n^2)$ to $O(n\exp n)$ in adjacency matrix reconstruction (Luo et al., 2022). In hyperspectral and turbulence modeling, spectral priors improve spectral angle (SAM) and energy spectrum alignment metrics by factors of 4–6 (Liu et al., 2023, Oommen et al., 2024).

Spectral-feature-preserving models such as InSPECT report 39.2% mean FID reduction and 45.8% IS improvement versus standard DDPM for fixed computation, with faster convergence and improved sample diversity (Yan et al., 19 Dec 2025).

In functional and physics-informed GSDMs, latent spectral compression and regularity preservation yield 3x–15x inference speedup and 2–4x reduction in mean PDE error relative to grid-based diffusion surrogates (Gallon et al., 10 Feb 2026).

---

By systematically extending diffusion models into spectral domains, GSDM establishes a versatile, theoretically justified paradigm for generative modeling, statistical inference, and inverse problems across structured data modalities, yielding substantial advances in sample quality, efficiency, and geometric alignment.