Extension of spectral-gap subspace-separation phenomenology to curved manifolds

Determine whether the spectral-gap-based subspace-separation phenomenology derived for generative diffusion models on linear manifolds extends to data supported on smooth curved manifolds; specifically, ascertain whether the eigenvalue spectrum of the negative Jacobian of the score function exhibits analogous intermediate and final gaps that reflect subspace structure in the tangent spaces of such curved manifolds during the reverse-time generative diffusion process.

Background

The paper analytically characterizes the Jacobian eigenvalue spectrum of the score function for generative diffusion models when data lie on low-dimensional linear manifolds, using random matrix theory. It identifies intermediate spectral gaps that separate tangent subspaces with different variances and a final gap revealing the full manifold dimension, and it organizes the generative dynamics into three phases (trivial, manifold coverage, consolidation).

While the results are proved for linear manifolds, the authors investigate trained networks on both synthetic linear datasets and natural images to compare phenomenology. They explicitly conjecture that the same subspace-separation behavior—visible through spectral gaps in the Jacobian spectrum—captures the main features present in tangent spaces of curved manifolds, leaving open the question of formal extension beyond the linear setting.