Riemannian Score-Based Generative Models

• Riemannian Score-Based Generative Models are frameworks that extend diffusion methods to complex data manifolds using intrinsic and score-derived Riemannian metrics.
• They integrate stochastic differential equations with geometric constructs such as geodesics and the Laplace–Beltrami operator to perform interpolation, sampling, and editing.
• Empirical evaluations show that RSGMs achieve competitive performance in image interpolation and generative tasks on both synthetic and structured manifolds.

Riemannian Score-Based Generative Models (RSGMs) generalize diffusion-based generative modeling beyond the Euclidean setting by endowing data spaces with nontrivial Riemannian metrics derived from either intrinsic data geometric structure or from the learned score function produced by neural score networks. This approach enables the principled treatment of interpolation, sampling, and generation of data lying on complex manifolds—whether these are known a priori (e.g., spheres, rotation groups, or mesh surfaces) or are implicitly defined by the learned density of powerful diffusion models. RSGMs combine the stochastic analysis of noising and denoising SDEs on manifolds with the geometric machinery of Riemannian metrics, geodesics, and associated differential operators.

1. Geometric Foundations of RSGMs

RSGMs operate on a smooth, possibly high-dimensional Riemannian manifold $(\mathcal M, g)$. The Riemannian metric $g$ is a smoothly varying family of inner products $g_x$ on each tangent space $T_x\mathcal{M}$, represented in local coordinates $x=(x^1,\ldots,x^d)$ by a symmetric positive definite matrix $G(x)=[g_{ij}(x)]$. The metric induces a volume form $$\mathrm{dVol}_g(x)=\sqrt{\det G(x)}\,dx^1\cdots dx^d$$, a gradient operator $\nabla_\mathcal{M}$, divergence $\operatorname{div}_\mathcal{M}$, and the Laplace–Beltrami operator $\Delta_\mathcal{M}$. These structures underlie both the forward diffusion ("noising") and the reverse-time generative ("denoising") processes in RSGMs (Bortoli et al., 2022).

A prototypical modeling choice is to let the manifold be a submanifold of ambient space: $M = \{x \in \mathbb{R}^n : \xi(x) = 0\}$ for some smooth map $\xi:\mathbb{R}^n\to\mathbb{R}^k$ of full rank on $M$. The tangent space at $x$ is $T_xM=\{v\in\mathbb{R}^n:\nabla\xi(x)^\top v = 0\}$, with the projection operator $$\Pi_x=I_n - \nabla\xi(x)[\nabla\xi(x)^\top\nabla\xi(x)]^{-1}\nabla\xi(x)^\top$$ (Liu et al., 7 May 2025).

2. Riemannian Metrics Derived from the Score Function

For models in which data lies in $\mathbb{R}^D$ with an unknown, complex structure (e.g., images), RSGMs define a data-space metric using the score function $s_\theta(x, t) \approx \nabla_x \log p_t(x)$. Two main constructions are found in recent work:

• Score Jacobian Metric: $g_x = G_x = J_x^\top J_x$, where $$J_x = \nabla_x s_\theta(x, t) \approx \nabla_x^2 \log p_t(x)$$. In coordinates,

$$g_{ij}(x) = \sum_{a=1}^{D} \partial_i s_\theta^a(x, t) \partial_j s_\theta^a(x, t)$$

This metric compresses directions with low change in the score and expands those with sharp score variations, aligning geodesics with natural semantic changes on the data manifold (Saito et al., 28 Apr 2025).

• Stein Score Metric Family: $g(x) = I_n + \lambda\, s(x) s(x)^\top$, $\lambda > 0$, where $s(x) = \nabla_x \log p(x)$. This metric leaves tangential directions to the score unchanged and stretches normal directions, effectively encoding the data manifold geometry implicitly learned by the diffusion model (Azeglio et al., 16 May 2025).

These score-derived metrics are symmetric positive definite when the score is nondegenerate and produce meaningful geometric structure for downstream tasks such as interpolation, extrapolation, and image editing.

3. Forward and Reverse Diffusions on Manifolds

The general RSGM paradigm involves a forward process (diffusion), which gradually corrupts data with noise, and a reverse-time process that generatively reconstructs data samples. On a Riemannian manifold, the forward SDE can be written as

$dX_t = f(t, X_t)\,dt + g(t)\,dB_t^\mathcal{M}$

where $B_t^\mathcal{M}$ is manifold Brownian motion and $g(t)$ is a (possibly time-dependent) scalar or matrix controlling the noise level. For compact manifolds, pure Brownian noising (drift $f=0$, $g=1$) is common; for noncompact manifolds, an Ornstein–Uhlenbeck drift or a Brownian bridge may be more appropriate (Bortoli et al., 2022).

The time-reversal of the diffusion yields a generative process: $$dY_t = \Big[ f(T-t, Y_t) - g(T-t)^2 \nabla_\mathcal{M} \log p_{T-t}(Y_t) \Big] dt + g(T-t)\, d\overline{B}_t^\mathcal{M}$$ with the so-called Riemannian score $\nabla_\mathcal{M} \log p_t(x)$. In submanifold settings, all drift and diffusion terms are projected onto the tangent space using $\Pi_x$ (Liu et al., 7 May 2025).

Score approximation is by neural networks $s_\theta(t, x)$, trained with denoising score matching (DSM) or sliced score matching (SSM) loss: $$L_{\mathrm{DSM}}(\theta) = \mathbb{E} \| s_\theta(t, X_t) - \nabla_\mathcal{M}\log p_{t|0}(X_t|X_0) \|^2$$ where the transition kernel $p_{t|0}$ may be tractable only in a few special cases.

4. Numerical Methods: Geodesic Computation and Interpolation

A distinguishing feature of recent data-space RSGMs is the explicit use of geodesics in the score-based metric for interpolation, editing, and exploration. Given a Riemannian metric $g_x$, geodesics $\gamma(s)$ minimize the path energy

$$E[\gamma] = \frac{1}{2} \int_0^1 \gamma'(s)^\top G_{\gamma(s)} \gamma'(s)\, ds$$

subject to boundary conditions. The geodesic equations involve Christoffel symbols: $$\frac{d^2\gamma^k}{ds^2} + \sum_{i,j} \Gamma^k_{ij}(\gamma(s)) \frac{d\gamma^i}{ds} \frac{d\gamma^j}{ds} = 0$$ with explicit expressions for $\Gamma^k_{ij}$ depending on the local behavior of the score (Saito et al., 28 Apr 2025, Azeglio et al., 16 May 2025).

In practical high-dimensional settings, geodesics are discretized, and the length or energy functional is minimized numerically. Energy-based optimization (using gradient-based solvers such as Adam and, in some cases, Riemannian Adam for normalization/transport) is favored over explicit ODE integration. Regularization by the variance of Euclidean step-lengths is often included to prevent path collapse (Saito et al., 28 Apr 2025).

For interpolation, real or generated data points are first encoded into the noise space (e.g., by DDIM inversion), geodesics are computed under the current metric, and the intermediates are decoded back to data space. For extrapolation, manifold-aware walks are performed by initializing a momentum along the last geodesic segment and updating with a convex combination of the momentum and the local score, enabling plausible traversal beyond empirical data (Azeglio et al., 16 May 2025).

5. Empirical Results and Comparisons

Experimental evaluations have demonstrated that RSGMs achieve superior or competitive performance to previous generative and interpolation schemes on diverse manifolds and high-dimensional data.

• Image Interpolation: On Stable Diffusion and MNIST, geodesic interpolation in the score-based metric yields smoother, more realistic, and semantically coherent transitions compared to linear (Lerp/Slerp) or latent-noise correction methods (NAO, NoiseDiffusion). Quantitatively, RSGMs match or outperform baselines on metrics such as MSE, LPIPS, DreamSim, and CLIP-based scores; for instance, the highest "Reality" (0.716), "Noisiness" (0.818), and competitive "Fidelity" (0.810) scores on Stable Diffusion were observed for the proposed RSGM approach (Saito et al., 28 Apr 2025).
• Synthetic and Structured Data: On synthetic spheres ($S^2$), RSGMs nearly recover ground-truth geodesics with relative error dropping from 4.9% (no geometric term) to 0.07% for large metric strength ($\lambda \geq 1000$) (Azeglio et al., 16 May 2025). On Rotated MNIST, improvements in PSNR and SSIM favor the geodesic method.
• Generative Modeling on Known Manifolds: For explicit Riemannian manifolds (e.g., Earth data on $S^2$, SO(10), or molecular configuration spaces), RSGMs achieve NLL and sample-quality competitive with or exceeding mixtures of Kent, Riemannian CNF, Moser flows, and Riemannian flow models (Bortoli et al., 2022, Liu et al., 7 May 2025).

A summary of typical quantitative results is given in the table below, highlighting image interpolation metrics on Stable Diffusion (four test examples) (Saito et al., 28 Apr 2025):

Method	MSE [$\times 10^{-3}$]	LPIPS [$\times 10^{-1}$]	Reality	Noisiness	Fidelity
Lerp	7.89	1.71	0.389	0.406	0.686
Slerp	7.89	1.71	0.704	0.765	0.784
NAO	64.04	6.03	0.617	0.766	0.815
NoiseDiff	14.13	2.35	0.600	0.646	0.783
Proposed RSGM	7.89	1.71	0.716	0.818	0.810

6. Extensions and Algorithmic Variants

RSGMs have been extended and adapted in several directions:

• Riemannian Schrödinger Bridge: The Riemannian Diffusion Schrödinger Bridge (RDSB) generalizes classical Schrödinger bridge algorithms to non-Euclidean settings, enabling entropic interpolation between distributions on manifolds. Alternating optimization of forward and backward drifts via iterative proportional fitting (IPF) reduces the number of simulation steps required and enhances sample quality, particularly for interpolation tasks (Thornton et al., 2022).
• Projection Schemes for General Submanifolds: Riemannian Denoising Diffusion Probabilistic Models (RDDPMs) leverage only the level-set definition and its derivatives to implement forward and reverse SDEs on general submanifolds, circumventing the need for explicit geodesic, eigenfunction, or heat kernel computations. Score approximation is conducted with projected neural networks, and stepwise projection via Newton's method enforces manifold constraints (Liu et al., 7 May 2025).
• Conditional/Richer Geodesics and Sampling: Potential extensions include Riemannian Langevin dynamics for curvature-aware generation, class-conditional or text-guided geodesics via additional forcing, and the application of frame-wise geodesic parallel transport for tasks such as video editing (Saito et al., 28 Apr 2025).
• Open Mathematical and Computational Challenges: Key directions for future research include efficient approximation of geometric quantities (e.g., Christoffel symbols, divergence corrections) in high dimension, metric extension to the full time-dependent SDE, and the analysis of manifold singularities arising from degenerate scores (Saito et al., 28 Apr 2025).

7. Computational Complexity, Limitations, and Outlook

RSGMs incur additional computational overhead relative to Euclidean SGMs due to the need for geometric operations (metric calculations, projections, geodesic shooting), but complexity is generally dominated by the cost of score network evaluation and optimization steps. For submanifold projection methods, the chief cost is Newton iterations for constraint satisfaction per SDE step (Liu et al., 7 May 2025). Sampling costs scale as $O(N)$ or $O(ndN)$, where $n$ is latent dimension, $d$ is data dimension, and $N$ is the number of discretization steps.

Limitations include the tractability of heat kernels or transition densities for generic manifolds, sensitivity to discretization choices, and the challenges posed by manifolds with boundary or singularity loci. Empirically, RSGMs achieve robust sample quality across a range of data modalities and geometries; a plausible implication is that score-based metrics capture semantically meaningful variations aligned with the true data manifold.

In summary, Riemannian Score-Based Generative Models synthesize differential geometry and modern generative modeling, providing a principled framework for generative tasks and interpolation on complex data manifolds, both known and implicitly learned (Bortoli et al., 2022, Saito et al., 28 Apr 2025, Azeglio et al., 16 May 2025, Liu et al., 7 May 2025, Thornton et al., 2022).