Score-Based Riemannian Geometry

• Score-based Riemannian geometry is the synthesis of deep generative models with geometric structures, enabling analysis of data concentrated on low-dimensional manifolds.
• It constructs Riemannian metrics from score functions, facilitating efficient computation of geodesics, projections, and optimization through local curvature and tangent space estimation.
• Applications include generative sampling, image interpolation, and dimensionality recovery, with theoretical guarantees ensuring robust manifold recovery and scalable algorithm performance.

Score-based Riemannian geometry refers to the synthesis of Riemannian geometric structures with score-based models—primarily denoising diffusion and score-based generative models—for the analysis, sampling, optimization, and representation of high-dimensional data presumed to be concentrated near, or supported on, low-dimensional manifolds embedded in ambient spaces. This field bridges advances in deep generative modeling (notably diffusion models) with foundational geometric techniques, enabling the extraction, manipulation, and computation of manifold geometry directly from learned score functions, and extending these methodologies to arbitrary Riemannian and sub-Riemannian settings.

1. Theoretical Foundations: Manifold Hypothesis and Score-Geometry Interaction

Under the manifold hypothesis, a high-dimensional data distribution $\mu_{\mathrm{data}}$ is supported on or near a compact, embedded submanifold $\mathcal{M} \subset \mathbb{R}^d$ of intrinsic dimension $k \ll d$. In generative frameworks, the data distribution is accessed only via samples or densities, not through explicit manifold parameterizations. Gaussian smoothing operations $$p_\sigma(x) = (N(0,\sigma^2 I) * \mu_{\mathrm{data}})(x)$$ lead to densities concentrated in tubular neighborhoods of $\mathcal{M}$, enabling the definition of a Stein score $s_\sigma(x) := \nabla_x \log p_\sigma(x)$.

A critical insight from recent work is the separation of information scales: the score $s_\sigma(x)$ contains $\Theta(\sigma^{-2})$-scale information about the normal direction (dictating the geometry of $\mathcal{M}$), while carrying only $O(1)$-scale information about the tangential, on-manifold density (Li et al., 29 Sep 2025). This scale separation underpins the feasibility of manifold recovery, manifold-aware sampling, and optimization via learned score functions, even when density estimation remains statistically or computationally harder.

2. Metric Construction: Score-Induced Riemannian Structures

Multiple constructions of Riemannian metrics from score functions have emerged:

• Ambient Stein-score metric: Given $\mathcal{M} \subset \mathbb{R}^d$0, define at $\mathcal{M} \subset \mathbb{R}^d$1,

$\mathcal{M} \subset \mathbb{R}^d$2

where $\mathcal{M} \subset \mathbb{R}^d$3 penalizes components in the (typically normal) score direction. This structure stretches ambient space in directions normal to the data manifold, preserving tangential distances (Azeglio et al., 16 May 2025).

• Pullback score metric: For a learned diffeomorphism $\mathcal{M} \subset \mathbb{R}^d$4 and convex $\mathcal{M} \subset \mathbb{R}^d$5 where $\mathcal{M} \subset \mathbb{R}^d$6,

$\mathcal{M} \subset \mathbb{R}^d$7

provides a Riemannian metric that admits closed-form geodesics in terms of $\mathcal{M} \subset \mathbb{R}^d$8 and $\mathcal{M} \subset \mathbb{R}^d$9, allowing analytic computation of exponential/log maps and distances (Diepeveen et al., 2024).

• Score Hessian/Jacobian-based metric: Using $k \ll d$0 (i.e., the Hessian of the log-density at noise level $k \ll d$1),

$k \ll d$2

and defining the Riemannian metric via the inner product $k \ll d$3 (Saito et al., 28 Apr 2025).

Each construction leverages the fact that, for well-trained generative models under the manifold hypothesis, $k \ll d$4 is (approximately) normal to the manifold, and its Jacobian encodes local curvature and tangential structure. These constructions are formulated intrinsically (on $k \ll d$5) or extrinsically (in ambient coordinates).

3. Algorithms: Geodesics, Optimization, and Sampling Schemes

Fundamental geometric operations—projection, retraction, geodesics, Riemannian gradients—can be approximated or computed solely from the score and its derivatives:

• Projection and tangent space approximation: The link function

$k \ll d$6

satisfies $k \ll d$7 ambient closest-point projection onto $k \ll d$8; the Hessian $k \ll d$9 approximates the tangent projector $$p_\sigma(x) = (N(0,\sigma^2 I) * \mu_{\mathrm{data}})(x)$$0 (Kharitenko et al., 27 Sep 2025).

• Geodesic computation: Minimizing the variational energy

$$p_\sigma(x) = (N(0,\sigma^2 I) * \mu_{\mathrm{data}})(x)$$1

yields score-geodesics, where movement off the learned manifold is penalized (Azeglio et al., 16 May 2025, Saito et al., 28 Apr 2025). Discrete optimization is achieved via gradient descent or Riemannian-Adam.

• Optimization via DLF and DRGD: Denoising Landing Flow (DLF) and Denoising Riemannian Gradient Descent (DRGD) perform Riemannian optimization over the implicit manifold using only the score and its Jacobian, yielding convergence and feasibility guarantees under mild regularity and error bounds (Kharitenko et al., 27 Sep 2025).
• Sampling/regression: Modifications to Langevin or predictor-corrector steps leveraging approximate scores, or tampered-score sampling, enable manifold-concentrated or uniform-on-manifold sampling (Li et al., 29 Sep 2025). Algorithms extend to estimation of Riemannian $$p_\sigma(x) = (N(0,\sigma^2 I) * \mu_{\mathrm{data}})(x)$$2-means/means or regression by exploiting analytic score-based surrogates for geodesic distances (Rygaard et al., 18 Feb 2025).

4. Applications: Generative Modeling, Data Interpolation, and Representation

• Generative sampling: Riemannian diffusion models generalize score-based generative modeling to arbitrary manifolds, with theory and implementation covering SDE reversals, likelihood estimation with Riemannian divergences, and sampling via geodesic random walks (Bortoli et al., 2022, Huang et al., 2022, Lou et al., 2023). Score-based Schrödinger bridges extend to interpolation between arbitrary marginal distributions on compact manifolds (Thornton et al., 2022).
• Manifold-aware image and data interpolation: Score-based metrics enable geodesic interpolation and extrapolation directly on data manifolds learned by diffusion models, yielding smoother, more structure-preserving transitions than Euclidean or spherical methods (Azeglio et al., 16 May 2025, Saito et al., 28 Apr 2025). This is effective in both latent and pixel/ambient spaces, as demonstrated on MNIST, natural images, and high-dimensional synthetic spheres.
• Dimension recovery and representation: Through score-based pullback geometry, Riemannian autoencoders can recover intrinsic manifold dimension and provide near-isometric charts for downstream tasks. Error bounds guarantee dimension recovery and near-isometric reconstruction (Diepeveen et al., 2024).
• Sub-Riemannian extensions: Score-based methods have been generalized to hypoelliptic, non-integrable geometries (e.g., the Heisenberg group) where the horizontal gradient replaces the full gradient and denoising losses must accommodate non-holonomic frames and the lack of closed-form heat kernels (Grong et al., 2024).

5. Theoretical Guarantees and Rate-Limiting Results

• Manifold recovery rates: Uniform approximation of projection and tangent projector $$p_\sigma(x) = (N(0,\sigma^2 I) * \mu_{\mathrm{data}})(x)$$3 and $$p_\sigma(x) = (N(0,\sigma^2 I) * \mu_{\mathrm{data}})(x)$$4 hold in a tubular neighborhood of $$p_\sigma(x) = (N(0,\sigma^2 I) * \mu_{\mathrm{data}})(x)$$5 (Kharitenko et al., 27 Sep 2025).
• Rate separations: Manifold support is learned at $$p_\sigma(x) = (N(0,\sigma^2 I) * \mu_{\mathrm{data}})(x)$$6 score error, while recovering on-manifold density or functional statistics requires $$p_\sigma(x) = (N(0,\sigma^2 I) * \mu_{\mathrm{data}})(x)$$7 score error—a scale separation central to manifold-geometric algorithm design (Li et al., 29 Sep 2025).
• Convergence of optimization and sampling: DLF and DRGD converge to points with $$p_\sigma(x) = (N(0,\sigma^2 I) * \mu_{\mathrm{data}})(x)$$8 manifold distance and Riemannian gradient norm. Langevin/tampered-score samplers with $$p_\sigma(x) = (N(0,\sigma^2 I) * \mu_{\mathrm{data}})(x)$$9 score errors yield measures concentrating uniformly on the manifold (Kharitenko et al., 27 Sep 2025, Li et al., 29 Sep 2025).
• Scalability: High-dimensional Riemannian symmetric spaces (e.g., $\mathcal{M}$0, $\mathcal{M}$1, high-dimensional spheres) admit efficient computational ansätze for heat kernel and score evaluation, with proven generative and likelihood performance (Lou et al., 2023).

6. Empirical and Algorithmic Validation

Experimental studies demonstrate:

• Substantial improvements in interpolation fidelity (LPIPS, FID, KID, SSIM) for score-geodesic paths over baselines—especially for image morphing and semantic transitions in synthetic and natural image data (Azeglio et al., 16 May 2025, Saito et al., 28 Apr 2025).
• Efficient computation of Fréchet and diffusion means, $\mathcal{M}$2-means clustering, and maximum-likelihood regression using score-matched estimates, avoiding the cubic scaling of traditional geodesic-based algorithms (Rygaard et al., 18 Feb 2025).
• Superior density estimation and sample quality for manifold-constrained data, outperforming traditional flows and variational methods on spheres, tori, and Lie groups (Huang et al., 2022, Lou et al., 2023).
• Robustness of manifold-concentrated sampling to score approximation errors, and practical applicability for large-scale models (e.g., Stable Diffusion 1.5) (Li et al., 29 Sep 2025).

7. Extensions and Open Directions

• Generalization to sub-Riemannian/hypoelliptic geometry: Horizontal score learning, divergence, and denoising losses adapted to non-holonomic distributions, with practical success on Heisenberg group bridges (Grong et al., 2024).
• Riemannian Schrödinger bridges and path-space interpolation: Variational and iterative proportional fitting techniques for interpolating distributions on $\mathcal{M}$3 have been developed and shown convergent both theoretically and in practical generative settings (Thornton et al., 2022).
• Automated dimension and chart discovery: Score-based Riemannian autoencoders guarantee dimension recovery, nearly isometric reconstruction, and closed-form chart computation, validated via synthetic and real datasets (Diepeveen et al., 2024).
• Computational efficiency and scale: Efficient divergence estimation (QR/Hutchinson trace), adaptive heat-kernel computation, and manifold-optimized neural architectures enable these methods to scale to large data and model sizes (Huang et al., 2022, Lou et al., 2023).
• Future research: Open challenges include boundary and singular manifold cases, further scaling in sub-Riemannian and non-symmetric settings, and deeper integration of geometry-aware statistics in deep generative models.

---

Score-based Riemannian geometry stands at the intersection of generative modeling, stochastic analysis, and computational geometry, enabling the recovery and exploitation of manifold structure directly from learned or sampled score fields. The field’s rapid development is driven both by methodological advances—precise geometric surrogates, scalable algorithms, provable guarantees—and by the demands of practical, high-dimensional generative and control tasks in modern machine learning.