DRGD: Denoising Riemannian Gradient Descent

• Denoising Riemannian Gradient Descent (DRGD) is a score-based optimization approach that approximates classical manifold operations using a pretrained denoising score network.
• It reconstructs tangent projection and retraction via Tweedie’s formula and score matching, bridging explicit geometric operators with data-driven manifold approximations.
• DRGD integrates diffusion-based scores into Riemannian gradient descent, achieving convergence guarantees and practical performance in complex, data-driven design problems.

Denoising Riemannian Gradient Descent (DRGD) is a score-based method for Riemannian optimization on a manifold that is not given by explicit equations, charts, or classical manifold oracles, but only implicitly through a data distribution concentrated near that manifold. In "Landing with the Score: Riemannian Optimization through Denoising" (Kharitenko et al., 27 Sep 2025), DRGD is formulated under the data manifold hypothesis and uses a denoising score network to approximate the two operations that classical Riemannian gradient descent requires: tangent-space projection and retraction. The method is positioned for data-driven design problems central to modern generative AI, where the feasible set is represented by samples rather than by an analytically specified manifold.

1. Geometric setting and optimization problem

The underlying geometric assumption is an unknown, compact, embedded, boundary-free $C^2$ submanifold $\mathcal M \subset \mathbb R^d$ of intrinsic dimension $k \ll d$. For any point $x$ in a tubular neighborhood $\mathcal T(\tau)$ of $\mathcal M$, there is a unique closest-point projection $\pi(x) \in \mathcal M$, and the squared-distance function is defined by

$d(x)=\tfrac12\|x-\pi(x)\|^2.$

At a point $p \in \mathcal M$, the tangent-space projector is $P_{T_p\mathcal M}$, and the normal projector is $\mathcal M \subset \mathbb R^d$0 (Kharitenko et al., 27 Sep 2025).

The optimization problem is Riemannian optimization over $\mathcal M \subset \mathbb R^d$1 when $\mathcal M \subset \mathbb R^d$2 is only implicitly specified by the data distribution. This differs from classical settings in which retractions, tangent projectors, exponential maps, or local coordinates are available analytically. The paper frames this as a broad class of data-driven design problems. A plausible implication is that the feasible geometry is to be inferred from the statistics of the data rather than from explicit geometric modeling.

2. Score-based recovery of manifold operations

Let $\mathcal M \subset \mathbb R^d$3 denote the ground-truth distribution supported on $\mathcal M \subset \mathbb R^d$4. For $\mathcal M \subset \mathbb R^d$5, the Gaussian-smoothed density is

$\mathcal M \subset \mathbb R^d$6

with Stein score

$\mathcal M \subset \mathbb R^d$7

The central construction is the link function

$\mathcal M \subset \mathbb R^d$8

Using Tweedie’s formula, the paper connects this link function to geometric operations on the manifold, and as $\mathcal M \subset \mathbb R^d$9, under mild regularity,

$k \ll d$0

Thus, a denoising score can approximate closest-point projection, while derivatives of the score can approximate tangent-space projection (Kharitenko et al., 27 Sep 2025).

In practice, the score is not assumed known analytically. Instead, a neural network $k \ll d$1 is trained by denoising score matching. This places DRGD directly in the score-based diffusion-model ecosystem. The significance of this link is that the optimization method can use well-studied parameterizations, efficient training procedures, and even pretrained score networks from the diffusion-model literature. This suggests that a pretrained score network can serve as a geometric surrogate for manifold operations that are otherwise unavailable.

3. DRGD as a surrogate for classical Riemannian gradient descent

Classical Riemannian gradient descent on a known manifold takes the form

$k \ll d$2

where $k \ll d$3 is a retraction and $k \ll d$4 is a step size. DRGD replaces both unavailable operations by score-based approximations.

The paper defines a “Tweedie-retraction”

$k \ll d$5

and a “tangent-projector”

$k \ll d$6

The DRGD update is then

$k \ll d$7

Here, $k \ll d$8 plays the role of the Riemannian gradient, and the post-mapping by $k \ll d$9 approximately retracts the iterate back onto $x$0 (Kharitenko et al., 27 Sep 2025).

This formulation is structurally close to classical Riemannian gradient descent, but its geometry is entirely recovered from the score network. It should therefore be understood not as Euclidean gradient descent with an ad hoc denoiser, but as an approximation to a manifold method in which both tangent and retraction information are inferred from the smoothed data distribution.

4. Iteration mechanics, hyperparameters, and computational profile

The algorithm takes as input a pretrained score network $x$1 for fixed $x$2, an objective $x$3 with Lipschitz gradient $x$4, an initialization $x$5 such as a nearest neighbor from the data, and a step-size schedule $x$6. The output is an approximate solution $x$7.

At each iteration, the Euclidean gradient $x$8 is computed, and the Jacobian-vector product $x$9 is obtained by backpropagation through

$\mathcal T(\tau)$0

A tangent step

$\mathcal T(\tau)$1

is then followed by one denoising step

$\mathcal T(\tau)$2

The principal hyperparameters are the noise level $\mathcal T(\tau)$3, the step-size schedule $\mathcal T(\tau)$4, where $\mathcal T(\tau)$5 is the smoothness of $\mathcal T(\tau)$6, and the number of iterations $\mathcal T(\tau)$7 (Kharitenko et al., 27 Sep 2025).

The practical cost profile is explicit. Each iteration requires one forward pass and one backward pass, with cost comparable to two Euclidean gradient evaluations, and memory dominated by storing network activations for the backward pass. No fine-tuning of $\mathcal T(\tau)$8 is needed: DRGD is an inference-time procedure using only forward and input-gradient queries to the score network.

5. Feasibility and stationarity guarantees

The theoretical analysis assumes uniform sup-norm approximation bounds on the tubular neighborhood $\mathcal T(\tau)$9: $\mathcal M$0 Under these assumptions, the paper gives guarantees for both feasibility and optimality (Kharitenko et al., 27 Sep 2025).

For the continuous-time denoising landing flow,

$\mathcal M$1

every accumulation point $\mathcal M$2 satisfies

$\mathcal M$3

and

$\mathcal M$4

For discrete DRGD, if $\mathcal M$5, then the iterates stay in $\mathcal M$6, and

$\mathcal M$7

Hence after $\mathcal M$8 steps, the method attains an $\mathcal M$9-approximate stationary point. The proof sketch given in the paper relies on approximate smoothness of $\pi(x) \in \mathcal M$0, error bounds on $\pi(x) \in \mathcal M$1 and $\pi(x) \in \mathcal M$2, tubular-neighborhood barrier arguments for feasibility, and the identity

$\pi(x) \in \mathcal M$3

6. Empirical behavior, strengths, and limitations

The reported experiments cover two settings. On the Brockett cost over the $\pi(x) \in \mathcal M$4 manifold, the score is trained on $\pi(x) \in \mathcal M$5 uniform samples of $\pi(x) \in \mathcal M$6, using MLP score architectures of width $\pi(x) \in \mathcal M$7 for $\pi(x) \in \mathcal M$8 and $\pi(x) \in \mathcal M$9 for $d(x)=\tfrac12\|x-\pi(x)\|^2.$0. In that setting, the DRGD objective is reported as $d(x)=\tfrac12\|x-\pi(x)\|^2.$1 the best training cost and improves as $d(x)=\tfrac12\|x-\pi(x)\|^2.$2. On finite-horizon reference tracking tasks in data-driven control, the experiments use double pendulum and unicycle models with horizon $d(x)=\tfrac12\|x-\pi(x)\|^2.$3, and a 1D UNet style architecture on input/output trajectories. There, DRGD finds trajectories that generalize beyond training examples, yield significantly lower tracking error, and remain close to the true behavior manifold, reflected in a small simulation-reconstruction gap (Kharitenko et al., 27 Sep 2025).

The paper identifies several strengths. DRGD requires only inference of pretrained diffusion scores and no new training loop; it avoids explicit manifold equations or charts; and it is a feasible method in the sense that iterates remain near $d(x)=\tfrac12\|x-\pi(x)\|^2.$4 while yielding approximate Riemannian criticality. These features explain its relevance to data-driven generative and design applications in which the feasible set is represented statistically rather than analytically.

The limitations are equally explicit. Uniform sup-norm approximation of the score and its Jacobian may be stringent in high dimensions. The convergence error scales as

$d(x)=\tfrac12\|x-\pi(x)\|^2.$5

so choosing small $d(x)=\tfrac12\|x-\pi(x)\|^2.$6 entails a trade-off against score accuracy. Step sizes must also respect the tubular-neighborhood radius $d(x)=\tfrac12\|x-\pi(x)\|^2.$7. The practical guidelines follow directly from these constraints: choose $d(x)=\tfrac12\|x-\pi(x)\|^2.$8 as small as the score-network fidelity allows, warm-start $d(x)=\tfrac12\|x-\pi(x)\|^2.$9 from a nearby data sample, use a decreasing step-size schedule or line-search within the safe range $p \in \mathcal M$0, and monitor $p \in \mathcal M$1 via one denoising step to ensure the iterates remain in $p \in \mathcal M$2.

Taken together, DRGD defines a data-driven approximation to Riemannian gradient descent in which retraction and tangent projection are recovered from denoising scores. Its distinctive contribution is not merely to regularize Euclidean optimization with a generative prior, but to reconstruct the core geometric operators of manifold optimization from the score of a Gaussian-blurred data distribution.