Riemannian Hamiltonian Steepest Descent

• Riemannian Hamiltonian steepest descent is an optimization framework that leverages manifold geometry and Hamiltonian structures to guide descent directions on curved spaces.
• It employs exponential maps, retractions, and projection techniques across diverse model spaces such as spheres, positive-definite matrices, and hyperbolic spaces for feasible updates.
• The method integrates Hamiltonian proxy objectives and dynamical formulations to achieve linear convergence guarantees and robust performance in both deterministic and stochastic settings.

Riemannian Hamiltonian steepest descent concerns optimization on curved parameter spaces where the descent direction is defined by a Riemannian metric and the update is constrained to remain on the manifold. Its basic ingredients are a manifold $\mathcal M$, tangent spaces $T_x\mathcal M$, a metric $g$, and a rule for moving from a tangent vector back to $\mathcal M$ through an exponential map, a retraction, or a projection. In the recent literature, the topic encompasses at least three closely related constructions: classical Riemannian steepest descent $x_{k+1}=\exp_{x_k}(-\alpha_k\nabla_g f(x_k))$, Hamiltonian proxy methods that minimize $H=\frac12\|\operatorname{grad}f\|^2$, and metric-dependent dynamical or quantum Hamiltonian systems in which the force term is the natural gradient $g^{-1}\nabla V$ (A. et al., 27 Feb 2025, Han et al., 2022, Abe et al., 30 Mar 2026).

1. Geometric framework

The fundamental Riemannian identity is

$$g_x(\nabla_g f(x),v)=df(x)[v], \qquad \forall v\in T_x\mathcal M,$$

which defines the Riemannian gradient as the unique tangent vector representing the differential through the metric. The corresponding steepest-descent update uses the exponential map,

$x_{k+1}=\exp_{x_k}(-\alpha_k\nabla_g f(x_k)),$

so that the iterate moves along the geodesic determined by the negative gradient. When exact geodesics are inconvenient, the same role is played by a retraction or a local projection back to the manifold (A. et al., 27 Feb 2025).

This geometry appears in several concrete model spaces. On the unit sphere $\mathbb S^2\subset\mathbb R^3$, the Riemannian gradient is obtained by projecting the ambient Euclidean gradient onto the tangent space, and a normalized update

$T_x\mathcal M$0

is used as a small-step approximation to the exponential map. On the manifold $T_x\mathcal M$1 of positive definite Hermitian matrices, the metric is

$T_x\mathcal M$2

and the exponential map is

$T_x\mathcal M$3

In the hyperboloid model of hyperbolic space $T_x\mathcal M$4, the tangent space is defined by Minkowski orthogonality and the exact exponential map has the closed form

$T_x\mathcal M$5

In infinite-dimensional Hartree–Fock theory, orthonormal orbitals are treated as points on an $T_x\mathcal M$6-embedded Stiefel manifold $T_x\mathcal M$7 with Grassmann quotient $T_x\mathcal M$8, so that orthonormality constraints become geometric rather than purely algebraic (Duan et al., 2019, Wilson et al., 2018, Dinvay, 16 Mar 2026).

2. Classical Riemannian steepest descent and its variants

For smooth objectives on manifolds, one convergence template is based on the generalized smoothness condition

$T_x\mathcal M$9

where $g$0 is geodesic distance. A geodesic Taylor expansion then yields

$g$1

With the constant step size $g$2, this becomes

$g$3

and the stated objective-gap estimate is

$g$4

The same paper gives an illustrative example on $g$5 for minimizing $g$6, where the iterates descend along great-circle geometry toward the south pole (A. et al., 27 Feb 2025).

The term “steepest descent” is not restricted to the metric-induced negative gradient. In manifold constrained steepest descent, the direction is defined by the norm-constrained linear problem

$g$7

followed by the projected update

$g$8

On the Stiefel manifold with spectral norm, this yields SPEL,

$g$9

and the paper emphasizes that this direction need not lie in the tangent space before projection. Under its smoothness and projection-regularity assumptions, MCSD attains an $\mathcal M$0 deterministic stationarity rate and an $\mathcal M$1 stochastic rate for its momentum variant (Yang et al., 29 Jan 2026).

For nonconvex, non-Lipschitz objectives on complete submanifolds, the Riemannian smoothing steepest descent method replaces $\mathcal M$2 by a smoothing $\mathcal M$3, uses

$\mathcal M$4

and performs Armijo backtracking along a retracted curve while reducing the smoothing and stationarity thresholds when the gradient norm is small. The stated results are subsequential: accumulation points are stationary for the smoothing limit, and under Riemannian gradient sub-consistency they are Riemannian limiting stationary points of the original problem (Morimoto et al., 2021).

3. Hamiltonian as a stationarity-residual objective

A distinct use of “Hamiltonian” appears in min-max optimization on product manifolds $\mathcal M$5. The Riemannian Hamiltonian is defined by

$\mathcal M$6

with $\mathcal M$7. Minimizing $\mathcal M$8 is then a proxy for solving

$\mathcal M$9

A key identity is

$x_{k+1}=\exp_{x_k}(-\alpha_k\nabla_g f(x_k))$0

so steepest descent on $x_{k+1}=\exp_{x_k}(-\alpha_k\nabla_g f(x_k))$1 becomes

$x_{k+1}=\exp_{x_k}(-\alpha_k\nabla_g f(x_k))$2

Under the Riemannian Polyak–Łojasiewicz condition for $x_{k+1}=\exp_{x_k}(-\alpha_k\nabla_g f(x_k))$3 and the fixed step $x_{k+1}=\exp_{x_k}(-\alpha_k\nabla_g f(x_k))$4, the paper proves

$x_{k+1}=\exp_{x_k}(-\alpha_k\nabla_g f(x_k))$5

equivalently linear decay of $x_{k+1}=\exp_{x_k}(-\alpha_k\nabla_g f(x_k))$6 (Han et al., 2022).

This Hamiltonian construction is motivated by the failure of direct descent-ascent in curved min-max problems. For geodesic-bilinear objectives, the paper states that the min-max gradient

$x_{k+1}=\exp_{x_k}(-\alpha_k\nabla_g f(x_k))$7

is orthogonal to $x_{k+1}=\exp_{x_k}(-\alpha_k\nabla_g f(x_k))$8, explaining why Riemannian gradient descent-ascent can orbit or cycle while Hamiltonian descent points toward the correct saddle set. The framework also includes consensus regularization

$x_{k+1}=\exp_{x_k}(-\alpha_k\nabla_g f(x_k))$9

and stochastic variants based on a symmetric two-sample estimator of $H=\frac12\|\operatorname{grad}f\|^2$0 (Han et al., 2022).

4. Dynamical and quantum Hamiltonian formulations

A second Hamiltonian interpretation is genuinely dynamical. Quantum Riemannian Hamiltonian Descent introduces a position-dependent metric $H=\frac12\|\operatorname{grad}f\|^2$1 into the kinetic term and starts from the Lagrangian

$H=\frac12\|\operatorname{grad}f\|^2$2

The corresponding classical Hamiltonian is

$H=\frac12\|\operatorname{grad}f\|^2$3

and the quantum Hamiltonian replaces the flat Laplacian by the Laplace–Beltrami operator. The classical equation of motion is

$H=\frac12\|\operatorname{grad}f\|^2$4

so the system is a damped geodesic flow driven by the natural gradient. Operator-ordering and measure effects generate curvature-dependent corrections in the effective potential, but the paper argues that these appear with factors $H=\frac12\|\operatorname{grad}f\|^2$5 and $H=\frac12\|\operatorname{grad}f\|^2$6, hence are suppressed at late times when $H=\frac12\|\operatorname{grad}f\|^2$7 grows, for example $H=\frac12\|\operatorname{grad}f\|^2$8. Near a local minimizer, the local convergence rate is controlled by the smallest eigenvalue of the Riemannian preconditioned Hessian $H=\frac12\|\operatorname{grad}f\|^2$9 (Abe et al., 30 Mar 2026).

Double-bracket quantum imaginary-time evolution reaches the Riemannian steepest-descent direction through a different route. The relevant manifold is the adjoint-unitary orbit

$g^{-1}\nabla V$0

and the Riemannian gradient of the Hilbert–Schmidt loss on this orbit is

$g^{-1}\nabla V$1

The resulting flow

$g^{-1}\nabla V$2

is Brockett’s double-bracket flow. For imaginary-time evolution, the paper identifies the energy descent law

$g^{-1}\nabla V$3

where $g^{-1}\nabla V$4 is the energy variance, so the speed of descent is the squared norm of the Riemannian gradient. Discrete quantum steps are implemented through group-commutator product formulas, and the numerics show bottlenecks near eigenstates because the variance, hence the gradient, becomes very small there. The paper is explicit that this is a Riemannian steepest-descent interpretation rather than a developed Hamiltonian or variational formulation (Zander et al., 1 Apr 2025).

5. Representative manifolds and applications

Several application areas instantiate the general scheme by pairing a problem-specific manifold with a problem-specific metric.

Domain	Objective or geometry	Characteristic conclusion
$g^{-1}\nabla V$5	$g^{-1}\nabla V$6; Karcher mean $g^{-1}\nabla V$7	Riemannian gradient follows geodesics; natural gradient is reported faster in simulations
Hyperbolic $g^{-1}\nabla V$8	Fréchet mean $g^{-1}\nabla V$9	Exact hyperboloid exponential updates outperform Poincaré-ball retractions
$$g_x(\nabla_g f(x),v)=df(x)[v], \qquad \forall v\in T_x\mathcal M,$$0 Stiefel/Grassmann	Hartree–Fock energy $$g_x(\nabla_g f(x),v)=df(x)[v], \qquad \forall v\in T_x\mathcal M,$$1 under orthonormality	Riemannian steepest descent with Armijo and kinetic-energy preconditioning is robust from random initial guesses
Stiefel in machine learning	PCA, orthogonality-constrained CNNs, manifold-constrained LLM adapter tuning	MCSD and SPEL give single-loop manifold-constrained steepest descent

On $$g_x(\nabla_g f(x),v)=df(x)[v], \qquad \forall v\in T_x\mathcal M,$$2, the affine-invariant metric makes the squared geodesic distance

$$g_x(\nabla_g f(x),v)=df(x)[v], \qquad \forall v\in T_x\mathcal M,$$3

the central cost. For the control problem, the Riemannian gradient step is written directly on the matrix manifold,

$$g_x(\nabla_g f(x),v)=df(x)[v], \qquad \forall v\in T_x\mathcal M,$$4

while the natural gradient acts on the control coordinates as

$$g_x(\nabla_g f(x),v)=df(x)[v], \qquad \forall v\in T_x\mathcal M,$$5

For Karcher means of Toeplitz positive definite Hermitian matrices, both the Riemannian gradient and natural gradient converge to the same barycenter, with the latter reported faster in the simulations (Duan et al., 2019).

In hyperbolic space, the hyperboloid model makes exact Riemannian updates computationally simple because both the tangent projection and the exponential map are closed form. For Fréchet mean computation, the paper reports that exact exponential updates converge faster than Poincaré-ball retraction updates (Wilson et al., 2018).

In Hartree–Fock theory, the energy minimization problem is posed directly in $$g_x(\nabla_g f(x),v)=df(x)[v], \qquad \forall v\in T_x\mathcal M,$$6, orthonormality is encoded by the Stiefel manifold

$$g_x(\nabla_g f(x),v)=df(x)[v], \qquad \forall v\in T_x\mathcal M,$$7

and the basic update is

$$g_x(\nabla_g f(x),v)=df(x)[v], \qquad \forall v\in T_x\mathcal M,$$8

The paper also introduces the preconditioner

$$g_x(\nabla_g f(x),v)=df(x)[v], \qquad \forall v\in T_x\mathcal M,$$9

motivated by inversion of the kinetic-energy operator. Here the target is the Hamiltonian expectation value in the variational quantum-chemical sense; the paper explicitly states that the resulting dynamics are gradient-flow-like rather than Hamiltonian time evolution (Dinvay, 16 Mar 2026).

6. Terminology, distinctions, and recurrent misconceptions

The literature uses “Hamiltonian” in materially different senses. In Riemannian min-max optimization, the Hamiltonian is the residual objective $x_{k+1}=\exp_{x_k}(-\alpha_k\nabla_g f(x_k)),$0. In QRHD, it is a time-dependent classical or quantum Hamiltonian with metric-dependent kinetic energy. In Hartree–Fock, the minimized quantity is the Hamiltonian expectation value on a manifold of orthonormal orbitals, but the paper states that the descent dynamics are gradient-flow-like, not Hamiltonian time evolution. By contrast, several steepest-descent papers are explicit that they do not introduce a Hamiltonian structure at all: the generalized steepest-descent paper on manifolds does not develop a Hamiltonian or variational formulation for its manifold method, the positive-definite-matrix paper does not formulate a Hamiltonian function, symplectic flow, or energy-conserving dynamics, and DB-QITE interprets imaginary-time evolution as Riemannian steepest descent rather than a Hamiltonian descent method in the strict sense (A. et al., 27 Feb 2025, Duan et al., 2019, Dinvay, 16 Mar 2026, Zander et al., 1 Apr 2025).

A second misconception is that steepest descent on a manifold must always mean the negative tangent gradient followed by an exponential map. The surveyed methods show several alternatives: tangent projection of an ambient gradient, norm-induced oracle directions followed by manifold projection, smoothing-based descent for non-Lipschitz objectives, and damped second-order dynamics whose forcing term is the natural gradient. Exact geodesic motion is sometimes simpler than approximate retractions, as in the hyperboloid model, while in other settings efficient first-order retractions such as Löwdin orthonormalization or polar projection are preferred. Vanishing gradient also need not indicate a minimizer: DB-QITE exhibits stationary saddle behavior near eigenstates, and the min-max Hamiltonian framework therefore assumes that stationary points of $x_{k+1}=\exp_{x_k}(-\alpha_k\nabla_g f(x_k)),$1 correspond to global saddle points when $x_{k+1}=\exp_{x_k}(-\alpha_k\nabla_g f(x_k)),$2 is minimized to zero (Yang et al., 29 Jan 2026, Wilson et al., 2018, Han et al., 2022).

This suggests that Riemannian Hamiltonian steepest descent is best understood not as a single algorithmic formula but as a geometric design principle. The common thread is that descent is defined relative to a manifold metric, feasibility is enforced by geodesic or retraction-based motion, and Hamiltonian structure—when present—serves either as a residual objective, as a dynamical generator, or as a variational energy whose constrained minimization is carried out by Riemannian steepest descent.