Riemannian Online Gradient Descent

• Riemannian Online Gradient Descent generalizes traditional online methods to curved manifold settings by replacing Euclidean updates with intrinsic geodesic moves.
• The method leverages the exponential map or retraction to ensure efficient updates with theoretical guarantees on convergence and regret under geodesic convexity.
• It extends to various applications including decentralized optimization, stochastic approximation, and structured models such as hyperbolic spaces, SPD matrices, and tensor manifolds.

Searching arXiv for relevant papers on Riemannian Online Gradient Descent and closely related variants. Riemannian Online Gradient Descent (ROGD) generalizes online and stochastic first-order methods to curved spaces by replacing Euclidean additive updates with intrinsic geodesic moves on a manifold. Given a Riemannian manifold $M$, a sequence of per-round losses $f_t:M\to\mathbb{R}$, and an iterate $x_t\in M$, the canonical update advances along the geodesic in the negative Riemannian gradient direction,

$$x_{t+1}=\operatorname{Exp}_{x_t}\!\big(-\eta_t\,\operatorname{grad} f_t(x_t)\big),$$

where $\eta_t>0$ is a learning rate and $\operatorname{Exp}_{x_t}$ is the exponential map at $x_t$ (Wilson et al., 2018). In the literature, this template appears in adversarial online optimization, stochastic approximation, decentralized optimization, optimistic algorithms, and structured manifold models such as hyperbolic spaces, symmetric positive definite manifolds, and fixed-rank tensor-train manifolds (Bonnabel, 2011, Sakai et al., 2023, Chen et al., 2024, Wang et al., 2023, Sahinoglu et al., 14 Sep 2025, Cai et al., 6 May 2026).

1. Geometric setting and first-order structure

ROGD is formulated on a smooth Riemannian manifold $(M,g)$, where each tangent space $T_xM$ carries an inner product induced by the metric $g$, and the geodesic distance is denoted $f_t:M\to\mathbb{R}$0. A central class is the Hadamard manifold: complete, simply connected, and with nonpositive sectional curvature. In this setting, unique geodesics connect any two points, the exponential map is a diffeomorphism, the logarithm map $f_t:M\to\mathbb{R}$1 is globally well-defined, and $f_t:M\to\mathbb{R}$2 is geodesically convex (Chen et al., 2024, Sakai et al., 2023).

The Riemannian gradient $f_t:M\to\mathbb{R}$3 is defined by the metric duality relation

$f_t:M\to\mathbb{R}$4

for all $f_t:M\to\mathbb{R}$5 (Bonnabel, 2011). For geodesically convex functions, the first-order inequality takes the intrinsic form

$f_t:M\to\mathbb{R}$6

which replaces the Euclidean affine lower bound (Chen et al., 2024, Wang et al., 2023). In constrained problems one typically assumes a closed, bounded, geodesically convex feasible set $f_t:M\to\mathbb{R}$7, with geodesic projection

$f_t:M\to\mathbb{R}$8

On Hadamard manifolds this projection is single-valued and nonexpansive (Chen et al., 2024, Sahinoglu et al., 14 Sep 2025).

Curvature enters the analysis through comparison inequalities. On Hadamard manifolds with sectional curvatures lower bounded by $f_t:M\to\mathbb{R}$9, a recurring factor is

$x_t\in M$0

which quantifies curvature-dependent distortion in triangle inequalities and regret constants (Chen et al., 2024). In analyses that allow positive curvature, one instead restricts the feasible set to a strongly convex ball, typically requiring $x_t\in M$1 when the curvature upper bound satisfies $x_t\in M$2 (Sahinoglu et al., 9 Sep 2025).

2. Canonical updates and algorithmic forms

The basic intrinsic update uses the exponential map. In stochastic approximation this is written as

$x_t\in M$3

where $x_t\in M$4 is an unbiased stochastic tangent gradient estimate and the step sizes satisfy the Robbins–Monro conditions $x_t\in M$5 and $x_t\in M$6 (Bonnabel, 2011). When exact geodesics are expensive, a retraction $x_t\in M$7 can replace $x_t\in M$8, yielding

$x_t\in M$9

with first-order local agreement $$x_{t+1}=\operatorname{Exp}_{x_t}\!\big(-\eta_t\,\operatorname{grad} f_t(x_t)\big),$$0 (Bonnabel, 2011). Retractions are standard on fixed-rank matrix and tensor manifolds, where truncated SVD, TT-rounding, and TT-SVD are natural choices (Cai et al., 6 May 2026).

Mini-batch Riemannian stochastic gradient descent (RSGD) replaces the stochastic oracle by a batch average. If

$$x_{t+1}=\operatorname{Exp}_{x_t}\!\big(-\eta_t\,\operatorname{grad} f_t(x_t)\big),$$1

then the estimator remains unbiased and its conditional variance scales as $$x_{t+1}=\operatorname{Exp}_{x_t}\!\big(-\eta_t\,\operatorname{grad} f_t(x_t)\big),$$2, so mini-batching reduces variance by a factor $$x_{t+1}=\operatorname{Exp}_{x_t}\!\big(-\eta_t\,\operatorname{grad} f_t(x_t)\big),$$3 (Sakai et al., 2023).

Projected online updates combine a local gradient step with projection onto $$x_{t+1}=\operatorname{Exp}_{x_t}\!\big(-\eta_t\,\operatorname{grad} f_t(x_t)\big),$$4. In decentralized projected Riemannian gradient descent, each agent forms the half-step

$$x_{t+1}=\operatorname{Exp}_{x_t}\!\big(-\eta_t\,\operatorname{grad} f_t(x_t)\big),$$5

before a consensus phase (Chen et al., 2024). In optimistic online variants, the gradient is augmented by a parallel-transported prediction term. The Riemannian Online Optimistic Gradient Descent update is

$$x_{t+1}=\operatorname{Exp}_{x_t}\!\big(-\eta_t\,\operatorname{grad} f_t(x_t)\big),$$6

where $$x_{t+1}=\operatorname{Exp}_{x_t}\!\big(-\eta_t\,\operatorname{grad} f_t(x_t)\big),$$7 and $$x_{t+1}=\operatorname{Exp}_{x_t}\!\big(-\eta_t\,\operatorname{grad} f_t(x_t)\big),$$8 denotes parallel transport along the geodesic from $$x_{t+1}=\operatorname{Exp}_{x_t}\!\big(-\eta_t\,\operatorname{grad} f_t(x_t)\big),$$9 to $\eta_t>0$0 (Wang et al., 2023).

A specialized but structurally important variant appears on the fixed-TT-rank manifold $\eta_t>0$1 for matrix product operator quantum state tomography. There the online Riemannian gradient descent iteration consists of tangent-space projection of the Euclidean gradient, a tangent step, an elementwise trimming step for incoherence control, and a TT retraction by TTSVD: $\eta_t>0$2 This formulation exploits the manifold geometry of real low-TT-rank coefficient tensors derived from Hermitian matrix product operators (Cai et al., 6 May 2026).

3. Regret, convergence, and step-size regimes

Two analytical traditions coexist. In stochastic optimization, the goal is convergence to critical points or small gradient norms. In adversarial online optimization, the goal is sublinear regret relative to a static or time-varying comparator sequence. The distinction is explicit: some Riemannian SGD analyses provide convergence bounds but do not present adversarial regret guarantees (Sakai et al., 2023).

Under geodesic $\eta_t>0$3-smoothness and unbiased mini-batch gradients, RSGD on a Hadamard manifold satisfies

$\eta_t>0$4

For a constant step size $\eta_t>0$5, this yields an average squared-gradient bound of order $\eta_t>0$6; with diminishing $\eta_t>0$7, one obtains $\eta_t>0$8 (Sakai et al., 2023). Earlier stochastic approximation results established almost sure convergence of intrinsic and retraction-based updates to critical points under compactness or Hadamard-type stability assumptions (Bonnabel, 2011).

In online convex optimization, the central quantity is regret. For a static comparator $\eta_t>0$9, the standard regret is

$\operatorname{Exp}_{x_t}$0

while dynamic regret compares against a sequence $\operatorname{Exp}_{x_t}$1 and incorporates the path variation

$\operatorname{Exp}_{x_t}$2

For decentralized online Riemannian optimization on Hadamard manifolds, the dynamic regret bound established for both exact and simplified consensus is

$\operatorname{Exp}_{x_t}$3

where $\operatorname{Exp}_{x_t}$4 is the second-largest singular value of the mixing matrix, so the spectral gap $\operatorname{Exp}_{x_t}$5 governs the network penalty (Chen et al., 2024). When $\operatorname{Exp}_{x_t}$6 or $\operatorname{Exp}_{x_t}$7, this reduces to centralized Riemannian OGD with regret order $\operatorname{Exp}_{x_t}$8 (Chen et al., 2024).

Optimistic Riemannian algorithms refine the comparator dependence by exploiting temporal regularity in the losses. For R-OOGD, if the losses are $\operatorname{Exp}_{x_t}$9-convex, $x_t$0-$x_t$1-Lipschitz, and $x_t$2-$x_t$3-smooth, then

$x_t$4

where $x_t$5, $x_t$6, and $x_t$7 are curvature-dependent constants (Wang et al., 2023). The adaptive meta-expert variant R-AOOGD achieves

$x_t$8

matching the Euclidean dependence on regularity measures up to curvature constants (Wang et al., 2023).

A distinct line of work replaces geodesic convexity by horospherical convexity on Hadamard manifolds. For $x_t$9-convex losses, ROGD with fixed $(M,g)$0 satisfies

$(M,g)$1

which yields $(M,g)$2 regret under $(M,g)$3-Lipschitzness, with no curvature term in the bound (Sahinoglu et al., 14 Sep 2025). For $(M,g)$4-strongly $(M,g)$5-convex losses and $(M,g)$6, the regret becomes

$(M,g)$7

again curvature-independent (Sahinoglu et al., 14 Sep 2025). This establishes that curvature dependence is not universal; it can disappear when the objective class is strengthened from $(M,g)$8-convexity to $(M,g)$9-convexity (Sahinoglu et al., 14 Sep 2025).

4. Consensus, networks, and decentralized extensions

Decentralized ROGD introduces a second geometric operation beyond gradient descent: manifold consensus. On Hadamard manifolds, the exact consensus step is formulated through the weighted Fréchet mean,

$T_xM$0

which is unique because the objective is geodesically convex (Chen et al., 2024). The corresponding algorithm is described as the first decentralized online Riemannian optimization algorithm on Hadamard manifolds (Chen et al., 2024).

The consensus step admits a linear variance reduction result: $T_xM$1 where $T_xM$2 denotes Fréchet variance (Chen et al., 2024). Since weighted Fréchet means require solving an inner minimization problem, a closed-form approximation based on log-exp averaging was proposed: $T_xM$3 With $T_xM$4, this also contracts variance linearly and attains the same dynamic regret order, although the contraction factor is weaker than for exact Fréchet averaging (Chen et al., 2024).

The network error can be bounded explicitly. For exact consensus,

$T_xM$5

while the closed-form consensus yields

$T_xM$6

where $T_xM$7 is the Fréchet mean of the network iterates (Chen et al., 2024). These formulas make explicit the role of curvature, Lipschitz continuity, network size, and spectral gap.

Beyond Hadamard manifolds, positive curvature obstructs global convexity of geodesic distance, projection nonexpansiveness, and consensus analysis. A curvature-aware consensus step on manifolds with bounded sectional curvature addresses this by restricting the feasible set to a convex ball and choosing a consensus step size $T_xM$8 from curvature-dependent constants $T_xM$9, $g$0, and distortion terms $g$1, $g$2 (Sahinoglu et al., 9 Sep 2025). The resulting one-step intrinsic averaging update is

$g$3

and it enjoys linear variance contraction

$g$4

together with $g$5 static regret in both full-information and two-point bandit settings (Sahinoglu et al., 9 Sep 2025). This shows that decentralization can be extended beyond nonpositive curvature, but only under explicit convexity-radius and distortion controls (Sahinoglu et al., 9 Sep 2025).

5. Model-specific realizations

In hyperbolic space, the hyperboloid model makes ROGD fully explicit. Hyperbolic $g$6-space is represented as

$g$7

with Lorentz inner product

$g$8

The tangent projection is $g$9, the distance is $f_t:M\to\mathbb{R}$00, and the exact exponential map is

$f_t:M\to\mathbb{R}$01

This yields exact geodesic gradient steps rather than retraction approximations (Wilson et al., 2018). For online Fréchet mean estimation with losses $f_t:M\to\mathbb{R}$02, the gradient satisfies $f_t:M\to\mathbb{R}$03, so the update is a geodesic step toward the incoming sample (Wilson et al., 2018).

On the manifold of symmetric positive definite matrices, several geometries and applications appear. Under the affine-invariant Riemannian metric,

$f_t:M\to\mathbb{R}$04

the manifold $f_t:M\to\mathbb{R}$05 is Hadamard and the exponential map is

$f_t:M\to\mathbb{R}$06

(Sakai et al., 2023). This geometry supports RSGD for the Riemannian centroid problem, nonlinear gossip updates that move neighboring covariance matrices toward each other along the Fisher geodesic, and online Tyler’s $f_t:M\to\mathbb{R}$07-estimation and online Fréchet mean computation under curvature-independent $f_t:M\to\mathbb{R}$08-convex analyses (Bonnabel, 2011, Sahinoglu et al., 14 Sep 2025). Empirically, increasing the batch size improves RSGD performance on the SPD centroid problem, and the number of steps required for convergence is a convex monotone decreasing function of batch size (Sakai et al., 2023).

On fixed-rank tensor-train manifolds, online Riemannian gradient descent has been specialized to quantum state tomography with matrix product operators. The key structural result is that Hermiticity of the MPO cores implies that the coefficient tensor under the Pauli or generalized Gell-Mann basis is real and low-TT-rank, making the reconstruction problem equivalent to noisy low-rank tensor completion on the manifold $f_t:M\to\mathbb{R}$09 of fixed TT rank (Cai et al., 6 May 2026). With proper initialization, the resulting oRGD converges linearly to the target MPO, the number of distinct measurement settings scales quadratically with the system size, and the per-iteration computational costs are $f_t:M\to\mathbb{R}$10 for contraction and tangent projection and $f_t:M\to\mathbb{R}$11 for TT retraction (Cai et al., 6 May 2026).

ROGD also appears in game-theoretic and regression settings. R-OGDA extends optimistic descent-ascent to zero-sum $f_t:M\to\mathbb{R}$12-convex-concave games on product manifolds and achieves average-iterate $f_t:M\to\mathbb{R}$13, best-iterate $f_t:M\to\mathbb{R}$14, and linear last-iterate convergence rates under strong $f_t:M\to\mathbb{R}$15-convexity-strong concavity (Wang et al., 2023). Online geodesic regression and robust geometry-aware PCA provide further examples in which intrinsic updates operate directly on spheres, Grassmannians, or SPD products (Wang et al., 2023).

6. Variants, misconceptions, and open problems

A common misconception is that “online” and “stochastic” are interchangeable in the Riemannian setting. They are not. Stochastic analyses typically assume i.i.d. sampling, unbiased gradient oracles, and convergence criteria based on expected gradient norms or variational measures $f_t:M\to\mathbb{R}$16; they do not automatically imply adversarial regret guarantees (Sakai et al., 2023, Bonnabel, 2011). By contrast, online convex optimization studies regret against static or dynamic comparators under adversarial sequences of losses (Chen et al., 2024, Wang et al., 2023, Sahinoglu et al., 14 Sep 2025).

A second misconception is that exact geodesic updates are invariably too costly to use. This is false in some important manifolds. In the hyperboloid model of hyperbolic space, the exponential and logarithm maps have closed forms and the tutorial evidence shows that exact hyperboloid exponential updates can outperform Poincaré-ball retraction updates in Fréchet mean problems (Wilson et al., 2018). At the same time, retractions remain essential in matrix and tensor manifolds, where truncated SVD, QR-based updates, or TT-rounding are computationally preferable (Bonnabel, 2011, Cai et al., 6 May 2026).

A third misconception is that curvature-dependent regret degradation is unavoidable. Geodesically convex analyses on manifolds with nonzero curvature often introduce curvature factors such as $f_t:M\to\mathbb{R}$17 or $f_t:M\to\mathbb{R}$18 through comparison geometry (Chen et al., 2024, Wang et al., 2023). However, under horospherical convexity on Hadamard manifolds, ROGD admits $f_t:M\to\mathbb{R}$19 and $f_t:M\to\mathbb{R}$20 regret bounds with no curvature term in the final rates (Sahinoglu et al., 14 Sep 2025). This does not eliminate curvature from the geometry; rather, it replaces the function class by the stronger notion of $f_t:M\to\mathbb{R}$21-convexity (Sahinoglu et al., 14 Sep 2025).

The present theory remains assumption-heavy. Hadamard geometry is often imposed to guarantee uniqueness of projections and Fréchet means, geodesic convexity and Lipschitz continuity are standard prerequisites for regret analysis, and positive-curvature settings require bounded convex balls with radius below the convexity threshold (Chen et al., 2024, Sahinoglu et al., 9 Sep 2025). In MPO-based tomography, Hermiticity and trace-one handling are addressed, but positivity is not enforced; incorporating PSD constraints would change the geometry and remains future work (Cai et al., 6 May 2026). In decentralized dynamic regret, tuned step sizes depend on the path variation $f_t:M\to\mathbb{R}$22, and designing adaptive policies without oracle knowledge remains open (Chen et al., 2024).

The stated extensions are correspondingly geometric and algorithmic: directed or time-varying graphs, coupled constraints, bandit feedback, acceleration, strong convexity and smoothness, adaptive consensus weights, second-order methods on Hadamard manifolds, tighter curvature dependence beyond Hadamard manifolds, and retraction-transport generalizations with explicit constants (Chen et al., 2024, Sahinoglu et al., 9 Sep 2025). This suggests that Riemannian Online Gradient Descent is best understood not as a single algorithm but as a geometric design pattern: update in a tangent space using a metric-compatible gradient, return to the manifold by exponential map or retraction, and control curvature either through comparison inequalities, Busemann-function envelopes, or model-specific structure.