Decentralized Riemannian Gradient Descent

• Decentralized online Riemannian gradient descent is a class of algorithms where networked agents update their local iterates on a manifold using Riemannian gradients and neighbor-to-neighbor consensus.
• It leverages intrinsic operations such as exponential maps, logarithm maps, and geodesic projections to handle curvature and control dynamic regret through curvature-dependent bounds.
• Applications include distributed Fréchet mean computation on Hadamard manifolds and bounded-curvature settings, demonstrating trade-offs between computational efficiency and convergence guarantees.

Decentralized online Riemannian gradient descent denotes a family of algorithms in which multiple networked agents maintain local iterates on a manifold, update those iterates using local Riemannian gradient information, and exchange only neighbor-to-neighbor messages, with performance evaluated by regret rather than solely by asymptotic stationarity. In the strict online sense, the first decentralized online Riemannian framework on Hadamard manifolds was developed through decentralized projected Riemannian gradient descent, which combines projected Riemannian gradient steps with manifold consensus and proves a dynamic regret bound controlled by path variation and network connectivity; subsequent work extends the same agenda beyond Hadamard manifolds to bounded-curvature settings and to two-point bandit feedback (Chen et al., 2024, Sahinoglu et al., 9 Sep 2025).

1. Problem formulations and regret criteria

The canonical decentralized online Riemannian setup places $n$ agents on a communication graph and lets each agent $i$ maintain a local iterate $x_{i,t}\in \mathcal X\subseteq \mathcal M$ at round $t$. In the Hadamard-manifold formulation, agent $i$ receives a local loss $f_{i,t}:\mathcal M\to\mathbb R$, and the global loss is

$f_t(x)=\frac{1}{n}\sum_{i=1}^n f_{i,t}(x).$

The objective is decentralized tracking of a comparator sequence $\{u_t\}_{t=1}^T\subset \mathcal X$, rather than maintenance of a single centralized iterate. The associated dynamic regret is

$\mathrm{D\mbox{-}Regret}(\{x_{i,t}\}) = \frac{1}{n}\sum_{t=1}^T\sum_{i=1}^n f_t(x_{i,t}) - \sum_{t=1}^T f_t(u_t),$

and the nonstationarity of the environment is quantified by the path variation

$P_T=\sum_{t=1}^T d(u_{t+1},u_t).$

This formulation makes the effect of drifting minimizers explicit: $i$0 corresponds to the static case, whereas larger $i$1 increases tracking difficulty (Chen et al., 2024).

A distinct, later line studies static regret against a fixed comparator $i$2 on manifolds with bounded sectional curvature. There the decentralized full-information regret is

$i$3

with the same global loss aggregation

$i$4

This is still online learning, but with a static comparator rather than a drifting one (Sahinoglu et al., 9 Sep 2025).

The communication layer is standard across these formulations. The network is represented by a symmetric doubly stochastic matrix $i$5, with $i$6 when agents communicate. The spectral quantity $i$7, the second largest singular value, encodes connectivity: smaller $i$8 implies faster mixing, while the spectral gap $i$9 measures consensus difficulty (Chen et al., 2024).

2. Geometric setting and why curvature matters

The geometric assumptions determine which consensus constructions and regret arguments are available. In the Hadamard-manifold setting, $x_{i,t}\in \mathcal X\subseteq \mathcal M$0 is complete, simply connected, and has nonpositive sectional curvature. This yields a collection of structural properties used throughout the analysis: unique geodesics between points, global invertibility of $x_{i,t}\in \mathcal X\subseteq \mathcal M$1 with inverse $x_{i,t}\in \mathcal X\subseteq \mathcal M$2, geodesic convexity of squared distance, uniqueness of Fréchet means, and single-valued nonexpansive projection onto a closed geodesically convex feasible set $x_{i,t}\in \mathcal X\subseteq \mathcal M$3. Projection is defined by

$x_{i,t}\in \mathcal X\subseteq \mathcal M$4

Local losses are assumed geodesically convex, differentiable, and $x_{i,t}\in \mathcal X\subseteq \mathcal M$5-Lipschitz, so the analysis uses true Riemannian gradients $x_{i,t}\in \mathcal X\subseteq \mathcal M$6 rather than subgradients (Chen et al., 2024).

The first-order inequality for differentiable geodesically convex functions has the familiar intrinsic form

$x_{i,t}\in \mathcal X\subseteq \mathcal M$7

This substitutes for Euclidean convexity and permits online-gradient-style telescoping arguments on curved spaces. The Hadamard assumption is especially strong because it globalizes $x_{i,t}\in \mathcal X\subseteq \mathcal M$8 and removes geodesic ambiguity (Chen et al., 2024).

Going beyond Hadamard manifolds requires substantially tighter localization. The bounded-curvature formulation assumes sectional curvature bounds

$x_{i,t}\in \mathcal X\subseteq \mathcal M$9

on a geodesically convex domain $t$0 of diameter $t$1, with the additional restriction

$t$2

when $t$3. This regime preserves unique geodesics, well-defined logarithm maps, and uniqueness of Fréchet means inside $t$4, but only locally rather than globally. The analysis replaces global Hadamard convexity by curvature-dependent comparison inequalities, including generalized cosine-law bounds and logarithm-map distortion estimates (Sahinoglu et al., 9 Sep 2025).

A central consequence is that geometry enters regret bounds through explicit curvature constants. In negatively curved Hadamard settings this dependence appears through

$t$5

whereas the beyond-Hadamard analysis uses constants

$t$6

together with additional distortion constants $t$7 (Chen et al., 2024, Sahinoglu et al., 9 Sep 2025).

3. Algorithmic structure and consensus mechanisms

The basic decentralized online Riemannian update alternates a local gradient step and a consensus step. On Hadamard manifolds, decentralized projected Riemannian gradient descent starts from a common initialization $t$8 and applies

$t$9

followed by the weighted Fréchet mean consensus step

$i$0

This second step is the intrinsic analogue of Euclidean weighted averaging: in Euclidean space the minimizer would be $i$1, whereas on a manifold linear averaging is replaced by minimization of weighted squared geodesic distances (Chen et al., 2024).

A central object in the consensus analysis is the Fréchet variance

$i$2

For exact weighted Fréchet averaging on a Hadamard manifold, the consensus map satisfies the linear contraction

$i$3

which is the Riemannian counterpart of linear consensus contraction in Euclidean space (Chen et al., 2024).

Because solving a Fréchet mean minimization at every round can be expensive, the same work introduces a simplified closed-form consensus step,

$i$4

with

$i$5

This replaces exact averaging by one Riemannian gradient step toward consensus in the tangent space. The contraction weakens to

$i$6

but the dynamic-regret order remains unchanged (Chen et al., 2024).

Beyond Hadamard manifolds, intrinsic consensus is kept but becomes explicitly curvature-aware. The consensus operator is

$i$7

which is interpretable as one intrinsic gradient step on the local consensus objective

$i$8

Choosing

$i$9

yields linear variance reduction with rate

$f_{i,t}:\mathcal M\to\mathbb R$0

namely

$f_{i,t}:\mathcal M\to\mathbb R$1

where $f_{i,t}:\mathcal M\to\mathbb R$2 is the Fréchet mean of $f_{i,t}:\mathcal M\to\mathbb R$3 (Sahinoglu et al., 9 Sep 2025).

4. Regret guarantees and performance bounds

The principal dynamic-regret theorem on Hadamard manifolds states that, under a lower sectional-curvature bound $f_{i,t}:\mathcal M\to\mathbb R$4,

$f_{i,t}:\mathcal M\to\mathbb R$5

where

$f_{i,t}:\mathcal M\to\mathbb R$6

With the prescribed choice of $f_{i,t}:\mathcal M\to\mathbb R$7, this yields

$f_{i,t}:\mathcal M\to\mathbb R$8

The same paper shows that the simplified closed-form consensus variant achieves the same order,

$f_{i,t}:\mathcal M\to\mathbb R$9

despite its weaker per-round contraction (Chen et al., 2024).

This bound cleanly separates three sources of difficulty. Geometry appears through curvature-dependent constants such as $f_t(x)=\frac{1}{n}\sum_{i=1}^n f_{i,t}(x).$0, network structure appears through $f_t(x)=\frac{1}{n}\sum_{i=1}^n f_{i,t}(x).$1, and temporal drift appears through $f_t(x)=\frac{1}{n}\sum_{i=1}^n f_{i,t}(x).$2. Two special cases clarify the interpretation: when $f_t(x)=\frac{1}{n}\sum_{i=1}^n f_{i,t}(x).$3, corresponding to complete averaging, the bound reduces to the minimax-optimal $f_t(x)=\frac{1}{n}\sum_{i=1}^n f_{i,t}(x).$4; when $f_t(x)=\frac{1}{n}\sum_{i=1}^n f_{i,t}(x).$5, it becomes $f_t(x)=\frac{1}{n}\sum_{i=1}^n f_{i,t}(x).$6 (Chen et al., 2024).

On manifolds with bounded sectional curvature, the static-regret theory changes comparator structure but preserves the $f_t(x)=\frac{1}{n}\sum_{i=1}^n f_{i,t}(x).$7 rate. With step sizes

$f_t(x)=\frac{1}{n}\sum_{i=1}^n f_{i,t}(x).$8

the full-information algorithm satisfies

$f_t(x)=\frac{1}{n}\sum_{i=1}^n f_{i,t}(x).$9

The corresponding bound can be written more explicitly as

$\{u_t\}_{t=1}^T\subset \mathcal X$0

The same work extends the framework to two-point bandit feedback by using the intrinsic estimator

$\{u_t\}_{t=1}^T\subset \mathcal X$1

with perturbations generated through exponential maps, and proves

$\{u_t\}_{t=1}^T\subset \mathcal X$2

when $\{u_t\}_{t=1}^T\subset \mathcal X$3 (Sahinoglu et al., 9 Sep 2025).

A nearby but distinct result concerns decentralized stochastic Riemannian optimization with diminishing step sizes on manifolds with bounded sectional curvature. There the problem is static rather than online, and the guarantees are an $\{u_t\}_{t=1}^T\subset \mathcal X$4 consensus bound and an $\{u_t\}_{t=1}^T\subset \mathcal X$5 ergodic optimality gap. This establishes exact non-asymptotic optimality-gap decay for intrinsic decentralized stochastic Riemannian methods, but not regret against a sequence of changing losses (Nguyen et al., 17 Mar 2026).

5. Neighboring formulations and related methodological lines

The phrase “decentralized online Riemannian gradient descent” is used narrowly in regret-minimization work and more broadly in streaming or stochastic manifold optimization. The literature therefore separates into three adjacent categories: strict online regret methods, online or streaming stationarity methods, and offline decentralized manifold optimization.

Work	Setting	Main guarantee or distinction
(Chen et al., 2024)	Decentralized online geodesically convex optimization on Hadamard manifolds	Dynamic regret $\{u_t\}_{t=1}^T\subset \mathcal X$6
(Sahinoglu et al., 9 Sep 2025)	Decentralized online optimization beyond Hadamard; full information and two-point bandit	Static regret $\{u_t\}_{t=1}^T\subset \mathcal X$7 in both settings
(Deng et al., 2024)	Online stochastic nonconvex optimization on compact submanifolds with streaming data	Oracle complexity $\{u_t\}_{t=1}^T\subset \mathcal X$8, not regret
(Nguyen et al., 17 Mar 2026)	Static decentralized stochastic Riemannian diffusion with diminishing step size	Consensus $\{u_t\}_{t=1}^T\subset \mathcal X$9, ergodic optimality gap $\mathrm{D\mbox{-}Regret}(\{x_{i,t}\}) = \frac{1}{n}\sum_{t=1}^T\sum_{i=1}^n f_t(x_{i,t}) - \sum_{t=1}^T f_t(u_t),$0
(Deng et al., 2023)	Offline smooth optimization on compact submanifolds	DPRGD $\mathrm{D\mbox{-}Regret}(\{x_{i,t}\}) = \frac{1}{n}\sum_{t=1}^T\sum_{i=1}^n f_t(x_{i,t}) - \sum_{t=1}^T f_t(u_t),$1, DPRGT $\mathrm{D\mbox{-}Regret}(\{x_{i,t}\}) = \frac{1}{n}\sum_{t=1}^T\sum_{i=1}^n f_t(x_{i,t}) - \sum_{t=1}^T f_t(u_t),$2 stationarity
(Hu et al., 2023)	Offline decentralized natural-gradient ERM on manifolds	Stationarity $\mathrm{D\mbox{-}Regret}(\{x_{i,t}\}) = \frac{1}{n}\sum_{t=1}^T\sum_{i=1}^n f_t(x_{i,t}) - \sum_{t=1}^T f_t(u_t),$3 with Kronecker RFIM tracking

A recurrent source of confusion is that streaming or stochastic decentralized manifold methods are not automatically online-learning methods in the regret sense. The stochastic diffusion method with diminishing step size explicitly optimizes a static objective $\mathrm{D\mbox{-}Regret}(\{x_{i,t}\}) = \frac{1}{n}\sum_{t=1}^T\sum_{i=1}^n f_t(x_{i,t}) - \sum_{t=1}^T f_t(u_t),$4 under unbiased gradient noise rather than a sequence of changing losses (Nguyen et al., 17 Mar 2026). Likewise, the decentralized projected Riemannian stochastic recursive momentum method processes local data in real time as it streams in and uses only $\mathrm{D\mbox{-}Regret}(\{x_{i,t}\}) = \frac{1}{n}\sum_{t=1}^T\sum_{i=1}^n f_t(x_{i,t}) - \sum_{t=1}^T f_t(u_t),$5 stochastic gradient evaluations per iteration, but its guarantees are nonconvex stationarity and oracle complexity rather than regret (Deng et al., 2024).

Outside the strict online setting, several methodological branches enlarge the design space. Gradient-tracking methods on compact submanifolds establish exact convergence with constant step size (Deng et al., 2023). Retraction-based Stiefel-manifold algorithms provide earlier decentralized gradient and gradient-tracking templates (Chen et al., 2021). Natural-gradient methods incorporate RFIM-based preconditioning with Kronecker-factor communication to obtain $\mathrm{D\mbox{-}Regret}(\{x_{i,t}\}) = \frac{1}{n}\sum_{t=1}^T\sum_{i=1}^n f_t(x_{i,t}) - \sum_{t=1}^T f_t(u_t),$6 stationarity for decentralized ERM on Stiefel and Grassmann manifolds (Hu et al., 2023). Other work treats weakly convex nonsmooth problems by decentralized Riemannian subgradient methods (Wang et al., 2023), composite optimization by decentralized proximal gradient tracking (Wang et al., 2024), quantized communication by quantized Riemannian gradient tracking (Chen et al., 9 Jun 2025), and momentum-augmented tracking on compact submanifolds (Chen et al., 14 Feb 2026).

6. Applications, computational tradeoffs, and limitations

The first online Hadamard-manifold framework is validated on nonstationary decentralized Fréchet mean computation over two model spaces: hyperbolic space $\mathrm{D\mbox{-}Regret}(\{x_{i,t}\}) = \frac{1}{n}\sum_{t=1}^T\sum_{i=1}^n f_t(x_{i,t}) - \sum_{t=1}^T f_t(u_t),$7 and the manifold of symmetric positive definite matrices. In hyperbolic space, with $\mathrm{D\mbox{-}Regret}(\{x_{i,t}\}) = \frac{1}{n}\sum_{t=1}^T\sum_{i=1}^n f_t(x_{i,t}) - \sum_{t=1}^T f_t(u_t),$8 agents on a sparse nearest-neighbor network and local objectives

$\mathrm{D\mbox{-}Regret}(\{x_{i,t}\}) = \frac{1}{n}\sum_{t=1}^T\sum_{i=1}^n f_t(x_{i,t}) - \sum_{t=1}^T f_t(u_t),$9

the reported regimes are static, abruptly changing, and mixed abrupt/gradual drift. In $P_T=\sum_{t=1}^T d(u_{t+1},u_t).$0 with the affine-invariant metric, the manifold operations are

$P_T=\sum_{t=1}^T d(u_{t+1},u_t).$1

$P_T=\sum_{t=1}^T d(u_{t+1},u_t).$2

$P_T=\sum_{t=1}^T d(u_{t+1},u_t).$3

Across both spaces, the exact and simplified consensus variants exhibit similar dynamic-regret behavior, while the closed-form consensus step has much lower runtime (Chen et al., 2024).

The beyond-Hadamard online framework demonstrates that positively curved geometry can also be handled, provided the feasible set stays inside a sufficiently small geodesically convex region. The numerical example in that work uses the unit sphere and supports the curvature-aware consensus design, while the bandit extension shows that intrinsic smoothing and two-point estimation can preserve the $P_T=\sum_{t=1}^T d(u_{t+1},u_t).$4 rate (Sahinoglu et al., 9 Sep 2025). A plausible implication is that decentralized online Riemannian learning is no longer confined to globally nonpositively curved domains, but the price is stricter radius control and curvature-sensitive constants.

The dominant computational tradeoff in online regret methods lies in consensus design. Exact weighted Fréchet averaging is geometrically natural and yields the stronger contraction

$P_T=\sum_{t=1}^T d(u_{t+1},u_t).$5

but it requires solving an inner minimization problem at every iteration. Closed-form log-exp consensus is cheaper per round and avoids that inner problem, though its contraction is weaker (Chen et al., 2024). In online settings, where per-round latency matters, this difference is structurally important.

The main limitations are now clear in the literature. Hadamard-based theory depends on global nonpositive curvature; beyond-Hadamard theory requires domain diameter restrictions such as

$P_T=\sum_{t=1}^T d(u_{t+1},u_t).$6

when $P_T=\sum_{t=1}^T d(u_{t+1},u_t).$7, and the constants degrade with curvature and network sparsity (Sahinoglu et al., 9 Sep 2025). Most analyses assume synchronous communication, a fixed symmetric doubly stochastic mixing matrix, and exact access to $P_T=\sum_{t=1}^T d(u_{t+1},u_t).$8, $P_T=\sum_{t=1}^T d(u_{t+1},u_t).$9, and metric projection. Moreover, much of the broader decentralized Riemannian literature remains offline, stochastic-static, or stationarity-oriented rather than regret-oriented. This suggests that the current topic is best understood as a still-emerging intersection of online convex optimization, decentralized control, and intrinsic manifold optimization, with the most mature regret theory presently concentrated in geodesically convex settings (Chen et al., 2024, Sahinoglu et al., 9 Sep 2025).