Recurrent Manifold Optimization (RMO)

• Recurrent Manifold Optimization (RMO) is a class of methods that iteratively refines solutions on Riemannian manifolds while preserving geometric constraints.
• It encompasses approaches such as orthogonal RNN training on the Stiefel manifold, SPD-valued recurrence, and closed-loop Bayesian feature refinement.
• RMO enhances stability, supports faster convergence in high-dimensional nonconvex problems, and finds applications in deep learning, wireless communications, and computer vision.

Recurrent Manifold Optimization (RMO) denotes a class of methods in which recurrent or iterative refinement is performed with explicit respect for non-Euclidean geometry. In recent cross-modal registration, RBE-Flow formulates dense flow estimation as a closed-loop recurrent Bayesian estimation problem on learned feature manifolds, with a Recurrent Manifold Optimization block that produces both local updates and uncertainty estimates (Ding et al., 29 Jun 2026). In geometric optimization for deep learning, RMO also refers to training recurrent networks whose hidden-to-hidden weights remain on the Stiefel manifold, thereby preserving orthogonality during learning (Fei et al., 2023). In manifold-valued sequence modeling, the SPD-SRU formulates recurrence directly on $SPD(n)$ and defines learning as a constrained optimization problem over orthogonal actions and simplex-constrained weights (Chakraborty et al., 2018). These usages suggest that RMO is best understood as an umbrella term for recurrent procedures that embed optimization steps, state evolution, or parameter learning in a Riemannian or manifold-constrained setting.

1. Terminological scope and recurrent formulations

A common source of ambiguity is that the acronym “RMO” is used in more than one way. Li et al. use RMO to mean “Riemannian manifold optimization” in advanced wireless communications, where optimization is performed directly on manifolds such as the complex sphere, Stiefel manifold, Grassmann manifold, and Hermitian positive-definite manifold (Li et al., 9 Feb 2026). By contrast, RBE-Flow uses “Recurrent Manifold Optimization” for a specific iterative module inside a recurrent Bayesian estimator on learned feature manifolds (Ding et al., 29 Jun 2026). The SPD-SRU exposition labels its learning problem “Recurrent Manifold Optimization” because the recurrent cell and its optimization operate in a manifold-valued setting, specifically $SPD(n)$ with weighted Fréchet means and orthogonal group actions (Chakraborty et al., 2018). The survey of geometric optimization for deep learning uses the term for training recurrent networks whose recurrent weight matrix $W$ is constrained by $W^T W = I_m$, so that optimization proceeds on $\St(m,m)$ rather than in Euclidean space (Fei et al., 2023).

Setting	Manifold or constraint	Recurrent role
Orthogonal RNN training	$W \in \St(m,m)$, $W^T W = I_m$	Hidden-to-hidden transition is manifold-constrained
SPD-SRU	$SPD(n)$, $g \in O(n)$, simplex-normalized weights	Hidden state and moving means are manifold-valued
RBE-Flow	Learned feature manifolds, dense flow with covariance	Iterative refinement and Bayesian fusion are closed-loop
Wireless RMO	Complex sphere, Stiefel, Grassmann, HPD, circle manifolds	Not recurrent in the strict RNN sense; supplies the geometric optimization backbone

This suggests that “recurrent” can refer either to temporal sequence processing, as in recurrent neural networks and SPD-SRU, or to iterative closed-loop refinement, as in RBE-Flow. It also suggests that the invariant element across these formulations is not a single algorithm but a shared commitment to manifold-respecting updates.

2. Geometric foundations and algorithmic primitives

The geometric basis is the standard constrained problem

$\min_{x \in \mathcal M} f(x),$

where $SPD(n)$0 is a Riemannian submanifold and $SPD(n)$1 is the restriction of $SPD(n)$2 to $SPD(n)$3 (Li et al., 9 Feb 2026). A real manifold $SPD(n)$4 of dimension $SPD(n)$5 is a topological space locally homeomorphic to $SPD(n)$6 and endowed with a smooth structure. If $SPD(n)$7 is embedded in $SPD(n)$8 or $SPD(n)$9, each tangent space $W$0 can be equipped with the inner product induced by the ambient Euclidean metric, yielding a Riemannian manifold $W$1 (Li et al., 9 Feb 2026).

For embedded manifolds, the Riemannian gradient is the orthogonal projection of the Euclidean gradient onto the tangent space: $W$2 On the sphere $W$3, the tangent space is

$W$4

with projector

$W$5

The Riemannian Hessian is

$W$6

and the exponential map on the sphere is

$W$7

Because exact geodesic updates are often expensive, practical algorithms use retractions. Examples include sphere normalization,

$W$8

and Stiefel retractions based on QR factorization or polar decomposition (Li et al., 9 Feb 2026).

The recurrent formulations of RMO draw on this same toolkit. The survey treatment of orthogonal RNNs makes the Stiefel geometry explicit, while the SPD-SRU relies on manifold-valued averages and orthogonal actions, and RBE-Flow uses local Gauss–Newton steps whose damping is adapted by posterior covariance (Fei et al., 2023). This suggests that recurrence does not replace manifold optimization; rather, it composes manifold-respecting local updates over time or across refinement iterations.

3. Orthogonality-constrained recurrent networks on the Stiefel manifold

In the orthogonal-RNN formulation, the recurrent model is

$W$9

with loss

$W^T W = I_m$0

subject to

$W^T W = I_m$1

The purpose of the constraint is explicit: keeping $W^T W = I_m$2 orthogonal stabilizes long-range gradient flow and avoids vanishing or exploding norms (Fei et al., 2023).

The tangent space at $W^T W = I_m$3 is

$W^T W = I_m$4

and the Riemannian gradient is obtained by projecting the Euclidean gradient $W^T W = I_m$5 onto that tangent space: $W^T W = I_m$6 where $W^T W = I_m$7. An equivalent expression is

$W^T W = I_m$8

After computing a descent direction in $W^T W = I_m$9, the update is returned to the manifold with the QR retraction

$\St(m,m)$0

so that

$\St(m,m)$1

remains on $\St(m,m)$2. When momentum is used, a tangent vector $\St(m,m)$3 can be transported to $\St(m,m)$4 by projection: $\St(m,m)$5 The survey presents a one-epoch Riemannian SGD-with-momentum loop built from these ingredients: Euclidean backpropagation through time, tangent projection, momentum update in the tangent space, retraction for $\St(m,m)$6, and ordinary Euclidean updates for $\St(m,m)$7 and $\St(m,m)$8 (Fei et al., 2023).

The same source states that treating $\St(m,m)$9 as a compact Riemannian manifold with bounded curvature gives standard convergence results for Riemannian SGD-with-momentum. It also reports that expRNN uses this QR retraction and parallel transport and shows stable long-term memory on copying and addition tasks, while GORU augments the orthogonal RNN with forget gates and achieves $W \in \St(m,m)$0 on permuted-MNIST, outperforming Euclidean LSTMs, uRNNs, and soft-orthogonal variants (Fei et al., 2023). The built-in QR retraction has cost $W \in \St(m,m)$1, and the same source notes that GeoTorch or McTorch can be used to implement manifold-constrained training in PyTorch.

4. SPD-valued recurrence and statistical recurrent models

The SPD-SRU generalizes recurrence from Euclidean hidden states to the manifold

$W \in \St(m,m)$2

whose tangent space at $W \in \St(m,m)$3 is

$W \in \St(m,m)$4

Two metrics are emphasized: the affine-invariant metric

$W \in \St(m,m)$5

with geodesic

$W \in \St(m,m)$6

distance

$W \in \St(m,m)$7

and corresponding exponential and logarithm; and the Stein metric

$W \in \St(m,m)$8

which is described as computationally cheaper and is used for the weighted Fréchet means in the cell (Chakraborty et al., 2018).

Given an input sequence $W \in \St(m,m)$9, the SPD-SRU maintains moving means $W^T W = I_m$0 over a collection of scales $W^T W = I_m$1. Its forward pass is defined by seven manifold operations: combining past scales into $W^T W = I_m$2 via weighted Fréchet mean, applying the orthogonal translation action $W^T W = I_m$3 to obtain $W^T W = I_m$4, fusing $W^T W = I_m$5 with the input $W^T W = I_m$6 to get $W^T W = I_m$7, translating to $W^T W = I_m$8, updating each moving average $W^T W = I_m$9, aggregating them into $SPD(n)$0, and finally producing

$SPD(n)$1

The hidden state is $SPD(n)$2 (Chakraborty et al., 2018).

The optimization problem is explicit: $SPD(n)$3 with respect to

$SPD(n)$4

subject to $SPD(n)$5, $SPD(n)$6, and $SPD(n)$7. The paper reduces optimization to Euclidean updates by parametrizing $SPD(n)$8 as $SPD(n)$9 with $g \in O(n)$0, and by parametrizing the weights through unconstrained $g \in O(n)$1 followed by normalization $g \in O(n)$2. Standard backpropagation computes $g \in O(n)$3, $g \in O(n)$4, $g \in O(n)$5, and $g \in O(n)$6, after which $g \in O(n)$7 is reprojected to enforce skew-symmetry and the normalized weights are recomputed (Chakraborty et al., 2018).

The statistical analysis is based on an isometric mapping $g \in O(n)$8 into a Hilbert sphere of normalized Gaussian densities, with

$g \in O(n)$9

A recursive weighted Fréchet mean estimator on the positive orthant of $\min_{x \in \mathcal M} f(x),$0 is given in closed form, and Proposition 5 states that $\min_{x \in \mathcal M} f(x),$1 as $\min_{x \in \mathcal M} f(x),$2 and that the convergence rate is super-linear (Chakraborty et al., 2018).

The empirical results are reported in three domains. On Moving MNIST, SPD-SRU with hidden SPD dimension $\min_{x \in \mathcal M} f(x),$3 uses $\min_{x \in \mathcal M} f(x),$4 parameters and $\min_{x \in \mathcal M} f(x),$5 seconds per epoch, and achieves $\min_{x \in \mathcal M} f(x),$6 for the $\min_{x \in \mathcal M} f(x),$7-class $\min_{x \in \mathcal M} f(x),$8 setting, $\min_{x \in \mathcal M} f(x),$9 for the $SPD(n)$00-class $SPD(n)$01 setting, and $SPD(n)$02 for the $SPD(n)$03-class $SPD(n)$04 setting. The same table states that SPD-SRU uses approximately $SPD(n)$05 fewer parameters than SRU/LSTM while maintaining near-perfect accuracy as the angle gap shrinks. On UCF11 action recognition, SPD-SRU uses $SPD(n)$06 parameters, about $SPD(n)$07 seconds per epoch, and reaches test accuracy $SPD(n)$08, equal to TT-RNN with approximately $SPD(n)$09 fewer parameters. In a brain-imaging group-difference test on Parkinson’s versus control tracts, a permutation test with $SPD(n)$10 permutations yields $SPD(n)$11 for Left M1 and $SPD(n)$12 for Right M1, whereas a baseline based on point-wise Fréchet mean plus Cramér’s test yields $SPD(n)$13 (Chakraborty et al., 2018).

5. Closed-loop Bayesian refinement on learned feature manifolds

RBE-Flow reformulates dense cross-modal flow estimation as a recurrent Bayesian estimation problem on learned feature manifolds. At each pixel $SPD(n)$14, with feature maps $SPD(n)$15, the residual is

$SPD(n)$16

Over a local neighborhood, residuals are stacked into $SPD(n)$17 and Jacobians into $SPD(n)$18. The Recurrent Manifold Optimization block solves at iteration $SPD(n)$19 the damped non-linear least-squares problem

$SPD(n)$20

which under Gauss–Newton linearization gives

$SPD(n)$21

The resulting flow observation is

$SPD(n)$22

and the block also emits a local measurement covariance $SPD(n)$23, with $SPD(n)$24 in the measurement model (Ding et al., 29 Jun 2026).

The probabilistic update stage is the Uncertainty-Adaptive Probabilistic Update (UAPU). From prior mean $SPD(n)$25 and covariance $SPD(n)$26, sigma points are formed using the Cholesky factor of $SPD(n)$27, propagated through a learned observation network $SPD(n)$28, and used to compute predicted observation mean $SPD(n)$29, innovation covariance $SPD(n)$30, and cross-covariance $SPD(n)$31. The Kalman-style gain is

$SPD(n)$32

and the posterior updates are

$SPD(n)$33

$SPD(n)$34

A defining feature of this formulation is the feedback of posterior uncertainty into the next optimization step: $SPD(n)$35 The paper states that in regions or iterations of high uncertainty, the solver automatically shifts toward smaller, more conservative update steps. To stabilize training, it introduces a geometry-aware rectified NLL loss, motivated by the observation that a naive Gaussian NLL of the form $SPD(n)$36 often collapses $SPD(n)$37. The full loop includes multi-scale features $SPD(n)$38, a Global Flow Init stage that builds a $SPD(n)$39D correlation at $SPD(n)$40 scale and applies a softmax to obtain $SPD(n)$41, recurrent refinement in which ConvGRU predicts $SPD(n)$42, and a total loss $SPD(n)$43 (Ding et al., 29 Jun 2026).

The abstract reports extensive experiments on OSdataset, WHU-OPT-SAR, and RoadScene, stating that RBE-Flow consistently achieves state-of-the-art performance and outperforms existing methods by a significant margin, particularly under strict sub-pixel criteria (Ding et al., 29 Jun 2026). The paper’s own summary interprets the method as converting each feature-metric refinement into a local Gauss–Newton measurement with uncertainty, then fusing these non-linear observations into a global Bayesian belief.

6. Broader Riemannian optimization context, applications, and practical interpretation

The broader Riemannian-optimization literature clarifies why manifold-aware recurrence is attractive in high-dimensional nonconvex problems. Li et al. describe next-generation wireless systems such as massive MIMO, reconfigurable intelligent surfaces, integrated sensing and communication, and fluid antenna systems as settings in which large-scale optimization with nonconvex constraints is central, and they argue that conventional Euclidean-space methods rely on approximations or relaxations that degrade performance and incur substantial computational costs (Li et al., 9 Feb 2026). Their formulation of RMO operates directly on the manifold defined by the constraints, satisfying the constraints at every optimization step.

The paper gives three baseline manifold algorithms. Riemannian Gradient Descent computes $SPD(n)$44, chooses a step size, and updates by retraction $SPD(n)$45. Its worst-case overall complexity is $SPD(n)$46 for Stiefel-type settings with $SPD(n)$47, or $SPD(n)$48 for the sphere. Riemannian Conjugate Gradient augments the descent direction with a transported previous direction and a conjugacy coefficient, with per-iteration complexity of the same order as RGD plus vector-transport cost $SPD(n)$49. Riemannian Trust-Region solves a quadratic model over the tangent space and uses the ratio

$SPD(n)$50

to accept steps and adapt the trust-region radius (Li et al., 9 Feb 2026).

The convergence guarantees are stated under standard assumptions: $SPD(n)$51 is $SPD(n)$52, $SPD(n)$53 is Lipschitz, and $SPD(n)$54 is compact. Under these conditions, RGD with exact line search converges to critical points with global sublinear rate $SPD(n)$55 in $SPD(n)$56; RCG often shows superlinear convergence locally under nondegeneracy; and RTR obtains quadratic convergence near a nondegenerate local minimum and global convergence to second-order critical points (Li et al., 9 Feb 2026).

The wireless case studies provide concrete manifold assignments. In FAS-assisted secure beamforming in NOMA, the beamformer $SPD(n)$57 satisfies $SPD(n)$58, so the problem is cast on the complex sphere with secrecy-rate objective

$SPD(n)$59

This is solved by RGD and RTR. The reported performance comparison states that RTR improves average secrecy rate by approximately $SPD(n)$60 over MM and $SPD(n)$61 over SDR, while average runtime to convergence is $SPD(n)$62 s for RTR, $SPD(n)$63 s for RGD, $SPD(n)$64 s for SDR, and $SPD(n)$65 s for MM. In massive MIMO precoding, the objective

$SPD(n)$66

is posed on the Stiefel manifold, and RCG is described as having linear per-iteration cost in $SPD(n)$67 and $SPD(n)$68, outperforming $SPD(n)$69 SDR. In RIS phase-shift design, optimization is carried out on the complex circle manifold with $SPD(n)$70 or $SPD(n)$71 cost versus $SPD(n)$72 for MM. In ISAC waveform and phase optimization, the covariance matrix $SPD(n)$73 is optimized on the HPD manifold by RTR, avoiding costly PSD projections (Li et al., 9 Feb 2026).

The implementation guidelines in the same source are also directly relevant to recurrent formulations. Manifold choice should match the physical or geometric constraint: unit modulus to the circle manifold, orthonormal columns to the Stiefel manifold, subspace invariance to the Grassmann manifold, and PSD covariance to the HPD manifold. Retraction choice depends on the manifold and computational regime: normalization on the sphere; QR on Stiefel when $SPD(n)$74; polar decomposition when higher numerical accuracy is needed; and matrix-exponential or symmetric polar maps for PSD-type settings. Armijo backtracking with $SPD(n)$75 and initial $SPD(n)$76 is recommended for step-size tuning; trust-region radius can be adjusted by halving when $SPD(n)$77 and doubling when $SPD(n)$78; and the Polak–Ribière coefficient is reported to often outperform Fletcher–Reeves in practice (Li et al., 9 Feb 2026).

Taken together, these results suggest a unifying interpretation of Recurrent Manifold Optimization. In one line of work, the manifold constraint is imposed on recurrent parameters, as with orthogonal RNNs on $SPD(n)$79. In another, the hidden state itself evolves on a manifold, as in the SPD-SRU on $SPD(n)$80. In a third, the recurrent element is an uncertainty-aware refinement loop on a learned manifold, as in RBE-Flow. Across all of these settings, the recurring technical components are tangent-space computation, manifold-compatible retraction, transport or covariance propagation, and update rules that preserve the defining geometry rather than approximating it away.