Product Riemannian Manifold Optimization

• PRMO is a framework that exploits the product structure of Riemannian manifolds to enable block-wise adaptivity and tailored metric design.
• It generalizes adaptive methods like Adagrad and Adam to non-Euclidean spaces using manifold operations such as exponential maps and parallel transport.
• Preconditioning techniques in PRMO accelerate convergence by reducing the condition number of the Riemannian Hessian through strategic metric choices.

Product Riemannian Manifold Optimization (PRMO) denotes optimization in which the decision variable lies on a Cartesian product of manifolds and the optimization method exploits that product structure in the metric, gradient, transport, and update rules. In the arXiv literature, the term is used in several closely related senses: adaptive first-order optimization on $$\mathcal M=\mathcal M_1\times\cdots\times\mathcal M_n$$, optimization on product manifolds under a preconditioned metric, optimization on products of generalized Stiefel manifolds with a non-standard metric, and application-specific manifold algorithms such as penalty-based RBFGS on mixed Euclidean/oblique/circle product spaces. Across these usages, the common premise is that tangent spaces split factor-wise and that manifold operations can be organized blockwise, while the chosen metric materially affects convergence behavior (Bécigneul et al., 2018, Gao et al., 2023, Shustin et al., 2019, Geng et al., 5 Sep 2025).

1. Product-manifold structure and geometric preliminaries

A standard PRMO setting begins with complete Riemannian manifolds $(\mathcal M_i,\rho^i)$, $i=1,\dots,n$, and forms the product manifold

$$\mathcal M=\mathcal M_1\times\cdots\times\mathcal M_n.$$

In the adaptive-optimization formulation, the product metric is written as

$$\rho_x\bigl((u^1,\dots,u^n),(v^1,\dots,v^n)\bigr)=\sum_{i=1}^n \rho^i_{x^i}(u^i,v^i),$$

and the tangent space decomposes as

$$T_x\mathcal M=\bigoplus_{i=1}^n T_{x^i}\mathcal M_i.$$

Distance, exponential map, logarithm, and parallel transport act factor-wise. In the embedded-manifold framework, the same product structure is expressed as $$M:=M_1\times\cdots\times M_K\subset E_1\times\cdots\times E_K$$, with

$T_xM=T_{x_1}M_1\times\cdots\times T_{x_K}M_K,$

and the standard product metric

$$g_x^{(e)}(\xi,\eta)=\sum_{k=1}^K \langle \xi_k,\eta_k\rangle_{E_k}.$$

These formulations are equivalent in emphasizing separability at the manifold level (Bécigneul et al., 2018, Gao et al., 2023).

Geodesic convexity is the principal convexity notion used in the first-order theory. If $\mathcal X_i\subset \mathcal M_i$ is geodesically convex, then $(\mathcal M_i,\rho^i)$0 is the feasible set. A differentiable $(\mathcal M_i,\rho^i)$1 is geodesically convex on $(\mathcal M_i,\rho^i)$2 iff for every $(\mathcal M_i,\rho^i)$3,

$(\mathcal M_i,\rho^i)$4

This replaces linear convexity by a relation between the Riemannian gradient and the inverse exponential map. A plausible implication is that PRMO inherits much of the formal structure of block-coordinate Euclidean optimization, but only after replacing vector-space operations by their manifold counterparts.

2. Adaptive first-order PRMO on $(\mathcal M_i,\rho^i)$5

In "Riemannian Adaptive Optimization Methods" (Bécigneul et al., 2018), PRMO is the setting in which adaptive schemes such as Adagrad, Adam, and Amsgrad are generalized to products of manifolds, with adaptivity implemented across manifolds in the cartesian product. Writing $(\mathcal M_i,\rho^i)$6, $(\mathcal M_i,\rho^i)$7, and $(\mathcal M_i,\rho^i)$8, the Riemannian Adagrad update is

$(\mathcal M_i,\rho^i)$9

For Riemannian Adam, one introduces

$i=1,\dots,n$0

and updates

$i=1,\dots,n$1

Riemannian Amsgrad keeps $i=1,\dots,n$2 and uses $i=1,\dots,n$3 in the denominator; when constraints are present, the iterate is projected back to $i=1,\dots,n$4.

The nontrivial step is momentum transport. On a general manifold, $i=1,\dots,n$5 must be moved to $i=1,\dots,n$6 before it can be combined with $i=1,\dots,n$7. The update therefore becomes

$i=1,\dots,n$8

where $i=1,\dots,n$9 is parallel transport. Because $$\mathcal M=\mathcal M_1\times\cdots\times\mathcal M_n.$$0 is an isometry, norms and inner products are preserved. This guarantees that adaptive momentum and variance estimates stay in the correct tangent spaces.

The convergence theory is stated for geodesically complete manifolds with sectional curvature $$\mathcal M=\mathcal M_1\times\cdots\times\mathcal M_n.$$1, geodesically convex feasible sets of diameter $$\mathcal M=\mathcal M_1\times\cdots\times\mathcal M_n.$$2, uniformly bounded gradients $$\mathcal M=\mathcal M_1\times\cdots\times\mathcal M_n.$$3, and geodesically convex losses $$\mathcal M=\mathcal M_1\times\cdots\times\mathcal M_n.$$4. Regret is

$$\mathcal M=\mathcal M_1\times\cdots\times\mathcal M_n.$$5

For Ramsgrad with $$\mathcal M=\mathcal M_1\times\cdots\times\mathcal M_n.$$6 and $$\mathcal M=\mathcal M_1\times\cdots\times\mathcal M_n.$$7, the regret bound contains the curvature factor $$\mathcal M=\mathcal M_1\times\cdots\times\mathcal M_n.$$8. If all $$\mathcal M=\mathcal M_1\times\cdots\times\mathcal M_n.$$9, the method recovers Euclidean Amsgrad bounds; setting $$\rho_x\bigl((u^1,\dots,u^n),(v^1,\dots,v^n)\bigr)=\sum_{i=1}^n \rho^i_{x^i}(u^i,v^i),$$0 yields Riemannian Adagrad with regret $$\rho_x\bigl((u^1,\dots,u^n),(v^1,\dots,v^n)\bigr)=\sum_{i=1}^n \rho^i_{x^i}(u^i,v^i),$$1. The paper explicitly states that the generalization is tight: choosing Euclidean space as Riemannian manifold yields the same algorithms and regret bounds as the standard Euclidean methods.

3. Preconditioned metrics as a PRMO acceleration mechanism

"Optimization on product manifolds under a preconditioned metric" (Gao et al., 2023) treats PRMO as a metric-design framework. The central claim is that local linear convergence of first-order Riemannian methods is governed by the condition number of the Riemannian Hessian at a minimizer,

$$\rho_x\bigl((u^1,\dots,u^n),(v^1,\dots,v^n)\bigr)=\sum_{i=1}^n \rho^i_{x^i}(u^i,v^i),$$2

and that one can accelerate optimization by endowing the product manifold with a preconditioned metric

$$\rho_x\bigl((u^1,\dots,u^n),(v^1,\dots,v^n)\bigr)=\sum_{i=1}^n \rho^i_{x^i}(u^i,v^i),$$3

where $$\rho_x\bigl((u^1,\dots,u^n),(v^1,\dots,v^n)\bigr)=\sum_{i=1}^n \rho^i_{x^i}(u^i,v^i),$$4 is a self-adjoint, uniformly SPD linear operator approximating the diagonal blocks of the Riemannian Hessian under the standard product metric.

Three preconditioner-design strategies are proposed. Exact block-diagonal preconditioning sets $$\rho_x\bigl((u^1,\dots,u^n),(v^1,\dots,v^n)\bigr)=\sum_{i=1}^n \rho^i_{x^i}(u^i,v^i),$$5, the restriction of the Hessian along the $$\rho_x\bigl((u^1,\dots,u^n),(v^1,\dots,v^n)\bigr)=\sum_{i=1}^n \rho^i_{x^i}(u^i,v^i),$$6-th factor. Left and right preconditioning exploit factorized diagonal blocks $$\rho_x\bigl((u^1,\dots,u^n),(v^1,\dots,v^n)\bigr)=\sum_{i=1}^n \rho^i_{x^i}(u^i,v^i),$$7 or $$\rho_x\bigl((u^1,\dots,u^n),(v^1,\dots,v^n)\bigr)=\sum_{i=1}^n \rho^i_{x^i}(u^i,v^i),$$8, using either $$\rho_x\bigl((u^1,\dots,u^n),(v^1,\dots,v^n)\bigr)=\sum_{i=1}^n \rho^i_{x^i}(u^i,v^i),$$9, $$T_x\mathcal M=\bigoplus_{i=1}^n T_{x^i}\mathcal M_i.$$0, or a symmetric choice $$T_x\mathcal M=\bigoplus_{i=1}^n T_{x^i}\mathcal M_i.$$1. Gauss–Newton type preconditioning applies when $$T_x\mathcal M=\bigoplus_{i=1}^n T_{x^i}\mathcal M_i.$$2, setting

$$T_x\mathcal M=\bigoplus_{i=1}^n T_{x^i}\mathcal M_i.$$3

which induces the metric

$$T_x\mathcal M=\bigoplus_{i=1}^n T_{x^i}\mathcal M_i.$$4

Under this metric, the Riemannian gradient becomes

$$T_x\mathcal M=\bigoplus_{i=1}^n T_{x^i}\mathcal M_i.$$5

with factor-wise expression

$$T_x\mathcal M=\bigoplus_{i=1}^n T_{x^i}\mathcal M_i.$$6

The Hessian is

$$T_x\mathcal M=\bigoplus_{i=1}^n T_{x^i}\mathcal M_i.$$7

while retractions and vector transports remain blockwise: $$T_x\mathcal M=\bigoplus_{i=1}^n T_{x^i}\mathcal M_i.$$8

This framework leads directly to PRMO-RGD and PRMO-RCG. Their local linear rate is stated as

$$T_x\mathcal M=\bigoplus_{i=1}^n T_{x^i}\mathcal M_i.$$9

for some $$M:=M_1\times\cdots\times M_K\subset E_1\times\cdots\times E_K$$0. The principal theorem further states that if $$M:=M_1\times\cdots\times M_K\subset E_1\times\cdots\times E_K$$1, then $$M:=M_1\times\cdots\times M_K\subset E_1\times\cdots\times E_K$$2. This suggests that, in PRMO, the metric is not an incidental geometric choice but an algorithmic object analogous to a Euclidean preconditioner.

4. PRMO on products of generalized Stiefel manifolds

"Riemannian optimization with a preconditioning scheme on the generalized Stiefel manifold" (Shustin et al., 2019) develops the geometric components needed when the feasible set is a product of generalized Stiefel manifolds. A single generalized Stiefel factor is

$$M:=M_1\times\cdots\times M_K\subset E_1\times\cdots\times E_K$$3

with tangent space

$$M:=M_1\times\cdots\times M_K\subset E_1\times\cdots\times E_K$$4

For $$M:=M_1\times\cdots\times M_K\subset E_1\times\cdots\times E_K$$5 blocks $$M:=M_1\times\cdots\times M_K\subset E_1\times\cdots\times E_K$$6, the feasible set is

$$M:=M_1\times\cdots\times M_K\subset E_1\times\cdots\times E_K$$7

The paper then equips each tangent space with the nonstandard metric

$$M:=M_1\times\cdots\times M_K\subset E_1\times\cdots\times E_K$$8

where $$M:=M_1\times\cdots\times M_K\subset E_1\times\cdots\times E_K$$9 is the preconditioner.

If $T_xM=T_{x_1}M_1\times\cdots\times T_{x_K}M_K,$0 is an ambient extension of the cost, the ambient gradient under this metric is

$T_xM=T_{x_1}M_1\times\cdots\times T_{x_K}M_K,$1

and the manifold gradient is

$T_xM=T_{x_1}M_1\times\cdots\times T_{x_K}M_K,$2

With $T_xM=T_{x_1}M_1\times\cdots\times T_{x_K}M_K,$3, the projector is given by

$T_xM=T_{x_1}M_1\times\cdots\times T_{x_K}M_K,$4

where $T_xM=T_{x_1}M_1\times\cdots\times T_{x_K}M_K,$5 solves the Sylvester equation

$T_xM=T_{x_1}M_1\times\cdots\times T_{x_K}M_K,$6

For constant $T_xM=T_{x_1}M_1\times\cdots\times T_{x_K}M_K,$7, the Hessian is expressed through the projected ambient Hessian and the Weingarten term; the paper gives the simplified formula

$T_xM=T_{x_1}M_1\times\cdots\times T_{x_K}M_K,$8

Retractions are given explicitly. The polar-based retraction is

$T_xM=T_{x_1}M_1\times\cdots\times T_{x_K}M_K,$9

and the $$g_x^{(e)}(\xi,\eta)=\sum_{k=1}^K \langle \xi_k,\eta_k\rangle_{E_k}.$$0-QR-based retraction is

$$g_x^{(e)}(\xi,\eta)=\sum_{k=1}^K \langle \xi_k,\eta_k\rangle_{E_k}.$$1

A simple vector transport is projection after lift,

$$g_x^{(e)}(\xi,\eta)=\sum_{k=1}^K \langle \xi_k,\eta_k\rangle_{E_k}.$$2

The paper states linear convergence for gradient descent under uniform Hessian conditioning, local superlinear convergence for conjugate gradient under exact line-search, and local quadratic or superlinear convergence for trust-region methods with a second-order retraction. Its numerical examples include generalized eigenvalue subspace computation and CCA; on the MEDIAMILL dataset, the full preconditioner $$g_x^{(e)}(\xi,\eta)=\sum_{k=1}^K \langle \xi_k,\eta_k\rangle_{E_k}.$$3 yields approximately $$g_x^{(e)}(\xi,\eta)=\sum_{k=1}^K \langle \xi_k,\eta_k\rangle_{E_k}.$$4 fewer iterations than $$g_x^{(e)}(\xi,\eta)=\sum_{k=1}^K \langle \xi_k,\eta_k\rangle_{E_k}.$$5, and condition-number estimates at the optimum are reported as $$g_x^{(e)}(\xi,\eta)=\sum_{k=1}^K \langle \xi_k,\eta_k\rangle_{E_k}.$$6 for $$g_x^{(e)}(\xi,\eta)=\sum_{k=1}^K \langle \xi_k,\eta_k\rangle_{E_k}.$$7 versus $$g_x^{(e)}(\xi,\eta)=\sum_{k=1}^K \langle \xi_k,\eta_k\rangle_{E_k}.$$8 for $$g_x^{(e)}(\xi,\eta)=\sum_{k=1}^K \langle \xi_k,\eta_k\rangle_{E_k}.$$9.

5. Penalty-based PRMO and RBFGS on mixed product manifolds

In "Movable IRS-Aided ISAC Systems: Joint Beamforming and Position Optimization" (Geng et al., 5 Sep 2025), PRMO is instantiated for a mixed-variable constrained problem whose decision variables are the transmit beamforming matrix $\mathcal X_i\subset \mathcal M_i$0, the receive filter matrix $\mathcal X_i\subset \mathcal M_i$1, the IRS phase-shift vector $\mathcal X_i\subset \mathcal M_i$2, and the array-wise position-parameter vector $\mathcal X_i\subset \mathcal M_i$3. The objective is

$\mathcal X_i\subset \mathcal M_i$4

subject to communication-rate constraints, sensing-SINR constraints, unit-norm receive filters, constant-modulus phase shifts, region confinement through the sigmoid projection $\mathcal X_i\subset \mathcal M_i$5, and minimum element-distance constraints.

The feasible set is the product manifold

$\mathcal X_i\subset \mathcal M_i$6

where

$\mathcal X_i\subset \mathcal M_i$7

and

$\mathcal X_i\subset \mathcal M_i$8

The tangent space constraints are

$\mathcal X_i\subset \mathcal M_i$9

All inequality constraints $(\mathcal M_i,\rho^i)$00 are absorbed into the smooth penalty objective

$(\mathcal M_i,\rho^i)$01

where $(\mathcal M_i,\rho^i)$02 is the linear-quadratic smoothing of $(\mathcal M_i,\rho^i)$03: $(\mathcal M_i,\rho^i)$04

The Riemannian metric is the product of Euclidean metrics,

$(\mathcal M_i,\rho^i)$05

Retractions are blockwise: $(\mathcal M_i,\rho^i)$06, $(\mathcal M_i,\rho^i)$07 is column-renormalized, $(\mathcal M_i,\rho^i)$08, and $(\mathcal M_i,\rho^i)$09. The Riemannian gradient is obtained by projecting Euclidean gradients onto each block tangent space, including

$(\mathcal M_i,\rho^i)$10

and

$(\mathcal M_i,\rho^i)$11

The search direction is computed by RBFGS,

$(\mathcal M_i,\rho^i)$12

followed by Armijo line search and retraction. The cautious update condition is

$(\mathcal M_i,\rho^i)$13

after which the inverse-Hessian approximation is updated blockwise. In practice, a limited-memory two-loop recursion is used. The outer exact-penalty loop increases $(\mathcal M_i,\rho^i)$14 if any $(\mathcal M_i,\rho^i)$15, while reducing the smoothing and tolerance parameters. The paper states that the Armijo line search guarantees monotone decrease of $(\mathcal M_i,\rho^i)$16 and that, for sufficiently large $(\mathcal M_i,\rho^i)$17 and small $(\mathcal M_i,\rho^i)$18, the limit point satisfies all original inequalities. Its overall cost is reported as

$(\mathcal M_i,\rho^i)$19

6. Applications, scope, and open questions

The application range of PRMO is broad but structurally coherent. In the adaptive first-order setting, the illustrative experiment is WordNet embedding in the Poincaré ball $(\mathcal M_i,\rho^i)$20, using the transitive closure of WordNet nouns with approximately $(\mathcal M_i,\rho^i)$21 nodes and approximately $(\mathcal M_i,\rho^i)$22 edges. In that geometry, the metric factor is $(\mathcal M_i,\rho^i)$23, the Riemannian gradient is $(\mathcal M_i,\rho^i)$24, and exponential, logarithm, generalized addition, and parallel transport have closed forms. The reported empirical outcome is that Radam and Ramsgrad converge faster and to lower training loss than RSGD, attain higher Mean Average Precision in reconstruction and link prediction, and that full exponential-map updates yield the best final loss, whereas retraction-based versions converge faster in the early stage but slightly underperform in final MAP (Bécigneul et al., 2018).

In the preconditioned-metric literature, the main benchmark problems are CCA, truncated SVD, and tensor-ring completion. For CCA on $(\mathcal M_i,\rho^i)$25, the left preconditioner $(\mathcal M_i,\rho^i)$26, $(\mathcal M_i,\rho^i)$27 and the refined LR12 metric are derived from the Hessian block structure. The experiments report that unpreconditioned RGD took $(\mathcal M_i,\rho^i)$28 iterations at approximately $(\mathcal M_i,\rho^i)$29, whereas LR12-RGD took approximately $(\mathcal M_i,\rho^i)$30 iterations at $(\mathcal M_i,\rho^i)$31; unpreconditioned RCG took approximately $(\mathcal M_i,\rho^i)$32 iterations at approximately $(\mathcal M_i,\rho^i)$33, whereas LR12-RCG took approximately $(\mathcal M_i,\rho^i)$34 iterations at $(\mathcal M_i,\rho^i)$35. For TSVD, RGD(E) required $(\mathcal M_i,\rho^i)$36 iterations and $(\mathcal M_i,\rho^i)$37, while RGD(R12) required $(\mathcal M_i,\rho^i)$38 iterations and $(\mathcal M_i,\rho^i)$39; RCG(E) required $(\mathcal M_i,\rho^i)$40 iterations and $(\mathcal M_i,\rho^i)$41, while RCG(R12) required $(\mathcal M_i,\rho^i)$42 iterations and $(\mathcal M_i,\rho^i)$43. For TSVD, the reported condition number changes from approximately $(\mathcal M_i,\rho^i)$44 to approximately $(\mathcal M_i,\rho^i)$45 under the new metric. For tensor-ring completion, Gauss–Newton PRMO converges in fewer outer iterations but each step is more expensive, and is competitive for lower ranks (Gao et al., 2023).

The MIRS-aided ISAC application demonstrates a different interpretation of PRMO: joint optimization over a constructed product Riemannian manifold space, solved by penalty-based transformation and RBFGS. Simulation results are summarized as follows: the proposed MIRS outperforms conventional IRS in power minimization with both element-wise control and array-wise control; the minimum power is achieved by element-wise control, while array-wise control yields a suboptimal solution and higher computational efficiency. The array-wise formulation reduces the dimension of $(\mathcal M_i,\rho^i)$46 by $(\mathcal M_i,\rho^i)$47 and the number of minimum-distance constraints by approximately $(\mathcal M_i,\rho^i)$48, which greatly speeds up position updates (Geng et al., 5 Sep 2025).

Several limitations are explicit in the literature. The adaptive first-order theory requires product structure: intrinsic coordinate-wise adaptivity in a single general manifold remains elusive, because one must be able to identify factors along which adaptation is meaningful. Some regret terms scale with the number of factors $(\mathcal M_i,\rho^i)$49, parallel transport may be expensive on complicated manifolds, and curvature-dependent constants $(\mathcal M_i,\rho^i)$50 degrade bounds when curvature is large in magnitude. Open questions listed in the literature include combining adaptivity with Riemannian Nesterov-style acceleration and extending convergence guarantees to non-convex objectives (Bécigneul et al., 2018). A broader caution follows from the surveyed papers: PRMO is not a single universal algorithm, but a family of manifold-optimization constructions whose shared feature is a product geometry and whose concrete realizations range from adaptive first-order methods to metric-preconditioned gradient and conjugate-gradient methods, trust-region schemes, and penalty-based quasi-Newton methods.