---
title: Product Riemannian Manifold Optimization
url: https://www.emergentmind.com/topics/product-riemannian-manifold-optimization-prmo
type: topic
---

# Product Riemannian Manifold Optimization

Product Riemannian Manifold Optimization (PRMO) denotes optimization in which the decision variable lies on a Cartesian product of manifolds and the optimization method exploits that product structure in the metric, gradient, transport, and update rules. In the arXiv literature, the term is used in several closely related senses: adaptive first-order optimization on \(\mathcal M=\mathcal M_1\times\cdots\times\mathcal M_n\), optimization on product manifolds under a preconditioned metric, optimization on products of generalized Stiefel manifolds with a non-standard metric, and application-specific manifold algorithms such as penalty-based RBFGS on mixed Euclidean/oblique/circle product spaces. Across these usages, the common premise is that tangent spaces split factor-wise and that manifold operations can be organized blockwise, while the chosen metric materially affects convergence behavior [1810.00760] [2306.08873] [1902.01635] [2509.04873].

## 1. Product-manifold structure and geometric preliminaries

A standard PRMO setting begins with complete Riemannian manifolds \((\mathcal M_i,\rho^i)\), \(i=1,\dots,n\), and forms the product manifold
\[
\mathcal M=\mathcal M_1\times\cdots\times\mathcal M_n.
\]
In the adaptive-optimization formulation, the product metric is written as
\[
\rho_x\bigl((u^1,\dots,u^n),(v^1,\dots,v^n)\bigr)=\sum_{i=1}^n \rho^i_{x^i}(u^i,v^i),
\]
and the tangent space decomposes as
\[
T_x\mathcal M=\bigoplus_{i=1}^n T_{x^i}\mathcal M_i.
\]
Distance, exponential map, logarithm, and parallel transport act factor-wise. In the embedded-manifold framework, the same product structure is expressed as \(M:=M_1\times\cdots\times M_K\subset E_1\times\cdots\times E_K\), with
\[
T_xM=T_{x_1}M_1\times\cdots\times T_{x_K}M_K,
\]
and the standard product metric
\[
g_x^{(e)}(\xi,\eta)=\sum_{k=1}^K \langle \xi_k,\eta_k\rangle_{E_k}.
\]
These formulations are equivalent in emphasizing separability at the manifold level [1810.00760] [2306.08873].

Geodesic convexity is the principal convexity notion used in the first-order theory. If \(\mathcal X_i\subset \mathcal M_i\) is geodesically convex, then \(\mathcal X=\mathcal X_1\times\cdots\times\mathcal X_n\) is the feasible set. A differentiable \(f:\mathcal M\to\mathbb R\) is geodesically convex on \(\mathcal X\) iff for every \(x,y\in\mathcal X\),
\[
f(x)-f(y)\le \rho_x\bigl(\mathrm{grad}\,f(x),-\log_x(y)\bigr).
\]
This replaces linear convexity by a relation between the Riemannian gradient and the inverse exponential map. A plausible implication is that PRMO inherits much of the formal structure of block-coordinate Euclidean optimization, but only after replacing vector-space operations by their manifold counterparts.

## 2. Adaptive first-order PRMO on \(\mathcal M_1\times\cdots\times\mathcal M_n\)

In "Riemannian Adaptive Optimization Methods" [1810.00760], PRMO is the setting in which adaptive schemes such as Adagrad, Adam, and Amsgrad are generalized to products of manifolds, with adaptivity implemented across manifolds in the cartesian product. Writing \(x_t=(x_t^1,\dots,x_t^n)\), \(g_t=\mathrm{grad}\,f_t(x_t)\), and \(g_t^i\in T_{x_t^i}\mathcal M_i\), the Riemannian Adagrad update is
\[
x_{t+1}^i=\mathrm{Exp}_{x_t^i}^i\!\Bigl(-\alpha_t\,\frac{g_t^i}{\sqrt{\sum_{k=1}^t \|g_k^i\|_{x_k^i}^2}}\Bigr).
\]
For Riemannian Adam, one introduces
\[
m_t^i=\beta_1\,m_{t-1}^i+(1-\beta_1)\,g_t^i,\qquad
v_t^i=\beta_2\,v_{t-1}^i+(1-\beta_2)\,\|g_t^i\|_{x_t^i}^2,
\]
and updates
\[
x_{t+1}^i=\mathrm{Exp}_{x_t^i}^i\!\Bigl(-\alpha_t\,\frac{m_t^i}{\sqrt{v_t^i}}\Bigr).
\]
Riemannian Amsgrad keeps \(\hat v_t^i=\max(\hat v_{t-1}^i,v_t^i)\) and uses \(\sqrt{\hat v_t^i}\) in the denominator; when constraints are present, the iterate is projected back to \(\mathcal X_i\).

The nontrivial step is momentum transport. On a general manifold, \(m_{t-1}^i\in T_{x_{t-1}^i}\mathcal M_i\) must be moved to \(T_{x_t^i}\mathcal M_i\) before it can be combined with \(g_t^i\). The update therefore becomes
\[
m_t^i=\beta_{1t}\,P^i_{x_{t-1}^i\to x_t^i}(m_{t-1}^i)+(1-\beta_{1t})\,g_t^i,
\]
where \(P^i\) is parallel transport. Because \(P^i\) is an isometry, norms and inner products are preserved. This guarantees that adaptive momentum and variance estimates stay in the correct tangent spaces.

The convergence theory is stated for geodesically complete manifolds with sectional curvature \(\ge \kappa_i\le 0\), geodesically convex feasible sets of diameter \(D_\infty\), uniformly bounded gradients \(\|g_t^i\|_{x_t^i}\le G_\infty\), and geodesically convex losses \(f_t\). Regret is
\[
R_T=\sum_{t=1}^T f_t(x_t)-\min_{x\in\mathcal X}\sum_{t=1}^T f_t(x).
\]
For Ramsgrad with \(\alpha_t=\alpha/\sqrt t\) and \(\gamma=\beta_1/\sqrt{\beta_2}<1\), the regret bound contains the curvature factor \(\zeta(\kappa_i,D_\infty)\). If all \(\kappa_i=0\), the method recovers Euclidean Amsgrad bounds; setting \(\beta_{1t}=0\) yields Riemannian Adagrad with regret \(\mathcal O\bigl(D_\infty\sqrt{\sum_{i,t}\|g_t^i\|^2}\bigr)\). The paper explicitly states that the generalization is tight: choosing Euclidean space as Riemannian manifold yields the same algorithms and regret bounds as the standard Euclidean methods.

## 3. Preconditioned metrics as a PRMO acceleration mechanism

"Optimization on product manifolds under a preconditioned metric" [2306.08873] treats PRMO as a metric-design framework. The central claim is that local linear convergence of first-order Riemannian methods is governed by the condition number of the Riemannian Hessian at a minimizer,
\[
\kappa_g(\mathrm{Hess}_g f(x^*))=\frac{\lambda_{\max}(\mathrm{Hess}_g f(x^*))}{\lambda_{\min}(\mathrm{Hess}_g f(x^*))},
\]
and that one can accelerate optimization by endowing the product manifold with a preconditioned metric
\[
g_x(\xi,\eta)=\langle \xi,\,P(x)[\eta]\rangle,
\]
where \(P(x)\) is a self-adjoint, uniformly SPD linear operator approximating the diagonal blocks of the Riemannian Hessian under the standard product metric.

Three preconditioner-design strategies are proposed. Exact block-diagonal preconditioning sets \(P_k(x)=H_{kk}(x)\), the restriction of the Hessian along the \(k\)-th factor. Left and right preconditioning exploit factorized diagonal blocks \(H_{kk}=L_k(x)R_k(x)\) or \(R_k(x)L_k(x)\), using either \(P_k=L_k\), \(P_k=R_k\), or a symmetric choice \(P_k=\frac12(L_kR_k+R_kL_k)\). Gauss–Newton type preconditioning applies when \(f(x)=\frac12\|F(x)\|^2\), setting
\[
P(x)=(DF(x))^\top DF(x),
\]
which induces the metric
\[
g_x(\xi,\eta)=\langle DF(x)[\xi],\,DF(x)[\eta]\rangle.
\]

Under this metric, the Riemannian gradient becomes
\[
\mathrm{grad}_g f(x)=\Pi_{g,x}\bigl(P(x)^{-1}[\nabla \bar f(x)]\bigr),
\]
with factor-wise expression
\[
\bigl(\Pi_{g^1,x_1}(P_1(x)^{-1}[\partial_1 f(x)]),\dots,\Pi_{g^K,x_K}(P_K(x)^{-1}[\partial_K f(x)])\bigr).
\]
The Hessian is
\[
\mathrm{Hess}_g f(x)[\eta]
=
\Pi_{g,x}\bigl(P(x)^{-1}(D^2\bar f(x)[\eta])+(DP(x)^{-1}[\eta])(\nabla\bar f(x))\bigr),
\]
while retractions and vector transports remain blockwise:
\[
R_x(\eta)=\bigl(R_{x_1}^1(\eta_1),\dots,R_{x_K}^K(\eta_K)\bigr),\qquad
T_{y\leftarrow x}(\eta)=\Pi_{g,y}(\eta).
\]

This framework leads directly to PRMO-RGD and PRMO-RCG. Their local linear rate is stated as
\[
f(x^t)-f(x^*)\le (1-c/\kappa_g)\,\bigl(f(x^{t-1})-f(x^*)\bigr),
\]
for some \(c>0\). The principal theorem further states that if \(P\approx H\), then \(\kappa_g\approx 1\). This suggests that, in PRMO, the metric is not an incidental geometric choice but an algorithmic object analogous to a Euclidean preconditioner.

## 4. PRMO on products of generalized Stiefel manifolds

"Riemannian optimization with a preconditioning scheme on the generalized Stiefel manifold" [1902.01635] develops the geometric components needed when the feasible set is a product of generalized Stiefel manifolds. A single generalized Stiefel factor is
\[
\mathrm{St}_B(p,n):=\{X\in\mathbb R^{n\times p}: X^\top B X=I_p\},
\]
with tangent space
\[
T_X\mathrm{St}_B(p,n)=\{Z\in\mathbb R^{n\times p}: Z^\top B X+X^\top B Z=0_p\}.
\]
For \(k\) blocks \(X_1,\dots,X_k\), the feasible set is
\[
M=\mathrm{St}_{B_1}(p_1,n_1)\times\cdots\times \mathrm{St}_{B_k}(p_k,n_k).
\]
The paper then equips each tangent space with the nonstandard metric
\[
g_X(\xi,\eta)=\mathrm{Tr}(\xi^\top M_X \eta),
\]
where \(M_X=M(X)\succeq 0\) is the preconditioner.

If \(\bar f\) is an ambient extension of the cost, the ambient gradient under this metric is
\[
\mathrm{grad}_e\,\bar f(X)=M_X^{-1}\nabla\bar f(X),
\]
and the manifold gradient is
\[
\mathrm{grad}\,f(X)=\Pi_X\bigl(M_X^{-1}\nabla\bar f(X)\bigr).
\]
With \(Z=M_X^{-1}\nabla\bar f(X)\), the projector is given by
\[
\Pi_X(Z)=Z-M_X^{-1}BX\,S_Z,
\]
where \(S_Z\) solves the Sylvester equation
\[
(X^\top B M_X^{-1} B X)S_Z + S_Z(X^\top B M_X^{-1} B X)
=
X^\top B Z + (X^\top B Z)^\top.
\]
For constant \(M_X\equiv M\), the Hessian is expressed through the projected ambient Hessian and the Weingarten term; the paper gives the simplified formula
\[
\mathrm{Hess}\,f(X)[\eta]
=
\Pi_X\!\bigl(M^{-1}\nabla^2\bar f(X)\eta - M^{-1}B\eta\,(X^\top\nabla\bar f(X)-X^\top M\,\mathrm{grad}\,f(X))\bigr).
\]

Retractions are given explicitly. The polar-based retraction is
\[
R_X(\xi)=(X+\xi)(I+\xi^\top B \xi)^{-1/2},
\]
and the \(B\)-QR-based retraction is
\[
R_X(\xi)=qf_B(X+\xi)=B^{-1}\,qf\bigl(B^{1}(X+\xi)\bigr).
\]
A simple vector transport is projection after lift,
\[
\tau_{\eta_X}(\xi_X):=\Pi_{R_X(\eta_X)}(\xi_X).
\]
The paper states linear convergence for gradient descent under uniform Hessian conditioning, local superlinear convergence for conjugate gradient under exact line-search, and local quadratic or superlinear convergence for trust-region methods with a second-order retraction. Its numerical examples include generalized eigenvalue subspace computation and CCA; on the MEDIAMILL dataset, the full preconditioner \(M=\Sigma\) yields approximately \(10\times\) fewer iterations than \(M=I\), and condition-number estimates at the optimum are reported as \(\kappa\approx 4.0\) for \(M=\Sigma\) versus \(\kappa\approx 60\) for \(M=I\).

## 5. Penalty-based PRMO and RBFGS on mixed product manifolds

In "Movable IRS-Aided ISAC Systems: Joint Beamforming and Position Optimization" [2509.04873], PRMO is instantiated for a mixed-variable constrained problem whose decision variables are the transmit beamforming matrix \(W\in\mathbb C^{M\times(K_c+M)}\), the receive filter matrix \(V=[v_1,\dots,v_{K_t}]\in\mathbb C^{M(M+K_c)\times K_t}\), the IRS phase-shift vector \(\phi=[e^{j\phi_1},\dots,e^{j\phi_N}]^\top\in\mathbb C^N\), and the array-wise position-parameter vector \(\tilde u\in\mathbb R^{2N/a^2}\). The objective is
\[
\min_{W,V,\phi,\tilde u} f(X)=\|W\|_F^2,
\]
subject to communication-rate constraints, sensing-SINR constraints, unit-norm receive filters, constant-modulus phase shifts, region confinement through the sigmoid projection \(u=p(\tilde u)=A\cdot \mathrm{sig}(\tilde u)\), and minimum element-distance constraints.

The feasible set is the product manifold
\[
M=\mathbb C^{M\times(K_c+M)} \times M_V \times M_\phi \times \mathbb R^{2N/a^2},
\]
where
\[
M_V=\{V\in\mathbb C^{M(M+K_c)\times K_t}\mid \mathrm{diag}(V^\mathrm H V)=I_{K_t}\},
\]
and
\[
M_\phi=\{\phi\in\mathbb C^N\mid |\phi_n|=1,\ \forall n\}.
\]
The tangent space constraints are
\[
T_V M_V=\{\zeta_V\mid \mathrm{diag}(V^\mathrm H\zeta_V)=0\},
\qquad
T_\phi M_\phi=\{\zeta_\phi\mid \mathrm{Re}(\zeta_\phi^*\circ\phi)=0\}.
\]
All inequality constraints \(h_i(X)\le 0\) are absorbed into the smooth penalty objective
\[
g(X)=f(X)+\rho\sum_{i\in I} P(h_i(X),u),
\]
where \(P(x,u)\) is the linear-quadratic smoothing of \(\max\{0,x\}\):
\[
P(x,u)=0\ \text{if }x\le 0,\qquad
P(x,u)=x^2/(2u)\ \text{if }0<x\le u,\qquad
P(x,u)=x-u/2\ \text{if }x>u.
\]

The Riemannian metric is the product of Euclidean metrics,
\[
\langle \zeta_X,\zeta_X'\rangle
=
\mathrm{Re}\,\mathrm{Tr}(\zeta_W^\mathrm H\zeta_W')
+
\mathrm{Re}\,\mathrm{Tr}(\zeta_V^\mathrm H\zeta_V')
+
\mathrm{Re}(\zeta_\phi^\mathrm H\zeta_\phi')
+
\zeta_{\tilde u}^\top \zeta_{\tilde u}'.
\]
Retractions are blockwise: \(W\leftarrow W+\alpha d_W\), \(V\) is column-renormalized, \(\phi\leftarrow (\phi+\alpha d_\phi)\circ/|\phi+\alpha d_\phi|\), and \(\tilde u\leftarrow \tilde u+\alpha d_{\tilde u}\). The Riemannian gradient is obtained by projecting Euclidean gradients onto each block tangent space, including
\[
\mathrm{grad}_V g=\nabla_{V^*}g - V\cdot \mathrm{Re}\,\mathrm{ddiag}(V^\mathrm H\nabla_{V^*}g),
\]
and
\[
\mathrm{grad}_\phi g=\nabla_{\phi^*}g-\mathrm{Re}(\nabla_{\phi^*}g\circ\phi^*)\circ\phi.
\]

The search direction is computed by RBFGS,
\[
d_X^l=-H_X^l[\mathrm{grad}_X g(X^l)],
\]
followed by Armijo line search and retraction. The cautious update condition is
\[
\langle s_X^l,y_X^l\rangle \ge 10^{-4}\,\langle s_X^l,s_X^l\rangle \cdot \|\mathrm{grad}\,g(X^l)\|,
\]
after which the inverse-Hessian approximation is updated blockwise. In practice, a limited-memory two-loop recursion is used. The outer exact-penalty loop increases \(\rho\) if any \(h_i(X^{l+1})>0\), while reducing the smoothing and tolerance parameters. The paper states that the Armijo line search guarantees monotone decrease of \(g(X^l)\) and that, for sufficiently large \(\rho\) and small \(u\), the limit point satisfies all original inequalities. Its overall cost is reported as
\[
O\!\Bigl(M^{3.5}(K_c+M)^{3.5} + I_1I_2 [ M^4K_t + N^3 + K_c^4M + K_t^2M^3 + K_t^2N^2M + S(M^2K_t+K_cK_tM+N) ] \Bigr).
\]

## 6. Applications, scope, and open questions

The application range of PRMO is broad but structurally coherent. In the adaptive first-order setting, the illustrative experiment is WordNet embedding in the Poincaré ball \(\mathbb D^n\), using the transitive closure of WordNet nouns with approximately \(82\,\mathrm{k}\) nodes and approximately \(743\,\mathrm{k}\) edges. In that geometry, the metric factor is \(\lambda_x=2/(1-\|x\|^2)\), the Riemannian gradient is \(\mathrm{grad}\,f(x)=\frac1{\lambda_x^2}\nabla_E f(x)\), and exponential, logarithm, generalized addition, and parallel transport have closed forms. The reported empirical outcome is that Radam and Ramsgrad converge faster and to lower training loss than RSGD, attain higher Mean Average Precision in reconstruction and link prediction, and that full exponential-map updates yield the best final loss, whereas retraction-based versions converge faster in the early stage but slightly underperform in final MAP [1810.00760].

In the preconditioned-metric literature, the main benchmark problems are CCA, truncated SVD, and tensor-ring completion. For CCA on \(M=\mathrm{St}_{\Sigma_{xx}}(m,d_x)\times \mathrm{St}_{\Sigma_{yy}}(m,d_y)\), the left preconditioner \(P_1=\Sigma_{xx}\), \(P_2=\Sigma_{yy}\) and the refined LR12 metric are derived from the Hessian block structure. The experiments report that unpreconditioned RGD took \(10\,000\) iterations at approximately \(249\,\mathrm{s}\), whereas LR12-RGD took approximately \(6\,600\) iterations at \(195\,\mathrm{s}\); unpreconditioned RCG took approximately \(1\,745\) iterations at approximately \(31\,\mathrm{s}\), whereas LR12-RCG took approximately \(410\) iterations at \(15\,\mathrm{s}\). For TSVD, RGD(E) required \(7781\) iterations and \(117\,\mathrm{s}\), while RGD(R12) required \(387\) iterations and \(3.4\,\mathrm{s}\); RCG(E) required \(478\) iterations and \(5.4\,\mathrm{s}\), while RCG(R12) required \(105\) iterations and \(1.45\,\mathrm{s}\). For TSVD, the reported condition number changes from approximately \(2430\) to approximately \(95\) under the new metric. For tensor-ring completion, Gauss–Newton PRMO converges in fewer outer iterations but each step is more expensive, and is competitive for lower ranks [2306.08873].

The MIRS-aided ISAC application demonstrates a different interpretation of PRMO: joint optimization over a constructed product Riemannian manifold space, solved by penalty-based transformation and RBFGS. Simulation results are summarized as follows: the proposed MIRS outperforms conventional IRS in power minimization with both element-wise control and array-wise control; the minimum power is achieved by element-wise control, while array-wise control yields a suboptimal solution and higher computational efficiency. The array-wise formulation reduces the dimension of \(\tilde u\) by \(1/a^2\) and the number of minimum-distance constraints by approximately \(1/a^4\), which greatly speeds up position updates [2509.04873].

Several limitations are explicit in the literature. The adaptive first-order theory requires product structure: intrinsic coordinate-wise adaptivity in a single general manifold remains elusive, because one must be able to identify factors along which adaptation is meaningful. Some regret terms scale with the number of factors \(n\), parallel transport may be expensive on complicated manifolds, and curvature-dependent constants \(\zeta(\kappa_i,D_\infty)\) degrade bounds when curvature is large in magnitude. Open questions listed in the literature include combining adaptivity with Riemannian Nesterov-style acceleration and extending convergence guarantees to non-convex objectives [1810.00760]. A broader caution follows from the surveyed papers: PRMO is not a single universal algorithm, but a family of manifold-optimization constructions whose shared feature is a product geometry and whose concrete realizations range from adaptive first-order methods to metric-preconditioned gradient and conjugate-gradient methods, trust-region schemes, and penalty-based quasi-Newton methods.

Source: https://www.emergentmind.com/topics/product-riemannian-manifold-optimization-prmo