---
title: Riemannian Augmented Lagrangian Method
url: https://www.emergentmind.com/topics/riemannian-augmented-lagrangian-method-ralm
type: topic
---

# Riemannian Augmented Lagrangian Method

Riemannian Augmented Lagrangian Method (RALM) is a class of augmented Lagrangian methods for constrained optimization problems whose primal variable lies on a Riemannian manifold. In its intrinsic form, RALM extends Powell–Hestenes–Rockafellar augmented Lagrangians, manifold-adapted Karush–Kuhn–Tucker conditions, sequential optimality conditions, and weak constraint-qualification theory from nonlinear programming to optimization on smooth manifolds with equality and inequality constraints. Closely related variants also treat nonsmooth composite objectives through variable splitting and Moreau-envelope smoothing, establish local linear convergence under manifold variational sufficiency, and derive first-order oracle complexity bounds for inexact schemes [2306.14345; 1911.09900; 2308.06793; 2410.00482].

## 1. Intrinsic formulation on Riemannian manifolds

A standard RALM setting considers a smooth, complete \(n\)-dimensional Riemannian manifold \((M,\langle\cdot,\cdot\rangle)\) with associated norm \(\|\cdot\|\), together with smooth data
\[
f: M \to \mathbb{R},\qquad h=(h_1,\dots,h_s):M\to\mathbb{R}^s,\qquad g=(g_1,\dots,g_m):M\to\mathbb{R}^m,
\]
and the constrained Riemannian optimization problem
\[
\begin{array}{ll}
\text{Minimize} & f(x),\\
\text{subject to} & h(x)=0,\quad g(x)\le 0,\quad x\in M.
\end{array}
\]
Its feasible set is
\[
\Omega := \{x\in M:\ h(x)=0,\ g(x)\le 0\}.
\]
The Riemannian gradient of a smooth scalar function \(\phi\) is the unique tangent vector \(\operatorname{grad}\phi(x)\in T_xM\) satisfying
\[
\langle \operatorname{grad}\phi(x),\xi\rangle = d\phi(x)[\xi]\qquad \forall \xi\in T_xM.
\]
For vector-valued constraints, Jacobians act on tangent vectors through the metric,
\[
Dh(x)[\xi]=\big(\langle \operatorname{grad} h_1(x),\xi\rangle,\dots,\langle \operatorname{grad} h_s(x),\xi\rangle\big),
\]
\[
Dg(x)[\xi]=\big(\langle \operatorname{grad} g_1(x),\xi\rangle,\dots,\langle \operatorname{grad} g_m(x),\xi\rangle\big).
\]
If \(M\) is an embedded submanifold of \(\mathbb{R}^n\) with induced metric, the Riemannian gradient is the orthogonal projection of the Euclidean gradient onto \(T_xM\) [2306.14345].

The intrinsic KKT system has the same algebraic structure as in Euclidean nonlinear programming, but all differential objects live in tangent spaces. A feasible point \(x^*\in\Omega\) satisfies the KKT conditions if there exist multipliers \((\lambda^*,\mu^*)\in\mathbb{R}^s\times\mathbb{R}_+^m\) with \(\mu_j^*=0\) whenever \(g_j(x^*)<0\), such that
\[
\operatorname{grad} f(x^*)+\sum_{i=1}^s \lambda_i^*\,\operatorname{grad} h_i(x^*)
+\sum_{j=1}^m \mu_j^*\,\operatorname{grad} g_j(x^*)=0,
\]
together with
\[
h(x^*)=0,\qquad g(x^*)\le 0,\qquad \mu^*\ge 0,\qquad \mu_j^*g_j(x^*)=0\ \forall j.
\]
At a feasible point \(x\), the active set is \(\mathcal{A}(x):=\{j:g_j(x)=0\}\), the intrinsic linearization cone is
\[
\mathcal{L}(x):=\Big\{\xi\in T_xM:\ \langle \operatorname{grad} h_i(x),\xi\rangle=0,\ \forall i;\ \langle \operatorname{grad} g_j(x),\xi\rangle\le 0,\ \forall j\in\mathcal{A}(x)\Big\},
\]
and its polar is
\[
\mathcal{L}(x)^\circ=\Big\{v\in T_xM:\ v=\sum_{i=1}^s \lambda_i\,\operatorname{grad} h_i(x)+\sum_{j\in\mathcal{A}(x)} \mu_j\,\operatorname{grad} g_j(x),\ \mu_j\ge 0\Big\}.
\]
These constructions are central because the manifold-adapted constraint qualifications and convergence theory are stated directly in terms of tangent-space geometry rather than ambient Euclidean coordinates [2306.14345].

## 2. Augmented Lagrangian construction and safeguarded iteration

The canonical intrinsic augmented Lagrangian in the smooth constrained setting is the Powell–Hestenes–Rockafellar form
\[
\mathcal{L}_\rho(x,\lambda,\mu)
:= f(x)+\frac{\rho}{2}\left\|h(x)+\frac{\lambda}{\rho}\right\|^2
+\frac{\rho}{2}\left\|\left[g(x)+\frac{\mu}{\rho}\right]_+\right\|^2,
\]
where \(\rho>0\), \((\lambda,\mu)\in\mathbb{R}^s\times\mathbb{R}_+^m\), and \([\cdot]_+\) is the componentwise Euclidean projection onto \(\mathbb{R}_+^m\). This expression implicitly includes the linear multiplier terms and quadratic penalties. Its Riemannian gradient is
\[
\operatorname{grad}\mathcal{L}_\rho(x,\lambda,\mu)
=\operatorname{grad} f(x)
+\sum_{i=1}^s(\lambda_i+\rho h_i(x))\,\operatorname{grad} h_i(x)
+\sum_{j=1}^m \rho\left[g_j(x)+\frac{\mu_j}{\rho}\right]_+\,\operatorname{grad} g_j(x).
\]
The Euclidean norms in the penalty terms are taken in \(\mathbb{R}^s\) and \(\mathbb{R}^m\), but differentiation with respect to \(x\) is intrinsic [2306.14345].

A standard safeguarded RALM iteration proceeds as follows. At iteration \(k\), given safeguarded multipliers \((\bar\lambda^k,\bar\mu^k)\) and penalty \(\rho_k\), one computes an approximate stationary point \(x^k\in M\) of \(\mathcal{L}_{\rho_k}(\cdot,\bar\lambda^k,\bar\mu^k)\) so that
\[
\big\|\operatorname{grad}\mathcal{L}_{\rho_k}(x^k,\bar\lambda^k,\bar\mu^k)\big\|\le \epsilon_k,
\]
using a Riemannian optimization routine such as gradient descent, conjugate gradients, or trust-region, together with retractions and vector transports. The unsafeguarded multiplier update is
\[
\lambda^k:=\bar\lambda^k+\rho_k h(x^k),\qquad
\mu^k:=\big[\bar\mu^k+\rho_k g(x^k)\big]_+.
\]
With
\[
V^k:=\frac{\mu^k-\bar\mu^k}{\rho_k},
\]
the penalty is kept fixed if
\[
\max\big\{\|h(x^k)\|_2,\|V^k\|_2\big\}
\le \tau \max\big\{\|h(x^{k-1})\|_2,\|V^{k-1}\|_2\big\},
\]
and otherwise increased by \(\rho_{k+1}:=\gamma \rho_k\), where \(\tau\in[0,1)\) and \(\gamma>1\). Safeguarding then projects multipliers onto prescribed boxes,
\[
\bar\lambda^{k+1}:=\Pi_{[\lambda_{\min},\lambda_{\max}]^s}(\lambda^k),\qquad
\bar\mu^{k+1}:=\Pi_{[0,\mu_{\max}]^m}(\mu^k).
\]
The safeguard uses the infeasibility measure \(\max\{\|h(x)\|,\|V\|\}\) to control penalty escalation and ensure progress toward feasibility [2306.14345].

The manifold-specific implementation enters through the update mechanism, not through the multiplier algebra. A retraction \(R_x:T_xM\to M\) satisfies \(R_x(0_x)=x\) and \(DR_x(0_x)=\mathrm{id}_{T_xM}\); a vector transport \(\mathcal{T}_\eta:T_xM\to T_{R_x(\eta)}M\) transports tangent vectors along \(\eta\). For second-order subproblem solvers, the Riemannian Hessian of \(\mathcal{L}_\rho\) involves covariant derivatives and, in embedded settings, curvature corrections via Weingarten terms. Typical stopping conditions monitor stationarity and feasibility through quantities such as
\[
\|\operatorname{grad}\mathcal{L}_{\rho_k}(x^k,\bar\lambda^k,\bar\mu^k)\|\le \varepsilon_g,\qquad
\|h(x^k)\|\le \varepsilon_h,\qquad
\|g(x^k)_+\|\le \varepsilon_c.
\]
When \(M\) is Euclidean, gradients become Euclidean gradients, retractions become identity updates, and vector transports are trivial, so the intrinsic construction recovers classical Euclidean ALM exactly [2306.14345].

## 3. Constraint qualifications, sequential stationarity, and global convergence

A distinctive feature of the intrinsic theory is its systematic transfer of weak Euclidean constraint qualifications to the manifold setting. At a feasible point \(x\), with
\[
A(x,\mathcal{I},\mathcal{J})
:=\{\operatorname{grad} h_i(x):i\in\mathcal{I}\}\cup \{\operatorname{grad} g_j(x):j\in\mathcal{J}\},
\]
the framework defines manifold versions of LICQ, MFCQ, CRCQ, CPLD, RCRCQ, RCPLD, CRSC, and quasinormality. LICQ requires linear independence of equality gradients together with active inequality gradients. MFCQ is stated as positive-linear independence. CRCQ and RCRCQ impose local constant-rank conditions. CPLD and RCPLD use positive-linear dependence at the reference point and nearby linear dependence. CRSC is formulated through the set
\[
\mathcal{J}_-(x):=\{j\in\mathcal{A}(x): -\operatorname{grad}g_j(x)\in \mathcal{L}(x)^\circ\},
\]
and quasinormality rules out nonzero abnormal multipliers satisfying specific sign-compatibility conditions along nearby infeasible points [2306.14345].

The logical relations among these conditions are strict in several directions:
**LICQ \(\Rightarrow\) CRCQ**, **LICQ \(\Rightarrow\) MFCQ**, **CRCQ \(\Rightarrow\) RCRCQ**, **MFCQ \(\Rightarrow\) CPLD**, **CRCQ \(\Rightarrow\) CPLD**, **CPLD \(\Rightarrow\) RCPLD**, and **RCRCQ \(\Rightarrow\) RCPLD**. CRSC is weaker than RCRCQ and MFCQ and independent of RCPLD. Quasinormality is strictly weaker than CPLD and independent of RCPLD/CRSC, yet it implies boundedness of certain dual sequences. This hierarchy is important because the strongest global convergence statements are phrased under RCPLD or CRSC, while multiplier boundedness is recovered under the weaker quasinormality assumption [2306.14345].

The convergence analysis is organized around sequential optimality conditions. A feasible point is an AKKT point if there exist sequences \((x^k,\lambda^k,\mu^k)\) with \(x^k\to x\), \(\operatorname{grad}L(x^k,\lambda^k,\mu^k)\to 0\), and \(\mu_j^k=0\) for large \(k\) whenever \(j\notin\mathcal{A}(x)\). A PAKKT point adds sign-control conditions on scaled multipliers, and Scaled-PAKKT relaxes stationarity to
\[
\frac{\|\operatorname{grad}L(x^k,\lambda^k,\mu^k)\|}{\gamma_k}\to 0,
\qquad \gamma_k:=\|(1,\lambda^k,\mu^k)\|_\infty.
\]
Every local minimizer is both AKKT and PAKKT. For the safeguarded RALM itself, any feasible accumulation point of \((x^k)\) is AKKT, and the dual sequence generated by the multiplier update supplies the witnessing AKKT multipliers. Moreover, feasible accumulation points satisfy PAKKT, or Scaled-PAKKT if the subproblems are solved only to scaled tolerances [2306.14345].

These sequential conditions yield the main intrinsic convergence theorems. If an accumulation point is AKKT and satisfies RCPLD or CRSC, then it satisfies the KKT conditions, even if the Lagrange multiplier set at that point is unbounded. If a feasible accumulation point satisfies quasinormality and is PAKKT, then the associated dual sequence is bounded; every dual limit point is a Lagrange multiplier, and multiplier convergence follows. The paper summarizes the result informally as strong global convergence: under smoothness of \(f,h,g\) and completeness of \(M\), feasible accumulation points produced by safeguarded RALM are KKT points under LICQ, MFCQ, CRCQ, CPLD, RCRCQ, RCPLD, or CRSC, while under quasinormality the method also guarantees bounded duals and convergence of multipliers without assuming boundedness of the multiplier set a priori [2306.14345].

## 4. Nonsmooth composite and split-variable extensions

A major extension of RALM addresses nonsmooth composite objectives by separating manifold geometry from nonsmoothness. In the manifold inexact augmented Lagrangian method (MIALM), the problem is
\[
\min_{X\in\mathcal{M}} \ F(X):=f(X)+g(AX),
\]
where \(\mathcal{M}\) is an embedded Riemannian submanifold of a Euclidean space, \(f\) is smooth, \(A\) is linear, and \(g\) is convex and locally Lipschitz. Introducing an auxiliary Euclidean variable \(Y\) yields the separable constrained formulation
\[
\min_{X\in\mathcal{M},\,Y\in\mathbb{E}} f(X)+g(Y)\qquad \text{s.t.}\quad AX=Y,
\]
with augmented Lagrangian
\[
\mathcal{L}_\rho(X,Y;Z)
:=f(X)+g(Y)-\langle Z,AX-Y\rangle+\frac{\rho}{2}\|AX-Y\|_F^2.
\]
Minimizing over \(Y\) gives the exact proximal update
\[
Y^*(X;Z)=\operatorname{prox}_{\frac{1}{\rho}g}\left(AX-\frac{1}{\rho}Z\right)
\]
and the reduced smooth objective
\[
\psi_Z(X):=f(X)+M_{\frac{1}{\rho}g}\left(AX-\frac{1}{\rho}Z\right)-\frac{1}{2\rho}\|Z\|_F^2.
\]
Thus each outer iteration decomposes into a smooth Riemannian subproblem in \(X\), an exact Euclidean proximal step in \(Y\), and a Euclidean multiplier update. Under local first-order inexactness and LICQ at a cluster point, the algorithm converges to a KKT point of the separable problem; under global inexactness and \(\epsilon_{\min}=0\), any limit point is a global minimizer of the separable reformulation, and \(X^*\) is a global minimizer of the original composite problem [1911.09900].

A related inexact framework studies
\[
\min_{x\in\mathcal{M}} \ \Phi(x):=f(x)+h(A(x)),
\]
with \(\mathcal{M}\) a smooth embedded Riemannian submanifold, \(f\) continuously differentiable, \(A\) smooth, and \(h\) convex, Lipschitz continuous, and proximable. After introducing \(y\) and the equality constraint \(A(x)-y=0\), the augmented Lagrangian becomes
\[
\mathcal{L}_\sigma(x,y;z)=f(x)+h(y)+\langle z,A(x)-y\rangle+\frac{\sigma}{2}\|A(x)-y\|^2.
\]
Eliminating \(y\) leads to the differentiable inner objective
\[
L_k(x)=f(x)+M_{h/\sigma_k}\!\left(A(x)+\frac{z_k}{\sigma_k}\right),
\]
whose Euclidean gradient is
\[
\nabla L_k(x)=\nabla f(x)+\sigma_k\nabla A(x)^\top\!\left(A(x)+\frac{z_k}{\sigma_k}
-\operatorname{prox}_{h/\sigma_k}\!\left(A(x)+\frac{z_k}{\sigma_k}\right)\right).
\]
The algorithm uses Riemannian gradient descent for the \(x\)-subproblem, the exact proximal update
\[
y_{k+1}=\operatorname{prox}_{h/\sigma_k}\!\left(A(x_{k+1})+\frac{z_k}{\sigma_k}\right),
\]
and the classical dual step
\[
z_{k+1}=z_k+\sigma_k\big(A(x_{k+1})-y_{k+1}\big).
\]
This split-variable construction shows that, in many nonsmooth settings, the manifold part of RALM is concentrated entirely in the smooth primal subproblem, while multipliers and proximal maps remain Euclidean [2410.00482].

## 5. Local rates, generalized Hessians, and oracle complexity

The local convergence theory for nonsmooth manifold problems introduces the manifold variational sufficient condition (MVSC). For
\[
\min_{x\in M} f(x)+\theta(g(x)),
\]
with \(f,g\) continuously differentiable and \(\theta\) proper closed convex, the augmented Lagrangian is
\[
\ell_\rho(x,y):=\inf_{u\in Y}\Big\{f(x)+\theta(g(x)+u)-(y,u)+\frac{\rho}{2}|u|^2\Big\}.
\]
The strong MVSC is shown to be equivalent to the manifold strong second-order sufficient condition in certain circumstances. More precisely, if \(\theta\) is polyhedral convex, strong MVSC is equivalent to M-SSOSC; if \(\theta\) is the indicator of a second-order cone or positive semidefinite cone, strong MVSC is equivalent to the corresponding second-order condition involving the support function of the second-order tangent set. Under MVSC, one can construct a local dual problem, apply the Euclidean proximal point algorithm to that dual, and obtain a linear convergence rate for RALM without imposing uniqueness of the multiplier. The same framework also shows that, under suitable assumptions, M-SSOSC is equivalent to nonsingularity of the generalized Hessian of the augmented Lagrangian, which is the property needed by semismooth Newton-type methods for the primal subproblems [2308.06793].

Complexity theory for inexact RALM has focused on Moreau-smoothed composite models. In deterministic settings, ManIAL solves
\[
\min_{x\in\mathcal{M}} f(x)+h(Ax)
\]
on a compact embedded submanifold by combining geometric penalty growth with Riemannian gradient descent on the smooth inner subproblems. With carefully chosen penalty parameters and inner termination criteria, ManIAL attains an \(\mathcal{O}(\epsilon^{-3})\) oracle complexity result, matching the best-known complexity result. In stochastic expectation settings, StoManIAL replaces deterministic inner descent by a Riemannian recursive momentum method and attains an oracle complexity of \(\tilde{\mathcal{O}}(\epsilon^{-3.5})\), improving on the previously best-known \(\mathcal{O}(\epsilon^{-4})\) result [2404.05121].

For general nonlinear \(A(x)\), the Riemannian inexact augmented Lagrangian method (RiAL) establishes, for the first time, the oracle complexity of a Riemannian inexact augmented Lagrangian method with the classical dual update. Using Riemannian gradient descent with the stopping condition
\[
\|\operatorname{grad}L_k(x_{k,t})\|\le \varepsilon_k,
\]
the method finds an \(\varepsilon\)-stationary point with \(\mathcal{O}(\varepsilon^{-3})\) calls to the first-order oracle. The analysis emphasizes that the classical dual stepsize is crucial to the high efficiency of the method: the update
\[
z_{k+1}=z_k+\sigma_k(A(x_{k+1})-y_{k+1})
\]
implies \(z_{k+1}\in \partial h(y_{k+1})\), and since \(h\) is Lipschitz, the duals remain bounded by the Lipschitz constant while feasibility decreases geometrically with \(\sigma_k\) [2410.00482].

## 6. Geometric realizations and application domains

On standard manifolds, practical RALM implementations use established manifold primitives. For the sphere \(S^{n-1}\), the tangent space is \(T_xS^{n-1}=\{\xi:\langle x,\xi\rangle=0\}\), the Riemannian gradient is the Euclidean gradient projected onto \(T_x\), and a standard retraction is \(R_x(\xi)=(x+\xi)/\|x+\xi\|\). For the Stiefel manifold \(\mathrm{St}(p,n)\), the tangent space is \(T_X\mathrm{St}=\{Z:X^\top Z+Z^\top X=0\}\), the tangent projection is \(\Pi_{T_X}(G)=G-X\,\mathrm{sym}(X^\top G)\), and QR-based or polar retractions are typical. Grassmann optimization is handled through Stiefel representatives and a quotient metric. These recipes allow RALM to combine manifold constraints handled intrinsically with additional equality and inequality constraints handled by the augmented Lagrangian [2306.14345].

A structure-preserving control application is \(H^2\)-optimal reduction of positive networks. There the reduced dynamics matrix is parametrized as
\[
A_r=(J_r-R_r)Q_r,
\]
with \(J_r\in\mathrm{Skew}_r\), \(R_r,Q_r\in\mathrm{Sym}_r\) positive definite, so stability is preserved by construction. Equality constraints enforce the zero pattern inherited from a clustering-based initialization, and inequality constraints enforce off-diagonal Metzler structure and elementwise nonnegativity of \(B_r\) and \(C_r\). In the reported experiment, the clustering baseline achieved \(\mathrm{Err}_2=75.7\%\) and \(\mathrm{Err}_\infty=70.24\%\), whereas RALM achieved \(\mathrm{Err}_2=3.14\%\) and \(\mathrm{Err}_\infty=4.67\%\) while preserving stability, positivity, and interconnection structure [2112.15389].

Low-rank semidefinite programming provides another major specialization. The decomposition augmented Lagrangian method for low-rank SDP factors the semidefinite variable as \(X=R^\top R\), embeds part of the linear structure into a manifold
\[
\mathcal{M}:=\{R\in\mathbb{R}^{p\times n}: \mathcal{B}(R^\top R)=b_0\},
\]
and handles nonsmooth regularization and remaining linear constraints by an augmented Lagrangian with splitting. Each subproblem is then solved by a regularized Riemannian semismooth Newton method. On max-cut with cutting planes, the method solved 100% successfully on the Gset, with max node size \(n=20000\) and largest g81 solved in \(\sim 10.5\) minutes; SDPNAL+ solved 72.2% and was typically \(>10\times\) slower [2109.11707].

Doubly nonnegative relaxations of mixed-binary quadratic programs have motivated several recent RALM designs. RNNAL is a globally convergent Riemannian augmented Lagrangian method that penalizes nonnegativity and complementarity constraints while preserving the remaining constraints as an algebraic variety. After a modified Burer–Monteiro factorization, the feasible set becomes
\[
\mathcal{M}_r=\{R\in\mathbb{R}^{n\times r}: AR=be_1^\top,\ \operatorname{diag}_B(RR^\top)=R_Be_1\},
\]
and the metric projection retraction is reduced to a convex optimization problem under regularity conditions. The method serves as a prototype algorithm for solving general DNN problems and reports scalability up to \(n\) of order \(10^4\) in numerical experiments [2502.13849]. RiNNAL+ sharpens this direction by proving that the DNN relaxation with matrix dimension \((n+l+1)\) is equivalent to an SDP-RLT relaxation with smaller matrix dimension \((n+1)\), and by alternating between a low-rank Riemannian phase and a single projected-gradient rank-update phase. Reported results include up to \(180\times\) speedup over SDPNAL+ on BIQ-S, successful solution of \(n=5000\) instances in 18 minutes where SDPNAL+ failed, and 40–100× speedups on medium-large \(\theta_+\) instances [2507.13776].

Optimal transport furnishes a different type of manifold-constrained RALM. For the projection robust Wasserstein distance, ReALM reformulates the problem on \(\mathbb{R}^n\times\mathbb{R}^n\times \mathrm{St}(k,d)\) with nonlinear inequality constraints and uses an exponential augmented Lagrangian
\[
\widetilde L_\eta(x,y,\pi)
= r^\top\alpha+c^\top\beta+y+\eta\sum_{i,j}\pi_{ij}\exp\!\left(-\frac{\varphi_{ij}(x)+y}{\eta}\right).
\]
Its inner solver, iRBBS, combines inexact Riemannian Barzilai–Borwein steps on the Stiefel manifold with a flexible number of Sinkhorn iterations. The paper shows that iRBBS can return an \(\epsilon\)-stationary point of the original PRW distance problem within \(\mathcal{O}(\epsilon^{-3})\) iterations, and reports that iRBBS with moderate inner accuracy is 5–12× faster on average than R(A)BCD while ReALM is about 2–5× faster than continuation or penalty-only variants [2211.13386].

RALM has also been developed on possibly infinite-dimensional shape manifolds. In stochastic shape optimization, the augmented Lagrangian is
\[
\mathcal{L}_\rho(x,\lambda):=f(x)+\frac{\rho}{2}\operatorname{dist}_K(h̃(x)+\lambda)^2,
\]
where projection onto a cone \(K\) unifies equality and inequality constraints. The inner loop uses randomized mini-batch Riemannian SGD with random stopping, and the outer loop updates multipliers through cone projection. Under boundedness and smoothness assumptions, bounded multiplier sequences yield manifold KKT limit points; if multipliers diverge, feasible limit points satisfy AKKT. The method was demonstrated on a multi-shape optimization problem with geometric constraints in a Riemannian shape manifold [2303.17404].

## 7. Scope, misconceptions, and open problems

A recurring misconception is that RALM requires boundedness of the set of Lagrange multipliers to prove convergence. The intrinsic global theory explicitly avoids this requirement: under RCPLD or CRSC, feasible accumulation points are KKT even when the multiplier set is unbounded, and under quasinormality the dual sequence itself is proved bounded and convergent [2306.14345]. A second misconception is that “Riemannian” implies manifold-valued multipliers. In essentially all of the formulations summarized above, the manifold structure is confined to the primal variable; multipliers live in Euclidean spaces and are updated by classical ALM rules, projections, or proximal maps [1911.09900; 2410.00482].

Another point of clarification concerns terminology. In ReALM for projection robust Wasserstein distance, “exponential” refers to the exponential penalty in the augmented Lagrangian, not to use of the manifold exponential map for the primal step. The manifold step is implemented by a retraction, specifically QR-retraction in the experiments [2211.13386]. More generally, many successful RALM implementations rely on retractions rather than exact geodesics, which is consistent with both the convergence theory and the complexity analyses.

The current limitations are also well defined in the literature. The intrinsic smooth-constrained theory does not provide complexity guarantees or rates; those depend on the chosen Riemannian subproblem solver [2306.14345]. In the manifold extension of weak Euclidean CQ theory, error bounds under CRSC or RCPLD on manifolds are conjectured and remain open, as do second-order optimality conditions under relaxed CQs and invariance questions for quadratic forms [2306.14345]. In low-rank DNN methods, rank selection remains delicate, and constraint relaxations that ensure smoothness may enlarge the lifted dimension [2502.13849]. In SDP-RLT methods, high-rank solutions reduce the advantage of low-rank manifold phases because projected-gradient rank updates and eigendecompositions become dominant [2507.13776]. In the local nonsmooth theory, the analysis is fundamentally local and depends on neighborhoods where MVSC holds; extending equivalence between primal proximal point schemes and dual ALM under manifold retractional convexity remains open [2308.06793].

Taken together, these results place RALM at the intersection of nonlinear programming, manifold optimization, nonsmooth analysis, and low-rank conic optimization. Its defining pattern is stable across variants: preserve manifold feasibility intrinsically, shift nonsmooth or hard side constraints into an augmented Lagrangian, solve smooth or smoothed manifold subproblems with geometry-aware methods, and recover global or local guarantees through sequential stationarity, weak constraint qualifications, variational sufficiency, or first-order complexity analysis.

Source: https://www.emergentmind.com/topics/riemannian-augmented-lagrangian-method-ralm