Riemannian Augmented Lagrangian Method

• RALM is a constrained optimization method on Riemannian manifolds that extends classical Powell–Hestenes–Rockafellar techniques using intrinsic KKT conditions.
• It employs safeguarded multiplier updates, retractions, and vector transports to solve equality and inequality constraints while ensuring feasibility.
• Extensions of RALM address nonsmooth composite objectives via variable splitting and Moreau-envelope smoothing, yielding local linear convergence and oracle complexity bounds.

Riemannian Augmented Lagrangian Method (RALM) is a class of augmented Lagrangian methods for constrained optimization problems whose primal variable lies on a Riemannian manifold. In its intrinsic form, RALM extends Powell–Hestenes–Rockafellar augmented Lagrangians, manifold-adapted Karush–Kuhn–Tucker conditions, sequential optimality conditions, and weak constraint-qualification theory from nonlinear programming to optimization on smooth manifolds with equality and inequality constraints. Closely related variants also treat nonsmooth composite objectives through variable splitting and Moreau-envelope smoothing, establish local linear convergence under manifold variational sufficiency, and derive first-order oracle complexity bounds for inexact schemes (Andreani et al., 2023, Kangkang et al., 2019, Zhou et al., 2023, Xu et al., 2024).

1. Intrinsic formulation on Riemannian manifolds

A standard RALM setting considers a smooth, complete $n$-dimensional Riemannian manifold $(M,\langle\cdot,\cdot\rangle)$ with associated norm $\|\cdot\|$, together with smooth data

$$f: M \to \mathbb{R},\qquad h=(h_1,\dots,h_s):M\to\mathbb{R}^s,\qquad g=(g_1,\dots,g_m):M\to\mathbb{R}^m,$$

and the constrained Riemannian optimization problem

$$\begin{array}{ll} \text{Minimize} & f(x),\ \text{subject to} & h(x)=0,\quad g(x)\le 0,\quad x\in M. \end{array}$$

Its feasible set is

$\Omega := \{x\in M:\ h(x)=0,\ g(x)\le 0\}.$

The Riemannian gradient of a smooth scalar function $\phi$ is the unique tangent vector $\operatorname{grad}\phi(x)\in T_xM$ satisfying

$$\langle \operatorname{grad}\phi(x),\xi\rangle = d\phi(x)[\xi]\qquad \forall \xi\in T_xM.$$

For vector-valued constraints, Jacobians act on tangent vectors through the metric,

$$Dh(x)[\xi]=\big(\langle \operatorname{grad} h_1(x),\xi\rangle,\dots,\langle \operatorname{grad} h_s(x),\xi\rangle\big),$$

$(M,\langle\cdot,\cdot\rangle)$0

If $(M,\langle\cdot,\cdot\rangle)$1 is an embedded submanifold of $(M,\langle\cdot,\cdot\rangle)$2 with induced metric, the Riemannian gradient is the orthogonal projection of the Euclidean gradient onto $(M,\langle\cdot,\cdot\rangle)$3 (Andreani et al., 2023).

The intrinsic KKT system has the same algebraic structure as in Euclidean nonlinear programming, but all differential objects live in tangent spaces. A feasible point $(M,\langle\cdot,\cdot\rangle)$4 satisfies the KKT conditions if there exist multipliers $(M,\langle\cdot,\cdot\rangle)$5 with $(M,\langle\cdot,\cdot\rangle)$6 whenever $(M,\langle\cdot,\cdot\rangle)$7, such that

$(M,\langle\cdot,\cdot\rangle)$8

together with

$(M,\langle\cdot,\cdot\rangle)$9

At a feasible point $\|\cdot\|$0, the active set is $\|\cdot\|$1, the intrinsic linearization cone is

$\|\cdot\|$2

and its polar is

$\|\cdot\|$3

These constructions are central because the manifold-adapted constraint qualifications and convergence theory are stated directly in terms of tangent-space geometry rather than ambient Euclidean coordinates (Andreani et al., 2023).

2. Augmented Lagrangian construction and safeguarded iteration

The canonical intrinsic augmented Lagrangian in the smooth constrained setting is the Powell–Hestenes–Rockafellar form

$\|\cdot\|$4

where $\|\cdot\|$5, $\|\cdot\|$6, and $\|\cdot\|$7 is the componentwise Euclidean projection onto $\|\cdot\|$8. This expression implicitly includes the linear multiplier terms and quadratic penalties. Its Riemannian gradient is

$\|\cdot\|$9

The Euclidean norms in the penalty terms are taken in $$f: M \to \mathbb{R},\qquad h=(h_1,\dots,h_s):M\to\mathbb{R}^s,\qquad g=(g_1,\dots,g_m):M\to\mathbb{R}^m,$$0 and $$f: M \to \mathbb{R},\qquad h=(h_1,\dots,h_s):M\to\mathbb{R}^s,\qquad g=(g_1,\dots,g_m):M\to\mathbb{R}^m,$$1, but differentiation with respect to $$f: M \to \mathbb{R},\qquad h=(h_1,\dots,h_s):M\to\mathbb{R}^s,\qquad g=(g_1,\dots,g_m):M\to\mathbb{R}^m,$$2 is intrinsic (Andreani et al., 2023).

A standard safeguarded RALM iteration proceeds as follows. At iteration $$f: M \to \mathbb{R},\qquad h=(h_1,\dots,h_s):M\to\mathbb{R}^s,\qquad g=(g_1,\dots,g_m):M\to\mathbb{R}^m,$$3, given safeguarded multipliers $$f: M \to \mathbb{R},\qquad h=(h_1,\dots,h_s):M\to\mathbb{R}^s,\qquad g=(g_1,\dots,g_m):M\to\mathbb{R}^m,$$4 and penalty $$f: M \to \mathbb{R},\qquad h=(h_1,\dots,h_s):M\to\mathbb{R}^s,\qquad g=(g_1,\dots,g_m):M\to\mathbb{R}^m,$$5, one computes an approximate stationary point $$f: M \to \mathbb{R},\qquad h=(h_1,\dots,h_s):M\to\mathbb{R}^s,\qquad g=(g_1,\dots,g_m):M\to\mathbb{R}^m,$$6 of $$f: M \to \mathbb{R},\qquad h=(h_1,\dots,h_s):M\to\mathbb{R}^s,\qquad g=(g_1,\dots,g_m):M\to\mathbb{R}^m,$$7 so that

$$f: M \to \mathbb{R},\qquad h=(h_1,\dots,h_s):M\to\mathbb{R}^s,\qquad g=(g_1,\dots,g_m):M\to\mathbb{R}^m,$$8

using a Riemannian optimization routine such as gradient descent, conjugate gradients, or trust-region, together with retractions and vector transports. The unsafeguarded multiplier update is

$$f: M \to \mathbb{R},\qquad h=(h_1,\dots,h_s):M\to\mathbb{R}^s,\qquad g=(g_1,\dots,g_m):M\to\mathbb{R}^m,$$9

With

$$\begin{array}{ll} \text{Minimize} & f(x),\ \text{subject to} & h(x)=0,\quad g(x)\le 0,\quad x\in M. \end{array}$$0

the penalty is kept fixed if

$$\begin{array}{ll} \text{Minimize} & f(x),\ \text{subject to} & h(x)=0,\quad g(x)\le 0,\quad x\in M. \end{array}$$1

and otherwise increased by $$\begin{array}{ll} \text{Minimize} & f(x),\ \text{subject to} & h(x)=0,\quad g(x)\le 0,\quad x\in M. \end{array}$$2, where $$\begin{array}{ll} \text{Minimize} & f(x),\ \text{subject to} & h(x)=0,\quad g(x)\le 0,\quad x\in M. \end{array}$$3 and $$\begin{array}{ll} \text{Minimize} & f(x),\ \text{subject to} & h(x)=0,\quad g(x)\le 0,\quad x\in M. \end{array}$$4. Safeguarding then projects multipliers onto prescribed boxes,

$$\begin{array}{ll} \text{Minimize} & f(x),\ \text{subject to} & h(x)=0,\quad g(x)\le 0,\quad x\in M. \end{array}$$5

The safeguard uses the infeasibility measure $$\begin{array}{ll} \text{Minimize} & f(x),\ \text{subject to} & h(x)=0,\quad g(x)\le 0,\quad x\in M. \end{array}$$6 to control penalty escalation and ensure progress toward feasibility (Andreani et al., 2023).

The manifold-specific implementation enters through the update mechanism, not through the multiplier algebra. A retraction $$\begin{array}{ll} \text{Minimize} & f(x),\ \text{subject to} & h(x)=0,\quad g(x)\le 0,\quad x\in M. \end{array}$$7 satisfies $$\begin{array}{ll} \text{Minimize} & f(x),\ \text{subject to} & h(x)=0,\quad g(x)\le 0,\quad x\in M. \end{array}$$8 and $$\begin{array}{ll} \text{Minimize} & f(x),\ \text{subject to} & h(x)=0,\quad g(x)\le 0,\quad x\in M. \end{array}$$9; a vector transport $\Omega := \{x\in M:\ h(x)=0,\ g(x)\le 0\}.$0 transports tangent vectors along $\Omega := \{x\in M:\ h(x)=0,\ g(x)\le 0\}.$1. For second-order subproblem solvers, the Riemannian Hessian of $\Omega := \{x\in M:\ h(x)=0,\ g(x)\le 0\}.$2 involves covariant derivatives and, in embedded settings, curvature corrections via Weingarten terms. Typical stopping conditions monitor stationarity and feasibility through quantities such as

$\Omega := \{x\in M:\ h(x)=0,\ g(x)\le 0\}.$3

When $\Omega := \{x\in M:\ h(x)=0,\ g(x)\le 0\}.$4 is Euclidean, gradients become Euclidean gradients, retractions become identity updates, and vector transports are trivial, so the intrinsic construction recovers classical Euclidean ALM exactly (Andreani et al., 2023).

3. Constraint qualifications, sequential stationarity, and global convergence

A distinctive feature of the intrinsic theory is its systematic transfer of weak Euclidean constraint qualifications to the manifold setting. At a feasible point $\Omega := \{x\in M:\ h(x)=0,\ g(x)\le 0\}.$5, with

$\Omega := \{x\in M:\ h(x)=0,\ g(x)\le 0\}.$6

the framework defines manifold versions of LICQ, MFCQ, CRCQ, CPLD, RCRCQ, RCPLD, CRSC, and quasinormality. LICQ requires linear independence of equality gradients together with active inequality gradients. MFCQ is stated as positive-linear independence. CRCQ and RCRCQ impose local constant-rank conditions. CPLD and RCPLD use positive-linear dependence at the reference point and nearby linear dependence. CRSC is formulated through the set

$\Omega := \{x\in M:\ h(x)=0,\ g(x)\le 0\}.$7

and quasinormality rules out nonzero abnormal multipliers satisfying specific sign-compatibility conditions along nearby infeasible points (Andreani et al., 2023).

The logical relations among these conditions are strict in several directions: LICQ $\Omega := \{x\in M:\ h(x)=0,\ g(x)\le 0\}.$8 CRCQ, LICQ $\Omega := \{x\in M:\ h(x)=0,\ g(x)\le 0\}.$9 MFCQ, CRCQ $\phi$0 RCRCQ, MFCQ $\phi$1 CPLD, CRCQ $\phi$2 CPLD, CPLD $\phi$3 RCPLD, and RCRCQ $\phi$4 RCPLD. CRSC is weaker than RCRCQ and MFCQ and independent of RCPLD. Quasinormality is strictly weaker than CPLD and independent of RCPLD/CRSC, yet it implies boundedness of certain dual sequences. This hierarchy is important because the strongest global convergence statements are phrased under RCPLD or CRSC, while multiplier boundedness is recovered under the weaker quasinormality assumption (Andreani et al., 2023).

The convergence analysis is organized around sequential optimality conditions. A feasible point is an AKKT point if there exist sequences $\phi$5 with $\phi$6, $\phi$7, and $\phi$8 for large $\phi$9 whenever $\operatorname{grad}\phi(x)\in T_xM$0. A PAKKT point adds sign-control conditions on scaled multipliers, and Scaled-PAKKT relaxes stationarity to

$\operatorname{grad}\phi(x)\in T_xM$1

Every local minimizer is both AKKT and PAKKT. For the safeguarded RALM itself, any feasible accumulation point of $\operatorname{grad}\phi(x)\in T_xM$2 is AKKT, and the dual sequence generated by the multiplier update supplies the witnessing AKKT multipliers. Moreover, feasible accumulation points satisfy PAKKT, or Scaled-PAKKT if the subproblems are solved only to scaled tolerances (Andreani et al., 2023).

These sequential conditions yield the main intrinsic convergence theorems. If an accumulation point is AKKT and satisfies RCPLD or CRSC, then it satisfies the KKT conditions, even if the Lagrange multiplier set at that point is unbounded. If a feasible accumulation point satisfies quasinormality and is PAKKT, then the associated dual sequence is bounded; every dual limit point is a Lagrange multiplier, and multiplier convergence follows. The paper summarizes the result informally as strong global convergence: under smoothness of $\operatorname{grad}\phi(x)\in T_xM$3 and completeness of $\operatorname{grad}\phi(x)\in T_xM$4, feasible accumulation points produced by safeguarded RALM are KKT points under LICQ, MFCQ, CRCQ, CPLD, RCRCQ, RCPLD, or CRSC, while under quasinormality the method also guarantees bounded duals and convergence of multipliers without assuming boundedness of the multiplier set a priori (Andreani et al., 2023).

4. Nonsmooth composite and split-variable extensions

A major extension of RALM addresses nonsmooth composite objectives by separating manifold geometry from nonsmoothness. In the manifold inexact augmented Lagrangian method (MIALM), the problem is

$\operatorname{grad}\phi(x)\in T_xM$5

where $\operatorname{grad}\phi(x)\in T_xM$6 is an embedded Riemannian submanifold of a Euclidean space, $\operatorname{grad}\phi(x)\in T_xM$7 is smooth, $\operatorname{grad}\phi(x)\in T_xM$8 is linear, and $\operatorname{grad}\phi(x)\in T_xM$9 is convex and locally Lipschitz. Introducing an auxiliary Euclidean variable $$\langle \operatorname{grad}\phi(x),\xi\rangle = d\phi(x)[\xi]\qquad \forall \xi\in T_xM.$$0 yields the separable constrained formulation

$$\langle \operatorname{grad}\phi(x),\xi\rangle = d\phi(x)[\xi]\qquad \forall \xi\in T_xM.$$1

with augmented Lagrangian

$$\langle \operatorname{grad}\phi(x),\xi\rangle = d\phi(x)[\xi]\qquad \forall \xi\in T_xM.$$2

Minimizing over $$\langle \operatorname{grad}\phi(x),\xi\rangle = d\phi(x)[\xi]\qquad \forall \xi\in T_xM.$$3 gives the exact proximal update

$$\langle \operatorname{grad}\phi(x),\xi\rangle = d\phi(x)[\xi]\qquad \forall \xi\in T_xM.$$4

and the reduced smooth objective

$$\langle \operatorname{grad}\phi(x),\xi\rangle = d\phi(x)[\xi]\qquad \forall \xi\in T_xM.$$5

Thus each outer iteration decomposes into a smooth Riemannian subproblem in $$\langle \operatorname{grad}\phi(x),\xi\rangle = d\phi(x)[\xi]\qquad \forall \xi\in T_xM.$$6, an exact Euclidean proximal step in $$\langle \operatorname{grad}\phi(x),\xi\rangle = d\phi(x)[\xi]\qquad \forall \xi\in T_xM.$$7, and a Euclidean multiplier update. Under local first-order inexactness and LICQ at a cluster point, the algorithm converges to a KKT point of the separable problem; under global inexactness and $$\langle \operatorname{grad}\phi(x),\xi\rangle = d\phi(x)[\xi]\qquad \forall \xi\in T_xM.$$8, any limit point is a global minimizer of the separable reformulation, and $$\langle \operatorname{grad}\phi(x),\xi\rangle = d\phi(x)[\xi]\qquad \forall \xi\in T_xM.$$9 is a global minimizer of the original composite problem (Kangkang et al., 2019).

A related inexact framework studies

$$Dh(x)[\xi]=\big(\langle \operatorname{grad} h_1(x),\xi\rangle,\dots,\langle \operatorname{grad} h_s(x),\xi\rangle\big),$$0

with $$Dh(x)[\xi]=\big(\langle \operatorname{grad} h_1(x),\xi\rangle,\dots,\langle \operatorname{grad} h_s(x),\xi\rangle\big),$$1 a smooth embedded Riemannian submanifold, $$Dh(x)[\xi]=\big(\langle \operatorname{grad} h_1(x),\xi\rangle,\dots,\langle \operatorname{grad} h_s(x),\xi\rangle\big),$$2 continuously differentiable, $$Dh(x)[\xi]=\big(\langle \operatorname{grad} h_1(x),\xi\rangle,\dots,\langle \operatorname{grad} h_s(x),\xi\rangle\big),$$3 smooth, and $$Dh(x)[\xi]=\big(\langle \operatorname{grad} h_1(x),\xi\rangle,\dots,\langle \operatorname{grad} h_s(x),\xi\rangle\big),$$4 convex, Lipschitz continuous, and proximable. After introducing $$Dh(x)[\xi]=\big(\langle \operatorname{grad} h_1(x),\xi\rangle,\dots,\langle \operatorname{grad} h_s(x),\xi\rangle\big),$$5 and the equality constraint $$Dh(x)[\xi]=\big(\langle \operatorname{grad} h_1(x),\xi\rangle,\dots,\langle \operatorname{grad} h_s(x),\xi\rangle\big),$$6, the augmented Lagrangian becomes

$$Dh(x)[\xi]=\big(\langle \operatorname{grad} h_1(x),\xi\rangle,\dots,\langle \operatorname{grad} h_s(x),\xi\rangle\big),$$7

Eliminating $$Dh(x)[\xi]=\big(\langle \operatorname{grad} h_1(x),\xi\rangle,\dots,\langle \operatorname{grad} h_s(x),\xi\rangle\big),$$8 leads to the differentiable inner objective

$$Dh(x)[\xi]=\big(\langle \operatorname{grad} h_1(x),\xi\rangle,\dots,\langle \operatorname{grad} h_s(x),\xi\rangle\big),$$9

whose Euclidean gradient is

$(M,\langle\cdot,\cdot\rangle)$00

The algorithm uses Riemannian gradient descent for the $(M,\langle\cdot,\cdot\rangle)$01-subproblem, the exact proximal update

$(M,\langle\cdot,\cdot\rangle)$02

and the classical dual step

$(M,\langle\cdot,\cdot\rangle)$03

This split-variable construction shows that, in many nonsmooth settings, the manifold part of RALM is concentrated entirely in the smooth primal subproblem, while multipliers and proximal maps remain Euclidean (Xu et al., 2024).

5. Local rates, generalized Hessians, and oracle complexity

The local convergence theory for nonsmooth manifold problems introduces the manifold variational sufficient condition (MVSC). For

$(M,\langle\cdot,\cdot\rangle)$04

with $(M,\langle\cdot,\cdot\rangle)$05 continuously differentiable and $(M,\langle\cdot,\cdot\rangle)$06 proper closed convex, the augmented Lagrangian is

$(M,\langle\cdot,\cdot\rangle)$07

The strong MVSC is shown to be equivalent to the manifold strong second-order sufficient condition in certain circumstances. More precisely, if $(M,\langle\cdot,\cdot\rangle)$08 is polyhedral convex, strong MVSC is equivalent to M-SSOSC; if $(M,\langle\cdot,\cdot\rangle)$09 is the indicator of a second-order cone or positive semidefinite cone, strong MVSC is equivalent to the corresponding second-order condition involving the support function of the second-order tangent set. Under MVSC, one can construct a local dual problem, apply the Euclidean proximal point algorithm to that dual, and obtain a linear convergence rate for RALM without imposing uniqueness of the multiplier. The same framework also shows that, under suitable assumptions, M-SSOSC is equivalent to nonsingularity of the generalized Hessian of the augmented Lagrangian, which is the property needed by semismooth Newton-type methods for the primal subproblems (Zhou et al., 2023).

Complexity theory for inexact RALM has focused on Moreau-smoothed composite models. In deterministic settings, ManIAL solves

$(M,\langle\cdot,\cdot\rangle)$10

on a compact embedded submanifold by combining geometric penalty growth with Riemannian gradient descent on the smooth inner subproblems. With carefully chosen penalty parameters and inner termination criteria, ManIAL attains an $(M,\langle\cdot,\cdot\rangle)$11 oracle complexity result, matching the best-known complexity result. In stochastic expectation settings, StoManIAL replaces deterministic inner descent by a Riemannian recursive momentum method and attains an oracle complexity of $(M,\langle\cdot,\cdot\rangle)$12, improving on the previously best-known $(M,\langle\cdot,\cdot\rangle)$13 result (Deng et al., 2024).

For general nonlinear $(M,\langle\cdot,\cdot\rangle)$14, the Riemannian inexact augmented Lagrangian method (RiAL) establishes, for the first time, the oracle complexity of a Riemannian inexact augmented Lagrangian method with the classical dual update. Using Riemannian gradient descent with the stopping condition

$(M,\langle\cdot,\cdot\rangle)$15

the method finds an $(M,\langle\cdot,\cdot\rangle)$16-stationary point with $(M,\langle\cdot,\cdot\rangle)$17 calls to the first-order oracle. The analysis emphasizes that the classical dual stepsize is crucial to the high efficiency of the method: the update

$(M,\langle\cdot,\cdot\rangle)$18

implies $(M,\langle\cdot,\cdot\rangle)$19, and since $(M,\langle\cdot,\cdot\rangle)$20 is Lipschitz, the duals remain bounded by the Lipschitz constant while feasibility decreases geometrically with $(M,\langle\cdot,\cdot\rangle)$21 (Xu et al., 2024).

6. Geometric realizations and application domains

On standard manifolds, practical RALM implementations use established manifold primitives. For the sphere $(M,\langle\cdot,\cdot\rangle)$22, the tangent space is $(M,\langle\cdot,\cdot\rangle)$23, the Riemannian gradient is the Euclidean gradient projected onto $(M,\langle\cdot,\cdot\rangle)$24, and a standard retraction is $(M,\langle\cdot,\cdot\rangle)$25. For the Stiefel manifold $(M,\langle\cdot,\cdot\rangle)$26, the tangent space is $(M,\langle\cdot,\cdot\rangle)$27, the tangent projection is $(M,\langle\cdot,\cdot\rangle)$28, and QR-based or polar retractions are typical. Grassmann optimization is handled through Stiefel representatives and a quotient metric. These recipes allow RALM to combine manifold constraints handled intrinsically with additional equality and inequality constraints handled by the augmented Lagrangian (Andreani et al., 2023).

A structure-preserving control application is $(M,\langle\cdot,\cdot\rangle)$29-optimal reduction of positive networks. There the reduced dynamics matrix is parametrized as

$(M,\langle\cdot,\cdot\rangle)$30

with $(M,\langle\cdot,\cdot\rangle)$31, $(M,\langle\cdot,\cdot\rangle)$32 positive definite, so stability is preserved by construction. Equality constraints enforce the zero pattern inherited from a clustering-based initialization, and inequality constraints enforce off-diagonal Metzler structure and elementwise nonnegativity of $(M,\langle\cdot,\cdot\rangle)$33 and $(M,\langle\cdot,\cdot\rangle)$34. In the reported experiment, the clustering baseline achieved $(M,\langle\cdot,\cdot\rangle)$35 and $(M,\langle\cdot,\cdot\rangle)$36, whereas RALM achieved $(M,\langle\cdot,\cdot\rangle)$37 and $(M,\langle\cdot,\cdot\rangle)$38 while preserving stability, positivity, and interconnection structure (Misawa et al., 2021).

Low-rank semidefinite programming provides another major specialization. The decomposition augmented Lagrangian method for low-rank SDP factors the semidefinite variable as $(M,\langle\cdot,\cdot\rangle)$39, embeds part of the linear structure into a manifold

$(M,\langle\cdot,\cdot\rangle)$40

and handles nonsmooth regularization and remaining linear constraints by an augmented Lagrangian with splitting. Each subproblem is then solved by a regularized Riemannian semismooth Newton method. On max-cut with cutting planes, the method solved 100% successfully on the Gset, with max node size $(M,\langle\cdot,\cdot\rangle)$41 and largest g81 solved in $(M,\langle\cdot,\cdot\rangle)$42 minutes; SDPNAL+ solved 72.2% and was typically $(M,\langle\cdot,\cdot\rangle)$43 slower (Wang et al., 2021).

Doubly nonnegative relaxations of mixed-binary quadratic programs have motivated several recent RALM designs. RNNAL is a globally convergent Riemannian augmented Lagrangian method that penalizes nonnegativity and complementarity constraints while preserving the remaining constraints as an algebraic variety. After a modified Burer–Monteiro factorization, the feasible set becomes

$(M,\langle\cdot,\cdot\rangle)$44

and the metric projection retraction is reduced to a convex optimization problem under regularity conditions. The method serves as a prototype algorithm for solving general DNN problems and reports scalability up to $(M,\langle\cdot,\cdot\rangle)$45 of order $(M,\langle\cdot,\cdot\rangle)$46 in numerical experiments (Hou et al., 19 Feb 2025). RiNNAL+ sharpens this direction by proving that the DNN relaxation with matrix dimension $(M,\langle\cdot,\cdot\rangle)$47 is equivalent to an SDP-RLT relaxation with smaller matrix dimension $(M,\langle\cdot,\cdot\rangle)$48, and by alternating between a low-rank Riemannian phase and a single projected-gradient rank-update phase. Reported results include up to $(M,\langle\cdot,\cdot\rangle)$49 speedup over SDPNAL+ on BIQ-S, successful solution of $(M,\langle\cdot,\cdot\rangle)$50 instances in 18 minutes where SDPNAL+ failed, and 40–100× speedups on medium-large $(M,\langle\cdot,\cdot\rangle)$51 instances (Hou et al., 18 Jul 2025).

Optimal transport furnishes a different type of manifold-constrained RALM. For the projection robust Wasserstein distance, ReALM reformulates the problem on $(M,\langle\cdot,\cdot\rangle)$52 with nonlinear inequality constraints and uses an exponential augmented Lagrangian

$(M,\langle\cdot,\cdot\rangle)$53

Its inner solver, iRBBS, combines inexact Riemannian Barzilai–Borwein steps on the Stiefel manifold with a flexible number of Sinkhorn iterations. The paper shows that iRBBS can return an $(M,\langle\cdot,\cdot\rangle)$54-stationary point of the original PRW distance problem within $(M,\langle\cdot,\cdot\rangle)$55 iterations, and reports that iRBBS with moderate inner accuracy is 5–12× faster on average than R(A)BCD while ReALM is about 2–5× faster than continuation or penalty-only variants (Jiang et al., 2022).

RALM has also been developed on possibly infinite-dimensional shape manifolds. In stochastic shape optimization, the augmented Lagrangian is

$(M,\langle\cdot,\cdot\rangle)$56

where projection onto a cone $(M,\langle\cdot,\cdot\rangle)$57 unifies equality and inequality constraints. The inner loop uses randomized mini-batch Riemannian SGD with random stopping, and the outer loop updates multipliers through cone projection. Under boundedness and smoothness assumptions, bounded multiplier sequences yield manifold KKT limit points; if multipliers diverge, feasible limit points satisfy AKKT. The method was demonstrated on a multi-shape optimization problem with geometric constraints in a Riemannian shape manifold (Geiersbach et al., 2023).

7. Scope, misconceptions, and open problems

A recurring misconception is that RALM requires boundedness of the set of Lagrange multipliers to prove convergence. The intrinsic global theory explicitly avoids this requirement: under RCPLD or CRSC, feasible accumulation points are KKT even when the multiplier set is unbounded, and under quasinormality the dual sequence itself is proved bounded and convergent (Andreani et al., 2023). A second misconception is that “Riemannian” implies manifold-valued multipliers. In essentially all of the formulations summarized above, the manifold structure is confined to the primal variable; multipliers live in Euclidean spaces and are updated by classical ALM rules, projections, or proximal maps (Kangkang et al., 2019, Xu et al., 2024).

Another point of clarification concerns terminology. In ReALM for projection robust Wasserstein distance, “exponential” refers to the exponential penalty in the augmented Lagrangian, not to use of the manifold exponential map for the primal step. The manifold step is implemented by a retraction, specifically QR-retraction in the experiments (Jiang et al., 2022). More generally, many successful RALM implementations rely on retractions rather than exact geodesics, which is consistent with both the convergence theory and the complexity analyses.

The current limitations are also well defined in the literature. The intrinsic smooth-constrained theory does not provide complexity guarantees or rates; those depend on the chosen Riemannian subproblem solver (Andreani et al., 2023). In the manifold extension of weak Euclidean CQ theory, error bounds under CRSC or RCPLD on manifolds are conjectured and remain open, as do second-order optimality conditions under relaxed CQs and invariance questions for quadratic forms (Andreani et al., 2023). In low-rank DNN methods, rank selection remains delicate, and constraint relaxations that ensure smoothness may enlarge the lifted dimension (Hou et al., 19 Feb 2025). In SDP-RLT methods, high-rank solutions reduce the advantage of low-rank manifold phases because projected-gradient rank updates and eigendecompositions become dominant (Hou et al., 18 Jul 2025). In the local nonsmooth theory, the analysis is fundamentally local and depends on neighborhoods where MVSC holds; extending equivalence between primal proximal point schemes and dual ALM under manifold retractional convexity remains open (Zhou et al., 2023).

Taken together, these results place RALM at the intersection of nonlinear programming, manifold optimization, nonsmooth analysis, and low-rank conic optimization. Its defining pattern is stable across variants: preserve manifold feasibility intrinsically, shift nonsmooth or hard side constraints into an augmented Lagrangian, solve smooth or smoothed manifold subproblems with geometry-aware methods, and recover global or local guarantees through sequential stationarity, weak constraint qualifications, variational sufficiency, or first-order complexity analysis.