Alternating Riemannian Manifold Optimization

• ARMO is defined as a framework that alternately optimizes variable subsets on Riemannian manifolds while ensuring adherence to intrinsic constraints like orthogonality and norm preservation.
• Methodologies employ blockwise schemes, augmented Lagrangian approaches, and manifold-specific operations such as tangent space projections and retractions to guarantee feasibility and convergence.
• Practical applications include robust matrix decomposition, sparse PCA, and RIS-aided channel estimation, showcasing improved computational efficiency and solution accuracy.

Alternating Riemannian Manifold Optimization (ARMO) encompasses algorithmic strategies that exploit the geometric structure of Riemannian manifolds by alternately optimizing over subsets of variables or problem blocks, each subject to manifold constraints. These approaches enable efficient solution of high-dimensional and structured optimization problems encountered in signal processing, machine learning, and applied mathematics, where intrinsic geometric or non-Euclidean constraints play a central role.

1. Fundamental Principles and Motivation

ARMO refers to the class of algorithms in which optimization alternates between different variable blocks, each living on (potentially distinct) Riemannian manifolds or subject to manifold-induced constraints. The core premise is to enforce structural feasibility (e.g., orthogonality, norm constraints, or more general nonlinear constraints) via manifold operations such as tangent space projection and retraction, while decomposing the problem into tractable subproblems amenable to efficient update schemes.

Manifold constraints naturally arise in applications such as eigenvalue computation, principal component analysis (PCA), matrix factorization, dictionary learning, and optimization for reconfigurable intelligent surfaces in wireless communications. In many modern instances, optimization problems combine smooth and nonsmooth components, high-dimensionality, and additional side constraints (such as sparsity, nonnegativity, or simplex constraints), further necessitating specialized optimization mechanisms that can maintain feasibility and scalability.

2. Prototypical Algorithms and Methodological Frameworks

A representative taxonomy of ARMO approaches, classified by their alternating structure and Riemannian mechanisms, includes:

• Blockwise Alternating and Proximal Schemes: These decompose the problem into blocks (e.g., low-rank and sparse parts in robust matrix completion (Huang et al., 2020)), alternately applying Riemannian (geometric) updates to one block while holding others fixed, then switching roles. Each block's update may employ first- or second-order manifold gradient steps, proximal mappings for nonsmooth terms, or closed-form truncations (such as SVDs for subspace blocks (Goyens et al., 2021)).
• Augmented Lagrangian and Penalty-Based Methods: Intrinsic extensions of the augmented Lagrangian (ALM) and exact penalty methods maintain manifold feasibility throughout the ADMM or penalty iteration by replacing Euclidean gradient and Hessian steps with their Riemannian analogs—ensuring that extrinsic constraints such as equality and inequality are handled via penalty or augmented terms, while intrinsic feasibility is retained by geometric operations (Liu et al., 2019).
• Regularized Newton and Trust-Region Schemes: Regularized second-order methods construct local quadratic models in the ambient Euclidean space, supplemented by regularization terms for model control and convergence guarantees, and use retraction mappings to project updates onto the manifold (Hu et al., 2017). Constrained trust-region subproblems are approximately solved on the tangent space, with specific mechanisms for detecting and exploiting negative curvature.
• Alternating Direction Methods of Multipliers (ADMM): Riemannian ADMM and its adaptive extensions handle composite objectives (smooth plus nonsmooth convex terms) by alternating between manifold-constrained gradient-like steps (with retraction), nonsmooth proximal updates, and adaptive dual variable updates, with each primal update retaining manifold feasibility. These approaches circumvent smoothing penalties and achieve optimal complexity for KKT convergence under suitable parameter choice (Li et al., 2022, Deng et al., 21 Oct 2025).
• Variational and Acceleration Schemes: Time-adaptive and symplectic integrator-based algorithms, derived from variational principles and Bregman Lagrangian/Hamiltonian systems, alternate (explicitly or implicitly) between momentum, position, and constraint handling (via projection) on the Riemannian space, providing structure-preserving and accelerated convergence (Duruisseaux et al., 2021, Duruisseaux et al., 2021, Duruisseaux et al., 2022).
• Randomized and Block-Submanifold Updates: For large-scale manifolds, updates can be confined to random low-dimensional submanifolds (“random submanifold descent”), vastly reducing per-iteration cost while maintaining favorable convergence properties (Han et al., 18 May 2025).

3. Theoretical Guarantees and Complexity

Convergence and complexity analyses for ARMO methods exhibit the following features:

• Global Convergence: Under standard regularity assumptions (Lipschitz gradients, compactness, suitable penalty parameter growth), iterates converge to stationary points—either in a global sense (every limit point is stationary) or (for augmented Lagrangian/penalty methods) to Karush-Kuhn-Tucker (KKT) points for constrained problems (Hu et al., 2017, Liu et al., 2019, Li et al., 2022, Deng et al., 21 Oct 2025).
• Iteration Complexity: ARMO variants exhibit different rates depending on smoothness and regularization strategies. For example, ARADMM achieves $\mathcal{O}(\epsilon^{-3})$ complexity for $\epsilon$-KKT solutions in the nonsmooth composite setting without smoothing (Deng et al., 21 Oct 2025), matching best-known rates for smoothed Riemannian ADMM methods. Manifold proximal gradient alternation schemes for nonsmooth objectives attain iteration complexity of $O(1/\epsilon^2)$ for stationarity (Huang et al., 2020). Riemannian ADMM has $O(\epsilon^{-4})$ iteration complexity for achieving $\epsilon$-stationarity (Li et al., 2022).
• Local Superlinear Convergence: Under stronger structural assumptions (such as positive-definite Riemannian Hessian at a solution and high-accuracy subproblem solutions), regularized Newton-type methods achieve superlinear or even quadratic local convergence rates (Hu et al., 2017).

4. Practical Applications and Numerical Evidence

ARMO techniques are empirically validated on a variety of high-dimensional, structured problems:

• Robust Low-Rank and Sparse Matrix Decomposition: Alternating manifold proximal gradient continuation achieves reliable background-foreground separation in video surveillance, outperforming nuclear-norm minimization and subgradient methods in efficiency (Huang et al., 2020).
• Nonlinear Matrix Recovery: Alternating schemes alternating between projected gradient steps for variable blocks and manifold SVD truncations enable high-accuracy recovery from partial or nonlinear measurements, with provable guarantees to unique stationary solutions via the Kurdyka–Łojasiewicz property (Goyens et al., 2021).
• Sparse Principal Component Analysis, Fair PCA, and Spectral Clustering: Alternating Riemannian descent-ascent frameworks for minimax formulations show superior computational and statistical performance compared to nested or single-loop alternatives, particularly in enforcing sparsity and fairness (Xu et al., 29 Sep 2024).
• Optimization with Orthogonality and Sparse Simplex Constraints: Adaptations exploiting oblique and Stiefel manifold geometry via multiplicative or random-submanifold updates produce improved solution sparsity and lower computational requirements relative to projection-based methods (Esposito et al., 31 Mar 2025, Han et al., 18 May 2025).
• RIS-Aided Channel Estimation: Alternating updates of receiver combiners and phase-shift matrices (subject to modulus constraints on the circle manifold) maximize Bayesian mutual information for enhanced channel estimation, demonstrating gains in sample efficiency and estimation accuracy (Zhang et al., 18 Oct 2025).

5. Algorithmic Components: Manifold-Specific Operations

ARMO methodologies rely on the following geometric primitives and operator formulations characteristic of manifold optimization:

• Tangent Space Projection: For any point $x \in \mathcal{M}$ and ambient-space direction $v$, the tangent component $P_{T_x\mathcal{M}}(v)$ is used in projected (Riemannian) gradient steps, e.g., $$x_{\text{new}} = \mathscr{R}_x(-\eta\, P_{T_x\mathcal{M}}(\nabla f(x)))$$.
• Retraction Operators: These smoothly map a tangent vector back onto the manifold, approximating the exponential map (e.g., QR-based retraction for the Stiefel manifold or normalization for the oblique manifold).
• Proximal Maps in Riemannian Contexts: Nonsmooth convex terms (e.g., $\ell_1$ norms) utilize proximal mappings, and for some ARMO algorithms, alternating between manifold gradient and proximal updates allows direct treatment of nonsmooth structure (Li et al., 2022, Deng et al., 21 Oct 2025).
• Curvature-Aware Second-Order Direction Computation: Steihaug-type conjugate gradient methods within tangent spaces, with explicit treatment of negative curvature directions, augment search directions beyond those obtainable by first order information alone (Hu et al., 2017).
• Manifold-Alternation Strategies: For variable blocks defined on different manifolds (e.g., a subspace factor and a sparse matrix), updates are alternated while maintaining feasibility and leveraging specific structure—such as SVD for subspace projection—yielding reduced per-iteration complexity (Goyens et al., 2021, Huang et al., 2020).

6. Challenges, Limitations, and Future Directions

Though ARMO frameworks achieve strong results across a variety of problems, several open fronts remain:

• Parameter Tuning: Penalty, smoothing, and step-size parameters (e.g., $\rho$, regularization, or continuation schedules) require careful handling to balance convergence speed and stability, particularly as ill-conditioning or constraint multiplicity increases (Liu et al., 2019).
• Scalability: For very high-dimensional manifolds, random submanifold block updates and efficient retraction approximations provide relief, but attaining the same accuracy as full-manifold methods remains challenging for certain problems (Han et al., 18 May 2025).
• Nonsmooth and Tame Geometry: Extending convergence guarantees to more general nonsmooth settings, possibly leveraging o-minimal geometry and variational stratification, is an area of ongoing interest, especially with respect to modern deep learning landscapes (Aspman et al., 2023).
• Composite and Minimax Optimization: Recent alternation frameworks for nonconvex-linear minimax problems on manifolds attain best-known complexity, suggesting that single-loop alternation with appropriate value function design can circumvent costly nested solution steps; further generalization to nonconvex–nonconcave objectives is a key avenue (Xu et al., 29 Sep 2024).

This synthesis underlines ARMO's central role in enabling efficient, structure-preserving optimization for manifold-valued problems, with a rich algorithmic toolkit tailored to the demands of modern data analysis, signal processing, and physical modeling.