Grassmannian Optimization Techniques

• Grassmannian optimization is a framework that performs optimization over the manifold of k-dimensional subspaces, enabling precise subspace estimation in various applications.
• It leverages differential geometry to compute Riemannian gradients, retractions, and tangent space updates that respect the intrinsic non-Euclidean structure.
• Applications include robust PCA, tensor completion, noncoherent MIMO design, and multi-view learning, demonstrating its versatility in high-dimensional problems.

Grassmannian optimization refers to the family of optimization techniques, algorithms, and analysis frameworks where the feasible set is the Grassmann manifold, the Riemannian manifold of $k$-dimensional linear subspaces of $\mathbb{R}^n$ (denoted $\operatorname{Gr}(k,n)$). This structure arises naturally in problems involving subspace estimation, dimensionality reduction, source separation, signal and image processing, multi-view learning, robust PCA, noncoherent MIMO design, manifold learning, and factorization of tensors and matrices under subspace constraints. Optimization on the Grassmannian leverages differential geometry and matrix manifold algorithms, yielding globally meaningful updates that respect intrinsic non-Euclidean geometry.

1. Formal Definitions and Manifold Structure

The Grassmann manifold $\operatorname{Gr}(k,n)$ is the set of all $k$-dimensional linear subspaces of $\mathbb{R}^n$. Equivalent representations include:

• The homogeneous space model: $\operatorname{Gr}(k,n) = O(n)/(O(k) \times O(n-k))$
• The projection-matrix model: $$\operatorname{Gr}(k,n) = \{P \in \mathbb{R}^{n \times n}: P^T = P = P^2, \operatorname{tr}(P) = k \}$$
• The orthonormal basis (Stiefel) model: equivalence classes of matrices $U \in \mathbb{R}^{n \times k}$ with $U^T U = I_k$ under right multiplication by $\mathbb{R}^n$0

The geodesic (intrinsic) distance between two points $\mathbb{R}^n$1 is given by the principal angles $\mathbb{R}^n$2 between the subspaces: $\mathbb{R}^n$3 where $\mathbb{R}^n$4, with $\mathbb{R}^n$5 denoting the singular values.

Tangent spaces are characterized (in the Stiefel model) by matrices $\mathbb{R}^n$6 with $\mathbb{R}^n$7. Projector-based and involution models (which identify $\mathbb{R}^n$8) admit simple, closed-form formulas for tangent vectors, gradients, and geodesics (Lai et al., 2020).

2. Core Algorithmic Primitives

Riemannian Optimization Paradigm

Many Grassmannian optimization problems are formulated as: $\mathbb{R}^n$9 with $\operatorname{Gr}(k,n)$0 depending only on the subspace, e.g., $\operatorname{Gr}(k,n)$1 (Rayleigh quotient), $\operatorname{Gr}(k,n)$2 (projection loss), or more complex functionals in robust PCA and clustering.

Essential algorithmic tools (Bendokat et al., 2020):

• Riemannian gradient: projection of the Euclidean gradient onto the Grassmann tangent space, e.g., $\operatorname{Gr}(k,n)$3 in the orthonormal basis model.
• Retraction: mapping a tangent vector back to the manifold, often via QR ($\operatorname{Gr}(k,n)$4), polar decomposition, or the exponential map.
• Parallel transport: ensures conjugate-gradient and quasi-Newton updates honor the manifold structure.
• Line search and trust-region methods: optimize step size along geodesics or in local quadratic models.

Generic algorithms include steepest descent, conjugate-gradient, and quasi-Newton (e.g., L-BFGS) adapted to the manifold. For Newton's method, analytic formulas for the Riemannian Hessian are available in models such as the involution algebra (Lai et al., 2020).

The computational complexity per iteration is typically $\operatorname{Gr}(k,n)$5 (in large-$\operatorname{Gr}(k,n)$6, fixed-$\operatorname{Gr}(k,n)$7), owing to QR/SVD steps and matrix multiplications (Bendokat et al., 2020).

3. Specialized Algorithms and Structural Insights

Averaging and Fréchet Means

A major structure-specific task is computing the average (Fréchet mean, Karcher mean) of several subspaces. Given $\operatorname{Gr}(k,n)$8, the Riemannian mean $\operatorname{Gr}(k,n)$9 minimizes the sum of squared geodesic distances: $\operatorname{Gr}(k,n)$0 Efficient averaging is critical for distributed, streaming, and federated learning; RGrAv and DRGrAv use Chebyshev polynomial acceleration to compute the induced arithmetic mean (IAM) by projecting the average of projection matrices back to the manifold (Ancelin et al., 2024).

Robust Optimization, Tensor and Geodesic constraints

• Subspace tracking and robust estimation: Algorithms such as t-GRASTA incorporate sparse error and transformation modeling, using geodesic gradient descent on the Grassmannian for robust alignment (He et al., 2013).
• Online tensor completion: Generalizes incremental Grassmannian optimization to the product of Grassmannians/tensor Grassmannians for t-SVD tensor decompositions, with local linear convergence guarantees in the streaming regime (Gilman et al., 2020).
• Min-max robust tracking: GeRoST solves a min-max problem where the adversary perturbs the data within a Grassmannian ball; closed-form robust updates avoid iterative convex relaxations (Bharadwaj et al., 1 Apr 2026).
• Geodesic models for dynamic subspaces: Batch estimation of smooth, time-varying subspaces using geodesic parameterizations yields improved denoising and tracking performance compared to unconstrained SVD (Blocker et al., 2023).

Multi-view and Product Manifold Optimization

Multi-view and tensor-product tasks optimize over a product of Grassmannians. Generalized Rayleigh quotients on such manifolds unify low-rank tensor approximation, quantum state separability, and subspace clustering, with derived formulae for Riemannian gradient, Hessian, and block-coordinate updates (Curtef et al., 2010). Multi-view clustering embeds orthogonality-constrained spectral clustering into unconstrained Riemannian optimization on products of Grassmannians, using trust-region solvers (Yang et al., 8 Mar 2025).

Evolutionary and Heuristic Optimization

Population-based global optimization (e.g., Differential Evolution) can operate on the Grassmannian by projecting mutated/fused individuals onto the manifold (via QR) rather than applying local Riemannian gradient steps. DE achieves escape from local minima and is robust against multimodal, nonconvex landscapes, though with higher per-iteration cost (Lesniewski, 27 Mar 2025).

4. Applications and Implications

Grassmannian optimization underpins a wide range of methods and applications:

• Robust PCA, image alignment, video separation: Optimization of low-rank manifolds/sparsity/transformations (He et al., 2013)
• MIMO systems: Grassmannian constellation design for noncoherent transmission, using autoencoders with embedded manifold constraints (Fu et al., 2021)
• Tensor completion and streaming tracking: t-Grassmannian factorization and tracking (Gilman et al., 2020)
• Dimensionality reduction: Unconstrained Riemannian methods learn discriminative submanifold embeddings (Liu et al., 2017)
• Frame theory: Coherence-optimal frames, ETFs, line packing, and their Grassmannian-optimality regimes (IV et al., 2017)
• Dynamic subspace estimation in statistical learning and neuroscience data (Blocker et al., 2023), federated learning, and distributed optimization (Ancelin et al., 2024).

5. Complexity, Theory, and Practical Limitations

Quadratic optimization over the real Grassmannian is NP-hard in all parameter regimes, even when $\operatorname{Gr}(k,n)$1 is fixed or minimal ($\operatorname{Gr}(k,n)$2), and for unconstrained (homogeneous) quadratic functions. There does not exist any fully-polynomial-time approximation scheme (FPTAS) unless P=NP. This computational barrier extends (via reductions) to the Stiefel and orthogonal group manifolds and to the Cartan manifold ($\operatorname{Gr}(k,n)$3) (Lai et al., 2024). In contrast, trace-linear problems (e.g., subspace projection, SVD-type) remain efficiently solvable.

A table comparing key complexity and optimization regimes:

Problem class	Complexity	Example
Linear objectives	P	PCA, subspace estimation, Rayleigh quotient
Quadratic objectives	NP-hard	Unconstrained quadratic forms on $\operatorname{Gr}(k,n)$4
Global approximation	No FPTAS	Clique number, line-packing via quadratic optimization
Riemannian local	Efficient*	Local minima found rapidly (descent/CG/trust-regions), global = NP-hard

*for typical smooth losses and with favorable initialization

A plausible implication is that practitioners must rely on local methods (conjugate gradient, trust-region, QR/polar retraction, manifold-aware optimization) and exploit problem-specific structure and initialization to ensure convergence to satisfactory solutions (Bendokat et al., 2020, Lai et al., 2024).

6. Model Variants, Representations, and Implementation Considerations

Grassmannian optimization admits several coordinate representations, each with computational and analytic advantages:

• Stiefel coordinate (orthonormal basis): Standard for most algorithms, allows efficient QR-based retraction (Bendokat et al., 2020).
• Projection/involution matrix: Enables explicit block-structure formulas for gradient, Hessian, and geodesics; avoids SVD, useful for machine precision and implementation efficiency (Lai et al., 2020).
• Affine Grassmannian: Embedding into a larger Grassmannian allows standard algorithms to apply to affine subspaces with bias (Lim et al., 2016).
• Product/Tensor/Block-diagonal structures: Optimization over $\operatorname{Gr}(k,n)$5 (tensor product, multi-view) admits block-wise updates and coupled Riemannian geometry (Curtef et al., 2010, Yang et al., 8 Mar 2025).

Common algorithmic and numerical features include:

• Retractions via QR or polar decomposition for O($\operatorname{Gr}(k,n)$6) per-iteration cost.
• Riemannian conjugate gradient and Newton schemes, with cost determined by tangent-space operations and often benefiting from "horizontal" coordinate representations (Bendokat et al., 2020, Lai et al., 2020).
• Acceleration of subspace averaging via Chebyshev polynomials (Ancelin et al., 2024).

7. Emerging Directions and Synthesis

Modern developments leverage Grassmannian optimization principles for neural network training (manifold-based variable projection in PINNs and deep regression) by exploiting the separable structure and absence of spurious local minima on the Grassmannian (Dus, 30 Jan 2026). In robust and high-dimensional settings, hybrid approaches combine non-Euclidean geometry, sharp perturbation bounds, Fréchet mean analysis, and probabilistic deviation estimates for principled streaming or inference algorithms (Eftekhari et al., 2016).

A plausible implication is that further extensions (e.g., to infinite-dimensional Hilbert Grassmannians, more general quotient manifolds, or coupled Einstein metrics) will benefit from the mature algorithmic infrastructure and analytic clarity that Grassmannian optimization methodologies provide for subspace- and manifold-valued statistical modeling.