Geometrically Robust Least Squares

• Geometrically robust least squares are optimization formulations that integrate geometric constraints to stabilize solutions amid data perturbations and structural uncertainty.
• They employ exact penalty and smooth approximation methods to transform nonconvex constraints into a tractable Riemannian optimization framework.
• Applications in signal processing, control, and robotics demonstrate how these methods ensure stability and uniqueness even near singularities.

A Geometrically Robust Least Squares Problem refers to a family of optimization formulations and algorithmic strategies designed to maintain the stability, accuracy, and well-posedness of least squares solutions under constraints imposed by geometry, data perturbation, and structural uncertainty. The concept spans algebraic, manifold, and residual-based methodologies, addressing both classic issues of sensitivity in algebraic systems and modern demands for robustness in geometric and control settings.

1. Mathematical Formulation and Geometric Context

The geometrically robust least squares (GRLS) paradigm arises when classical least squares, typically of the form

$\min_{x\in\mathbb{R}^n} \|A x - b\|_2^2,$

is augmented to enforce stability with respect to geometric or structural uncertainties. In applications where the operator $A$ or the feasible solution set must respect geometric constraints, GRLS imposes these via manifold structures or sets defined intrinsically by geometry.

A prototypical example considers model uncertainty over subspaces: $$\min_{x\in\mathbb{R}^n} \max_{S \in \mathcal{B}_\rho^d(\widehat{S}) \subset \operatorname{Gr}(k,n)} \|P_{S}x - b\|_2^2,$$ where $P_{S}$ is the orthogonal projector onto a $k$-dimensional subspace $S$ of $\mathbb{R}^n$, $\operatorname{Gr}(k,n)$ is the Grassmannian, and $\mathcal{B}_\rho^d(\widehat{S})$ denotes a geodesic ball of radius $\rho$ around a reference subspace $\widehat{S}$ in an appropriate manifold metric. This minimax structure captures robustness against worst-case subspace perturbations and is central to problems in signal processing and data-driven control (Coulson et al., 5 Nov 2025).

2. Reformulation via Exact Penalty and Smooth Approximation

Incorporating geometric constraints often renders the feasible set nonconvex and complicates direct optimization. The GRLS formulation in (Coulson et al., 5 Nov 2025) enforces constraints such as $y \in \mathcal{B}^d_\rho(\widehat{y})$ by introducing an exact penalty: $h(y) = \min\{0, \rho^2 - d(y, \widehat{y})^2\},$ with penalty parameter $\lambda \geq 0$. To facilitate practical optimization using differentiable algorithms, this hard constraint is smoothed by

$$h_u(y) = -u \log\left(1 + \exp\left(\frac{d(y, \widehat{y})^2 - \rho^2}{u}\right)\right),$$

for small $u>0$, leading to the smooth unconstrained minimax problem: $$\min_{x\in\mathbb{R}^n} \max_{y\in\operatorname{Gr}(k,n)} \left\| P_{y}x - b \right\|_2^2 - \lambda u \log\left(1 + \exp\left(\frac{d(y,\widehat{y})^2 - \rho^2}{u}\right)\right).$$ This relaxation preserves the geometric constraint in the limit $u \to 0$ and enables the use of Riemannian optimization over product manifolds (Coulson et al., 5 Nov 2025).

3. Optimization Algorithms: Riemannian Gradient Approaches

Given the smooth penalized structure, first-order algorithms operate naturally on the product manifold $\mathbb{R}^n \times \operatorname{Gr}(k,n)$. Tangent-space Riemannian gradient descent-ascent (TSRGDA) iterates as follows:

• For $k=0,1,\ldots$, compute gradients:

$\nabla_{x} \tilde{f} = 2 P_{Y}(P_{Y}x - b),$

$$\nabla_{Y} \tilde{f} = 2 \big[(P_Y x - b)x^\top + (P_Y x-b)b^\top\big] Y - \lambda \frac{2 d(Y,\widehat{Y})}{1+e^{-(\rho^2-d^2)/u}} \nabla_Y d,$$

where the Riemannian gradient on the Stiefel manifold is obtained via orthogonal projection. The updates are

$$x_{k+1} = x_k - \eta_x g_x^k, \quad Y_{k+1} = \mathrm{Retr}_{Y_k}(Y_k + \eta_y g_Y^k),$$

with $\mathrm{Retr}_{Y_k}$ a retraction (e.g., QR decomposition to orthonormal bases) and step sizes $\eta_x, \eta_y$ selected for convergence guarantees.

Convergence to first-order stationary points is established under standard smoothness and metric completeness hypotheses for the product manifold (Coulson et al., 5 Nov 2025).

4. Regularization and Stability Compared to Classical Methods

GRLS methods directly address instability and discontinuity in classical algebraic and least-squares solvers that occurs near singularities or under data perturbation. Traditional approaches may fail or produce unbounded solutions where the Jacobian degenerates (e.g., near multiple roots or rank drops). The geometric approach modulates the structure by:

• Constructing an augmented system $G(x,p) = [F(x,p); H(x,p)]$ that enforces necessary injectivity/surjectivity properties for local uniqueness and manifold regularity (Zeng, 2021).
• Ensuring the existence of a tubular neighborhood $\mathcal{T}$ around the structure-preserving manifold $M$ so that (i) every empirical data point in $\mathcal{T}$ has a unique Lipschitz projection onto $M$, and (ii) least-squares minimizers are both uniquely defined and stable with respect to small perturbations.

This overcomes classical issues by modeling both the geometric structure of the solution set and the projection mechanism via well-posed, regularized optimization (Zeng, 2021).

5. Applications in Signal Processing, Control, and Geometric Estimation

Geometrically robust least squares have foundational and practical roles across a spectrum of disciplines:

• Data-driven control and subspace tracking: The minimax-on-manifold formulation models worst-case behavior under bounded geometric uncertainty (e.g., Grassmannian balls) for controller synthesis, enabling robust performance even under modeling errors (Coulson et al., 5 Nov 2025).
• Geometric inverse and algebraic problems: The regularized least-squares strategy applies to multiple-root finding, approximate GCD/factorization, and defective eigenproblems. The scheme ensures stable recovery of "nearby" objects that preserve the relevant algebraic structure (Zeng, 2021).
• Robotics and SLAM: Closely related robust NLS and manifold-based solvers (with adaptive kernels) underpin state estimation, ICP, and bundle adjustment, leveraging geometric robustness both in the residual design and optimization strategy (Chebrolu et al., 2020, Jung et al., 2023).

6. Theoretical Guarantees and Numerical Behavior

Key properties arising from these frameworks include:

• Formal convergence to first-order stationary points for smooth penalized minimax algorithms under appropriate step sizes and manifold smoothness (Coulson et al., 5 Nov 2025).
• Existence and uniqueness of least-squares regularized projections onto structure-preserving manifolds in a Lipschitz neighborhood, with explicit geometric and analytic proof structure (Zeng, 2021).
• Empirical validation demonstrating that solutions trace continuous paths as data are perturbed, avoiding branch-switching and discontinuity typical of classical algebraic approaches.

Numerical experiments in (Coulson et al., 5 Nov 2025) with $n=2, k=1$ illustrate convergence to the correct solution on the boundary of the allowed subspace ball and smooth decay of the gradient norm, confirming theoretical expectations.

7. Relation to Alternative Robustness Techniques

Geometric robustness in least squares is complementary but distinct from outlier-robust M-estimators (e.g., adaptive kernel families, IRLS schemes in nonlinear least squares). While robust residuals address heavy-tailed noise and gross outliers, geometrically robust least squares (as in (Coulson et al., 5 Nov 2025, Zeng, 2021)) focus on the preservation of solution structure and stability under geometric transformations or data-parameter perturbations. A plausible implication is that these approaches can and should be combined for full-spectrum robustness in challenging applications.

---

Geometrically robust least squares comprise a mathematically principled and algorithmically tractable foundation for preserving stability, uniqueness, and smooth dependence on data in structurally constrained and uncertainty-aware settings, with broad applicability across contemporary problems in signal processing, system identification, control, and computational algebra.