---
title: Geometrically Robust Least Squares
url: https://www.emergentmind.com/topics/geometrically-robust-least-squares-problem
type: topic
---

# Geometrically Robust Least Squares

A Geometrically Robust Least Squares Problem refers to a family of optimization formulations and algorithmic strategies designed to maintain the stability, accuracy, and well-posedness of least squares solutions under constraints imposed by geometry, data perturbation, and structural uncertainty. The concept spans algebraic, manifold, and residual-based methodologies, addressing both classic issues of sensitivity in algebraic systems and modern demands for robustness in geometric and control settings.

## 1. Mathematical Formulation and Geometric Context

The geometrically robust least squares (GRLS) paradigm arises when classical least squares, typically of the form
\[
\min_{x\in\mathbb{R}^n} \|A x - b\|_2^2,
\]
is augmented to enforce stability with respect to geometric or structural uncertainties. In applications where the operator \(A\) or the feasible solution set must respect geometric constraints, GRLS imposes these via manifold structures or sets defined intrinsically by geometry.

A prototypical example considers model uncertainty over subspaces:
\[
\min_{x\in\mathbb{R}^n} \max_{S \in \mathcal{B}_\rho^d(\widehat{S}) \subset \operatorname{Gr}(k,n)} \|P_{S}x - b\|_2^2,
\]
where \(P_{S}\) is the orthogonal projector onto a \(k\)-dimensional subspace \(S\) of \(\mathbb{R}^n\), \(\operatorname{Gr}(k,n)\) is the Grassmannian, and \(\mathcal{B}_\rho^d(\widehat{S})\) denotes a geodesic ball of radius \(\rho\) around a reference subspace \(\widehat{S}\) in an appropriate manifold metric. This minimax structure captures robustness against worst-case subspace perturbations and is central to problems in signal processing and data-driven control [2511.03644].

## 2. Reformulation via Exact Penalty and Smooth Approximation

Incorporating geometric constraints often renders the feasible set nonconvex and complicates direct optimization. The GRLS formulation in [2511.03644] enforces constraints such as \(y \in \mathcal{B}^d_\rho(\widehat{y})\) by introducing an exact penalty:
\[
h(y) = \min\{0, \rho^2 - d(y, \widehat{y})^2\},
\]
with penalty parameter \(\lambda \geq 0\). To facilitate practical optimization using differentiable algorithms, this hard constraint is smoothed by
\[
h_u(y) = -u \log\left(1 + \exp\left(\frac{d(y, \widehat{y})^2 - \rho^2}{u}\right)\right),
\]
for small \(u>0\), leading to the smooth unconstrained minimax problem:
\[
\min_{x\in\mathbb{R}^n} \max_{y\in\operatorname{Gr}(k,n)} \left\| P_{y}x - b \right\|_2^2 - \lambda u \log\left(1 + \exp\left(\frac{d(y,\widehat{y})^2 - \rho^2}{u}\right)\right).
\]
This relaxation preserves the geometric constraint in the limit \(u \to 0\) and enables the use of Riemannian optimization over product manifolds [2511.03644].

## 3. Optimization Algorithms: Riemannian Gradient Approaches

Given the smooth penalized structure, first-order algorithms operate naturally on the product manifold \(\mathbb{R}^n \times \operatorname{Gr}(k,n)\). Tangent-space Riemannian gradient descent-ascent (TSRGDA) iterates as follows:

- For \(k=0,1,\ldots\), compute gradients:
  \[
  \nabla_{x} \tilde{f} = 2 P_{Y}(P_{Y}x - b),
  \]
  \[
  \nabla_{Y} \tilde{f} = 2 \big[(P_Y x - b)x^\top + (P_Y x-b)b^\top\big] Y - \lambda \frac{2 d(Y,\widehat{Y})}{1+e^{-(\rho^2-d^2)/u}} \nabla_Y d,
  \]
  where the Riemannian gradient on the Stiefel manifold is obtained via orthogonal projection. The updates are
  \[
  x_{k+1} = x_k - \eta_x g_x^k, \quad Y_{k+1} = \mathrm{Retr}_{Y_k}(Y_k + \eta_y g_Y^k),
  \]
  with \(\mathrm{Retr}_{Y_k}\) a retraction (e.g., QR decomposition to orthonormal bases) and step sizes \(\eta_x, \eta_y\) selected for convergence guarantees.

Convergence to first-order stationary points is established under standard smoothness and metric completeness hypotheses for the product manifold [2511.03644].

## 4. Regularization and Stability Compared to Classical Methods

GRLS methods directly address instability and discontinuity in classical algebraic and least-squares solvers that occurs near singularities or under data perturbation. Traditional approaches may fail or produce unbounded solutions where the Jacobian degenerates (e.g., near multiple roots or rank drops). The geometric approach modulates the structure by:

- Constructing an augmented system \(G(x,p) = [F(x,p); H(x,p)]\) that enforces necessary injectivity/surjectivity properties for local uniqueness and manifold regularity [2102.08330].
- Ensuring the existence of a tubular neighborhood \(\mathcal{T}\) around the structure-preserving manifold \(M\) so that (i) every empirical data point in \(\mathcal{T}\) has a unique Lipschitz projection onto \(M\), and (ii) least-squares minimizers are both uniquely defined and stable with respect to small perturbations.

This overcomes classical issues by modeling both the geometric structure of the solution set and the projection mechanism via well-posed, regularized optimization [2102.08330].

## 5. Applications in Signal Processing, Control, and Geometric Estimation

Geometrically robust least squares have foundational and practical roles across a spectrum of disciplines:

- **Data-driven control and subspace tracking:** The minimax-on-manifold formulation models worst-case behavior under bounded geometric uncertainty (e.g., Grassmannian balls) for controller synthesis, enabling robust performance even under modeling errors [2511.03644].
- **Geometric inverse and algebraic problems:** The regularized least-squares strategy applies to multiple-root finding, approximate GCD/factorization, and defective eigenproblems. The scheme ensures stable recovery of "nearby" objects that preserve the relevant algebraic structure [2102.08330].
- **Robotics and SLAM:** Closely related robust NLS and manifold-based solvers (with adaptive kernels) underpin state estimation, ICP, and bundle adjustment, leveraging geometric robustness both in the residual design and optimization strategy [2004.14938], [2305.06869].

## 6. Theoretical Guarantees and Numerical Behavior

Key properties arising from these frameworks include:

- Formal convergence to first-order stationary points for smooth penalized minimax algorithms under appropriate step sizes and manifold smoothness [2511.03644].
- Existence and uniqueness of least-squares regularized projections onto structure-preserving manifolds in a Lipschitz neighborhood, with explicit geometric and analytic proof structure [2102.08330].
- Empirical validation demonstrating that solutions trace continuous paths as data are perturbed, avoiding branch-switching and discontinuity typical of classical algebraic approaches.

Numerical experiments in [2511.03644] with \(n=2, k=1\) illustrate convergence to the correct solution on the boundary of the allowed subspace ball and smooth decay of the gradient norm, confirming theoretical expectations.

## 7. Relation to Alternative Robustness Techniques

Geometric robustness in least squares is complementary but distinct from outlier-robust M-estimators (e.g., adaptive kernel families, IRLS schemes in nonlinear least squares). While robust residuals address heavy-tailed noise and gross outliers, geometrically robust least squares (as in [2511.03644], [2102.08330]) focus on the preservation of solution structure and stability under geometric transformations or data-parameter perturbations. *A plausible implication is that these approaches can and should be combined for full-spectrum robustness in challenging applications.*

---

Geometrically robust least squares comprise a mathematically principled and algorithmically tractable foundation for preserving stability, uniqueness, and smooth dependence on data in structurally constrained and uncertainty-aware settings, with broad applicability across contemporary problems in signal processing, system identification, control, and computational algebra.

Source: https://www.emergentmind.com/topics/geometrically-robust-least-squares-problem