---
title: Minimum-Rank Universal LS Solvers
url: https://www.emergentmind.com/topics/minimum-rank-universal-least-squares-solvers
type: topic
---

# Minimum-Rank Universal LS Solvers

“Minimum-rank universal least-squares solvers” names a heterogeneous research area rather than a single algorithmic paradigm. In one line of work, the object of interest is a matrix \(H\) that maps every right-hand side \(b\) to a least-squares solution \(Hb\), and “minimum rank” refers to the smallest possible rank of such a universal solver. In another, the emphasis is on algorithms that remain valid for singular, inconsistent, or rank-deficient least-squares problems and return the canonical minimum-length solution \(A^\dagger b\). A third line studies minimum-rank or low-rank matrix recovery by converting rank-structured estimation into a sequence of weighted least-squares problems. A fourth exploits low-rank structure in updates, recursion, or total least-squares formulations. The common theme is universality across right-hand sides, regimes, or structural perturbations, but the meaning of “minimum rank” varies sharply across these settings [2509.04264].

## 1. Terminological scope and principal interpretations

The literature uses the phrase in several non-equivalent senses. The distinctions are structural, not cosmetic, because they determine the admissible unknowns, guarantees, and algorithmic machinery.

| Interpretation | Unknown | Canonical requirement |
|---|---|---|
| Universal solver matrix | \(H\in\mathbb{R}^{n\times m}\) | \(Hb\) solves LS for every \(b\) |
| Minimum-length universal LS algorithm | vector \(x\) | return \(A^\dagger b\) |
| Low-rank recovery via LS | matrix \(X\) | promote low rank through weighted LS |
| Low-rank-update LS solver | updated solution \(x\) | exploit \(A\mapsto A+UV^T\) |

In the generalized-inverse formulation, universality means that the same matrix \(H\) must solve the least-squares problem for all right-hand sides \(b\) [2509.04264]. In Krylov and randomized numerical linear algebra, universality instead means coverage of multiple regimes—consistent or inconsistent, full-rank or rank-deficient, overdetermined or underdetermined—while preserving the Moore–Penrose solution \(A^\dagger b\) [1301.2707]. In low-rank recovery, the target is typically not a universal solver matrix at all, but an unknown matrix \(X\) with small rank, recovered from linear measurements by repeatedly solving weighted least-squares subproblems [1010.2471]. Finally, some exact linear-algebra methods treat “low rank” as a property of the *update* \(UV^T\), so that the least-squares solver is reused under low-rank perturbations rather than recomputed from scratch [2406.15120].

This multiplicity of meanings is essential for reading the area correctly. A paper on minimum-length pseudoinverse solvers for singular systems is usually not a paper on minimum-rank matrix optimization; conversely, a paper on affine rank minimization may not produce a universal solver for varying right-hand sides.

## 2. Universal least-squares solvers as generalized inverses of minimum possible rank

The most literal realization of a minimum-rank universal least-squares solver appears in the generalized-inverse framework. For \(A\in\mathbb{R}^{m\times n}\), let \(H\in\mathbb{R}^{n\times m}\). The Penrose properties are
\[
AHA=A \tag{P1}
\]
\[
HAH=H \tag{P2}
\]
\[
(AH)^\top=AH \tag{P3}
\]
\[
(HA)^\top=HA. \tag{P4}
\]
A classical result recalled in recent work is that \(H\) is a universal least-squares solver iff it satisfies \((P1)+(P3)\), equivalently
\[
AH=AA^\dagger,
\]
or, again equivalently,
\[
A^\top A H = A^\top.
\]
The minimum-rank refinement is exact: among generalized inverses, \(\operatorname{rank}(H)\ge \operatorname{rank}(A)\), with equality iff \(H\) satisfies \((P2)\). Therefore a minimum-rank universal least-squares solver is precisely an ah-symmetric reflexive generalized inverse, i.e. a matrix satisfying
\[
(P1)+(P2)+(P3).
\]
This converts minimum rank from an optimization objective into an algebraic characterization [2509.04264].

The same work gives a reduced SVD parameterization. If
\[
A=U\Sigma V^\top,\qquad \Sigma=\begin{bmatrix} D&0\\0&0\end{bmatrix},
\]
with \(D\in\mathbb{R}^{r\times r}\) diagonal and positive, \(U=[U_1\;U_2]\), and \(V=[V_1\;V_2]\), then every minimum-rank universal least-squares solver has the form
\[
H = V_1D^{-1}U_1^\top + V_2ZU_1^\top,\qquad Z\in\mathbb{R}^{(n-r)\times r}.
\]
This shows that universality fixes the action on \(\mathcal R(A)\), while the free parameter \(Z\) acts through nullspace directions without increasing the rank beyond \(r=\operatorname{rank}(A)\) [2509.04264].

The same paper develops sparse formulations by minimizing the matrix entrywise \(1\)-norm over this feasible class, for example
\[
\min_H \left\{ \|H\|_1 : (P1)+(P2)+(P3)\right\},
\]
and an equivalent single linear characterization,
\[
A^\top H^\top A^\top + HAA^\dagger = A^\top + H.
\]
A Douglas–Rachford splitting method based on the linear formulation
\[
A^\top A H=A^\top,\qquad HAA^\dagger=H
\]
performed best computationally; the projection step admits the closed form
\[
\Pi_{\mathcal C}(V)=A^\dagger - A^\dagger A V A A^\dagger + V A A^\dagger.
\]
On test instances with \(n=0.5m\) and \(r=0.25m\), the fixed-point DRS variant solved all tested sizes up to \(m=5000\), with representative runtimes \(0.16\) sec at \(m=100\), \(4.33\) sec at \(m=500\), and \(538.69\) sec at \(m=5000\) [2509.04264].

A notable structural tension also emerges here. Enforcing minimum rank via \((P2)\) substantially limits sparsity gains: compared with \(A^\dagger\), the reported \(0\)-norm reduction for minimum-rank universal least-squares solvers was about \(8\%\)–\(16\%\), whereas plain universal least-squares solvers achieved about \(55\%\) reduction [2509.04264].

## 3. Universal least-squares algorithms for singular, inconsistent, and rank-deficient systems

A different strand of the literature treats universality operationally: one algorithm should solve
\[
\min_x \|Ax-b\|_2
\]
across compatible, incompatible, full-rank, and rank-deficient regimes, ideally returning the canonical minimum-length solution
\[
x^\dagger=A^\dagger b.
\]

For symmetric or Hermitian square systems, MINRES-QLP gives the clearest Krylov realization of this objective. It addresses
\[
Ax=b,\qquad A\in\mathbb{R}^{n\times n}\ \text{symmetric},\quad A\in\mathbb{C}^{n\times n}\ \text{Hermitian},
\]
possibly shifted to \(A-\sigma I\), through the unified least-squares problem
\[
\min_x \|Ax-b\|_2,
\]
and proves that the terminal iterate is the minimum-length solution of this problem. Its defining innovation is a right orthogonal transformation of the MINRES projected tridiagonal problem,
\[
L_k = R_k P_k,
\]
yielding a rank-revealing QLP decomposition without column pivoting. This lets the method identify the singular projected case and compute the pseudoinverse solution \(A^\dagger b\), whereas classical MINRES generally does not return the min-length solution for singular systems. The method remains a short-recurrence Lanczos solver and uses one product \(Av_k\) per iteration; however, its clean pseudoinverse interpretation is fundamentally an unpreconditioned Euclidean-norm statement, because SPD preconditioning changes the norm in which minimum length is defined [1301.2707].

LSRN extends universality to strongly rectangular systems. It solves
\[
\min_{x\in\mathbb{R}^n}\|Ax-b\|_2,\qquad A\in\mathbb{R}^{m\times n},
\]
for \(m\gg n\) or \(m\ll n\), allows rank deficiency, and returns the min-length solution \(A^\dagger b\). The method applies a random normal projection, computes an economy SVD of the sketch, and then solves the preconditioned problem by LSQR or the Chebyshev semi-iterative method. Its key theoretical feature is that the condition number of the preconditioned system depends on \(r/s\), not on the nonzero singular spectrum of \(A\), so iteration counts become predictable in terms of sketch size and rank. Because \(A\) is accessed only through matrix-matrix and matrix-vector products, the method accommodates dense, sparse, and operator-based representations [1109.5981].

Randomized Extended Kaczmarz pushes universality further toward arbitrary real matrices. It couples a row-action Kaczmarz iteration for \(x\) with a column-action projection process for an auxiliary vector \(z\), thereby removing the inconsistent component \(b-b_{\mathcal R(A)}\) online and maintaining iterates in \(\mathcal R(A^\top)\). The method converges in expectation to the minimum Euclidean norm least-squares solution \(x_{LS}=A^\dagger b\), with the bound
\[
\mathbb{E}\|x^{(T)}-x_{LS}\|_2^2 \le \left(1-\frac{1}{\kappa_F^2(A)}\right)^{\lfloor T/2\rfloor}(1+2k(A))\|x_{LS}\|_2^2.
\]
Its advertised computational appeal is strongest for sparse, well-conditioned, strongly rectangular systems [1205.5770].

A recent refinement shows that, for Hermitian and complex-symmetric systems, standard MINRES can be upgraded to a pseudoinverse solver with essentially no extra asymptotic cost. If \(x_g\) and \(r_g\) are the final MINRES iterate and residual, then for Hermitian \(A\),
\[
\hat{x}^\dagger = x_g - \frac{\langle r_g,x_g\rangle}{\|r_g\|^2}r_g
\]
equals \(A^\dagger b\). The correction removes the final null-space component of the MINRES least-squares solution and is attached only at termination. The same paper extends the idea to complex-symmetric systems and to PSD or even singular preconditioning, though in the preconditioned case exact recovery of the original \(A^\dagger b\) requires an additional range condition [2309.17096].

For arbitrary singular square systems, a different route is to solve
\[
A C A^{\rm T} z = b,\qquad x = C A^{\rm T} z,
\]
with \(C\) symmetric positive definite, and then apply RRGMRES to the transformed matrix \(A C A^{\rm T}\). Because \(A C A^{\rm T}\) is range-symmetric, RRGMRES is guaranteed to compute a least-squares solution for arbitrary singular \(A\) and arbitrary \(b\), including inconsistent cases. The same work proposes an NR-SSOR inner-iteration right preconditioner, yielding strong empirical reductions in the normal-equation residual. However, the returned solution is minimum-norm only for \(C=I\); for general \(C\neq I\), universality is retained at the level of least-squares solvability, not pseudoinverse canonicity [2504.09891].

## 4. Low-rank recovery by repeated least-squares and fixed-rank least-squares estimation

When “minimum rank” refers to the unknown matrix rather than the solver, least-squares typically appears as an inner engine. A canonical example is affine rank minimization,
\[
\min_{X\in M_{n\times p}} \operatorname{rank}(X)\quad \text{subject to } S(X)=M,
\]
with \(S:M_{n\times p}\to\mathbb C^m\). IRLS-M addresses this problem indirectly through nuclear-norm and smoothed nuclear-norm surrogates. At iteration \(\ell\), it solves
\[
X^\ell = \arg\min_{S(X)=M}\|(W^{\ell-1})^{1/2}X\|_F^2,
\]
with weights updated from the singular values of \(X^\ell\). The associated variational objective is
\[
\mathcal J(X,W)=\frac12\left(\|W^{1/2}X\|_F^2 + \|W^{-1/2}\|_F^2\right),
\]
and the smoothed surrogate converges toward the nuclear norm as \(\varepsilon\to 0\). Under the Strong Rank Null Space Property of order \(K\), if \(\varepsilon_\ell\to 0\), the iterates converge to the unique nuclear-norm minimizer; if \(\varepsilon_\ell\to \varepsilon>0\), every accumulation point minimizes the smoothed objective \(\mathcal J_\varepsilon\), with error controlled by the best rank-\(k\) approximation [1010.2471].

In high-dimensional trace regression, rank-constrained least squares is treated directly:
\[
\hat{\mathbf{\Theta}}_{L_0}(r)=\arg\min_{\operatorname{rank}(\mathbf{\Theta})\le r}\sum_{i=1}^n \left(y_i-\langle \mathbf X_i,\mathbf\Theta\rangle\right)^2.
\]
The primary contribution here is statistical rather than algorithmic: for \(r\ge r^\ast\), the in-sample prediction risk satisfies
\[
\mathcal R(\hat{\mathbf\Theta}_{L_0}(r)) \le c_1 \frac{rd\log(n)}{n}
\]
with high probability, and this bound is obtained without assumptions on the design matrix beyond independent observations and sub-Gaussian noise. The empirical optimization is carried out only approximately, by alternating minimization with multiple restarts and six initializations built from a coarse grid of nuclear-norm estimators and a spectral estimator. This is therefore a fixed-rank least-squares estimator with strong prediction and testing theory, not a universal minimum-rank solver [2111.14287].

A more explicitly algorithmic minimum-rank template is to search over ranks and solve a nonlinear least-squares feasibility problem at each rank. For differentiable matrix systems
\[
\phi(X)=b,
\]
the residuals are
\[
F_j(X)=\phi_j(X)-b_j.
\]
Low rank is imposed by factorization, e.g.
\[
X=Y^\top Z
\]
for arbitrary rank-\(r\) matrices or
\[
X=YY^\top
\]
for PSD matrices. The solver then applies Levenberg–Marquardt to
\[
\min \frac12\|F\|_2^2
\]
at a fixed rank \(r\), and increments \(r\) until a numerical solution is found. This realizes minimum rank operationally as “the first rank whose fixed-rank least-squares subproblem becomes numerically feasible.” The framework is broad enough to include matrix completion and Euclidean embedding, but its guarantees are numerical and local rather than global [1701.04118].

## 5. Low-rank updates, recursive rank-adaptive methods, and structured total least squares

Another interpretation of the topic focuses on algorithmic exploitation of low-rank structure rather than low-rank optimization itself. For full-column-rank rectangular matrices,
\[
A\in\mathbb{R}^{m\times n},\qquad m\ge n,\qquad \operatorname{rank}(A)=n,
\]
and low-rank updates
\[
\hat A=A+UV^T,
\]
an exact Woodbury-type pseudoinverse formula yields an updated least-squares solver without recomputing a fresh QR factorization. The central identity is
\[
(\hat A)^\dagger = A^\dagger-MA^\dagger +(I-M)(A^TA)^{-1}VU^T,
\]
with
\[
M=(A^TA)^{-1}X\bigl(I+Y^T(A^TA)^{-1}X\bigr)^{-1}Y^T,
\]
\[
X=[V,\;A^TU],\qquad Y=[\hat A^TU,\;V].
\]
Operationally, once a solver for \(A\) is available, the updated solve reduces to one old least-squares solve, \(2r\) solves with \(A^TA\), and one dense \(2r\times 2r\) system. In MATLAB experiments with \(m=10^5\), \(n=100,\dots,1000\), and update ranks \(r=10,20,30\), the reported speedup over recomputing QR ranged from about \(20\times\) to over \(130\times\), with relative forward error below \(3\times 10^{-14}\) in all tests [2406.15120].

For streaming or recursive least squares with rank-deficient matrices, rank-Greville maintains a rank factorization
\[
A_n=B_nC_n
\]
and updates the minimum-norm least-squares solution
\[
x_n=A_n^+Y_n
\]
directly. The two essential cases are separated by the rejection
\[
\Gamma^{rej}=\Gamma-C_n^\top \tilde C_n\Gamma.
\]
If the new row increases rank, then
\[
x_{n+1}=x_n+\frac{\Gamma^{rej}}{\|\Gamma^{rej}\|_2^2}\bigl(y_{n+1}-\Gamma^\top x_n\bigr);
\]
if rank is preserved, then
\[
x_{n+1}=x_n+\frac{\tilde C_n^\top\zeta}{1+\gamma^\top\zeta}\bigl(y_{n+1}-\Gamma^\top x_n\bigr).
\]
The per-observation update cost is \(O(mr)\), and solving from scratch by recursive processing costs \(O(nmr)\). The method is asymptotically advantageous when \(r\ll \min(n,m)\), with experiments suggesting it becomes fastest roughly when \(r\lesssim 0.15\,\min(n,m)\), though its numerical robustness remains below that of QR/SVD-based LAPACK solvers [2106.11594].

Low-rank structure also enters total least squares through rank-one perturbations. In Gauss–Newton TLS, the nonlinear residual map is
\[
f(x)=\mu(x)(Ax-b),\qquad \mu(x)=\frac{1}{\sqrt{1+x^Tx}},
\]
with Jacobian
\[
J(x)=\mu(x)\Bigl(A-\mu(x)^2(Ax-b)x^T\Bigr).
\]
Thus each Gauss–Newton step solves an ordinary least-squares problem with a rank-one update of \(A\). The paper derives an explicit step length
\[
\alpha = 1-\mu(x)^2 x^T h
\]
and proves global convergence of the resulting iteration for well-posed TLS, with
\[
\|x_k-x_{\mathrm{TLS}}\| = O\!\left(\left(\frac{\sigma_{n+1}}{\sigma_n}\right)^{2k}\right),
\qquad
|\eta(x_k)-\sigma_{n+1}| = O\!\left(\left(\frac{\sigma_{n+1}}{\sigma_n}\right)^{4k}\right).
\]
This is not a universal LS solver in the generalized-inverse sense, but it is a structurally exact example of low-rank-updated least-squares subproblems [1608.01619].

A matrix-valued total least-squares counterpart appears in positive semidefinite TLS. For
\[
DX\simeq T,\qquad D,T\in\mathbb{R}^{m\times n},\quad m\ge n,
\]
with \(X\in\mathbb{R}^{n\times n}\) symmetric PSD, the paper defines a new error model using \(X^\dagger\) and the range/null-space decomposition of \(X\). For known rank \(r\),
\[
X=YS^2Y^T,\qquad Y^TY=I,
\]
and the fixed-rank PSDTLS problem reduces to
\[
\min_{Y,S}\operatorname{tr}\!\big(Y^TAYS^2-Y^TCY+Y^TBYS^{-2}\big),
\]
with
\[
A=D^TD,\qquad B=T^TT,\qquad C=D^TT+T^TD.
\]
The solver alternates a closed-form update
\[
s_i=\left(\frac{y_i^TBy_i}{y_i^TAy_i}\right)^{1/4}
\]
with a Newton step on the Stiefel manifold, and is stated to converge locally at least quadratically. A rank-search wrapper then solves the minimum-rank problem by increasing \(r\) until the PSDTLS error falls below a prescribed threshold \(e\) [1407.3894].

## 6. Unifying themes, objective conflicts, and persistent misconceptions

The main conceptual fault line in this literature is between **minimum rank**, **minimum norm**, and **rank deficiency**. These are often adjacent in notation but distinct in substance.

First, minimum-rank universal least-squares solvers in the strict generalized-inverse sense are matrices \(H\) of minimum possible rank that solve least squares for every \(b\); their defining property is reflexivity \((P2)\) on top of the universal least-squares conditions \((P1)+(P3)\) [2509.04264]. By contrast, MINRES-QLP, LSRN, REK, and related methods are minimum-*length* least-squares solvers: their canonical output is \(A^\dagger b\), not a minimum-rank matrix [1301.2707]. This distinction is frequently blurred because both settings use the Moore–Penrose pseudoinverse, but the optimization target is different.

Second, the phrase “universal” is always conditional. MINRES-QLP is universal across nonsingular, singular compatible, and singular incompatible problems, but only within the symmetric/Hermitian square class [1301.2707]. LSRN is universal across strongly overdetermined and strongly underdetermined rectangular systems, dense/sparse/operator representations, and rank-deficient inputs, but it is designed specifically for strongly rectangular systems [1109.5981]. IRLS-M is operator-agnostic at the level of measurement maps \(S:M_{n\times p}\to\mathbb C^m\), yet its fastest implementations depend on separable structure such as matrix completion [1010.2471]. This suggests that universality in current usage is almost always *structured universality*, not solver-independence without assumptions.

Third, acceleration mechanisms can change the selected least-squares solution. In MINRES-QLP, SPD preconditioning alters the norm in which minimum length is defined, so the Euclidean pseudoinverse interpretation is no longer automatic [1301.2707]. In transformed RRGMRES, the map
\[
x=C A^{\rm T}z
\]
always yields a least-squares solution of \(Ax=b\), but it is minimum-norm only for \(C=I\) [2504.09891]. The same tradeoff appears in low-rank update and recursive methods: one can gain substantial speed or adaptivity, but the exact canonicality statement must be checked in the transformed variables.

Finally, sparsity and minimum rank conflict even when both are imposed on the solver rather than the recovered unknown. The 2025 generalized-inverse study shows that adding the exact minimum-rank condition \((P2)\) materially reduces the degree of sparsification achievable by \(1\)-norm minimization [2509.04264]. A plausible implication is that “simple” universal solvers are not simple in every sense simultaneously: rank, sparsity, numerical robustness, and algebraic universality do not align automatically.

Taken together, these results delimit the modern meaning of the field. Minimum-rank universal least-squares solvers are not one class of objects but a family of algebraic and algorithmic constructions organized around three recurring questions: how to solve least squares for *all* right-hand sides in a prescribed structural class, how to preserve a canonical solution such as \(A^\dagger b\) in rank-deficient regimes, and how to exploit low-rank structure—of the solver, the unknown, or the update—without sacrificing the defining least-squares property.

Source: https://www.emergentmind.com/topics/minimum-rank-universal-least-squares-solvers