Minimum-Rank Universal LS Solvers

• Minimum-rank universal least-squares solvers are defined either as matrices that deliver solutions for every right-hand side or as algorithms that compute minimum-length pseudoinverses.
• They employ varied methodologies including generalized inverse formulations, Krylov subspace techniques, and low-rank update strategies to address singular, inconsistent, or rank-deficient systems.
• Structured optimization and sparsity-promoting formulations are used to balance numerical robustness, universality, and efficiency in solving diverse least-squares problems.

“Minimum-rank universal least-squares solvers” names a heterogeneous research area rather than a single algorithmic paradigm. In one line of work, the object of interest is a matrix $H$ that maps every right-hand side $b$ to a least-squares solution $Hb$, and “minimum rank” refers to the smallest possible rank of such a universal solver. In another, the emphasis is on algorithms that remain valid for singular, inconsistent, or rank-deficient least-squares problems and return the canonical minimum-length solution $A^\dagger b$. A third line studies minimum-rank or low-rank matrix recovery by converting rank-structured estimation into a sequence of weighted least-squares problems. A fourth exploits low-rank structure in updates, recursion, or total least-squares formulations. The common theme is universality across right-hand sides, regimes, or structural perturbations, but the meaning of “minimum rank” varies sharply across these settings (Machado et al., 4 Sep 2025).

1. Terminological scope and principal interpretations

The literature uses the phrase in several non-equivalent senses. The distinctions are structural, not cosmetic, because they determine the admissible unknowns, guarantees, and algorithmic machinery.

Interpretation	Unknown	Canonical requirement
Universal solver matrix	$H\in\mathbb{R}^{n\times m}$	$Hb$ solves LS for every $b$
Minimum-length universal LS algorithm	vector $x$	return $A^\dagger b$
Low-rank recovery via LS	matrix $X$	promote low rank through weighted LS
Low-rank-update LS solver	updated solution $b$0	exploit $b$1

In the generalized-inverse formulation, universality means that the same matrix $b$2 must solve the least-squares problem for all right-hand sides $b$3 (Machado et al., 4 Sep 2025). In Krylov and randomized numerical linear algebra, universality instead means coverage of multiple regimes—consistent or inconsistent, full-rank or rank-deficient, overdetermined or underdetermined—while preserving the Moore–Penrose solution $b$4 (Choi et al., 2013). In low-rank recovery, the target is typically not a universal solver matrix at all, but an unknown matrix $b$5 with small rank, recovered from linear measurements by repeatedly solving weighted least-squares subproblems (Fornasier et al., 2010). Finally, some exact linear-algebra methods treat “low rank” as a property of the update $b$6, so that the least-squares solver is reused under low-rank perturbations rather than recomputed from scratch (Güttel et al., 2024).

This multiplicity of meanings is essential for reading the area correctly. A paper on minimum-length pseudoinverse solvers for singular systems is usually not a paper on minimum-rank matrix optimization; conversely, a paper on affine rank minimization may not produce a universal solver for varying right-hand sides.

2. Universal least-squares solvers as generalized inverses of minimum possible rank

The most literal realization of a minimum-rank universal least-squares solver appears in the generalized-inverse framework. For $b$7, let $b$8. The Penrose properties are

$b$9

$Hb$0

$Hb$1

$Hb$2

A classical result recalled in recent work is that $Hb$3 is a universal least-squares solver iff it satisfies $Hb$4, equivalently

$Hb$5

or, again equivalently,

$Hb$6

The minimum-rank refinement is exact: among generalized inverses, $Hb$7, with equality iff $Hb$8 satisfies $Hb$9. Therefore a minimum-rank universal least-squares solver is precisely an ah-symmetric reflexive generalized inverse, i.e. a matrix satisfying

$A^\dagger b$0

This converts minimum rank from an optimization objective into an algebraic characterization (Machado et al., 4 Sep 2025).

The same work gives a reduced SVD parameterization. If

$A^\dagger b$1

with $A^\dagger b$2 diagonal and positive, $A^\dagger b$3, and $A^\dagger b$4, then every minimum-rank universal least-squares solver has the form

$A^\dagger b$5

This shows that universality fixes the action on $A^\dagger b$6, while the free parameter $A^\dagger b$7 acts through nullspace directions without increasing the rank beyond $A^\dagger b$8 (Machado et al., 4 Sep 2025).

The same paper develops sparse formulations by minimizing the matrix entrywise $A^\dagger b$9-norm over this feasible class, for example

$H\in\mathbb{R}^{n\times m}$0

and an equivalent single linear characterization,

$H\in\mathbb{R}^{n\times m}$1

A Douglas–Rachford splitting method based on the linear formulation

$H\in\mathbb{R}^{n\times m}$2

performed best computationally; the projection step admits the closed form

$H\in\mathbb{R}^{n\times m}$3

On test instances with $H\in\mathbb{R}^{n\times m}$4 and $H\in\mathbb{R}^{n\times m}$5, the fixed-point DRS variant solved all tested sizes up to $H\in\mathbb{R}^{n\times m}$6, with representative runtimes $H\in\mathbb{R}^{n\times m}$7 sec at $H\in\mathbb{R}^{n\times m}$8, $H\in\mathbb{R}^{n\times m}$9 sec at $Hb$0, and $Hb$1 sec at $Hb$2 (Machado et al., 4 Sep 2025).

A notable structural tension also emerges here. Enforcing minimum rank via $Hb$3 substantially limits sparsity gains: compared with $Hb$4, the reported $Hb$5-norm reduction for minimum-rank universal least-squares solvers was about $Hb$6–$Hb$7, whereas plain universal least-squares solvers achieved about $Hb$8 reduction (Machado et al., 4 Sep 2025).

3. Universal least-squares algorithms for singular, inconsistent, and rank-deficient systems

A different strand of the literature treats universality operationally: one algorithm should solve

$Hb$9

across compatible, incompatible, full-rank, and rank-deficient regimes, ideally returning the canonical minimum-length solution

$b$0

For symmetric or Hermitian square systems, MINRES-QLP gives the clearest Krylov realization of this objective. It addresses

$b$1

possibly shifted to $b$2, through the unified least-squares problem

$b$3

and proves that the terminal iterate is the minimum-length solution of this problem. Its defining innovation is a right orthogonal transformation of the MINRES projected tridiagonal problem,

$b$4

yielding a rank-revealing QLP decomposition without column pivoting. This lets the method identify the singular projected case and compute the pseudoinverse solution $b$5, whereas classical MINRES generally does not return the min-length solution for singular systems. The method remains a short-recurrence Lanczos solver and uses one product $b$6 per iteration; however, its clean pseudoinverse interpretation is fundamentally an unpreconditioned Euclidean-norm statement, because SPD preconditioning changes the norm in which minimum length is defined (Choi et al., 2013).

LSRN extends universality to strongly rectangular systems. It solves

$b$7

for $b$8 or $b$9, allows rank deficiency, and returns the min-length solution $x$0. The method applies a random normal projection, computes an economy SVD of the sketch, and then solves the preconditioned problem by LSQR or the Chebyshev semi-iterative method. Its key theoretical feature is that the condition number of the preconditioned system depends on $x$1, not on the nonzero singular spectrum of $x$2, so iteration counts become predictable in terms of sketch size and rank. Because $x$3 is accessed only through matrix-matrix and matrix-vector products, the method accommodates dense, sparse, and operator-based representations (Meng et al., 2011).

Randomized Extended Kaczmarz pushes universality further toward arbitrary real matrices. It couples a row-action Kaczmarz iteration for $x$4 with a column-action projection process for an auxiliary vector $x$5, thereby removing the inconsistent component $x$6 online and maintaining iterates in $x$7. The method converges in expectation to the minimum Euclidean norm least-squares solution $x$8, with the bound

$x$9

Its advertised computational appeal is strongest for sparse, well-conditioned, strongly rectangular systems (Zouzias et al., 2012).

A recent refinement shows that, for Hermitian and complex-symmetric systems, standard MINRES can be upgraded to a pseudoinverse solver with essentially no extra asymptotic cost. If $A^\dagger b$0 and $A^\dagger b$1 are the final MINRES iterate and residual, then for Hermitian $A^\dagger b$2,

$A^\dagger b$3

equals $A^\dagger b$4. The correction removes the final null-space component of the MINRES least-squares solution and is attached only at termination. The same paper extends the idea to complex-symmetric systems and to PSD or even singular preconditioning, though in the preconditioned case exact recovery of the original $A^\dagger b$5 requires an additional range condition (Liu et al., 2023).

For arbitrary singular square systems, a different route is to solve

$A^\dagger b$6

with $A^\dagger b$7 symmetric positive definite, and then apply RRGMRES to the transformed matrix $A^\dagger b$8. Because $A^\dagger b$9 is range-symmetric, RRGMRES is guaranteed to compute a least-squares solution for arbitrary singular $X$0 and arbitrary $X$1, including inconsistent cases. The same work proposes an NR-SSOR inner-iteration right preconditioner, yielding strong empirical reductions in the normal-equation residual. However, the returned solution is minimum-norm only for $X$2; for general $X$3, universality is retained at the level of least-squares solvability, not pseudoinverse canonicity (Sugihara et al., 14 Apr 2025).

4. Low-rank recovery by repeated least-squares and fixed-rank least-squares estimation

When “minimum rank” refers to the unknown matrix rather than the solver, least-squares typically appears as an inner engine. A canonical example is affine rank minimization,

$X$4

with $X$5. IRLS-M addresses this problem indirectly through nuclear-norm and smoothed nuclear-norm surrogates. At iteration $X$6, it solves

$X$7

with weights updated from the singular values of $X$8. The associated variational objective is

$X$9

and the smoothed surrogate converges toward the nuclear norm as $b$00. Under the Strong Rank Null Space Property of order $b$01, if $b$02, the iterates converge to the unique nuclear-norm minimizer; if $b$03, every accumulation point minimizes the smoothed objective $b$04, with error controlled by the best rank-$b$05 approximation (Fornasier et al., 2010).

In high-dimensional trace regression, rank-constrained least squares is treated directly: $b$06 The primary contribution here is statistical rather than algorithmic: for $b$07, the in-sample prediction risk satisfies

$b$08

with high probability, and this bound is obtained without assumptions on the design matrix beyond independent observations and sub-Gaussian noise. The empirical optimization is carried out only approximately, by alternating minimization with multiple restarts and six initializations built from a coarse grid of nuclear-norm estimators and a spectral estimator. This is therefore a fixed-rank least-squares estimator with strong prediction and testing theory, not a universal minimum-rank solver (Law et al., 2021).

A more explicitly algorithmic minimum-rank template is to search over ranks and solve a nonlinear least-squares feasibility problem at each rank. For differentiable matrix systems

$b$09

the residuals are

$b$10

Low rank is imposed by factorization, e.g.

$b$11

for arbitrary rank-$b$12 matrices or

$b$13

for PSD matrices. The solver then applies Levenberg–Marquardt to

$b$14

at a fixed rank $b$15, and increments $b$16 until a numerical solution is found. This realizes minimum rank operationally as “the first rank whose fixed-rank least-squares subproblem becomes numerically feasible.” The framework is broad enough to include matrix completion and Euclidean embedding, but its guarantees are numerical and local rather than global (Le, 2017).

5. Low-rank updates, recursive rank-adaptive methods, and structured total least squares

Another interpretation of the topic focuses on algorithmic exploitation of low-rank structure rather than low-rank optimization itself. For full-column-rank rectangular matrices,

$b$17

and low-rank updates

$b$18

an exact Woodbury-type pseudoinverse formula yields an updated least-squares solver without recomputing a fresh QR factorization. The central identity is

$b$19

with

$b$20

$b$21

Operationally, once a solver for $b$22 is available, the updated solve reduces to one old least-squares solve, $b$23 solves with $b$24, and one dense $b$25 system. In MATLAB experiments with $b$26, $b$27, and update ranks $b$28, the reported speedup over recomputing QR ranged from about $b$29 to over $b$30, with relative forward error below $b$31 in all tests (Güttel et al., 2024).

For streaming or recursive least squares with rank-deficient matrices, rank-Greville maintains a rank factorization

$b$32

and updates the minimum-norm least-squares solution

$b$33

directly. The two essential cases are separated by the rejection

$b$34

If the new row increases rank, then

$b$35

if rank is preserved, then

$b$36

The per-observation update cost is $b$37, and solving from scratch by recursive processing costs $b$38. The method is asymptotically advantageous when $b$39, with experiments suggesting it becomes fastest roughly when $b$40, though its numerical robustness remains below that of QR/SVD-based LAPACK solvers (Staub et al., 2021).

Low-rank structure also enters total least squares through rank-one perturbations. In Gauss–Newton TLS, the nonlinear residual map is

$b$41

with Jacobian

$b$42

Thus each Gauss–Newton step solves an ordinary least-squares problem with a rank-one update of $b$43. The paper derives an explicit step length

$b$44

and proves global convergence of the resulting iteration for well-posed TLS, with

$b$45

This is not a universal LS solver in the generalized-inverse sense, but it is a structurally exact example of low-rank-updated least-squares subproblems (Fasino et al., 2016).

A matrix-valued total least-squares counterpart appears in positive semidefinite TLS. For

$b$46

with $b$47 symmetric PSD, the paper defines a new error model using $b$48 and the range/null-space decomposition of $b$49. For known rank $b$50,

$b$51

and the fixed-rank PSDTLS problem reduces to

$b$52

with

$b$53

The solver alternates a closed-form update

$b$54

with a Newton step on the Stiefel manifold, and is stated to converge locally at least quadratically. A rank-search wrapper then solves the minimum-rank problem by increasing $b$55 until the PSDTLS error falls below a prescribed threshold $b$56 (Bagherpour et al., 2014).

6. Unifying themes, objective conflicts, and persistent misconceptions

The main conceptual fault line in this literature is between minimum rank, minimum norm, and rank deficiency. These are often adjacent in notation but distinct in substance.

First, minimum-rank universal least-squares solvers in the strict generalized-inverse sense are matrices $b$57 of minimum possible rank that solve least squares for every $b$58; their defining property is reflexivity $b$59 on top of the universal least-squares conditions $b$60 (Machado et al., 4 Sep 2025). By contrast, MINRES-QLP, LSRN, REK, and related methods are minimum-length least-squares solvers: their canonical output is $b$61, not a minimum-rank matrix (Choi et al., 2013). This distinction is frequently blurred because both settings use the Moore–Penrose pseudoinverse, but the optimization target is different.

Second, the phrase “universal” is always conditional. MINRES-QLP is universal across nonsingular, singular compatible, and singular incompatible problems, but only within the symmetric/Hermitian square class (Choi et al., 2013). LSRN is universal across strongly overdetermined and strongly underdetermined rectangular systems, dense/sparse/operator representations, and rank-deficient inputs, but it is designed specifically for strongly rectangular systems (Meng et al., 2011). IRLS-M is operator-agnostic at the level of measurement maps $b$62, yet its fastest implementations depend on separable structure such as matrix completion (Fornasier et al., 2010). This suggests that universality in current usage is almost always structured universality, not solver-independence without assumptions.

Third, acceleration mechanisms can change the selected least-squares solution. In MINRES-QLP, SPD preconditioning alters the norm in which minimum length is defined, so the Euclidean pseudoinverse interpretation is no longer automatic (Choi et al., 2013). In transformed RRGMRES, the map

$b$63

always yields a least-squares solution of $b$64, but it is minimum-norm only for $b$65 (Sugihara et al., 14 Apr 2025). The same tradeoff appears in low-rank update and recursive methods: one can gain substantial speed or adaptivity, but the exact canonicality statement must be checked in the transformed variables.

Finally, sparsity and minimum rank conflict even when both are imposed on the solver rather than the recovered unknown. The 2025 generalized-inverse study shows that adding the exact minimum-rank condition $b$66 materially reduces the degree of sparsification achievable by $b$67-norm minimization (Machado et al., 4 Sep 2025). A plausible implication is that “simple” universal solvers are not simple in every sense simultaneously: rank, sparsity, numerical robustness, and algebraic universality do not align automatically.

Taken together, these results delimit the modern meaning of the field. Minimum-rank universal least-squares solvers are not one class of objects but a family of algebraic and algorithmic constructions organized around three recurring questions: how to solve least squares for all right-hand sides in a prescribed structural class, how to preserve a canonical solution such as $b$68 in rank-deficient regimes, and how to exploit low-rank structure—of the solver, the unknown, or the update—without sacrificing the defining least-squares property.