Randomized Golub–Kahan Methods

• Randomized Golub–Kahan approaches are Krylov subspace methods that use random sketching to approximate high-dimensional inner products in large-scale least-squares and inverse problems.
• They modify the classical Golub–Kahan bidiagonalization by replacing short recurrences with sketched inner products, resulting in upper Hessenberg and triangular matrices instead of strict bidiagonality.
• These methods enable efficient hybrid regularization—such as in rLSQR and sFLSMR—showing promise in applications like image deblurring, seismic tomography, and other inverse problems when sketch sizes are optimally chosen.

Searching arXiv for the cited papers to ground the article. arxiv_search(query="^^^^^^^^3randomized Golub-Kahan inverse problems sketched LSQR LSMR3^^^^^^^^", max_results=^^^^3\3randomized Golub-Kahan inverse problems sketched LSQR LSMR3^^^^) arxiv_search(query="^^^^^^^^3\3 Krylov methods for inverse problems^^^^^^^^3\3 OR ^^^^^^^^3\3 Flexible LSQR and LSMR with applications to inverse problems^^^^^^^^3\3 max_results=^^^^3\3randomized Golub-Kahan inverse problems sketched LSQR LSMR3^^^^) Randomized Golub–Kahan approach denotes a family of Krylov subspace procedures for large-scale linear least-squares and inverse problems in which the classical Golub–Kahan bidiagonalization is modified by random sketching, sketched inner products, or related randomized dimension-reduction devices. In the most direct formulation, the method constructs coupled approximation spaces for rectangular PRESERVED_PLACEHOLDER_^^^^3randomized Golub-Kahan inverse problems sketched LSQR LSMR3^^^^ while replacing expensive high-dimensional inner products by inner products of sketched vectors, thereby producing small projected problems suitable for least-squares solves and hybrid Tikhonov regularization (&&&^^^^3randomized Golub-Kahan inverse problems sketched LSQR LSMR3^^^^&&&). In a broader usage, the label also covers sketched flexible LSQR/LSMR variants and nearby settings in which deterministic Golub–Kahan bidiagonalization is contrasted or combined with randomized SVD, randomized range finding, or randomized sampling rather than replaced outright (&&&^^^^3\3^^^^&&&).

^^^^3\3^^^^. Classical Golub–Kahan foundation

The classical Golub–Kahan bidiagonalization (GKB) is a Lanczos-type process for rectangular matrices. For PRESERVED_PLACEHOLDER_^^^^3\3^^^^ and PRESERVED_PLACEHOLDER_^^^^3 OR \3^^^^, it builds orthonormal bases $V_k\in\mathbb{R}^{n\times k}$ and $U_{k+1}\in\mathbb{R}^{m\times (k+1)}$ such that

$$A V_k = U_{k+1} B_k,\qquad A^T U_{k+1} = V_{k+1} B_k^T,$$

with $B_k$ lower bidiagonal. The associated subspaces satisfy

$$R(V_k)=\mathcal{K}_k(A^T A,A^T b),\qquad R(U_{k+1})=\mathcal{K}_{k+1}(A A^T,b).$$

This structure underlies LSQR and LSMR: LSQR computes $x_k=V_k y_k$ with

$$y_k=\arg\min_{y\in\mathbb{R}^k}\|B_k y-\beta_1 e_1\|_2,$$

while LSMR minimizes a projected form of the normal-equations residual PRESERVED_PLACEHOLDER_^^^^3\3randomized Golub-Kahan inverse problems sketched LSQR LSMR3^^^^ on the same Krylov space (&&&^^^^3randomized Golub-Kahan inverse problems sketched LSQR LSMR3^^^^&&&).

The importance of this deterministic baseline is structural rather than merely historical. Bidiagonality yields short recurrences, inexpensive projected problems, and a direct connection between the spectral content of PRESERVED_PLACEHOLDER_^^^^3\3\3^^^^ and the regularization behavior of iterative solvers. Randomized Golub–Kahan methods preserve the alternating PRESERVED_PLACEHOLDER_^^^^3\3 OR \3^^^^- and PRESERVED_PLACEHOLDER_^^^^3\33^^^^-driven subspace construction, but alter how orthogonality and projection are enforced.

^^^^3 OR \3^^^^. Randomized Golub–Kahan in the strict sense

In the formulation of Chung and Gazzola, randomized Golub–Kahan (rGK) is a sketched version of GKB for general rectangular matrices. Two sketching matrices are introduced,

PRESERVED_PLACEHOLDER_^^^^3\3^^^^4

with PRESERVED_PLACEHOLDER_^^^^3\3^^^^5 and PRESERVED_PLACEHOLDER_^^^^3\3^^^^6, so that inner products in PRESERVED_PLACEHOLDER_^^^^3\3^^^^7 and PRESERVED_PLACEHOLDER_^^^^3\3^^^^8 are approximated by sketched inner products. The relevant probabilistic object is an PRESERVED_PLACEHOLDER_^^^^3\3^^^^9-embedding: for a subspace PRESERVED_PLACEHOLDER_^^^^3 OR \3randomized Golub-Kahan inverse problems sketched LSQR LSMR3^^^^,

PRESERVED_PLACEHOLDER_^^^^3 OR \3\3^^^^

The paper uses PRESERVED_PLACEHOLDER_^^^^3 OR \3 OR \3^^^^-oblivious subspace embeddings, such as Gaussian or SRHT sketches, so that these relations hold for any subspace of dimension up to PRESERVED_PLACEHOLDER_^^^^3 OR \33^^^^ with probability at least PRESERVED_PLACEHOLDER_^^^^3 OR \3^^^^4 (&&&^^^^3randomized Golub-Kahan inverse problems sketched LSQR LSMR3^^^^&&&).

The resulting factorization differs fundamentally from classical bidiagonalization. After PRESERVED_PLACEHOLDER_^^^^3 OR \3^^^^5 iterations, rGK produces

PRESERVED_PLACEHOLDER_^^^^3 OR \3^^^^6

where PRESERVED_PLACEHOLDER_^^^^3 OR \3^^^^7 is upper Hessenberg and PRESERVED_PLACEHOLDER_^^^^3 OR \3^^^^8 is upper triangular. The bases are orthogonal only with respect to the sketched inner products, not the original Euclidean inner products. This destroys the symmetry relations that force strict bidiagonality in classical GKB. As a consequence, the randomized variant uses long Gram–Schmidt-type recurrences rather than three-term recurrences. The loss of bidiagonality is not an implementation detail; it is the defining algebraic effect of replacing exact orthogonality by sketched orthogonality.

This factorization supports several solvers. Randomized LSQR (rLSQR) computes PRESERVED_PLACEHOLDER_^^^^3 OR \3^^^^9 from

$V_k\in\mathbb{R}^{n\times k}$^^^^3randomized Golub-Kahan inverse problems sketched LSQR LSMR3^^^^

which is equivalent to minimizing the sketched residual norm $V_k\in\mathbb{R}^{n\times k}$^^^^3\3^^^^ over the rGK Krylov subspace. Randomized CGLS (rCGLS) enforces a sketched Petrov–Galerkin condition on $V_k\in\mathbb{R}^{n\times k}$^^^^3 OR \3^^^^. Randomized LSMR (rLSMR) minimizes a sketched form of the normal-equations residual,

$V_k\in\mathbb{R}^{n\times k}$3

again on the rGK subspace. Hybrid variants regularize the projected problems rather than the full system, for example

$V_k\in\mathbb{R}^{n\times k}$4

Because the subspace $V_k\in\mathbb{R}^{n\times k}$5 does not depend on $V_k\in\mathbb{R}^{n\times k}$6, discrepancy principle, GCV, and WGCV can be applied cheaply on the small projected system. Numerical experiments in image deblurring and seismic tomography show that these randomized methods can reproduce the regularization behavior of deterministic methods when the sketch sizes are chosen appropriately, whereas very small $V_k\in\mathbb{R}^{n\times k}$7 lead to erratic behavior after a few iterations (&&&^^^^3randomized Golub-Kahan inverse problems sketched LSQR LSMR3^^^^&&&).

3. Sketched flexible LSQR and LSMR

A second major line replaces exact GKB not by sketched Gram–Schmidt alone, but by a flexible Golub–Kahan framework in which the approximation space itself is modified. Flexible LSQR and flexible LSMR use a basis

$V_k\in\mathbb{R}^{n\times k}$8

where $V_k\in\mathbb{R}^{n\times k}$9 may encode prior information, nonstationary preconditioning, low-rank truncation of image-shaped basis vectors, or the use of an unmatched transpose $U_{k+1}\in\mathbb{R}^{m\times (k+1)}$^^^^3randomized Golub-Kahan inverse problems sketched LSQR LSMR3^^^^. A representative example is the low-rank truncation operator

$U_{k+1}\in\mathbb{R}^{m\times (k+1)}$^^^^3\3^^^^

obtained by reshaping $U_{k+1}\in\mathbb{R}^{m\times (k+1)}$^^^^3 OR \3^^^^ as an image and keeping a rank-$U_{k+1}\in\mathbb{R}^{m\times (k+1)}$3 truncated SVD. In the flexible setting, the small projected matrices are no longer bidiagonal; upper triangular and upper Hessenberg factors appear, and long recurrences become the dominant cost (&&&^^^^3\3^^^^&&&).

The sketched flexible methods sFLSQR and sFLSMR recover short recurrences by combining partial orthogonalization with a sketching matrix $U_{k+1}\in\mathbb{R}^{m\times (k+1)}$4. Instead of solving the full residual minimization problem in $U_{k+1}\in\mathbb{R}^{m\times (k+1)}$5, sFLSQR solves

$U_{k+1}\in\mathbb{R}^{m\times (k+1)}$6

while sFLSMR solves

$U_{k+1}\in\mathbb{R}^{m\times (k+1)}$7

If $U_{k+1}\in\mathbb{R}^{m\times (k+1)}$8 is a $U_{k+1}\in\mathbb{R}^{m\times (k+1)}$9-oblivious $$A V_k = U_{k+1} B_k,\qquad A^T U_{k+1} = V_{k+1} B_k^T,$$^^^^3randomized Golub-Kahan inverse problems sketched LSQR LSMR3^^^^-subspace embedding for the relevant space, then for sFLSQR

$$A V_k = U_{k+1} B_k,\qquad A^T U_{k+1} = V_{k+1} B_k^T,$$^^^^3\3^^^^

Thus the sketched problem yields, up to the factor $$A V_k = U_{k+1} B_k,\qquad A^T U_{k+1} = V_{k+1} B_k^T,$$^^^^3 OR \3^^^^, the best residual attainable in the flexible subspace.

The distinction between sFLSQR and sFLSMR becomes important in noisy inverse problems. The analysis and experiments show that sFLSQR behaves well when the residual is small, but for moderate or large noise the residual inflation caused by sketching can be significant. By contrast, sFLSMR minimizes a sketched normal-equations residual; because $$A V_k = U_{k+1} B_k,\qquad A^T U_{k+1} = V_{k+1} B_k^T,$$3 damps components associated with small singular values, sFLSMR is more robust when the least-squares residual is not small. This spectral effect is visible in controlled ill-posed test problems and in ^^^^3 OR \3^^^^D and 3D CT experiments with unmatched transpose implementations on GPU, where sFLSMR attains residual norms and reconstruction errors comparable or slightly better than flexible deterministic variants while approximately halving runtime in reported tests (&&&^^^^3\3^^^^&&&).

4. Regularization and discrepancy control

Randomized Golub–Kahan methods are not merely accelerations of least-squares solves; they are projection frameworks for regularization. In the hybrid rLSQR setting, the projected Tikhonov problem

$$A V_k = U_{k+1} B_k,\qquad A^T U_{k+1} = V_{k+1} B_k^T,$$4

is equivalent to minimizing, over the rGK Krylov subspace, a sketched data-fit term together with a sketched regularization term,

$$A V_k = U_{k+1} B_k,\qquad A^T U_{k+1} = V_{k+1} B_k^T,$$5

Because $$A V_k = U_{k+1} B_k,\qquad A^T U_{k+1} = V_{k+1} B_k^T,$$6, the discrepancy principle and GCV-type criteria can be evaluated cheaply at each iteration. The numerical study reported that discrepancy principle and WGCV track the benchmark-optimal $$A V_k = U_{k+1} B_k,\qquad A^T U_{k+1} = V_{k+1} B_k^T,$$7 more reliably than GCV, while applying rLSQR directly to the damped least-squares form of Tikhonov yields poorer and more irregular performance than hybrid rLSQR on the projected problem (&&&^^^^3randomized Golub-Kahan inverse problems sketched LSQR LSMR3^^^^&&&).

A nearby deterministic development clarifies why Golub–Kahan projections are particularly suitable for discrepancy-based parameter choice. For iterated Tikhonov regularization with $$A V_k = U_{k+1} B_k,\qquad A^T U_{k+1} = V_{k+1} B_k^T,$$8, the residual norm can be written as

$$A V_k = U_{k+1} B_k,\qquad A^T U_{k+1} = V_{k+1} B_k^T,$$9

and partial Golub–Kahan bidiagonalization yields reduced quantities

$B_k$^^^^3randomized Golub-Kahan inverse problems sketched LSQR LSMR3^^^^

satisfying

$B_k$^^^^3\3^^^^

Gauss and Gauss–Radau quadrature then provide lower and upper bounds on the full residual norm, so that $B_k$^^^^3 OR \3^^^^ can be chosen without repeatedly solving large regularized systems. The same paper explicitly notes that this “partial” bidiagonalization is exactly what would be randomized in a future randomized Golub–Kahan, since controlling the Krylov dimension $B_k$3 trades approximation quality against cost. This suggests that discrepancy-principle enforcement via reduced spectral measures and quadrature is structurally compatible with randomized Golub–Kahan variants (Furchì et al., 29 Jun 2026).

5. Adjacent randomized and low-rank variants

The phrase “randomized Golub–Kahan approach” is used narrowly in some papers and only implicitly in others. Two related lines of work place deterministic GKB side by side with randomized low-rank approximation rather than replacing it by sketched inner products.

In image deblurring with Kronecker product sum approximations, enlarged Golub–Kahan bidiagonalization (EGKB) is used to compute a low-rank approximate SVD of the reordered blur matrix $B_k$4, while a randomized SVD (RSVD) serves as the randomized comparator. EGKB enlarges the Krylov dimension to $B_k$5, performs an SVD of a small bidiagonal matrix, and then truncates back to rank $B_k$6. A new stopping test based on $B_k$7 and $B_k$8 selects $B_k$9. The RSVD implementation uses the rank $$R(V_k)=\mathcal{K}_k(A^T A,A^T b),\qquad R(U_{k+1})=\mathcal{K}_{k+1}(A A^T,b).$$^^^^3randomized Golub-Kahan inverse problems sketched LSQR LSMR3^^^^ estimated by EGKB, so the deterministic Golub–Kahan process supplies the truncation dimension for the randomized method. Reported experiments on blur matrices of size $$R(V_k)=\mathcal{K}_k(A^T A,A^T b),\qquad R(U_{k+1})=\mathcal{K}_{k+1}(A A^T,b).$$^^^^3\3^^^^, $$R(V_k)=\mathcal{K}_k(A^T A,A^T b),\qquad R(U_{k+1})=\mathcal{K}_{k+1}(A A^T,b).$$^^^^3 OR \3^^^^, and $$R(V_k)=\mathcal{K}_k(A^T A,A^T b),\qquad R(U_{k+1})=\mathcal{K}_{k+1}(A A^T,b).$$3 show that EGKB and RSVD capture the dominant singular values of $$R(V_k)=\mathcal{K}_k(A^T A,A^T b),\qquad R(U_{k+1})=\mathcal{K}_{k+1}(A A^T,b).$$4 with comparable quality, while EGKB is often cheaper; the paper states that EGKB “competes favorably with the randomized singular value decomposition for estimating the approximate singular value decomposition” (Alsubhi et al., 2024).

In FFT-based gravity and magnetic inversion, the comparison is again between deterministic GKB and RSVD, both using fast matrix operations derived from BTTB structure and ^^^^3 OR \3^^^^D FFT embeddings. The study concludes that randomized algorithms are to be preferred for gravity inversion, with projected spaces of size approximately $$R(V_k)=\mathcal{K}_k(A^T A,A^T b),\qquad R(U_{k+1})=\mathcal{K}_{k+1}(A A^T,b).$$5, whereas GKB leads to more efficient implementations for magnetic inversion, again with spaces of size approximately $$R(V_k)=\mathcal{K}_k(A^T A,A^T b),\qquad R(U_{k+1})=\mathcal{K}_{k+1}(A A^T,b).$$6, and even $$R(V_k)=\mathcal{K}_k(A^T A,A^T b),\qquad R(U_{k+1})=\mathcal{K}_{k+1}(A A^T,b).$$7 when $$R(V_k)=\mathcal{K}_k(A^T A,A^T b),\qquad R(U_{k+1})=\mathcal{K}_{k+1}(A A^T,b).$$8. The paper does not explicitly use the phrase “randomized Golub–Kahan approach,” but it presents a concrete case in which GKB and randomized SVD occupy analogous roles as projection engines inside the same inverse-problem workflow. A plausible implication is that the advantages of randomization are problem dependent: gravity data favored randomized low-rank approximation, while magnetic data favored the fuller spectral access of GKB (&&&^^^^3\3randomized Golub-Kahan inverse problems sketched LSQR LSMR3^^^^&&&).

arXiv id	Setting	Relation to the topic
(&&&^^^^3randomized Golub-Kahan inverse problems sketched LSQR LSMR3^^^^&&&)	Rectangular inverse problems	Explicit randomized Golub–Kahan with sketched inner products
(&&&^^^^3\3^^^^&&&)	Flexible LSQR/LSMR	Sketched flexible GK-like recurrences
(Alsubhi et al., 2024)	Kronecker image deblurring	EGKB contrasted with RSVD; rank chosen by EGKB
(&&&^^^^3\3randomized Golub-Kahan inverse problems sketched LSQR LSMR3^^^^&&&)	Gravity and magnetic inversion	GKB and RSVD compared in FFT-based projection methods

6. Generalizations, limitations, and scope

The Golub–Kahan structure has also been generalized beyond deterministic and sketched least-squares solvers. In large-scale hierarchical Bayesian inverse problems, generalized Golub–Kahan (genGK) bidiagonalization is used to construct low-dimensional approximations of conditional means and conditional covariances for Metropolis–Hastings within Gibbs. The proposal distributions are Gaussian, built either from a low-rank approximation to the conditional covariance or from a preconditioned Lanczos sampler. The paper does not explicitly label this as a randomized Golub–Kahan method, but the overall procedure is stochastic because MCMC sampling, Gaussian proposal generation, and acceptance decisions are random. The authors also compare their genGK-based proposals to randomized SVD-based alternatives and note that genGK can provide comparable or better acceptance behavior without requiring explicit factorization of the prior covariance (&&&^^^^3\3^^^^5&&&).

A different extension appears in the computation of generalized matrix functions. Rational Krylov methods replace the polynomial Krylov spaces underlying classical Golub–Kahan by rational spaces for $$R(V_k)=\mathcal{K}_k(A^T A,A^T b),\qquad R(U_{k+1})=\mathcal{K}_{k+1}(A A^T,b).$$9 and $x_k=V_k y_k$^^^^3randomized Golub-Kahan inverse problems sketched LSQR LSMR3^^^^. The projected matrix is no longer bidiagonal, but the paper shows that its quasiseparable structure permits short recurrences that generalize the Golub–Kahan bidiagonalization. When all poles are chosen equal to infinity, the algorithm reduces to standard Golub–Kahan. This is not a randomized method, but it identifies an algebraic mechanism—structured projected matrices rather than exact bidiagonality—through which short recurrences can survive beyond the classical setting. That observation is relevant to randomized variants, especially those that seek to preserve structured small problems after sketching or compression (&&&^^^^3\3^^^^6&&&).

Several limitations recur across the literature. In strict rGK, too-small sketches lead to unstable or erratic behavior after a few iterations. In hybrid rLSQR, GCV tends to over-regularize, while discrepancy principle and WGCV behave more reliably. In sketched flexible methods, sFLSQR is effective when the residual is small, but sFLSMR is preferable in moderate-to-high noise regimes because it minimizes a sketched normal-equations residual. In adjacent deterministic-versus-randomized comparisons, RSVD is not uniformly superior: it is preferred for gravity inversion in the cited FFT-based study, whereas GKB is more efficient for magnetic inversion. These results indicate that “randomized Golub–Kahan” is not a single algorithmic recipe but a spectrum of methods whose success depends on the interaction between sketch size, spectral decay, noise level, and the role assigned to the projected subspace (&&&^^^^3randomized Golub-Kahan inverse problems sketched LSQR LSMR3^^^^&&&, &&&^^^^3\3^^^^&&&, &&&^^^^3\3randomized Golub-Kahan inverse problems sketched LSQR LSMR3^^^^&&&).

A common misconception is to identify randomized Golub–Kahan with randomized SVD. The literature distinguishes these notions. In the strict rGK sense, randomization enters the orthogonalization and projection process itself through sketched inner products and subspace embeddings. In the EGKB/RSVD and GKB/RSVD comparison papers, by contrast, GKB remains deterministic and randomization enters through a separate low-rank approximation engine. Another misconception is that short bidiagonal recurrences survive unchanged after randomization. In the strict sketched-inner-product formulation they do not: bidiagonality is replaced by upper Hessenberg and upper triangular small matrices. Short recurrences reappear only in different formulations, such as sketched flexible methods with partial orthogonalization or rational generalizations with quasiseparable projected structure.