Generalized Krylov Subspace Methods

• Generalized Krylov Subspace Methods are a family of projection techniques that construct low-dimensional approximation spaces using augmented, rational, or nonlinear basis constructions.
• They extend classical Krylov methods by modifying the operator, basis, and residual conditions, enabling efficient solutions for linear, eigenvalue, and inverse problems.
• These methods balance computational efficiency and accuracy by reducing high-dimensional problems to small projected systems, while managing trade-offs such as storage and orthogonalization costs.

Generalized Krylov Subspace Method (GKSM) denotes a family of projection methods that preserve the central Krylov idea—constructing an approximation in a low-dimensional subspace generated from an operator and initial data—while generalizing the classical setting by changing the operator, the basis construction, the residual or optimality condition, the preconditioning strategy, or the problem class itself. In the cited literature the term is not attached to a single universally fixed definition; rather, it functions as an umbrella description for methods that augment, recycle, rationalize, extend, block, sketch, or nonlinearize standard Krylov constructions, while retaining subspace expansion and projected solves as the organizing principle (1811.09025, Burke et al., 2022, Palitta et al., 16 Jun 2026).

1. Classical Krylov framework as the point of departure

For a linear system

$$Ax=b,\qquad A\in\mathbb{R}^{n\times n},\ b\in\mathbb{R}^n,$$

classical Krylov methods start from an initial guess $x_0$ and residual

$r_0=b-Ax_0,$

then form the $m$-dimensional Krylov subspace

$$\mathcal{K}_m(A,r_0)=\operatorname{span}\{r_0,Ar_0,A^2r_0,\dots,A^{m-1}r_0\}.$$

The approximate solution is sought in affine form

$x_m=x_0+V_my_m,$

where the columns of $V_m$ span $\mathcal{K}_m(A,r_0)$. The defining distinction between methods is the residual condition imposed on

$r_m=b-Ax_m.$

GMRES uses the Arnoldi relation

$AV_m=V_{m+1}\bar H_m$

and chooses $x_0$0 from the least-squares problem

$x_0$1

thereby minimizing the Euclidean residual norm. CG, by contrast, assumes that $x_0$2 is symmetric positive definite, generates $x_0$3-conjugate directions, and produces iterates in

$x_0$4

that are optimal in the $x_0$5-norm sense (1811.09025).

This framework establishes the ingredients that later become generalized: subspace generation, basis construction, projection, residual orthogonality or minimization, and preconditioning. The same paper also emphasizes two already-generalizing mechanisms within standard Krylov practice: preconditioning, which replaces $x_0$6 by operators such as $x_0$7, and restarted GMRES, which changes the algorithmic cycling while preserving the projection idea (1811.09025).

2. Principal senses in which Krylov methods are generalized

The phrase “generalized Krylov subspace method” is used across several, partly overlapping, directions. The common pattern is that the classical polynomial space $x_0$8 is replaced by a richer search space or a more flexible projection framework.

Generalization	Representative form	Example
Augmentation and recycling	$x_0$9	Unprojected augmented FOM, R$r_0=b-Ax_0,$0GMRES, AugCG (Burke et al., 2022)
Rational and extended spaces	$r_0=b-Ax_0,$1, $r_0=b-Ax_0,$2 with positive and negative powers	Biorthogonal rational/extended Krylov; Hamiltonian EKS (Buggenhout et al., 2018, Benner et al., 2022)
Nonlinear Krylov-type spaces	$r_0=b-Ax_0,$3, $r_0=b-Ax_0,$4	nlGCR, nlGMRESR, nlGCRO, nlLGMRES (Werner et al., 18 Nov 2025)
Matrix and global variants	$r_0=b-Ax_0,$5, block or matrix-valued bases	TV regularization and global nonlinear methods (Bentbib et al., 2018, Werner et al., 18 Nov 2025)
Sketched projected methods	Compressed projected problems for $r_0=b-Ax_0,$6 and $r_0=b-Ax_0,$7	sGKS for large-scale Tikhonov regularization (Palitta et al., 16 Jun 2026)

Augmented methods enlarge the search space by adding a fixed subspace $r_0=b-Ax_0,$8, often interpreted as a recycling or approximate invariant subspace. In the unprojected framework, the residual condition becomes a generalized Petrov–Galerkin constraint involving

$r_0=b-Ax_0,$9

where

$m$0

This replaces a standard Krylov method on a projected operator by an equivalent formulation on the original operator with modified test space (Burke et al., 2022).

Rational and extended variants replace polynomial powers by rational functions or by mixed positive and negative powers. The rational case uses poles $m$1, while extended Krylov subspaces mix $m$2 and $m$3. For non-Hermitian problems this can be coupled with oblique projection and biorthogonal bases, leading to structured pencils rather than a single Hessenberg matrix (Buggenhout et al., 2018). In the Hamiltonian setting, the subspace

$m$4

is paired with $m$5-orthogonality so that the reduced matrix remains Hamiltonian (Benner et al., 2022).

Nonlinear generalizations keep the subspace logic but replace a fixed operator by local Jacobian actions. The nonlinear GCR framework constructs bases $m$6 such that $m$7, then uses the local model

$m$8

to define projected steps (Werner et al., 18 Nov 2025).

3. Projection principles and basis construction

Despite their diversity, GKSM variants remain projection methods. The projected problem may be a least-squares system, a reduced generalized eigenproblem, a projected Tikhonov functional, or a local linearized rootfinding model.

In the classical linear case, GMRES reduces the large system to a small least-squares problem in the Arnoldi basis, while CG replaces Euclidean minimal residual conditions by $m$9-inner-product optimality (1811.09025). That template persists in generalized settings but the objects being projected change. In augmented methods, the block system for the coefficients $$\mathcal{K}_m(A,r_0)=\operatorname{span}\{r_0,Ar_0,A^2r_0,\dots,A^{m-1}r_0\}.$$0 can be reduced to a smaller projected problem involving $$\mathcal{K}_m(A,r_0)=\operatorname{span}\{r_0,Ar_0,A^2r_0,\dots,A^{m-1}r_0\}.$$1, with the augmentation correction recovered afterward (Burke et al., 2022).

In nonlinear GCR-type methods, the outer update has the form

$$\mathcal{K}_m(A,r_0)=\operatorname{span}\{r_0,Ar_0,A^2r_0,\dots,A^{m-1}r_0\}.$$2

with $$\mathcal{K}_m(A,r_0)=\operatorname{span}\{r_0,Ar_0,A^2r_0,\dots,A^{m-1}r_0\}.$$3. The basis is not generated by powers of one fixed matrix; it is generated by residuals, Jacobian actions, and possibly nested inner Krylov solves for local linearized problems

$$\mathcal{K}_m(A,r_0)=\operatorname{span}\{r_0,Ar_0,A^2r_0,\dots,A^{m-1}r_0\}.$$4

This yields nonlinear analogues of GMRESR, GCRO, and LGMRES, all of which fit a generalized Krylov interpretation because the search space is nested, flexible, and Jacobian-dependent (Werner et al., 18 Nov 2025).

A different projection pattern appears in nonconvex inverse problems. In compressed sensing MRI, GKSM replaces the nonlinear objective

$$\mathcal{K}_m(A,r_0)=\operatorname{span}\{r_0,Ar_0,A^2r_0,\dots,A^{m-1}r_0\}.$$5

by a quadratic surrogate

$$\mathcal{K}_m(A,r_0)=\operatorname{span}\{r_0,Ar_0,A^2r_0,\dots,A^{m-1}r_0\}.$$6

then restricts the step to a low-dimensional subspace

$$\mathcal{K}_m(A,r_0)=\operatorname{span}\{r_0,Ar_0,A^2r_0,\dots,A^{m-1}r_0\}.$$7

where $$\mathcal{K}_m(A,r_0)=\operatorname{span}\{r_0,Ar_0,A^2r_0,\dots,A^{m-1}r_0\}.$$8 is found from a small least-squares problem involving $$\mathcal{K}_m(A,r_0)=\operatorname{span}\{r_0,Ar_0,A^2r_0,\dots,A^{m-1}r_0\}.$$9 and $x_m=x_0+V_my_m,$0. The basis is expanded from surrogate gradients

$x_m=x_0+V_my_m,$1

so the generalized subspace is “gradient-driven” rather than generated by powers of a fixed linear operator (Hong et al., 15 Aug 2025).

In large-scale Tikhonov regularization with a general regularization matrix $x_m=x_0+V_my_m,$2, GKS builds a subspace $x_m=x_0+V_my_m,$3 and solves the projected problem

$x_m=x_0+V_my_m,$4

The new basis direction is the gradient of the Tikhonov functional,

$x_m=x_0+V_my_m,$5

which is then appended to the basis. The sketched variant replaces full QR factorizations of $x_m=x_0+V_my_m,$6 and $x_m=x_0+V_my_m,$7 by QR factorizations of compressed matrices and, crucially, does not rely on orthogonality of the basis at all (Palitta et al., 16 Jun 2026).

4. Structure-preserving and eigenvalue-oriented families

One major GKSM branch is rational and extended Krylov theory for spectral problems and matrix functions. Rational Krylov subspaces $x_m=x_0+V_my_m,$8 are generated by poles $x_m=x_0+V_my_m,$9, and their projections can be represented by structured pencils. In the biorthogonal setting, oblique projection onto right and left rational Krylov spaces yields a tridiagonal pencil

$V_m$0

whose subdiagonal and superdiagonal ratios encode the poles of the right and left subspaces. This is a direct generalization of classical non-Hermitian biorthogonal Krylov methods, where a single tridiagonal matrix appears instead of a pencil (Buggenhout et al., 2018).

The Hamiltonian extended Krylov subspace method specializes this logic to large sparse Hamiltonian matrices. It constructs a $V_m$1-orthogonal basis of

$V_m$2

and projects the Hamiltonian matrix onto a small Hamiltonian matrix, preserving the ambient symplectic structure. A notable feature is the existence of short recurrences involving at most five previously generated basis vectors, despite the extended nature of the subspace (Benner et al., 2022).

In generalized eigenvalue computation, the inverse-free preconditioned Krylov method of Golub and Ye builds, at iteration $V_m$3, a Krylov subspace for the shifted operator

$V_m$4

then applies Rayleigh–Ritz to the projected pencil. The recent acceleration study extends this framework by introducing Nesterov-like and heavy-ball-like extrapolated vectors into the subspace construction, and by proving convergence for a more generalized choice of subspace in the block case. The reported numerical results show convergence in fewer outer iterations than LOBPCG with the same subspace size in some clustered-eigenvalue regimes, and generally fewer iterations than the base method when solving for multiple clustered eigenvalues with small dimension size (Baker et al., 21 Mar 2026).

A conceptually different eigenproblem-related generalization appears in the generalized Gearhart–Koshy acceleration. Starting from block Kaczmarz for a consistent non-square system, the method is shown to be a Krylov subspace method for an equivalent square system

$V_m$5

where $V_m$6 is the block Kaczmarz iteration matrix. The iterate $V_m$7 is the point of minimal Euclidean error norm in the affine Krylov space

$V_m$8

rather than a residual-minimizing iterate. The paper proves that, in exact arithmetic, the method cannot break down prematurely and makes progress at every step (Hegland et al., 2023).

5. Nonlinear, matrix-valued, and application-specific realizations

In nonlinear algebra, the nlKrylov framework treats $V_m$9 and matrix-valued $\mathcal{K}_m(A,r_0)$0 by constructing generalized nonlinear Krylov spaces and embedding them into inexact Newton theory. Its outer iteration is always nlGCR-like, while the inner routine may be nlGMRESR, nlGCRO, or nlLGMRES. The same paper extends the framework to matrix-valued rootfinding using block bases, Frobenius inner products, and global nonlinear Krylov variants such as GL-nlGCR and GL-nlGMRESR (Werner et al., 18 Nov 2025).

In total-variation regularization, a generalized matrix Krylov subspace method arises inside an augmented Lagrangian and ADM splitting. The TV model is reformulated into constrained subproblems, and the $\mathcal{K}_m(A,r_0)$1-update becomes a generalized Sylvester equation

$\mathcal{K}_m(A,r_0)$2

Rather than solving each such equation from scratch, the method builds matrix-valued subspaces of increasing dimension from residual information and solves reduced least-squares problems in those subspaces. This is a matrix-valued analogue of Krylov projection and is explicitly described as a generalized matrix Krylov subspace method (Bentbib et al., 2018).

In compressed sensing MRI with gradient-driven denoisers, GKSM is introduced as an optimization method for the nonconvex objective

$\mathcal{K}_m(A,r_0)$3

The method uses one denoiser-gradient evaluation per iteration, a variable metric $\mathcal{K}_m(A,r_0)$4, and a low-dimensional subspace updated from the residual of the quadratic surrogate. The paper establishes convergence guarantees in nonconvex settings and reports numerical experiments on spiral and radial acquisitions. It also states that the proposed optimization method is applicable to any linear inverse problem (Hong et al., 15 Aug 2025).

Large-scale Tikhonov regularization provides another explicit GKSM instantiation. Standard GKS is a projection method for general $\mathcal{K}_m(A,r_0)$5; sGKS sketches the tall projected matrices, removes explicit reorthogonalization, and shows that, in the absence of sketching in the projected solve, it produces iterates identical to those of standard GKS. Numerical experiments on image deblurring, X-ray computerized tomography, seismic travel-time tomography, and dynamic computerized tomography show that sGKS matches the reconstruction quality of standard GKS while significantly reducing per-iteration costs and overall wall-clock time (Palitta et al., 16 Jun 2026).

Other domains use the same idea under different names. In deep learning, Krylov Subspace Descent constructs on each outer iteration a subspace generated by the gradient and Hessian- or Gauss–Newton-vector products, then optimizes the nonlinear loss in that subspace with BFGS. The paper does not use the term GKSM explicitly, but this suggests a generalized Krylov interpretation in which curvature-generated subspaces are used as reduced optimization spaces (Vinyals et al., 2011). In quantum computing, measurement-efficient quantum Krylov subspace diagonalisation generalizes the basis from pure powers of $\mathcal{K}_m(A,r_0)$6 to functional bases

$\mathcal{K}_m(A,r_0)$7

including the Gaussian-power basis

$\mathcal{K}_m(A,r_0)$8

and then solves the generalized eigenproblem $\mathcal{K}_m(A,r_0)$9 classically. This is a generalized Krylov construction in which basis design is driven by measurement efficiency and robustness to statistical fluctuations (Zhang et al., 2023).

6. Convergence, cost, and conceptual scope

A persistent source of ambiguity is terminological. Some papers introduce explicit algorithms named generalized Krylov subspace methods, while others use the phrase to characterize a broader framework that subsumes GMRES-like, GCR-like, augmented, rational, or nonlinear variants. This suggests that GKSM is best understood as a family resemblance concept rather than as a single canonical algorithm (1811.09025, Werner et al., 18 Nov 2025, Burke et al., 2022).

Convergence theory correspondingly varies by variant. In the GMRES prototype, convergence is linked to polynomial approximation over the spectrum of $r_m=b-Ax_m.$0 (1811.09025). In nlKrylov, the central condition is an inexact-Newton forcing bound

$r_m=b-Ax_m.$1

under which local convergence follows; the framework also extends to singular Jacobians and matrix-valued problems (Werner et al., 18 Nov 2025). In the CS-MRI GKSM, descent is established for suitable $r_m=b-Ax_m.$2, step sizes vanish, and after the subspace phase the method inherits KL-based convergence-rate statements through its connection to a quasi-Newton proximal scheme (Hong et al., 15 Aug 2025). In sGKS, the key guarantee is not a classical Ritz-value theorem but a quasi-optimal residual bound controlled by the embedding quality of the sketches; when basis deterioration becomes significant, a small number of iterative refinement steps in the projected solve recovers the accuracy of the unsketched method (Palitta et al., 16 Jun 2026).

Computational trade-offs are equally central. Krylov methods are attractive because they replace impossible full inversions by matrix–vector or operator–vector products and small projected problems. Generalization often improves modeling flexibility or convergence but introduces new costs: storage of long bases, repeated orthogonalization, shifted solves in rational or extended spaces, basis conditioning in nonorthogonal variants, or measurement overhead in quantum settings. The literature therefore recurrently couples GKSM ideas with preconditioning, restarting, truncation, recycling, sketching, or short recurrences to control memory and arithmetic cost (Buggenhout et al., 2018, Benner et al., 2022, Palitta et al., 16 Jun 2026).

In this broad sense, GKSM refers to methods that retain the Krylov principle—approximation by projection onto a systematically generated subspace—while allowing the generating mechanism, the geometry of orthogonality, the projected objective, or the ambient problem class to depart substantially from the classical polynomial linear-system setting.