Shifted Krylov Subspace Solver

Updated 19 April 2026

Shifted Krylov subspace solvers are iterative methods that use the invariance of Krylov bases to efficiently solve multiple linear systems of the form (A + σI)x = b.
They reduce computational overhead by reusing matrix-vector products across different shifts, cutting down the cost of independent solves.
These techniques are crucial in applications like quantum chromodynamics, matrix function evaluations, and inverse problems, offering significant speedups and memory savings.

A shifted Krylov subspace solver is a class of iterative algorithms designed to simultaneously solve large families of linear systems whose coefficient matrices differ by scalar multiples of the identity, i.e., systems of the form $(A + \sigma_j I) x_j = b$ for distinct shifts $\sigma_j$ . The approach fundamentally leverages the shift-invariance property of Krylov subspaces— $K_m(A, b) = K_m(A+\sigma I, b)$ —which enables the efficient reuse of Krylov bases and associated computational structures across all shifts. Shifted Krylov subspace solvers are indispensable in scientific applications requiring the solution of parameterized linear systems, evaluation of matrix functions, large-scale spectral computations, and inverse problems.

1. Mathematical Foundations and Shift-Invariance

The mathematical core of shifted Krylov subspace solvers is the invariance of the Krylov subspace under shifts of the coefficient matrix. Explicitly, for any $A \in \mathbb{C}^{n \times n}$ , $b\in \mathbb{C}^n$ , and $\sigma \in \mathbb{C}$ ,

$K_m(A, b) = \operatorname{span}\left\{b, Ab, \ldots, A^{m-1}b \right\} = K_m(A + \sigma I, b).$

This property allows algorithms (e.g., Arnoldi, Lanczos, Hessenberg, block Arnoldi) to build a single Krylov basis which is then leveraged to solve all systems $(A + \sigma_j I)x_j = b$ using only $\mathcal{O}(m)$ matrix-vector products instead of $\mathcal{O}(n_f m)$ if solved independently for $\sigma_j$ 0 shifts. Extensions to rational Krylov and block Krylov frameworks allow even further flexibility, such as the inclusion of multiple right-hand sides or rational functions in matrix-function applications (Saibaba et al., 2012, Bakhos et al., 2016, Hoshi et al., 2020, Birk et al., 2012, Soodhalter, 2014).

For non-Hermitian systems or unrelated right-hand sides, a Sylvester equation formulation encapsulates the family as $\sigma_j$ 1, with $\sigma_j$ 2, so that a block Krylov subspace built for $\sigma_j$ 3 and $\sigma_j$ 4 provides simultaneous solution capability for all shifts (Soodhalter, 2014, Burke, 2022).

2. Algorithmic Frameworks: Seed Methods, Recycling, and Preconditioning

Classic shifted Krylov solvers include:

Shifted CG/BiCG: For Hermitian or general systems, a seed system (typically with $\sigma_j$ 5) is solved using three-term recurrences (CG/BiCG), and cheap scalar recurrences are used to update all shifted iterates and residuals (Ohno et al., 2010, Hoshi et al., 2020).
Block and Rational Krylov Methods: These generalize the polynomial subspace to block or rational functions, incorporating multiple right-hand sides and/or adaptive poles, crucial in matrix equations and function evaluations (Kolesnikov et al., 2014, Daas et al., 30 Jun 2025, Benner et al., 2022).
Flexible and Multipreconditioned Solvers: Flexible GMRES/FOM allow change of (possibly shift-dependent) preconditioners at each step (Saibaba et al., 2012). In MPGMRES, multiple shift-and-invert preconditioners are used to build a larger, more robust search space growing linearly in the number of preconditioners, which is essential when the shifts are widely scattered or not well clustered (Bakhos et al., 2016).
Recycling Algorithms and Restarting: Krylov subspace recycling strategies (e.g., GCRO-DR, rsbGMRES) retain spectral information across related systems or across restarts, accelerating convergence for sequences of shifted (or slowly changing) systems (Soodhalter et al., 2013, Soodhalter, 2014, Burke, 2022). Recycled subspaces may be tuned adaptively, e.g., via shift-specific harmonic Ritz vector extraction (Burke, 2022).

A summary table of core approaches:

Framework	Shift Strategy	Multiple RHS	Preconditioning	Recycling
Shifted CG/BiCG	collinear res.	no	same for all	limited
Flexible GMRES/FOM	polyn./flex.	no	variable by step	possible
Block/Krylov Sylvester	none needed	yes	blockwise	compatible
Rational Krylov/Minimal res.	rational	yes/low-rank	shift dep./AMG	emerging
Recycled GMRES/FOM (rsb)	none needed	yes	block/flexible	state-of-art

3. Adaptive Subspace Construction and Residual Minimization

Many modern solvers adaptively construct the Krylov or rational subspace to accelerate convergence for the hardest shifts:

Minimal Residual Rational Krylov: An adaptive pole selection strategy sets the next rational Krylov pole $\sigma_j$ 6 to coincide with the shift $\sigma_j$ 7 of maximal current residual, ensuring that the most difficult shift is targeted in each expansion. Proven theorems guarantee monotonic (non-increasing) residuals and exactness if a pole matches a shift (Daas et al., 30 Jun 2025).
Flexible Preconditioning: By allowing the preconditioner to vary per step, convergence for challenging or widely separated shifts is greatly enhanced. This is particularly effective in ill-conditioned or high-frequency shift regimes (Saibaba et al., 2012, Bakhos et al., 2016).
Recycling and Deflation: Recycling subspaces can be updated adaptively at the end of each cycle by extracting harmonic or Ritz vectors tailored to the shift spectrum encountered, as in unprojected rsbGMRES approaches (Burke, 2022).

In block and matrix-equation contexts (e.g., Lyapunov/ADI techniques), an extended or rational Krylov subspace is grown so that all required shifted linear systems are approximately solved within a single subspace, and projected (small) equations are used to obtain approximate solutions for all shifts, with error estimation performed efficiently from small projected quantities (Benner et al., 2022, Kolesnikov et al., 2014).

4. Memory, Complexity, and Performance Comparisons

The primary computational advantage of shifted Krylov subspace solvers is the reduction of matrix-vector operations—often the computational bottleneck—from $\sigma_j$ 8 to $\sigma_j$ 9, plus minor overheads for small-dimension dense solves and collinearity recurrences:

Storage: One Krylov basis $K_m(A, b) = K_m(A+\sigma I, b)$ 0 (of size $K_m(A, b) = K_m(A+\sigma I, b)$ 1) and auxiliary structures (e.g., Hessenberg blocks, preconditioner factorizations). For block and recycling approaches, storage grows as $K_m(A, b) = K_m(A+\sigma I, b)$ 2 for $K_m(A, b) = K_m(A+\sigma I, b)$ 3 shifts or $K_m(A, b) = K_m(A+\sigma I, b)$ 4 for recycling dimension $K_m(A, b) = K_m(A+\sigma I, b)$ 5 (Soodhalter, 2014, Burke, 2022, Birk et al., 2012).
Per-iteration cost: One matrix-vector product (or block matrix-vector), one preconditioner solve per participating preconditioner, $K_m(A, b) = K_m(A+\sigma I, b)$ 6 or $K_m(A, b) = K_m(A+\sigma I, b)$ 7 small-dimension solves for all shifts, and $K_m(A, b) = K_m(A+\sigma I, b)$ 8 inner products for block Arnoldi (Bakhos et al., 2016, Saibaba et al., 2012).
Empirical performance: Numerical experiments consistently show order-of-magnitude speedups for large $K_m(A, b) = K_m(A+\sigma I, b)$ 9 (number of shifts), and robust scaling in realistic sparse-matrix applications (Saibaba et al., 2012, Bakhos et al., 2016, Soodhalter, 2014). For moderate block size and shift count, deflated block CG achieves up to 50% reduction in matvecs and wall time (Birk et al., 2012). Multipreconditioned GMRES reduces iteration counts by factors 2–4 compared to FGMRES, with total matvec counts nearly independent of $A \in \mathbb{C}^{n \times n}$ 0 (Bakhos et al., 2016).

5. Applications and Specialized Extensions

Shifted Krylov subspace solvers are used in a wide array of applications:

Quantum Chromodynamics (QCD) and lattice gauge theory: High-fidelity simulation of Dirac operators requires repeated solves for mass-shifted systems; block and shifted solvers provide essential scalability (Soodhalter, 2014, Birk et al., 2012, Soodhalter et al., 2013).
Matrix Functions: Evaluation of $A \in \mathbb{C}^{n \times n}$ 1 via contour quadrature—requiring solutions for $A \in \mathbb{C}^{n \times n}$ 2 with $A \in \mathbb{C}^{n \times n}$ 3 on a contour—is accomplished efficiently via shifted solvers, with the K $A \in \mathbb{C}^{n \times n}$ 4 library providing production capability for quantum lattice models (Hoshi et al., 2020, Ohno et al., 2010).
Lyapunov and Sylvester Matrix Equations: Extended/rational Krylov and ADI-integrated methods (e.g., (Benner et al., 2022, Kolesnikov et al., 2014)) use shift-invariance to simultaneously solve all required shifted systems, drastically reducing ADI iteration cost.
Eigenvalue Algorithms: FEAST and IFEAST compute spectral slices by contour integration, solving $A \in \mathbb{C}^{n \times n}$ 5 for several quadrature nodes $A \in \mathbb{C}^{n \times n}$ 6—efficiently handled with shifted Krylov solvers and supporting inexact, parallelizable inner iterations (Gavin et al., 2017).

Specialized algorithms address non-Hermitian systems (Gu et al., 2016), block multi-shift/multi-RHS cases (Birk et al., 2012), and rational Krylov formulations with challenging shift spectra (Daas et al., 30 Jun 2025).

6. Limitations, Open Problems, and Future Directions

While shift-invariance affords dramatic computational advantages, there are limitations and open areas:

Breakdown of Collinearity: In non-Hermitian or unrelated RHS settings, explicit collinearity must be enforced at restart (classical approaches) or circumvented by recasting the problem as a Sylvester equation to avoid shift-dependent restarts (Soodhalter, 2014, Burke, 2022).
Preconditioning Across Shifts: Sophisticated or shift-adapted preconditioners are required for heavily ill-conditioned or widely spaced shifts; multipreconditioned or flexible approaches mitigate but do not eliminate this challenge (Bakhos et al., 2016, Saibaba et al., 2012).
Recycling Subspace Design: The impossibility theorem shows that no single recycled augmented space of fixed dimension can provide exact minimal residual correction for all shifts simultaneously; thus, either per-shift deflation spaces or approximate corrections—possibly in recursive cycles—are needed (Soodhalter et al., 2013).
Large Block Size Overheads: For very large block size or number of shifts, orthogonalization and storage costs can become significant, making batched or adaptive block strategies preferred (Soodhalter, 2014).

Current research explores minimal-residual rational Krylov strategies for genuinely non-Hermitian, non-conjugate complex shifts (Daas et al., 30 Jun 2025), integration with advanced parallel computing systems (Burke, 2022), and subspace recycling theory for unprojected block methods.

7. Numerical Benchmarks and Empirical Results

For a 2D aquifer discretized with $A \in \mathbb{C}^{n \times n}$ 7, $A \in \mathbb{C}^{n \times n}$ 8 frequencies, flexible Arnoldi with $A \in \mathbb{C}^{n \times n}$ 9 preconditioners and $b\in \mathbb{C}^n$ 0 Arnoldi steps achieved total matvecs $b\in \mathbb{C}^n$ 1 for $b\in \mathbb{C}^n$ 2 and time up to $b\in \mathbb{C}^n$ 3 faster than direct factorization, with time nearly independent of $b\in \mathbb{C}^n$ 4 (Saibaba et al., 2012).
MPGMRES for a 3D aquifer, $b\in \mathbb{C}^n$ 5, $b\in \mathbb{C}^n$ 6, with preconditioners solved approximately, showed that MPGMRES with $b\in \mathbb{C}^n$ 7 required 20 matvecs and 36.7s, compared to 44 and 104.2s for FGMRES (Bakhos et al., 2016).
DSBlockCG for 4 right-hand sides and 7 shifts on a $b\in \mathbb{C}^n$ 8 lattice reduced matvecs from 22,337 to 2,766 and wall time by a factor of 5 (Birk et al., 2012).
Rational Krylov MR-RKSM converged in 2–5 $b\in \mathbb{C}^n$ 9 fewer iterations and $\sigma \in \mathbb{C}$ 0– $\sigma \in \mathbb{C}$ 1 less CPU time than classical Galerkin/extended-Krylov for complex, non-conjugate shifts, with asymptotically $\sigma \in \mathbb{C}$ 2 overhead per shift and minimal per-shift memory cost (Daas et al., 30 Jun 2025).
Block Krylov with recycling (rsbGMRES/unprojected) on lattice QCD and Poisson problems reduced total block matvecs by 20–35%, with runtime improvements of 1.2 $\sigma \in \mathbb{C}$ 3–1.5 $\sigma \in \mathbb{C}$ 4 compared to projected approaches for the same convergence (Burke, 2022).

In summary, shifted Krylov subspace solvers constitute a mature set of algorithms for efficiently solving large families of shifted linear systems. Modern advances integrate flexible and multipreconditioned solvers, rational Krylov subspaces, adaptive minimal residual projections, block (Sylvester-equation) approaches, and robust recycling frameworks tailored for large-scale, parallel scientific computations (Saibaba et al., 2012, Bakhos et al., 2016, Soodhalter et al., 2013, Burke, 2022, Soodhalter, 2014, Daas et al., 30 Jun 2025). New research continues to broaden their range, enhance their robustness for nonsymmetric or non-conjugate cases, and improve scalability for high-performance computing environments.