---
title: Shifted Krylov Subspace Solver
url: https://www.emergentmind.com/topics/shifted-krylov-subspace-solver
type: topic
---

# Shifted Krylov Subspace Solver

A shifted Krylov subspace solver is a class of iterative algorithms designed to simultaneously solve large families of linear systems whose coefficient matrices differ by scalar multiples of the identity, i.e., systems of the form $(A + \sigma_j I) x_j = b$ for distinct shifts $\sigma_j$. The approach fundamentally leverages the shift-invariance property of Krylov subspaces—$K_m(A, b) = K_m(A+\sigma I, b)$—which enables the efficient reuse of Krylov bases and associated computational structures across all shifts. Shifted Krylov subspace solvers are indispensable in scientific applications requiring the solution of parameterized linear systems, evaluation of matrix functions, large-scale spectral computations, and inverse problems.

## 1. Mathematical Foundations and Shift-Invariance

The mathematical core of shifted Krylov subspace solvers is the invariance of the Krylov subspace under shifts of the coefficient matrix. Explicitly, for any $A \in \mathbb{C}^{n \times n}$, $b\in \mathbb{C}^n$, and $\sigma \in \mathbb{C}$,
$$
K_m(A, b) = \operatorname{span}\left\{b, Ab, \ldots, A^{m-1}b \right\} = K_m(A + \sigma I, b).
$$
This property allows algorithms (e.g., Arnoldi, Lanczos, Hessenberg, block Arnoldi) to build a single Krylov basis which is then leveraged to solve all systems $(A + \sigma_j I)x_j = b$ using only $\mathcal{O}(m)$ matrix-vector products instead of $\mathcal{O}(n_f m)$ if solved independently for $n_f$ shifts. Extensions to rational Krylov and block Krylov frameworks allow even further flexibility, such as the inclusion of multiple right-hand sides or rational functions in matrix-function applications [1212.3660, 1603.08970, 2001.08707, 1205.0359, 1412.0393].

For non-Hermitian systems or unrelated right-hand sides, a Sylvester equation formulation encapsulates the family as $A X + X S = B$, with $S = \mathrm{diag}(\sigma_1, \ldots, \sigma_s)$, so that a block Krylov subspace built for $A$ and $B$ provides simultaneous solution capability for all shifts [1412.0393, 2209.06922].

## 2. Algorithmic Frameworks: Seed Methods, Recycling, and Preconditioning

Classic shifted Krylov solvers include:

- **Shifted CG/BiCG**: For Hermitian or general systems, a seed system (typically with $\sigma=0$) is solved using three-term recurrences (CG/BiCG), and cheap scalar recurrences are used to update all shifted iterates and residuals [1004.0292, 2001.08707].
- **Block and Rational Krylov Methods**: These generalize the polynomial subspace to block or rational functions, incorporating multiple right-hand sides and/or adaptive poles, crucial in matrix equations and function evaluations [1410.3335, 2507.00267, 2203.17174].
- **Flexible and Multipreconditioned Solvers**: Flexible GMRES/FOM allow change of (possibly shift-dependent) preconditioners at each step [1212.3660]. In MPGMRES, multiple shift-and-invert preconditioners are used to build a larger, more robust search space growing linearly in the number of preconditioners, which is essential when the shifts are widely scattered or not well clustered [1603.08970].
- **Recycling Algorithms and Restarting**: Krylov subspace recycling strategies (e.g., GCRO-DR, rsbGMRES) retain spectral information across related systems or across restarts, accelerating convergence for sequences of shifted (or slowly changing) systems [1301.2650, 1412.0393, 2209.06922]. Recycled subspaces may be tuned adaptively, e.g., via shift-specific harmonic Ritz vector extraction [2209.06922].

A summary table of core approaches:

| Framework                   | Shift Strategy  | Multiple RHS | Preconditioning   | Recycling    |
|-----------------------------|----------------|--------------|-------------------|--------------|
| Shifted CG/BiCG             | collinear res. | no           | same for all      | limited      |
| Flexible GMRES/FOM          | polyn./flex.   | no           | variable by step  | possible     |
| Block/Krylov Sylvester      | none needed    | yes          | blockwise         | compatible   |
| Rational Krylov/Minimal res.| rational       | yes/low-rank | shift dep./AMG    | emerging     |
| Recycled GMRES/FOM (rsb)    | none needed    | yes          | block/flexible    | state-of-art |

## 3. Adaptive Subspace Construction and Residual Minimization

Many modern solvers adaptively construct the Krylov or rational subspace to accelerate convergence for the hardest shifts:

- **Minimal Residual Rational Krylov**: An adaptive pole selection strategy sets the next rational Krylov pole $\xi_{m+1}$ to coincide with the shift $s_j$ of maximal current residual, ensuring that the most difficult shift is targeted in each expansion. Proven theorems guarantee monotonic (non-increasing) residuals and exactness if a pole matches a shift [2507.00267].
- **Flexible Preconditioning**: By allowing the preconditioner to vary per step, convergence for challenging or widely separated shifts is greatly enhanced. This is particularly effective in ill-conditioned or high-frequency shift regimes [1212.3660, 1603.08970].
- **Recycling and Deflation**: Recycling subspaces can be updated adaptively at the end of each cycle by extracting harmonic or Ritz vectors tailored to the shift spectrum encountered, as in unprojected rsbGMRES approaches [2209.06922].

In block and matrix-equation contexts (e.g., Lyapunov/ADI techniques), an extended or rational Krylov subspace is grown so that all required shifted linear systems are approximately solved within a single subspace, and projected (small) equations are used to obtain approximate solutions for all shifts, with error estimation performed efficiently from small projected quantities [2203.17174, 1410.3335].

## 4. Memory, Complexity, and Performance Comparisons

The primary computational advantage of shifted Krylov subspace solvers is the reduction of matrix-vector operations—often the computational bottleneck—from $\mathcal{O}(n_{\sigma} m)$ to $\mathcal{O}(m)$, plus minor overheads for small-dimension dense solves and collinearity recurrences:

- **Storage**: One Krylov basis $V_{m+1}$ (of size $n \times (m+1)$) and auxiliary structures (e.g., Hessenberg blocks, preconditioner factorizations). For block and recycling approaches, storage grows as $O(nms)$ for $s$ shifts or $O(n(m+k))$ for recycling dimension $k$ [1412.0393, 2209.06922, 1205.0359].
- **Per-iteration cost**: One matrix-vector product (or block matrix-vector), one preconditioner solve per participating preconditioner, $O(ms^2)$ or $O(sn)$ small-dimension solves for all shifts, and $O(ms^2)$ inner products for block Arnoldi [1603.08970, 1212.3660].
- **Empirical performance**: Numerical experiments consistently show order-of-magnitude speedups for large $n_\sigma$ (number of shifts), and robust scaling in realistic sparse-matrix applications [1212.3660, 1603.08970, 1412.0393]. For moderate block size and shift count, deflated block CG achieves up to 50% reduction in matvecs and wall time [1205.0359]. Multipreconditioned GMRES reduces iteration counts by factors 2–4 compared to FGMRES, with total matvec counts nearly independent of $n_{\sigma}$ [1603.08970].

## 5. Applications and Specialized Extensions

Shifted Krylov subspace solvers are used in a wide array of applications:

- **Quantum Chromodynamics (QCD)** and lattice gauge theory: High-fidelity simulation of Dirac operators requires repeated solves for mass-shifted systems; block and shifted solvers provide essential scalability [1412.0393, 1205.0359, 1301.2650].
- **Matrix Functions**: Evaluation of $f(A)b$ via contour quadrature—requiring solutions for $(z_j I - A)x_j = b$ with $z_j$ on a contour—is accomplished efficiently via shifted solvers, with the K$\omega$ library providing production capability for quantum lattice models [2001.08707, 1004.0292].
- **Lyapunov and Sylvester Matrix Equations**: Extended/rational Krylov and ADI-integrated methods (e.g., [2203.17174, 1410.3335]) use shift-invariance to simultaneously solve all required shifted systems, drastically reducing ADI iteration cost.
- **Eigenvalue Algorithms**: FEAST and IFEAST compute spectral slices by contour integration, solving $(z_j I - A)Y_j=X$ for several quadrature nodes $z_j$—efficiently handled with shifted Krylov solvers and supporting inexact, parallelizable inner iterations [1706.00692].

Specialized algorithms address non-Hermitian systems [1611.00288], block multi-shift/multi-RHS cases [1205.0359], and rational Krylov formulations with challenging shift spectra [2507.00267].

## 6. Limitations, Open Problems, and Future Directions

While shift-invariance affords dramatic computational advantages, there are limitations and open areas:

- **Breakdown of Collinearity**: In non-Hermitian or unrelated RHS settings, explicit collinearity must be enforced at restart (classical approaches) or circumvented by recasting the problem as a Sylvester equation to avoid shift-dependent restarts [1412.0393, 2209.06922].
- **Preconditioning Across Shifts**: Sophisticated or shift-adapted preconditioners are required for heavily ill-conditioned or widely spaced shifts; multipreconditioned or flexible approaches mitigate but do not eliminate this challenge [1603.08970, 1212.3660].
- **Recycling Subspace Design**: The impossibility theorem shows that no single recycled augmented space of fixed dimension can provide exact minimal residual correction for all shifts simultaneously; thus, either per-shift deflation spaces or approximate corrections—possibly in recursive cycles—are needed [1301.2650].
- **Large Block Size Overheads**: For very large block size or number of shifts, orthogonalization and storage costs can become significant, making batched or adaptive block strategies preferred [1412.0393].

Current research explores minimal-residual rational Krylov strategies for genuinely non-Hermitian, non-conjugate complex shifts [2507.00267], integration with advanced parallel computing systems [2209.06922], and subspace recycling theory for unprojected block methods.

## 7. Numerical Benchmarks and Empirical Results

- For a 2D aquifer discretized with $n\sim 90,000$, $n_f=200$ frequencies, flexible Arnoldi with $n_p=5$ preconditioners and $m=40$ Arnoldi steps achieved total matvecs $\sim 1 \times 200$ for $A$ and time up to $20\times$ faster than direct factorization, with time nearly independent of $n_f$ [1212.3660].
- MPGMRES for a 3D aquifer, $n=132,089$, $n_\sigma=100$, with preconditioners solved approximately, showed that MPGMRES with $n_p=5$ required 20 matvecs and 36.7s, compared to 44 and 104.2s for FGMRES [1603.08970].
- DSBlockCG for 4 right-hand sides and 7 shifts on a $16^4$ lattice reduced matvecs from 22,337 to 2,766 and wall time by a factor of 5 [1205.0359].
- Rational Krylov MR-RKSM converged in 2–5$\times$ fewer iterations and $3$–$5\times$ less CPU time than classical Galerkin/extended-Krylov for complex, non-conjugate shifts, with asymptotically $O(m^2)$ overhead per shift and minimal per-shift memory cost [2507.00267].
- Block Krylov with recycling (rsbGMRES/unprojected) on lattice QCD and Poisson problems reduced total block matvecs by 20–35%, with runtime improvements of 1.2$\times$–1.5$\times$ compared to projected approaches for the same convergence [2209.06922].

---

In summary, shifted Krylov subspace solvers constitute a mature set of algorithms for efficiently solving large families of shifted linear systems. Modern advances integrate flexible and multipreconditioned solvers, rational Krylov subspaces, adaptive minimal residual projections, block (Sylvester-equation) approaches, and robust recycling frameworks tailored for large-scale, parallel scientific computations [1212.3660, 1603.08970, 1301.2650, 2209.06922, 1412.0393, 2507.00267]. New research continues to broaden their range, enhance their robustness for nonsymmetric or non-conjugate cases, and improve scalability for high-performance computing environments.

Source: https://www.emergentmind.com/topics/shifted-krylov-subspace-solver