---
title: Invariant Krylov Subspace
url: https://www.emergentmind.com/topics/invariant-krylov-subspace
type: topic
---

# Invariant Krylov Subspace

An invariant Krylov subspace is a central concept in numerical linear algebra and computational mathematics, capturing the subspace structure generated by repeated action of an operator (typically a matrix) on an initial vector or set of vectors. Krylov subspaces are pivotal for iterative methods aimed at solving linear systems, eigenvalue problems, and for model reduction in high-dimensional settings. Their invariance properties are foundational both in classical and modern computational frameworks, including applications in large-scale optimization, eigensolvers, quantum algorithms, and scientific computing.

## 1. Definition of Invariant Krylov Subspace

A Krylov subspace of order $m$ associated with matrix $A \in \mathbb{C}^{n \times n}$ and vector $v$ is defined as
\[
\mathcal{K}_m(A, v) = \text{span} \{ v, Av, A^2v, \dots, A^{m-1}v \}
\]
For block or rational variants, the initial generator may be a matrix and the iterative actions may include rational functions or blocks.

The subspace is termed *invariant* under $A$ if $A \mathcal{K}_k(A, v) \subseteq \mathcal{K}_k(A, v)$ for some $k$. In practice, “invariant Krylov subspace” often refers to the smallest $k$ for which this property holds: at this point the sequence no longer produces new directions and the Krylov subspace encodes all information about $A$ acting on $v$ ([1811.09025]).

Krylov subspaces can also be generalized to rational forms,
\[
\mathcal{K}_m(A, b; \{\xi_j\}_{2:m}) = \text{span}\{ b, (A+\xi_2 I)^{-1} b, \dots, \prod_{j=2}^m (A+\xi_j I)^{-1}b \}
\]
which provide enhanced invariance properties for shifted systems and model reduction ([2507.00267]).

## 2. Construction and Properties

The standard Arnoldi iteration constructs an orthonormal basis $\{v_1, \ldots, v_m\}$ for the Krylov subspace. The procedure
- Starts with $v_1 = r_0/\|r_0\|$ (where $r_0$ is an initial residual),
- At step $j$ forms $w = Av_j$, orthogonalizes $w$ against previous $v_i$, and normalizes,
- Iterates until breakdown occurs ($\|w\| = 0$), which signals that $A\mathcal{K}_j \subseteq \mathcal{K}_j$, i.e., the subspace is invariant ([1811.09025]).

In specialized settings, the basis may be unnormalized (Lanczos triples: $(q_k, y_k, \delta_k)$ with $q_k = Hy_k + \delta_k c$) ([1409.4937]) or built to satisfy additional structure (e.g., $J$-orthogonal, symplectic) ([2202.12640], [1712.04047]).

For methods such as Krylov subspace descent for deep learning, basis construction involves preconditioning and explicit orthogonalization steps:
\[
v_1 = D^{-1}g / \|D^{-1}g\|, \quad v_{k+1} = \text{normalized}\left\{ D^{-1} (Hv_k) - \sum_{j=1}^k (u^\top v_j) v_j \right\}
\]
where $D$ is a diagonal preconditioner and $g$ the gradient ([1111.4259]).

An important property: for a matrix $A$, Krylov subspaces generated by $A$ (or shifted versions $(A + \sigma I)$ for any shift $\sigma$) from the same initial vector are *shift-invariant*, i.e., $\mathcal{K}_m(A, v) = \mathcal{K}_m(A + \sigma I, v)$ ([2209.06922], [2507.00267]).

## 3. Role in Iterative Solvers and Optimization

Krylov subspaces are the foundation for iterative methods including Conjugate Gradient (CG), GMRES, MINRES, and their variants ([1811.09025], [1409.4937]). These approaches iteratively seek solutions in growing Krylov subspaces and exploit their invariance:
- For $A$ symmetric positive definite, CG generates $A$-conjugate directions in $\mathcal{K}_k(A, r_0)$.
- GMRES, through the Arnoldi process, constructs a basis for $\mathcal{K}_m(A, r_0)$ and seeks the residual minimizer in this subspace ([1811.09025]).

In high-dimensional optimization, such as cubic-regularized Newton updates, Krylov subspaces built from the Hessian and gradient allow for low-dimensional approximations of the Newton step while maintaining dimension-independent convergence rates ([2401.03058]). The subspace $K_m(H_k, g_k)$ is invariant up to order $m$ and contains directions capturing the essential spectrum of the Hessian.

Enlarged Krylov subspaces—constructed by iteratively applying $A$ to multiple domain-decomposed blocks per iteration—capture a larger portion of the invariant subspace per iteration, leading to faster convergence and reduced communication overhead in parallel/distributed settings ([1804.10629]).

## 4. Invariance in Block, Rational, and Structure-Preserving Extensions

- **Block Krylov subspaces:** For $A \in \mathbb{C}^{n \times n}$ and block $V \in \mathbb{C}^{n \times r}$,
\[
\mathcal{K}_k(A, V) = \operatorname{span}\{ V, AV, \dots, A^{k-1}V \}
\]
Block Krylov subspaces support subspace expansions optimized to converge to target invariant subspaces with minimal principal angles ([2411.14578]).
  
- **Rational Krylov subspaces:** Allow inclusion of shifted/inverse directions providing enhanced approximation of spectra near poles and shift-invariant properties critical for sequences of shifted systems ([2507.00267]).
  
- **Structure-preserving Krylov subspaces:** In Hamiltonian systems, basis construction must be symplectic (e.g., $J$-orthogonal) to ensure preservation of invariants such as energy ([1712.04047], [2202.12640]). For skew-Hamiltonian matrices, Krylov subspaces are automatically isotropic and can be constructed to yield any Lagrangian subspace ([1910.12904]).

## 5. Augmentation, Recycling, and Adaptivity

Invariant subspace information can be amplified or recycled across multiple solves:
- **Augmented Krylov methods:** These enhance $\mathcal{K}_k(A, r_0)$ by an additional fixed or recycled subspace $U$ that approximates invariant directions—beneficial for accelerating convergence especially for slow modes associated with near-invariant subspaces ([2106.10050]).
- **Recycling for evolving systems:** In applications such as PDE-constrained optimization with changing meshes or matrices, previously computed invariant subspaces are mapped to new algebraic contexts via interpolation and further refined by warm-start Krylov–Schur cycles ([2010.11447]).
- **Pole selection/adaptive expansion:** In rational Krylov settings for sequences of shifted systems, adaptively choosing the next pole to match the shift with the largest residual error ensures that the new subspace captures the most difficult direction, preserving the property that the solution to certain shifted systems is exactly included after the pole is matched ([2507.00267]).

## 6. Connections to Quantum Algorithms and Model Reduction

In quantum simulation and quantum eigensolver algorithms:
- **Quantum Krylov subspace diagonalization (QKSD):** Approximates eigenstates in an invariant subspace spanned by $\{\hat{H}^k|\psi_0\rangle\}$, often with structured (e.g., Hankel) representations. Advanced techniques such as quantum signal processing are employed to prepare these states and extract molecular gradients and observables with measurement cost scaling reduced from $O(D^2)$ to $O(1)$ ([2501.05286]).
- **Measurement-efficient Gaussian-power Krylov bases:** Exploit the invariance of filtered Krylov subspaces to suppress statistical noise while retaining convergence properties ([2301.13353]).

In model reduction (e.g., Riccati and Lyapunov equations), invariant (Krylov or low-rank) subspaces yield numerical schemes for operator equations with reduced computational and storage costs while maintaining theoretical error bounds ([1910.13362]).

## 7. Optimality, Error Bounds, and Computable Algorithms

Optimal iterative expansion of subspaces to approximate invariant subspaces (e.g., those associated with dominant eigenvalues) may proceed via expansions of the form $V + A(W_0)$, where $W_0$ is chosen to minimize principal angles with the target invariant subspace. Theoretical results yield geometric convergence under spectral gap assumptions and lower bounds using Chebyshev polynomial filters ([2411.14578]). Practically, computable versions employ variants of Rayleigh–Ritz or refined projection methods for correction.

Bounds on residuals and error are often directly linked to the invariance property:
- If the Krylov subspace becomes exactly invariant, residuals vanish and iteration stops ([1811.09025]).
- In inner-product free iterative methods, the lack of orthogonality is controlled via condition numbers of well-defined factors, preserving approximation properties of the invariant subspace ([2409.05239]).

---

In summary, invariant Krylov subspaces encapsulate essential algebraic, geometric, and spectral properties necessary for efficient iterative methods in scientific computing, optimization, and quantum simulation. Their construction, structure, and adaptability underpin the design of efficient, robust, and theoretically justified solvers for high-dimensional and evolving problems across classical and quantum computational paradigms.

Source: https://www.emergentmind.com/topics/invariant-krylov-subspace