Invariant Krylov Subspace

• Invariant Krylov subspace is the space spanned by successive applications of an operator to a vector, ceasing to yield new directions once invariance is reached.
• It underlies iterative solvers like GMRES and CG, enabling efficient solutions of linear systems, eigenvalue problems, and model reduction in complex computations.
• Extensions such as block and rational Krylov subspaces enhance shift invariance and structure preservation, critical for quantum algorithms and large-scale optimization.

An invariant Krylov subspace is a central concept in numerical linear algebra and computational mathematics, capturing the subspace structure generated by repeated action of an operator (typically a matrix) on an initial vector or set of vectors. Krylov subspaces are pivotal for iterative methods aimed at solving linear systems, eigenvalue problems, and for model reduction in high-dimensional settings. Their invariance properties are foundational both in classical and modern computational frameworks, including applications in large-scale optimization, eigensolvers, quantum algorithms, and scientific computing.

1. Definition of Invariant Krylov Subspace

A Krylov subspace of order $m$ associated with matrix $A \in \mathbb{C}^{n \times n}$ and vector $v$ is defined as

$$\mathcal{K}_m(A, v) = \text{span} \{ v, Av, A^2v, \dots, A^{m-1}v \}$$

For block or rational variants, the initial generator may be a matrix and the iterative actions may include rational functions or blocks.

The subspace is termed invariant under $A$ if $$A \mathcal{K}_k(A, v) \subseteq \mathcal{K}_k(A, v)$$ for some $k$. In practice, “invariant Krylov subspace” often refers to the smallest $k$ for which this property holds: at this point the sequence no longer produces new directions and the Krylov subspace encodes all information about $A$ acting on $v$ (1811.09025).

Krylov subspaces can also be generalized to rational forms,

$$\mathcal{K}_m(A, b; \{\xi_j\}_{2:m}) = \text{span}\{ b, (A+\xi_2 I)^{-1} b, \dots, \prod_{j=2}^m (A+\xi_j I)^{-1}b \}$$

which provide enhanced invariance properties for shifted systems and model reduction (Daas et al., 30 Jun 2025).

2. Construction and Properties

The standard Arnoldi iteration constructs an orthonormal basis $\{v_1, \ldots, v_m\}$ for the Krylov subspace. The procedure

• Starts with $v_1 = r_0/\|r_0\|$ (where $r_0$ is an initial residual),
• At step $j$ forms $w = Av_j$, orthogonalizes $w$ against previous $v_i$, and normalizes,
• Iterates until breakdown occurs ($\|w\| = 0$), which signals that $A\mathcal{K}_j \subseteq \mathcal{K}_j$, i.e., the subspace is invariant (1811.09025).

In specialized settings, the basis may be unnormalized (Lanczos triples: $(q_k, y_k, \delta_k)$ with $q_k = Hy_k + \delta_k c$) (Forsgren et al., 2014) or built to satisfy additional structure (e.g., $J$-orthogonal, symplectic) (Benner et al., 2022, Koskela, 2017).

For methods such as Krylov subspace descent for deep learning, basis construction involves preconditioning and explicit orthogonalization steps: $$v_1 = D^{-1}g / \|D^{-1}g\|, \quad v_{k+1} = \text{normalized}\left\{ D^{-1} (Hv_k) - \sum_{j=1}^k (u^\top v_j) v_j \right\}$$ where $D$ is a diagonal preconditioner and $g$ the gradient (Vinyals et al., 2011).

An important property: for a matrix $A$, Krylov subspaces generated by $A$ (or shifted versions $(A + \sigma I)$ for any shift $\sigma$) from the same initial vector are shift-invariant, i.e., $$\mathcal{K}_m(A, v) = \mathcal{K}_m(A + \sigma I, v)$$ (Burke, 2022, Daas et al., 30 Jun 2025).

3. Role in Iterative Solvers and Optimization

Krylov subspaces are the foundation for iterative methods including Conjugate Gradient (CG), GMRES, MINRES, and their variants (1811.09025, Forsgren et al., 2014). These approaches iteratively seek solutions in growing Krylov subspaces and exploit their invariance:

• For $A$ symmetric positive definite, CG generates $A$-conjugate directions in $\mathcal{K}_k(A, r_0)$.
• GMRES, through the Arnoldi process, constructs a basis for $\mathcal{K}_m(A, r_0)$ and seeks the residual minimizer in this subspace (1811.09025).

In high-dimensional optimization, such as cubic-regularized Newton updates, Krylov subspaces built from the Hessian and gradient allow for low-dimensional approximations of the Newton step while maintaining dimension-independent convergence rates (Jiang et al., 5 Jan 2024). The subspace $K_m(H_k, g_k)$ is invariant up to order $m$ and contains directions capturing the essential spectrum of the Hessian.

Enlarged Krylov subspaces—constructed by iteratively applying $A$ to multiple domain-decomposed blocks per iteration—capture a larger portion of the invariant subspace per iteration, leading to faster convergence and reduced communication overhead in parallel/distributed settings (Moufawad, 2018).

4. Invariance in Block, Rational, and Structure-Preserving Extensions

• Block Krylov subspaces: For $A \in \mathbb{C}^{n \times n}$ and block $V \in \mathbb{C}^{n \times r}$,

$$\mathcal{K}_k(A, V) = \operatorname{span}\{ V, AV, \dots, A^{k-1}V \}$$

Block Krylov subspaces support subspace expansions optimized to converge to target invariant subspaces with minimal principal angles (Zuccalli et al., 21 Nov 2024).

• Rational Krylov subspaces: Allow inclusion of shifted/inverse directions providing enhanced approximation of spectra near poles and shift-invariant properties critical for sequences of shifted systems (Daas et al., 30 Jun 2025).
• Structure-preserving Krylov subspaces: In Hamiltonian systems, basis construction must be symplectic (e.g., $J$-orthogonal) to ensure preservation of invariants such as energy (Koskela, 2017, Benner et al., 2022). For skew-Hamiltonian matrices, Krylov subspaces are automatically isotropic and can be constructed to yield any Lagrangian subspace (Saltenberger et al., 2019).

5. Augmentation, Recycling, and Adaptivity

Invariant subspace information can be amplified or recycled across multiple solves:

• Augmented Krylov methods: These enhance $\mathcal{K}_k(A, r_0)$ by an additional fixed or recycled subspace $U$ that approximates invariant directions—beneficial for accelerating convergence especially for slow modes associated with near-invariant subspaces (Soodhalter, 2021).
• Recycling for evolving systems: In applications such as PDE-constrained optimization with changing meshes or matrices, previously computed invariant subspaces are mapped to new algebraic contexts via interpolation and further refined by warm-start Krylov–Schur cycles (Bolten et al., 2020).
• Pole selection/adaptive expansion: In rational Krylov settings for sequences of shifted systems, adaptively choosing the next pole to match the shift with the largest residual error ensures that the new subspace captures the most difficult direction, preserving the property that the solution to certain shifted systems is exactly included after the pole is matched (Daas et al., 30 Jun 2025).

6. Connections to Quantum Algorithms and Model Reduction

In quantum simulation and quantum eigensolver algorithms:

• Quantum Krylov subspace diagonalization (QKSD): Approximates eigenstates in an invariant subspace spanned by $\{\hat{H}^k|\psi_0\rangle\}$, often with structured (e.g., Hankel) representations. Advanced techniques such as quantum signal processing are employed to prepare these states and extract molecular gradients and observables with measurement cost scaling reduced from $O(D^2)$ to $O(1)$ (Oumarou et al., 9 Jan 2025).
• Measurement-efficient Gaussian-power Krylov bases: Exploit the invariance of filtered Krylov subspaces to suppress statistical noise while retaining convergence properties (Zhang et al., 2023).

In model reduction (e.g., Riccati and Lyapunov equations), invariant (Krylov or low-rank) subspaces yield numerical schemes for operator equations with reduced computational and storage costs while maintaining theoretical error bounds (Behr et al., 2019).

7. Optimality, Error Bounds, and Computable Algorithms

Optimal iterative expansion of subspaces to approximate invariant subspaces (e.g., those associated with dominant eigenvalues) may proceed via expansions of the form $V + A(W_0)$, where $W_0$ is chosen to minimize principal angles with the target invariant subspace. Theoretical results yield geometric convergence under spectral gap assumptions and lower bounds using Chebyshev polynomial filters (Zuccalli et al., 21 Nov 2024). Practically, computable versions employ variants of Rayleigh–Ritz or refined projection methods for correction.

Bounds on residuals and error are often directly linked to the invariance property:

• If the Krylov subspace becomes exactly invariant, residuals vanish and iteration stops (1811.09025).
• In inner-product free iterative methods, the lack of orthogonality is controlled via condition numbers of well-defined factors, preserving approximation properties of the invariant subspace (Brown et al., 8 Sep 2024).

---

In summary, invariant Krylov subspaces encapsulate essential algebraic, geometric, and spectral properties necessary for efficient iterative methods in scientific computing, optimization, and quantum simulation. Their construction, structure, and adaptability underpin the design of efficient, robust, and theoretically justified solvers for high-dimensional and evolving problems across classical and quantum computational paradigms.