Infinite-Dimensional QR: Theory & Applications

• Infinite-dimensional QR algorithms are generalizations of classical QR methods, extending eigenvalue computations to operator and function space settings.
• They exploit structured recurrences, low-rank approximations, and basis decompositions to handle integral, differential, and kernel operators effectively.
• These techniques achieve scalable performance in applications including numerical PDEs, Bayesian inference, and machine learning, offering enhanced stability and efficiency.

The Infinite-Dimensional QR Algorithm (IQR) refers to a class of techniques and conceptual extensions that generalize the classical finite-dimensional QR algorithm for eigenvalue computations to operator-theoretic and function-space settings. While no canonical "IQR" algorithm is present in the referenced corpus, this article rigorously surveys the key mathematical and algorithmic structures closely associated with infinite-dimensional and structure-exploiting QR-like algorithms, presenting their technical foundations, practical realizations in fast eigendecomposition, and their role in modern high-dimensional and operator-adapted computational frameworks.

1. Classical and Generalized QR Algorithm Structure

The finite-dimensional QR algorithm transforms a matrix $A \in \mathbb{C}^{n \times n}$ into Hessenberg form, and iteratively computes eigenvalues via similarity transformations $A_{k+1} = Q^\ast_k A_k Q_k$, where $Q_k R_k = A_k$ is the QR decomposition, converging to (block-)upper-triangular (Schur) form. Infinite-dimensional or large-scale analogues, such as those arising from integral, differential, or kernel operators, cannot be manipulated via explicit matrix arithmetic. This motivates structured, low-rank, and operator-adapted extensions to QR-type algorithms.

In hierarchical algorithms such as SuperDC, the efficiency comes from representing a (possibly large or infinite-dimensional) operator in a block or function basis in which it exhibits low-rank off-diagonal structure (e.g., HSS, HODLR formats), then recursively performing divide-and-conquer or block reduction steps that mimic QR/Schur-like transformations (Ou et al., 2021).

2. Structure-Exploiting Eigendecomposition: The Role of Structured Recursion

Explicit eigenvalue computation for structured matrices has been generalized from the classical tridiagonal (or block tridiagonal) setting to block-sparse and low-rank hierarchical formats. For instance, in the case of block tridiagonal matrices, the eigenproblem

$$A = \begin{pmatrix} B_0 & D_0 & \cdots & \ C_0 & B_1 & D_1 & \cdots \ & \ddots & \ddots & D_{L-2} \ & & C_{L-2} & B_{L-1} \end{pmatrix}$$

is solved by formulating a three-term recurrence for a matrix-valued polynomial and using zeros of the determinant of the last block polynomial to recover eigenvalues—eliminating the need for explicit QR or Schur iterations (Sandryhaila et al., 2013). When the blocks commute or are diagonal, the method approaches infinite-dimensional operator theory, recovering classic results for Sturm-Liouville and Toeplitz operators.

Such recurrences mirror the shift-implicit QR algorithm for tridiagonal forms, generalizing the implicit reduction "QR step" to block structured, and (by extension) operator-theoretic settings.

3. Dynamically Programmed Eigendecomposition in High Dimensions

In high-dimensional settings, such as those in Riemannian-manifold Hamiltonian Monte Carlo (RMHMC) for hierarchical Gaussian process models, QR-like stepwise eigendecomposition becomes computationally prohibitive due to the $O(d^3)$ scaling of dense matrix routines. The referenced work (Hayakawa et al., 9 Nov 2025) develops a dynamic-programming style fast eigensolver by leveraging the following:

• Symmetric Hessian Structure: At each integrator step, the Hessian $H(q) = \nabla_q^2[-\ln P(q)]$ is dense but changes slowly ($H(q(t+\epsilon)) \approx H(q(t))$ for small $\epsilon$).
• Warm-Start Cyclic Jacobi: By projecting $H(q(t+\epsilon))$ into the eigenbasis of $H(q(t))$, the matrix is already nearly diagonal; applying only 1-2 sweeps of cyclic Jacobi suffices to re-diagonalize.
• Amortized Re-orthonormalization: Periodic Gram-Schmidt orthonormalization offsets numerical drift, but the main cost per step is $O(d^2)$, dominated by the Jacobi sweep, an O(d)-fold speedup over naive dense QR or divide-and-conquer routines.

This approach replaces the classic shift QR step with a tailored, structure-exploiting update to the eigenbasis, a paradigm that is readily extensible to operator pencils provided the eigenspace varies smoothly under parameterization.

4. Infinite-Dimensional QR: Function Spaces, Basis Decomposition, and Operator Analogues

Analysis of infinite-dimensional QR is naturally situated in the setting of function spaces and their orthonormal basis decompositions. Spectral theory for compact normal operators (e.g., integral operators, Sturm-Liouville, or block Toeplitz operators) reduces the eigenvalue problem to that for an infinite matrix, with operator analogues of Householder reflections and Givens rotations being defined via their actions on function bases.

Practical QR-like algorithms in infinite or large-dimension settings thus exploit sparsity, decay, or low-rank structure in a chosen basis. SuperDC (Ou et al., 2021) explicitly implements divide-and-conquer on HSS matrices—whose off-diagonal blocks decay or are low-rank—enabling nearly linear-time ($O(r^2 n \log^2 n)$) recursive eigendecomposition, with recursion mimicking the block steps of infinite-dimensional QR.

For block tridiagonal operators, the three-term recurrence for the matrix-valued polynomial mimics the role of successive QR steps, each acting on an increasing "window" of the operator, and in the infinite-dimensional limit recovers the spectral mapping theorem for bounded self-adjoint operators.

5. Fast Eigendecomposition Algorithms Beyond Classical QR

Recent algorithmic advances have led to specialized methods that outperform generic QR/Schur for structured or infinite-dimensional problems:

• Warm Start Jacobi and Dynamic Programming (Hayakawa et al., 9 Nov 2025): Exploits near-constancy in the Hessian eigenbasis to accelerate repeated eigendecompositions.
• Rational Function Iterations for Unitary Matrices (Gawlik, 2020): Use minimax rational approximants (Zolotarev iterations) to achieve rapid, backward-stable unitary sign and spectral projectors, bypassing loss of orthogonality in classic Newton or Padé iterations.
• Divide-and-conquer with Fast Multipole Methods (Ou et al., 2021): Solves secular equations in nearly-linear time through triangular FMM, balancing rank updates to avoid exponential norm growth.
• Block Polynomial Recursion for Tridiagonal Matrices (Sandryhaila et al., 2013): Reduces the eigenproblem to a sequence of operator-valued polynomial recursions rather than explicit QR steps.
• Randomized Subspace Iteration for Block Hankel Structures (Minster et al., 2020): Uses FFT-based fast mat-vec and randomized SVD to dramatically reduce SVD/eigenvalue costs in system identification, achieving complexity $O(s\log s)$ instead of cubic in size.

6. Scaling, Performance, and Stability

Structure-exploiting and infinite-dimensional QR-style algorithms deliver substantial efficiency gains:

• Computational Complexity:
  • Classical dense QR: $O(n^3)$.
  • Dynamic Jacobi update: $O(d^2)$ per iteration, amortized over many steps (Hayakawa et al., 9 Nov 2025).
  • HSS/SuperDC: $O(r^2 n \log^2 n)$ (Ou et al., 2021).
  • Randomized SVD for block Hankel: $O(s \log s)$ (Minster et al., 2020).
• Stability: Techniques such as warm-started Jacobi, triangular FMM, and balancing of low-rank updates control both forward and backward errors, even in the presence of clustered or closely spaced eigenvalues, where standard QR can fail or lose orthogonality catastrophically (Gawlik, 2020, Ou et al., 2021).
• Parallelism and Hardware Mapping: Jacobi and structure-exploiting rotations map efficiently to SIMD/GPU cores, enabling real-time throughput for applications with high iterative demand (Hayakawa et al., 9 Nov 2025).

7. Connections, Applications, and Outlook

Infinite-dimensional QR algorithms and their finite but high-dimensional, structure-exploiting realizations underpin efficient computations in:

• Bayesian inference with Gaussian processes (Hayakawa et al., 9 Nov 2025)
• Signal processing and spectral methods on graphs (Sandryhaila et al., 2013)
• Machine learning kernel methods and large-scale manifold learning (Minster et al., 2020)
• Quantum and statistical physics, via operator-valued spectral recursions
• Numerical PDEs and boundary element methods (HSS/HODLR formats)

A plausible implication is that as operator-theoretic computations become more prevalent in applied mathematics, data science, and physics, infinite-dimensional QR concepts will continue to shape new algorithmic paradigms—blending spectral theory, hierarchical tensor algebra, and randomized algorithms to deliver tractable, stable, and scalable eigendecompositions far beyond the reach of classical dense QR.

---

Table: Summary of Key Structure-Exploiting QR-Like Algorithms

Algorithm Context	Structure/Innovation	Asymptotic Cost
Dynamic-Programming Jacobi (Hayakawa et al., 9 Nov 2025)	Warm-start on sequence of symmetric Hessians	$O(d^2)$ per iteration
SuperDC (HSS divide-conquer) (Ou et al., 2021)	Balanced low-rank recursion, FMM secular solves	$O(r^2 n \log^2 n)$
Rational Zolotarev sign (Gawlik, 2020)	Rational minimax function iteration	rapid (superlinear)
Block Poly Recurrence (Sandryhaila et al., 2013)	Recurrence on block-tridiagonal matrix polynomials	$O(N K^2)$
Randomized SVD (ERA) (Minster et al., 2020)	Fast mat-vec via FFT/block Hankel	$O(s\log s)$

All entries directly trace to the referenced arXiv papers. Each method generalizes or supremely augments classical QR, either in the direction of infinite-dimensional operator theory or by exploiting algebraic structure for efficiency and scalability.