Modified Randomized Arnoldi Process

• The Modified Randomized Arnoldi Process is a method that accelerates Krylov subspace algorithms by employing randomized sketching and non-traditional orthogonalization.
• It reduces computational and communication costs in large-scale problems by correcting the Hessenberg structure to recover theoretical guarantees.
• The process offers numerical stability and robust convergence for matrix function evaluations and eigenvalue problems, enabling efficient high-dimensional computations.

The Modified Randomized Arnoldi Process is a class of algorithms designed to accelerate Krylov subspace methods by employing randomized sketching and non-traditional orthogonalization, while preserving or restoring key theoretical properties of the standard Arnoldi process. These modifications are primarily motivated by the need to reduce the cost and communication bottlenecks associated with classical Arnoldi orthogonalization in large-scale scientific computing. The resulting algorithms achieve computational and parallel efficiency, while ensuring well-conditioned bases for Krylov subspaces and delivering robust convergence and stability for matrix function evaluations and eigenvalue problems (Cortinovis et al., 2022, Grigori et al., 15 Jan 2026, Damas et al., 17 Dec 2025).

1. Foundations: Arnoldi and the Need for Modification

The classical Arnoldi process builds an orthonormal basis $\{q_1, ..., q_m\}$ for the $m$-dimensional Krylov subspace

$$\mathcal{K}_m(A, b) = \mathrm{span}\{b,\, Ab,\, ...,\, A^{m-1}b\}$$

using Gram–Schmidt orthogonalization. This leads to a Hessenberg decomposition:

$A Q_m = Q_m G_m + g_{m+1,m} q_{m+1} e_m^H$

where $G_m$ is upper Hessenberg. While robust, the per-iteration cost is $O(n m)$ for vector operations and $O(n m^2)$ flops for orthogonalization, with $O(m)$ global communications per iteration—factors that constrain scalability (Grigori et al., 15 Jan 2026, Damas et al., 17 Dec 2025).

Recent approaches accelerate Arnoldi by forming bases that are only "sketch-orthonormal" or even non-orthonormal, replacing high-dimensional inner products with randomized projections, and offloading heavy operations to randomized least-squares routines, often yielding substantial speedups with provably well-conditioned bases (Damas et al., 17 Dec 2025, Cortinovis et al., 2022).

2. Core Algorithmic Structure

The Modified Randomized Arnoldi Process introduces random sketching matrices (e.g., SRHT or SRFT) to perform Gram–Schmidt or Householder-type orthogonalization in a lower-dimensional space. A typical algorithmic workflow (as in (Cortinovis et al., 2022)) is:

1. Sketching: Select a random embedding matrix $\Theta \in \mathbb{R}^{s \times n}$, with $s \approx 2m \ll n$.
2. Initialization: Normalize $b$, sketch it ($p_1 = \Theta b$), and set the first basis vector $v_1 = b / \|\Theta b\|$.
3. Iteration. For $i = 2, ..., m+1$:
   • Form $w_i = A v_{i-1}$, sketch $p_i = \Theta w_i$.
   • Gram–Schmidt (or Householder) re-orthogonalize $p_i$ against prior sketched basis vectors.
   • Apply corresponding updates in full space to $w_i$ to obtain $v_i$.
   • Grow low-dimensional coefficients (Hessenberg or $R$ factors).
4. Compressed Operator Construction:

$T_m = V_m^T A V_m$

Last column is corrected via a randomized least-squares:

$y = \arg\min_z \|V_m z - v_{m+1}\|_2$

1. Approximation:

$f(A)b \approx V_m f(T_m) e_1$

with $e_1$ the canonical vector, incorporating the norm of $b$.

These steps are realized in multiple architectures: Gram–Schmidt-based (cheap sketches in the inner product), Householder-based (randomized Householder reflectors; see (Grigori et al., 2024)), "sketch-and-select" (compressive projection pursuit as in (Güttel et al., 2023)), and with implicit restarting for eigenproblems (Damas et al., 2024).

3. Theoretical Guarantees and Similarity Restoration

While randomized orthogonalization produces well-conditioned bases, the resulting Hessenberg matrices differ from those of classical Arnoldi, impairing interpretation of Ritz values and the containment of the numerical range ($W(H_m) \subset W(A)$ may not hold). This leads to irregular convergence or loss of optimality for eigen- and matrix function problems (Grigori et al., 15 Jan 2026, Cortinovis et al., 2022).

A critical modification restores "Arnoldi similarity": after the randomized basis and $H_m$ are constructed, the last Arnoldi vector is re-projected to enforce

$U_m^H \widehat u_{m+1} = 0$

through a small $m \times m$ least-squares problem, yielding a corrected Hessenberg $\widehat H_m$ similar to the standard $G_m$:

$A U_m = U_m \widehat H_m + \widehat u_{m+1} c_m^H$

with $\widehat H_m = R^{-1} G_m R$ for some invertible $R$. This modification recovers the Ritz values, ensures $W(\widehat H_m) \subset W(A)$, and ensures that for any polynomial matrix function $f$, $f(\widehat H_m)$ gives identical projections as the classical Arnoldi method (Grigori et al., 15 Jan 2026).

4. Computational Complexity and Communication

The randomized approaches consistently achieve lower asymptotic and practical costs:

Process	Orthogonalization Flops	Mat-vec Calls	Communication per Step
Arnoldi (Gram–Schmidt)	$O(n m^2)$	$m$	$O(m)$
Randomized Gram–Schmidt	$O(n m^2 + n m \log n)$	$m$	1–2
RHQR-Arnoldi	$O(n m^2)$	$m$	1
Sketch-and-Select	$O(N m k + m\,N \log m)$	$m$	1

Randomized methods offload heavy inner-product and orthogonalization computations to lower-dimensional sketched problems and often exploit Level-3 BLAS for increased efficiency in the correction phase (Cortinovis et al., 2022, Grigori et al., 15 Jan 2026, Grigori et al., 2024, Güttel et al., 2023).

The sketch dimension $s$ (or $\ell$) is generally chosen as $s \approx 2m$; embedding tolerances in $[0.2, 0.5]$ typically suffice. When the basis becomes ill-conditioned, "whitening" or explicit QR recomputation in sketch space is used to restore conditioning (Cortinovis et al., 2022, Grigori et al., 2024).

5. Numerical Stability, Conditioning, and Parameter Selection

Randomized orthogonalization inherits the backward and forward stability guarantees of classical orthogonalization but shifts dependency from the matrix condition number to sketching error and embedding parameters (Damas et al., 17 Dec 2025, Grigori et al., 2024). Finite-precision analyses show, for example, that RHQR basis $Q$ satisfies

$$\mathrm{Cond}(\Psi Q) \leq \frac{1 + \epsilon}{1 - \epsilon}$$

with high probability, independently of the condition number of the matrix being factorized. The randomized process is unconditionally stable up to an error controlled by the sketch's embedding quality and the arithmetic precision used in sketch and small-matrix operations (Grigori et al., 2024). Parameter choices (sketch size $s$, tolerance $\epsilon$, mixed or low precision) are detailed with practical recommendations for high performance and stability.

6. Applications and Empirical Performance

The Modified Randomized Arnoldi Process is applicable to:

• Approximating matrix functions, e.g., matrix exponential, square root, fractional powers, with

$f(A) b \approx V_m f(T_m) e_1$

and convergence bounded as (cf. Crouzeix):

$$\|f(A)b - V_m f(T_m)e_1\|_2 \leq 2(1+\sqrt{2}) \min_{p \in \Pi_{m-1}} \|f - p\|_{W(A)}$$

(Cortinovis et al., 2022).

• Computing eigenvalues and eigenvectors in large sparse matrices, utilizing implicit restarting and polynomial filtering in a randomized fashion (cf. rIRA) (Damas et al., 2024).
• High-dimensional linear system solution frameworks (randomized GMRES, sketched orthogonalization methods) (Damas et al., 17 Dec 2025).

Empirical studies demonstrate that similarity-restoring modifications eliminate convergence pathologies (spikes, stagnation) observed in naive randomized Arnoldi, delivering convergence and final accuracy identical to classical orthogonalization, but at a fraction of the cost (Grigori et al., 15 Jan 2026, Cortinovis et al., 2022). Speedups of $2\times$ or better are reported for large $m$ and sparse $A$.

7. Extensions, Variants, and Related Algorithms

Several notable extensions and related approaches have been developed:

• Randomized Householder QR-based Arnoldi: Replaces traditional Gram–Schmidt with single-synchronization RHQR, matching or exceeding classic Householder QR stability, and reducing per-step communication to one global synchronization (Grigori et al., 2024).
• Sketch-and-select Arnoldi: At each iteration, only $k \ll m$ prior basis vectors are used for projection, with subset selection performed via compressive sensing heuristics, resulting in a process linear in $m$ with empirical stability close to fully orthogonalized bases (Güttel et al., 2023).
• Restarted and block variants: Efficiently combine randomized orthogonalization with implicit restarting schemes, supporting large-scale eigenvalue computations, and enabling high communication efficiency in distributed environments (Damas et al., 2024, Damas et al., 17 Dec 2025).

A common thread is the use of oblivious subspace embeddings (e.g., SRHT), randomized Gram–Schmidt or Householder procedures, and, when needed, explicit corrections to the Hessenberg structure to recover theoretical guarantees.

---

For complete algorithmic details, proofs of similarity restoration, recommended implementation parameters, and full empirical results, see the foundational works (Cortinovis et al., 2022, Grigori et al., 15 Jan 2026, Grigori et al., 2024, Güttel et al., 2023, Damas et al., 2024), and (Damas et al., 17 Dec 2025).