---
title: Modified Randomized Arnoldi Process
url: https://www.emergentmind.com/topics/modified-randomized-arnoldi-process
type: topic
---

# Modified Randomized Arnoldi Process

The Modified Randomized Arnoldi Process is a class of algorithms designed to accelerate Krylov subspace methods by employing randomized sketching and non-traditional orthogonalization, while preserving or restoring key theoretical properties of the standard Arnoldi process. These modifications are primarily motivated by the need to reduce the cost and communication bottlenecks associated with classical Arnoldi orthogonalization in large-scale scientific computing. The resulting algorithms achieve computational and parallel efficiency, while ensuring well-conditioned bases for Krylov subspaces and delivering robust convergence and stability for matrix function evaluations and eigenvalue problems [2212.12758, 2601.10248, 2512.15455].

## 1. Foundations: Arnoldi and the Need for Modification

The classical Arnoldi process builds an orthonormal basis $\{q_1, ..., q_m\}$ for the $m$-dimensional Krylov subspace
$$
\mathcal{K}_m(A, b) = \mathrm{span}\{b,\, Ab,\, ...,\, A^{m-1}b\}
$$
using Gram–Schmidt orthogonalization. This leads to a Hessenberg decomposition:
$$
A Q_m = Q_m G_m + g_{m+1,m} q_{m+1} e_m^H
$$
where $G_m$ is upper Hessenberg. While robust, the per-iteration cost is $O(n m)$ for vector operations and $O(n m^2)$ flops for orthogonalization, with $O(m)$ global communications per iteration—factors that constrain scalability [2601.10248, 2512.15455].

Recent approaches accelerate Arnoldi by forming bases that are only "sketch-orthonormal" or even non-orthonormal, replacing high-dimensional inner products with randomized projections, and offloading heavy operations to randomized least-squares routines, often yielding substantial speedups with provably well-conditioned bases [2512.15455, 2212.12758].

## 2. Core Algorithmic Structure

The Modified Randomized Arnoldi Process introduces random sketching matrices (e.g., SRHT or SRFT) to perform Gram–Schmidt or Householder-type orthogonalization in a lower-dimensional space. A typical algorithmic workflow (as in [2212.12758]) is:

1. **Sketching**: Select a random embedding matrix $\Theta \in \mathbb{R}^{s \times n}$, with $s \approx 2m \ll n$.
2. **Initialization**: Normalize $b$, sketch it ($p_1 = \Theta b$), and set the first basis vector $v_1 = b / \|\Theta b\|$.
3. **Iteration**. For $i = 2, ..., m+1$:
   - Form $w_i = A v_{i-1}$, sketch $p_i = \Theta w_i$.
   - Gram–Schmidt (or Householder) re-orthogonalize $p_i$ against prior sketched basis vectors.
   - Apply corresponding updates in full space to $w_i$ to obtain $v_i$.
   - Grow low-dimensional coefficients (Hessenberg or $R$ factors).
4. **Compressed Operator Construction**: 
   $$
   T_m = V_m^T A V_m
   $$
   Last column is corrected via a randomized least-squares:
   $$
   y = \arg\min_z \|V_m z - v_{m+1}\|_2
   $$
5. **Approximation**:
   $$
   f(A)b \approx V_m f(T_m) e_1
   $$
   with $e_1$ the canonical vector, incorporating the norm of $b$.

These steps are realized in multiple architectures: Gram–Schmidt-based (cheap sketches in the inner product), Householder-based (randomized Householder reflectors; see [2405.10923]), "sketch-and-select" (compressive projection pursuit as in [2306.03592]), and with implicit restarting for eigenproblems [2407.03208].

## 3. Theoretical Guarantees and Similarity Restoration

While randomized orthogonalization produces well-conditioned bases, the resulting Hessenberg matrices differ from those of classical Arnoldi, impairing interpretation of Ritz values and the containment of the numerical range ($W(H_m) \subset W(A)$ may not hold). This leads to irregular convergence or loss of optimality for eigen- and matrix function problems [2601.10248, 2212.12758]. 

A critical modification restores "Arnoldi similarity": after the randomized basis and $H_m$ are constructed, the last Arnoldi vector is re-projected to enforce
$$
U_m^H \widehat u_{m+1} = 0
$$
through a small $m \times m$ least-squares problem, yielding a corrected Hessenberg $\widehat H_m$ similar to the standard $G_m$:
$$
A U_m = U_m \widehat H_m + \widehat u_{m+1} c_m^H
$$
with $\widehat H_m = R^{-1} G_m R$ for some invertible $R$. This modification recovers the Ritz values, ensures $W(\widehat H_m) \subset W(A)$, and ensures that for any polynomial matrix function $f$, $f(\widehat H_m)$ gives identical projections as the classical Arnoldi method [2601.10248].

## 4. Computational Complexity and Communication

The randomized approaches consistently achieve lower asymptotic and practical costs:

| Process                  | Orthogonalization Flops | Mat-vec Calls | Communication per Step |
|--------------------------|------------------------|---------------|-----------------------|
| Arnoldi (Gram–Schmidt)   | $O(n m^2)$             | $m$           | $O(m)$                |
| Randomized Gram–Schmidt  | $O(n m^2 + n m \log n)$| $m$           | 1–2                   |
| RHQR-Arnoldi             | $O(n m^2)$             | $m$           | 1                     |
| Sketch-and-Select        | $O(N m k + m\,N \log m)$| $m$           | 1                     |

Randomized methods offload heavy inner-product and orthogonalization computations to lower-dimensional sketched problems and often exploit Level-3 BLAS for increased efficiency in the correction phase [2212.12758, 2601.10248, 2405.10923, 2306.03592].

The sketch dimension $s$ (or $\ell$) is generally chosen as $s \approx 2m$; embedding tolerances in $[0.2, 0.5]$ typically suffice. When the basis becomes ill-conditioned, "whitening" or explicit QR recomputation in sketch space is used to restore conditioning [2212.12758, 2405.10923].

## 5. Numerical Stability, Conditioning, and Parameter Selection

Randomized orthogonalization inherits the backward and forward stability guarantees of classical orthogonalization but shifts dependency from the matrix condition number to sketching error and embedding parameters [2512.15455, 2405.10923]. Finite-precision analyses show, for example, that RHQR basis $Q$ satisfies
$$
\mathrm{Cond}(\Psi Q) \leq \frac{1 + \epsilon}{1 - \epsilon}
$$
with high probability, independently of the condition number of the matrix being factorized. The randomized process is unconditionally stable up to an error controlled by the sketch's embedding quality and the arithmetic precision used in sketch and small-matrix operations [2405.10923]. Parameter choices (sketch size $s$, tolerance $\epsilon$, mixed or low precision) are detailed with practical recommendations for high performance and stability.

## 6. Applications and Empirical Performance

The Modified Randomized Arnoldi Process is applicable to:

- Approximating matrix functions, e.g., matrix exponential, square root, fractional powers, with
  $$
  f(A) b \approx V_m f(T_m) e_1
  $$
  and convergence bounded as (cf. Crouzeix):
  $$
  \|f(A)b - V_m f(T_m)e_1\|_2 \leq 2(1+\sqrt{2}) \min_{p \in \Pi_{m-1}} \|f - p\|_{W(A)}
  $$
  [2212.12758].
- Computing eigenvalues and eigenvectors in large sparse matrices, utilizing implicit restarting and polynomial filtering in a randomized fashion (cf. rIRA) [2407.03208].
- High-dimensional linear system solution frameworks (randomized GMRES, sketched orthogonalization methods) [2512.15455].

Empirical studies demonstrate that similarity-restoring modifications eliminate convergence pathologies (spikes, stagnation) observed in naive randomized Arnoldi, delivering convergence and final accuracy identical to classical orthogonalization, but at a fraction of the cost [2601.10248, 2212.12758]. Speedups of $2\times$ or better are reported for large $m$ and sparse $A$.

## 7. Extensions, Variants, and Related Algorithms

Several notable extensions and related approaches have been developed:

- **Randomized Householder QR-based Arnoldi**: Replaces traditional Gram–Schmidt with single-synchronization RHQR, matching or exceeding classic Householder QR stability, and reducing per-step communication to one global synchronization [2405.10923].
- **Sketch-and-select Arnoldi**: At each iteration, only $k \ll m$ prior basis vectors are used for projection, with subset selection performed via compressive sensing heuristics, resulting in a process linear in $m$ with empirical stability close to fully orthogonalized bases [2306.03592].
- **Restarted and block variants**: Efficiently combine randomized orthogonalization with implicit restarting schemes, supporting large-scale eigenvalue computations, and enabling high communication efficiency in distributed environments [2407.03208, 2512.15455].

A common thread is the use of oblivious subspace embeddings (e.g., SRHT), randomized Gram–Schmidt or Householder procedures, and, when needed, explicit corrections to the Hessenberg structure to recover theoretical guarantees.

---

For complete algorithmic details, proofs of similarity restoration, recommended implementation parameters, and full empirical results, see the foundational works [2212.12758], [2601.10248], [2405.10923], [2306.03592], [2407.03208], and [2512.15455].

Source: https://www.emergentmind.com/topics/modified-randomized-arnoldi-process