Randomized Two-Sided Gram-Schmidt Process

• The randomized two-sided Gram-Schmidt process replaces deterministic pivot selection with randomization to reduce orthogonalization costs and improve numerical stability.
• It employs both generalized factorization and sketch-based biorthogonalization approaches to construct well-conditioned bases with provable convergence and error bounds.
• Its integration in Krylov subspace methods and QR factorizations demonstrates practical benefits in reducing communication overhead and ensuring robust performance in large-scale computations.

The randomized two-sided Gram-Schmidt process denotes a class of orthogonalization and biorthogonalization procedures in which the deterministic inner-product or pivot-selection mechanism of classical two-sided Gram-Schmidt is replaced by randomization. In current numerical linear algebra, the term appears in two closely related senses. In one sense, it refers to a Jacobi-like or generalized QR procedure in which random pivot sets are selected and local transformations are applied on both sides within a unified factorization framework (Detherage et al., 4 May 2025). In the other, it refers to sketch-based biorthogonalization of two matrices $X$ and $Y$, producing bases $Q$ and $P$ whose sketches satisfy a biorthogonality relation such as $(\Omega Q)^T \Omega P = I$ rather than enforcing exact Euclidean biorthogonality in the ambient space (Grigori et al., 4 Sep 2025). Across these formulations, the central objectives are reduced orthogonalization cost, improved conditioning, and provable convergence or stability under randomized pivoting or oblivious subspace embeddings.

1. Conceptual scope and mathematical setting

The classical two-sided Gram-Schmidt process is associated with biorthogonalization: given two sequences or matrices, one constructs left and right bases satisfying a coupling condition such as $P^T Q = I$. Recent randomized variants relax or reinterpret that condition. In sketch-based formulations, the exact condition is replaced by sketch-biorthogonality, while in generalized factorization formulations the process is embedded in a broader class of one-sided and two-sided transformations acting on $A$ or on the Gram matrix $B = A^*A$ (Grigori et al., 4 Sep 2025).

A concise way to organize the modern usages is the following.

Formulation	Core objects	Target condition
Generalized factorization / Jacobi-like QR	$A$, $B=A^*A$, local $Y$0	Orthogonal columns, triangularity, or diagonalization
Sketch-based two-sided Gram-Schmidt	$Y$1, sketch $Y$2	$Y$3

In the generalized perspective, Gram-Schmidt, Gaussian elimination, Jacobi eigensolvers, and related decompositions are treated as instances of a single abstract iterative scheme (Detherage et al., 4 May 2025). In the sketch-based perspective, randomized orthogonalization constructs a well-conditioned basis whose sketch is orthonormal, and the two-sided extension applies the same principle to biorthogonal pairs (Damas et al., 17 Dec 2025). This suggests that the randomized two-sided Gram-Schmidt process is less a single algorithm than a family of algorithms unified by randomized local compression or randomized pivoting.

2. Generalized two-sided orthogonalization through randomized pivoting

The unified factorization framework of “A Unified Perspective on Orthogonalization and Diagonalization” begins from an abstract $Y$4-pivot algorithm. One initializes $Y$5 and $Y$6, then repeatedly selects a pivot set $Y$7 of size $Y$8, computes a local transformation $Y$9 so that $Q$0 has orthogonal columns, embeds $Q$1 into a global transformation $Q$2, and updates the iterate (Detherage et al., 4 May 2025). For the one-sided version,

$Q$3

whereas for the two-sided version,

$Q$4

The local structure imposed on $Q$5 determines the factorization target. The paper lists the following special cases: unitary $Q$6 with diagonal target for eigendecomposition or SVD; upper triangular $Q$7 with identity target for Cholesky or QR; unit upper triangular $Q$8 with diagonal target for LDL-type factorizations; and lower triangular $Q$9 with identity target for QL-type factorizations (Detherage et al., 4 May 2025). The Gram-Schmidt process is the QR-specialized instance obtained by taking $P$0 upper triangular. The two-sided Gram-Schmidt or Jacobi-like QR algorithm is included by acting with transformations on both left and right to target upper triangular or diagonal forms.

The randomized pivoting rule is the defining novelty of this framework. At each iteration, the pivot set $P$1 is chosen uniformly at random from all unordered pairs when $P$2, or from all subsets of size $P$3 in the block case. In the Gram-Schmidt interpretation, this means iteratively choosing random pairs of columns and orthogonalizing them. The paper emphasizes that this randomized rule differs substantially from the usual pivoting rules for Gaussian elimination or Gram-Schmidt, yet yields the same linear rate of convergence for each algorithm in the class, irrespective of which factorization it computes (Detherage et al., 4 May 2025).

For $P$4, the paper also records representative pairwise orthogonalization updates. One example is

$P$5

and another is the classical Gram-Schmidt step

$P$6

These formulas illustrate the local nature of the randomized pair update, even though the global process is two-sided in the generalized formulation (Detherage et al., 4 May 2025).

3. Sketch-based randomized biorthogonalization

The biorthogonalization formulation starts from two full-rank matrices $P$7. The objective is to compute $P$8 such that

$P$9

where $(\Omega Q)^T \Omega P = I$0 is a sketching matrix satisfying an oblivious subspace $(\Omega Q)^T \Omega P = I$1-embedding property (Grigori et al., 4 Sep 2025). Here the classical Euclidean biorthogonality condition $(\Omega Q)^T \Omega P = I$2 is replaced by an equivalent condition on the sketches.

Algorithmically, the process constructs columns $(\Omega Q)^T \Omega P = I$3 and $(\Omega Q)^T \Omega P = I$4 iteratively. In the formulation reproduced in the paper, one computes

$(\Omega Q)^T \Omega P = I$5

then sets

$(\Omega Q)^T \Omega P = I$6

If $(\Omega Q)^T \Omega P = I$7, the process breaks down. Otherwise one normalizes by

$(\Omega Q)^T \Omega P = I$8

so that $(\Omega Q)^T \Omega P = I$9 (Grigori et al., 4 Sep 2025).

The associated projectors are sketched oblique projectors. If $P^T Q = I$0 and $P^T Q = I$1 denote the partial bases, then the projector onto $P^T Q = I$2 acting sketch-orthogonally to $P^T Q = I$3 is

$P^T Q = I$4

When exact sketched biorthogonality is maintained at step $P^T Q = I$5, this simplifies to

$P^T Q = I$6

The algorithm admits several implementations, including rCGS, rMGS, and rCGS$P^T Q = I$7, the last of which explicitly maintains $P^T Q = I$8 when biorthogonality is lost in finite precision (Grigori et al., 4 Sep 2025).

The role of the sketch is geometric as well as computational. For vectors in the relevant subspace,

$P^T Q = I$9

and for sparse sign sketches a sketching dimension $A$0 suffices (Grigori et al., 4 Sep 2025). The overview paper on randomized orthogonalization places this construction within a broader framework in which randomized orthogonalization seeks a well-conditioned basis whose sketch is orthonormal, and identifies the two-sided extension as the natural sketched analogue of deterministic biorthogonalization (Damas et al., 17 Dec 2025).

4. Convergence, conditioning, and finite-precision behavior

In the generalized pivoting framework, convergence is expressed through the potential

$A$1

For randomized size-$A$2 pivoting,

$A$3

which gives linear decay in expectation. For pairwise updates,

$A$4

Accordingly, $A$5 iterations suffice to reach error $A$6, and the paper states that the one-sided Gram-Schmidt case exhibits analogous convergence of $A$7 at the same rate (Detherage et al., 4 May 2025).

The same paper also develops a finite-precision framework. For the randomized Gram-Schmidt process, running for $A$8 iterations with sufficiently small roundoff

$A$9

for normalized initial $B = A^*A$0, guarantees

$B = A^*A$1

and

$B = A^*A$2

with high probability (Detherage et al., 4 May 2025). The same stability framework yields, for randomized size-2 pivoting in the Jacobi/eigenvalue setting, a bound on the off-diagonal part after $B = A^*A$3 iterations and a bound on the maximal diagonal-normalized condition number of the iterates (Detherage et al., 4 May 2025). A second important consequence emphasized in the abstract is a provable, effective bound on the numerical stability of the Jacobi eigenvalue algorithm, addressing a longstanding open problem of Demmel and Veselić (Detherage et al., 4 May 2025).

In the sketch-based two-sided process, the dominant stability mechanism is different. Deterministic biorthogonalization is prone to near-breakdown because the next left-right inner product can become very small, producing vectors with very large norms. The randomized method reduces that risk probabilistically. For Gaussian sketches, Proposition 4.1 states

$B = A^*A$4

for unit vectors $B = A^*A$5, so nearly vanishing sketched couplings are unlikely (Grigori et al., 4 Sep 2025). The abstract summarizes the numerical consequence: the randomized two-sided Gram-Schmidt process is computationally less expensive than the classical two-sided Gram-Schmidt process, has better numerical stability, and the condition number of the computed bases $B = A^*A$6 is often smaller than in the deterministic case (Grigori et al., 4 Sep 2025).

5. Relation to randomized QR, sketch-orthogonality, and block variants

The randomized two-sided process is closely connected to one-sided randomized orthogonalization. In randomized Gram-Schmidt for QR, inner products and norms are replaced by sketched quantities. A typical single-sided formulation computes

$B = A^*A$7

then updates

$B = A^*A$8

The governing property is that $B = A^*A$9, and the resulting $A$0 satisfies singular-value bounds of the form

$A$1

when the sketch is an $A$2-embedding (Balabanov et al., 2020).

Randomized Cholesky QR provides a parallel viewpoint. Its column-oriented variant establishes the connection with the randomized Gram-Schmidt process, and if at each step the sketch of the current basis is refreshed, the column-oriented method becomes identical to randomized Gram-Schmidt in exact arithmetic (Balabanov, 2022). Randomized Householder QR pushes the same paradigm toward reflector-based QR: it computes a basis $A$3 such that the sketch $A$4 is numerically orthogonal, inherits the stability of Householder QR under mild probabilistic assumptions, and has a cost similar to randomized Gram-Schmidt overall (Grigori et al., 2024).

Block variants extend these ideas to BLAS3 regimes. The randomized block Gram-Schmidt process inherits the key characteristics of single-vector randomized Gram-Schmidt, while its low-synchronization variant can be implemented in a parallel environment using only one global reduction operation between processors per block (Balabanov et al., 2021). This broader ecosystem is relevant because the sketch-based two-sided process shares the same structural premise: a well-conditioned basis in the ambient space together with exact or approximate orthogonality in the sketch space (Damas et al., 17 Dec 2025).

A common misconception is that randomization merely approximates deterministic Gram-Schmidt with lower accuracy. The recent literature presents a different picture: the target object is often not an exactly orthonormal or exactly biorthogonal basis in $A$5, but a basis whose sketch is orthonormal or biorthogonal and whose ambient-space conditioning is controlled by the embedding (Grigori et al., 2024).

6. Applications, limitations, and current significance

The most explicit application of the randomized two-sided Gram-Schmidt process is to the nonsymmetric Lanczos algorithm. In that setting, one constructs left and right Krylov bases $A$6 and $A$7 satisfying

$A$8

then forms projected matrices such as

$A$9

The paper notes that these matrices are generally not tridiagonal and not transposes of each other, so the short-recurrence structure is lost (Grigori et al., 4 Sep 2025). Even so, the reported practical effect is favorable when explicit full biorthogonalization is required anyway: the randomized method is more stable, has less risk of breakdown, and numerical experiments show almost identical convergence of residuals and eigenvalue approximations between randomized and deterministic fully biorthogonalized Lanczos (Grigori et al., 4 Sep 2025).

More broadly, randomized orthogonalization techniques are used within Krylov subspace methods to mitigate orthogonalization cost. The overview paper discusses randomized Arnoldi relations and applications to linear systems of equations, eigenvalue problems, the evaluation of matrix functions, and matrix equations (Damas et al., 17 Dec 2025). One-sided randomized Gram-Schmidt has already been integrated into GMRES, while randomized global Gram-Schmidt with sketched Frobenius inner products has been used in RGl-GMRES for multiple right-hand sides (Balabanov et al., 2020, Badahmane et al., 16 Feb 2026). These developments suggest that two-sided randomized Gram-Schmidt is part of a larger shift from exact orthogonality in the ambient space toward sketch-controlled conditioning and communication-efficient orthogonalization.

The main limitations are structural rather than conceptual. In biorthogonal Krylov methods, the loss of short recurrences can offset some of the algorithmic simplicity of classical nonsymmetric Lanczos. In generalized pivoting methods, the random pivot rule changes the interpretation of progress from greedy local elimination to expected global decrease of a potential. Nonetheless, the current literature presents the randomized two-sided Gram-Schmidt process as a technically coherent response to two long-standing numerical difficulties: the instability of deterministic biorthogonalization and the communication cost of high-dimensional orthogonalization (Detherage et al., 4 May 2025).