---
title: Randomized Two-Sided Gram-Schmidt Process
url: https://www.emergentmind.com/topics/randomized-two-sided-gram-schmidt-process
type: topic
---

# Randomized Two-Sided Gram-Schmidt Process

The randomized two-sided Gram-Schmidt process denotes a class of orthogonalization and biorthogonalization procedures in which the deterministic inner-product or pivot-selection mechanism of classical two-sided Gram-Schmidt is replaced by randomization. In current numerical linear algebra, the term appears in two closely related senses. In one sense, it refers to a Jacobi-like or generalized QR procedure in which random pivot sets are selected and local transformations are applied on both sides within a unified factorization framework [2505.02023]. In the other, it refers to sketch-based biorthogonalization of two matrices \(X\) and \(Y\), producing bases \(Q\) and \(P\) whose sketches satisfy a biorthogonality relation such as \((\Omega Q)^T \Omega P = I\) rather than enforcing exact Euclidean biorthogonality in the ambient space [2509.04386]. Across these formulations, the central objectives are reduced orthogonalization cost, improved conditioning, and provable convergence or stability under randomized pivoting or oblivious subspace embeddings.

## 1. Conceptual scope and mathematical setting

The classical two-sided Gram-Schmidt process is associated with biorthogonalization: given two sequences or matrices, one constructs left and right bases satisfying a coupling condition such as \(P^T Q = I\). Recent randomized variants relax or reinterpret that condition. In sketch-based formulations, the exact condition is replaced by sketch-biorthogonality, while in generalized factorization formulations the process is embedded in a broader class of one-sided and two-sided transformations acting on \(A\) or on the Gram matrix \(B = A^*A\) [2509.04386].

A concise way to organize the modern usages is the following.

| Formulation | Core objects | Target condition |
|---|---|---|
| Generalized factorization / Jacobi-like QR | \(A\), \(B=A^*A\), local \(S \in \mathrm{GL}_k(\mathbb{F})\) | Orthogonal columns, triangularity, or diagonalization |
| Sketch-based two-sided Gram-Schmidt | \(X,Y \in \mathbb{R}^{n\times m}\), sketch \(\Omega\) | \((\Omega Q)^T \Omega P = I\) |

In the generalized perspective, Gram-Schmidt, Gaussian elimination, Jacobi eigensolvers, and related decompositions are treated as instances of a single abstract iterative scheme [2505.02023]. In the sketch-based perspective, randomized orthogonalization constructs a well-conditioned basis whose sketch is orthonormal, and the two-sided extension applies the same principle to biorthogonal pairs [2512.15455]. This suggests that the randomized two-sided Gram-Schmidt process is less a single algorithm than a family of algorithms unified by randomized local compression or randomized pivoting.

## 2. Generalized two-sided orthogonalization through randomized pivoting

The unified factorization framework of “A Unified Perspective on Orthogonalization and Diagonalization” begins from an abstract \(k\)-pivot algorithm. One initializes \(A^{(0)} = A\) and \(T^{(0)} = I\), then repeatedly selects a pivot set \(J \subset [n]\) of size \(k\), computes a local transformation \(S \in \mathrm{GL}_k(\mathbb{F})\) so that \(A_J S\) has orthogonal columns, embeds \(S^{-1}\) into a global transformation \(T\), and updates the iterate [2505.02023]. For the one-sided version,
\[
A \leftarrow A T^{-1},
\]
whereas for the two-sided version,
\[
B \leftarrow (T^{-1})^* B T^{-1}.
\]

The local structure imposed on \(S\) determines the factorization target. The paper lists the following special cases: unitary \(S\) with diagonal target for eigendecomposition or SVD; upper triangular \(S\) with identity target for Cholesky or QR; unit upper triangular \(S\) with diagonal target for LDL-type factorizations; and lower triangular \(S\) with identity target for QL-type factorizations [2505.02023]. The Gram-Schmidt process is the QR-specialized instance obtained by taking \(S\) upper triangular. The two-sided Gram-Schmidt or Jacobi-like QR algorithm is included by acting with transformations on both left and right to target upper triangular or diagonal forms.

The randomized pivoting rule is the defining novelty of this framework. At each iteration, the pivot set \(J\) is chosen uniformly at random from all unordered pairs when \(k=2\), or from all subsets of size \(k\) in the block case. In the Gram-Schmidt interpretation, this means iteratively choosing random pairs of columns and orthogonalizing them. The paper emphasizes that this randomized rule differs substantially from the usual pivoting rules for Gaussian elimination or Gram-Schmidt, yet yields the same linear rate of convergence for each algorithm in the class, irrespective of which factorization it computes [2505.02023].

For \(k=2\), the paper also records representative pairwise orthogonalization updates. One example is
\[
\begin{bmatrix} a_i^{\text{new}} & a_j^{\text{new}} \end{bmatrix}
:=
\begin{bmatrix}
\displaystyle \frac{a_i + a_j}{\sqrt{2 + 2\alpha}} &
\displaystyle \frac{a_i - a_j}{\sqrt{2 - 2\alpha}}
\end{bmatrix},
\quad
\alpha = \langle a_i, a_j \rangle,
\]
and another is the classical Gram-Schmidt step
\[
a_j \gets \frac{a_j - \langle a_i, a_j \rangle a_i}{\sqrt{1 - |\langle a_i, a_j \rangle|^2}}.
\]
These formulas illustrate the local nature of the randomized pair update, even though the global process is two-sided in the generalized formulation [2505.02023].

## 3. Sketch-based randomized biorthogonalization

The biorthogonalization formulation starts from two full-rank matrices \(X, Y \in \mathbb{R}^{n\times m}\). The objective is to compute \(Q, P \in \mathbb{R}^{n\times m}\) such that
\[
\mathrm{range}(X) = \mathrm{range}(Q), \qquad
\mathrm{range}(Y) = \mathrm{range}(P), \qquad
(\Omega Q)^T (\Omega P) = I,
\]
where \(\Omega \in \mathbb{R}^{s\times n}\) is a sketching matrix satisfying an oblivious subspace \(\varepsilon\)-embedding property [2509.04386]. Here the classical Euclidean biorthogonality condition \(Q^T P = I\) is replaced by an equivalent condition on the sketches.

Algorithmically, the process constructs columns \(\mathbf{q}_i\) and \(\mathbf{p}_i\) iteratively. In the formulation reproduced in the paper, one computes
\[
\mathbf{q}_i = \mathbf{x}_i - \mathcal{Q}_{i-1}\mathbf{x}_i, \qquad
\mathbf{p}_i = \mathbf{y}_i - \mathcal{P}_{i-1}\mathbf{y}_i,
\]
then sets
\[
d_i = \langle \Omega \mathbf{p}_i, \Omega \mathbf{q}_i \rangle.
\]
If \(d_i=0\), the process breaks down. Otherwise one normalizes by
\[
\mathbf{q}_i = \mathbf{q}_i/\sqrt{d_i}, \qquad
\mathbf{p}_i = \mathbf{p}_i/\sqrt{d_i}\cdot \mathrm{sign}(d_i),
\]
so that \((\Omega \mathbf{q}_i)^T(\Omega \mathbf{p}_i)=1\) [2509.04386].

The associated projectors are sketched oblique projectors. If \(Q_i\) and \(P_i\) denote the partial bases, then the projector onto \(\mathrm{range}(Q_i)\) acting sketch-orthogonally to \(\mathrm{range}(P_i)\) is
\[
\mathcal{Q}_i
=
Q_i \left((\Omega P_i)^T \Omega Q_i\right)^{-1} (\Omega P_i)^T \Omega.
\]
When exact sketched biorthogonality is maintained at step \(i-1\), this simplifies to
\[
\mathcal{Q}_{i-1} \mathbf{x}_i
=
Q_{i-1} (\Omega P_{i-1})^T \Omega \mathbf{x}_i.
\]
The algorithm admits several implementations, including rCGS, rMGS, and rCGS\(_O\), the last of which explicitly maintains \((\Omega P_{i-1})^T (\Omega Q_{i-1})\) when biorthogonality is lost in finite precision [2509.04386].

The role of the sketch is geometric as well as computational. For vectors in the relevant subspace,
\[
|\langle \mathbf{x},\mathbf{y}\rangle - \langle \Omega \mathbf{x}, \Omega \mathbf{y}\rangle|
\leq
\varepsilon \|\mathbf{x}\| \|\mathbf{y}\|,
\]
and for sparse sign sketches a sketching dimension \(s = O(\varepsilon^{-2} m \log m)\) suffices [2509.04386]. The overview paper on randomized orthogonalization places this construction within a broader framework in which randomized orthogonalization seeks a well-conditioned basis whose sketch is orthonormal, and identifies the two-sided extension as the natural sketched analogue of deterministic biorthogonalization [2512.15455].

## 4. Convergence, conditioning, and finite-precision behavior

In the generalized pivoting framework, convergence is expressed through the potential
\[
\Gamma(B) = \mathrm{tr}(B \odot B^{-1}) - n.
\]
For randomized size-\(k\) pivoting,
\[
\mathbb{E}[\Gamma(B(t))]
=
\left(1 - \frac{k(k-1)}{n(n-1)}\right)^t \Gamma(B^{(0)}),
\]
which gives linear decay in expectation. For pairwise updates,
\[
\mathbb{E}[\Gamma(B(t))]
=
\left(1 - \frac{2}{n(n-1)}\right)^t \Gamma(B^{(0)}).
\]
Accordingly, \(O(n^2 \log(n/\delta))\) iterations suffice to reach error \(\delta\), and the paper states that the one-sided Gram-Schmidt case exhibits analogous convergence of \(Q^*Q \to I\) at the same rate [2505.02023].

The same paper also develops a finite-precision framework. For the randomized Gram-Schmidt process, running for \(O(n^2 \log(n/\delta))\) iterations with sufficiently small roundoff
\[
\varepsilon = O\left(\delta \sigma_n(A)^4 / (n^6 \tau^4)\right),
\]
for normalized initial \(A\), guarantees
\[
\|Q^*Q - I\| \leq 3\sqrt{\delta}
\]
and
\[
\mathrm{dist}(\mathrm{col}(Q), \mathrm{col}(A)) \leq \delta
\]
with high probability [2505.02023]. The same stability framework yields, for randomized size-2 pivoting in the Jacobi/eigenvalue setting, a bound on the off-diagonal part after \(\tau = O(n^2 \log(n\hat\kappa(B)/\delta))\) iterations and a bound on the maximal diagonal-normalized condition number of the iterates [2505.02023]. A second important consequence emphasized in the abstract is a provable, effective bound on the numerical stability of the Jacobi eigenvalue algorithm, addressing a longstanding open problem of Demmel and Veselić [2505.02023].

In the sketch-based two-sided process, the dominant stability mechanism is different. Deterministic biorthogonalization is prone to near-breakdown because the next left-right inner product can become very small, producing vectors with very large norms. The randomized method reduces that risk probabilistically. For Gaussian sketches, Proposition 4.1 states
\[
\mathbb{P}\left(|\langle \Omega\mathbf{x}, \Omega\mathbf{y} \rangle| \leq \delta \right) \leq C s \delta
\]
for unit vectors \(\mathbf{x},\mathbf{y}\), so nearly vanishing sketched couplings are unlikely [2509.04386]. The abstract summarizes the numerical consequence: the randomized two-sided Gram-Schmidt process is computationally less expensive than the classical two-sided Gram-Schmidt process, has better numerical stability, and the condition number of the computed bases \(Q,P\) is often smaller than in the deterministic case [2509.04386].

## 5. Relation to randomized QR, sketch-orthogonality, and block variants

The randomized two-sided process is closely connected to one-sided randomized orthogonalization. In randomized Gram-Schmidt for QR, inner products and norms are replaced by sketched quantities. A typical single-sided formulation computes
\[
[R]_{(1:i-1,i)} = \underset{y}{\arg\min}\ \| S_{i-1} y - S w_i\|_2,
\]
then updates
\[
q'_i = w_i - Q_{i-1}[R]_{(1:i-1,i)}, \qquad
r_{i,i} = \|S q'_i\|, \qquad
q_i = q'_i/r_{i,i}.
\]
The governing property is that \(\langle x,y\rangle \approx \langle Sx,Sy\rangle\), and the resulting \(Q\) satisfies singular-value bounds of the form
\[
(1+\varepsilon)^{-1/2} \leq \sigma_{\min}(Q) \leq \sigma_{\max}(Q) \leq (1-\varepsilon)^{-1/2}
\]
when the sketch is an \(\varepsilon\)-embedding [2011.05090].

Randomized Cholesky QR provides a parallel viewpoint. Its column-oriented variant establishes the connection with the randomized Gram-Schmidt process, and if at each step the sketch of the current basis is refreshed, the column-oriented method becomes identical to randomized Gram-Schmidt in exact arithmetic [2210.09953]. Randomized Householder QR pushes the same paradigm toward reflector-based QR: it computes a basis \(Q\) such that the sketch \(\Psi Q\) is numerically orthogonal, inherits the stability of Householder QR under mild probabilistic assumptions, and has a cost similar to randomized Gram-Schmidt overall [2405.10923].

Block variants extend these ideas to BLAS3 regimes. The randomized block Gram-Schmidt process inherits the key characteristics of single-vector randomized Gram-Schmidt, while its low-synchronization variant can be implemented in a parallel environment using only one global reduction operation between processors per block [2111.14641]. This broader ecosystem is relevant because the sketch-based two-sided process shares the same structural premise: a well-conditioned basis in the ambient space together with exact or approximate orthogonality in the sketch space [2512.15455].

A common misconception is that randomization merely approximates deterministic Gram-Schmidt with lower accuracy. The recent literature presents a different picture: the target object is often not an exactly orthonormal or exactly biorthogonal basis in \(\mathbb{R}^n\), but a basis whose sketch is orthonormal or biorthogonal and whose ambient-space conditioning is controlled by the embedding [2405.10923].

## 6. Applications, limitations, and current significance

The most explicit application of the randomized two-sided Gram-Schmidt process is to the nonsymmetric Lanczos algorithm. In that setting, one constructs left and right Krylov bases \(Q_m\) and \(P_m\) satisfying
\[
(\Omega Q_m)^T (\Omega P_m) = I,
\]
then forms projected matrices such as
\[
H_m = (\Omega P_m)^T \Omega A Q_m, \qquad
T_m = (\Omega Q_m)^T \Omega A^T P_m.
\]
The paper notes that these matrices are generally not tridiagonal and not transposes of each other, so the short-recurrence structure is lost [2509.04386]. Even so, the reported practical effect is favorable when explicit full biorthogonalization is required anyway: the randomized method is more stable, has less risk of breakdown, and numerical experiments show almost identical convergence of residuals and eigenvalue approximations between randomized and deterministic fully biorthogonalized Lanczos [2509.04386].

More broadly, randomized orthogonalization techniques are used within Krylov subspace methods to mitigate orthogonalization cost. The overview paper discusses randomized Arnoldi relations and applications to linear systems of equations, eigenvalue problems, the evaluation of matrix functions, and matrix equations [2512.15455]. One-sided randomized Gram-Schmidt has already been integrated into GMRES, while randomized global Gram-Schmidt with sketched Frobenius inner products has been used in RGl-GMRES for multiple right-hand sides [2011.05090; 2602.14786]. These developments suggest that two-sided randomized Gram-Schmidt is part of a larger shift from exact orthogonality in the ambient space toward sketch-controlled conditioning and communication-efficient orthogonalization.

The main limitations are structural rather than conceptual. In biorthogonal Krylov methods, the loss of short recurrences can offset some of the algorithmic simplicity of classical nonsymmetric Lanczos. In generalized pivoting methods, the random pivot rule changes the interpretation of progress from greedy local elimination to expected global decrease of a potential. Nonetheless, the current literature presents the randomized two-sided Gram-Schmidt process as a technically coherent response to two long-standing numerical difficulties: the instability of deterministic biorthogonalization and the communication cost of high-dimensional orthogonalization [2505.02023].

Source: https://www.emergentmind.com/topics/randomized-two-sided-gram-schmidt-process