AgentClinic Benchmark Overview

• AgentClinic Benchmark is a framework that assesses various Gram–Schmidt orthogonalisation methods, comparing classical, modified, and block variants for numerical stability and efficiency.
• It examines a range of approaches—including randomized and hardware-aware variants—while analyzing finite-precision effects and loss of orthogonality in QR factorization.
• The benchmark highlights how algorithmic choices impact communication costs and synchronization requirements, offering actionable insights for both researchers and practitioners.

Searching arXiv for recent and relevant papers on Gram-Schmidt orthogonalisation to ground the article in the supplied literature. I’ll look up the cited arXiv records now. Gram–Schmidt orthogonalisation is the family of procedures that transforms a linearly independent set of vectors into an orthogonal or orthonormal basis spanning the same subspace, and, in matrix form, produces a thin or economic QR factorisation. For a full-column-rank matrix $A=[a_1,\dots,a_n]$, the standard target is

$A=QR,\qquad Q^TQ=I,$

with $R$ upper triangular; in block form, for $X=[X_1,\dots,X_p]\in\mathbb{R}^{m\times ps}$, the target is likewise $X=QR$ with $Q=[Q_1,\dots,Q_p]$. Modern treatments emphasise not only exact-arithmetic constructions, but also finite-precision orthogonality loss, communication cost, block structure, sketching, non-standard inner products, and application-specific variants (Carson et al., 2024).

1. Classical recurrences and the QR viewpoint

In its classical vector form, Gram–Schmidt starts from linearly independent vectors $a_1,\dots,a_k\in\mathbb{R}^\ell$ and constructs orthonormal vectors $q_1,\dots,q_k$ by

$r_{ij}=q_i^Ta_j,\qquad i=1,\dots,j-1,$

$v_j=a_j-\sum_{i=1}^{j-1}q_i r_{ij},$

$A=QR,\qquad Q^TQ=I,$0

Equivalently, with $A=QR,\qquad Q^TQ=I,$1, $A=QR,\qquad Q^TQ=I,$2, and upper triangular $A=QR,\qquad Q^TQ=I,$3, the process yields

$A=QR,\qquad Q^TQ=I,$4

A compact projector form writes

$A=QR,\qquad Q^TQ=I,$5

where $A=QR,\qquad Q^TQ=I,$6 (Carson et al., 2024).

Modified Gram–Schmidt replaces the single projection against the whole previously computed basis by sequential deflation. In block notation, one writes

$A=QR,\qquad Q^TQ=I,$7

for $A=QR,\qquad Q^TQ=I,$8, followed by $A=QR,\qquad Q^TQ=I,$9. In exact arithmetic, the classical and modified forms are equivalent; in floating-point arithmetic, their stability differs materially. This exact-arithmetic equivalence extends to block formulations and to determinant-based constructions of orthogonal polynomial systems (Balabanov et al., 2021).

A broader QR interpretation treats orthogonalisation as a project-and-normalise factorisation. Given an already computed basis $R$0 and a new block $R$1, the target relation can be written

$R$2

where $R$3. This formulation underlies communication-reducing algorithms based on CholeskyQR and TSQR, and makes explicit that orthogonalisation and normalisation are jointly a QR problem rather than two unrelated steps (Dreier et al., 2022).

2. Finite-precision behaviour, orthogonality loss, and reorthogonalisation

The central numerical issue is loss of orthogonality. In block settings, the principal measures used are

$R$4

the last being the relative Cholesky residual. Classical Gram–Schmidt can produce large orthogonality loss when the input is ill-conditioned; modified Gram–Schmidt is typically more stable; reorthogonalisation is the standard remedy (Carson et al., 2024).

The block reorthogonalised algorithm BCGS2 applies a two-pass block classical Gram–Schmidt step. For one block $R$5 against a previously computed block basis $R$6, the first pass computes

$R$7

and the second pass repeats the same projection-orthogonalisation pattern on the intermediate block. In exact arithmetic, the two-pass formulation is the block analogue of CGS2. Under appropriate assumptions on the diagonal blocks of $R$8, the floating-point analysis gives

$R$9

for full-column-rank $X=[X_1,\dots,X_p]\in\mathbb{R}^{m\times ps}$0 (Barlow et al., 2011).

The same reorthogonalisation principle is central in structured linear algebra applications. For symmetric saddle point systems, BCGS2 with thin Householder QR in each block step is proved backward stable under a mild assumption on the matrix $X=[X_1,\dots,X_p]\in\mathbb{R}^{m\times ps}$1. In that setting, the computed solution $X=[X_1,\dots,X_p]\in\mathbb{R}^{m\times ps}$2 is the exact solution of a nearby perturbed system, and the stability transfer relies on small factorisation residual and near-orthogonality of the computed $X=[X_1,\dots,X_p]\in\mathbb{R}^{m\times ps}$3 (Okulicka-Dłużewska et al., 2013).

A further refinement is the Pythagorean block classical Gram–Schmidt family. For block $X=[X_1,\dots,X_p]\in\mathbb{R}^{m\times ps}$4, the Pythagorean identity

$X=[X_1,\dots,X_p]\in\mathbb{R}^{m\times ps}$5

yields a Cholesky-based update

$X=[X_1,\dots,X_p]\in\mathbb{R}^{m\times ps}$6

For BCGS-PIP, the loss of orthogonality satisfies an $X=[X_1,\dots,X_p]\in\mathbb{R}^{m\times ps}$7 bound as long as $X=[X_1,\dots,X_p]\in\mathbb{R}^{m\times ps}$8. Two reorthogonalised Pythagorean variants, BCGS-PIP+ and BCGS-PIPI+, improve this to an $X=[X_1,\dots,X_p]\in\mathbb{R}^{m\times ps}$9 bound while retaining asymptotically two synchronization points per block column (Carson et al., 2024).

3. Block formulations, synchronization reduction, and the single-column exception

Block Gram–Schmidt treats a matrix as

$X=QR$0

with block columns $X=QR$1. A generic block step separates into a projection stage and an intraorthogonalization stage: $X=QR$2 This abstraction supports rigorous orthogonality-loss bounds in terms of projection error, QR backward error, and conditioning of the projected block (Carson et al., 2024).

A notable result is that, in the variant BCGS-A, a strong intrablock orthogonalization routine is needed only for the first block. If the first block is orthogonalized by a Householder-like routine and later blocks use a weaker but backward stable routine, then the entire basis can still satisfy

$X=QR$3

provided the condition number of $X=QR$4 is not too large. This isolates the role of the first block as the anchor of the induction used in the stability proof (Carson et al., 2024).

The same analysis clarifies the cost of reducing synchronization. Starting from the four-synchronization reorthogonalized method BCGSI+A, the paper derives 3-sync, 2-sync, and 1-sync variants by removing synchronization points one at a time. The theoretical and empirical conclusions are aligned: the first reduction already weakens stability, and the one-synchronization block method cannot be guaranteed stable in practice.

Variant	Synchronizations per block	Orthogonality bound / behaviour
BCGSI+A	4	$X=QR$5, stable
BCGSI+A-3S	3	$X=QR$6, stable until moderate $X=QR$7
BCGSI+A-2S	2	$X=QR$8, worse with quadratic growth
BCGSI+A-1S	1	$X=QR$9, unstable for large $Q=[Q_1,\dots,Q_p]$0

The important qualification is that the instability statement is specific to the block case. When the block size is $Q=[Q_1,\dots,Q_p]$1, every intraorthogonalization subproblem has $Q=[Q_1,\dots,Q_p]$2, and the recursive bounds simplify. Under $Q=[Q_1,\dots,Q_p]$3, the single-column versions corresponding to DCGS2 and CGS-2 satisfy

$Q=[Q_1,\dots,Q_p]$4

In this sense, DCGS2 and CGS-2 are proved as stable as Householder QR. This directly corrects a common overgeneralisation: one-sync block variants are not practically stable for ill-conditioned inputs, but one-sync single-column variants are an exception (Carson et al., 2024).

4. Randomised, hardware-aware, and non-standard-inner-product variants

Randomised Gram–Schmidt replaces exact inner products by sketched inner products. With sketching matrix $Q=[Q_1,\dots,Q_p]$5, $Q=[Q_1,\dots,Q_p]$6, the sketched inner product is

$Q=[Q_1,\dots,Q_p]$7

and the orthogonalisation coefficients are obtained from a small least-squares problem in sketch space. In one formulation,

$Q=[Q_1,\dots,Q_p]$8

where $Q=[Q_1,\dots,Q_p]$9 and $a_1,\dots,a_k\in\mathbb{R}^\ell$0. The resulting RGS process can be less computationally expensive than classical Gram–Schmidt while being at least as numerically stable as modified Gram–Schmidt; in the paper’s formulation, the computed basis is generally well conditioned rather than exactly orthonormal in Euclidean inner product, and the method integrates naturally into Arnoldi iteration and randomized GMRES (Balabanov et al., 2020).

Randomised block Gram–Schmidt extends the same idea to blocks and inherits the BLAS3 structure of block algorithms. For $a_1,\dots,a_k\in\mathbb{R}^\ell$1, the block coefficients solve

$a_1,\dots,a_k\in\mathbb{R}^\ell$2

with $a_1,\dots,a_k\in\mathbb{R}^\ell$3 and $a_1,\dots,a_k\in\mathbb{R}^\ell$4. A low-synchronization RBGS variant can be implemented in parallel with only one global reduction operation between processors per block, and the same block orthogonalisation is then used inside block Arnoldi, GMRES, FOM, and Rayleigh–Ritz procedures (Balabanov et al., 2021).

The hardware-aware orthogonalization framework reformulates orthogonalisation as a PQR problem and applies CholeskyQR and TSQR as orthogonalisation kernels. In this framework, the orthogonal basis is stored in a locally orthogonal representation, communication costs are reduced, and local and global levels may use different algorithms. Stability is inherited from the chosen subalgorithms: stable local orthogonalisation alone is not sufficient if the reduction algorithm is unstable (Dreier et al., 2022).

Modified Gram–Schmidt also admits a non-standard-inner-product formulation with respect to an SPD matrix $a_1,\dots,a_k\in\mathbb{R}^\ell$5, targeting

$a_1,\dots,a_k\in\mathbb{R}^\ell$6

For the $a_1,\dots,a_k\in\mathbb{R}^\ell$7-inner product $a_1,\dots,a_k\in\mathbb{R}^\ell$8, naive MGS requires $a_1,\dots,a_k\in\mathbb{R}^\ell$9 matrix-vector multiplications with $q_1,\dots,q_k$0. The high-accuracy and high-performance implementations reduce this to $q_1,\dots,q_k$1 matrix-vector multiplications while retaining competitive accuracy; the HA variant is accompanied by explicit error bounds, and the loss of $q_1,\dots,q_k$2-orthogonality is controlled in terms of $q_1,\dots,q_k$3 and $q_1,\dots,q_k$4 (Imakura et al., 2017).

5. Analytical and applied uses beyond numerical QR

In ultra-high-dimensional linear regression, Gram–Schmidt orthogonalisation supports forward variable selection by replacing raw marginal correlations with correlations against orthogonalised candidate predictors. In the GSFR method, for selected set $q_1,\dots,q_k$5, the orthogonal complement is

$q_1,\dots,q_k$6

and the selection score is the sample correlation between the current residual and the orthogonalised candidate: $q_1,\dots,q_k$7 The paper states that GSFR is theoretically equivalent to forward regression except for the stopping rule, achieves a convergence rate comparable to OGA, ensures variable selection consistency under mild conditions, and improves computational efficiency in ultra-high-dimensional settings (Chen et al., 7 Jul 2025).

Gram–Schmidt also appears in approximation theory and signal processing in a different role: not as the transformation to be computed exactly, but as an operation that may itself be approximated by the data matrix. For a full-column-rank matrix $q_1,\dots,q_k$8, with column-normalized version $q_1,\dots,q_k$9 and orthonormalisation $r_{ij}=q_i^Ta_j,\qquad i=1,\dots,j-1,$0, the paper studies

$r_{ij}=q_i^Ta_j,\qquad i=1,\dots,j-1,$1

with upper triangular correction matrices. When $r_{ij}=q_i^Ta_j,\qquad i=1,\dots,j-1,$2 is small, the diagonal correction is quadratic and the off-diagonal correction is essentially the negative Gram-matrix error. This explains why a high-dimensional Gaussian matrix can approximate the corresponding truncated Haar matrix, and the paper identifies applications in compressed subspace clustering and random projections (Li et al., 2016).

In polynomial spaces, Gram–Schmidt applied to monomials under weighted inner products yields the classical orthogonal polynomial systems. With

$r_{ij}=q_i^Ta_j,\qquad i=1,\dots,j-1,$3

the process generates Hermite, Laguerre, and Jacobi polynomials, and the paper gives determinant formulas for the resulting orthogonal vectors using generalized Vandermonde determinants with Gamma and Beta functions. The same work also gives a geometric formulation via the Hodge star operator, interpreting the orthogonal complement construction in exterior algebra (Wang, 2021).

These uses show that Gram–Schmidt is not confined to a single computational role. In some settings it is the primary QR engine; in others it serves as a modelling device for unique contributions, a determinant-based derivation of analytic objects, or a benchmark against which approximate orthogonalisation is assessed.

6. Generalisations outside the standard Euclidean setting

The $r_{ij}=q_i^Ta_j,\qquad i=1,\dots,j-1,$4-adic Gram–Schmidt orthogonalization process replaces inner-product orthogonality by norm-orthogonality over local fields. For a norm $r_{ij}=q_i^Ta_j,\qquad i=1,\dots,j-1,$5 on a finite-dimensional $r_{ij}=q_i^Ta_j,\qquad i=1,\dots,j-1,$6-vector space $r_{ij}=q_i^Ta_j,\qquad i=1,\dots,j-1,$7, orthogonality is defined by the max rule

$r_{ij}=q_i^Ta_j,\qquad i=1,\dots,j-1,$8

for vectors from prescribed one-dimensional subspaces. The constructive content of Weil’s existence theorem is turned into deterministic algorithms: given a basis, one recursively computes an $r_{ij}=q_i^Ta_j,\qquad i=1,\dots,j-1,$9-orthogonal basis, with the key step reduced to a closest vector problem in a $v_j=a_j-\sum_{i=1}^{j-1}q_i r_{ij},$0-adic lattice. The paper also gives a rank-2 lattice orthogonalization algorithm and points to applications in cryptography based on $v_j=a_j-\sum_{i=1}^{j-1}q_i r_{ij},$1-adic lattices (Deng, 2023).

A different generalisation is dimensional lifting. Given an ordered family $v_j=a_j-\sum_{i=1}^{j-1}q_i r_{ij},$2 in $v_j=a_j-\sum_{i=1}^{j-1}q_i r_{ij},$3 or $v_j=a_j-\sum_{i=1}^{j-1}q_i r_{ij},$4, one embeds the vectors into a higher-dimensional space $v_j=a_j-\sum_{i=1}^{j-1}q_i r_{ij},$5 and appends extra coordinates chosen recursively so that the lifted vectors become mutually orthogonal while their projections onto the original coordinate subspace recover the original family. The construction can accommodate linearly dependent vectors, repeated vectors, and zero vectors; it is order-dependent and non-unitary, and the paper motivates it by quantum decision and computing problems (Havlicek et al., 2016).

Taken together, these generalisations indicate that “Gram–Schmidt orthogonalisation” names a structural idea rather than a single algorithm: recursive removal of previously represented components, followed by normalisation or its analogue, can be formulated in Euclidean, block, sketched, weighted, $v_j=a_j-\sum_{i=1}^{j-1}q_i r_{ij},$6-adic, and lifted settings. The specific numerical and theoretical properties depend sharply on that ambient setting, on whether orthogonality is exact or approximate, and on how projection, normalisation, and communication are organised.