Shifted Cholesky QR Factorization

• Shifted Cholesky QR is a stabilized algorithm that adds a positive diagonal shift to the Gram matrix, ensuring a safe Cholesky factorization for ill-conditioned tall-skinny matrices.
• The method employs a three-stage (Shifted CholeskyQR3) approach that extends the condition-number range from O(u^(-1/2)) to O(u^(-1)), achieving near-machine-precision orthogonality.
• Refinements in shift selection, including column-based and probabilistic variants, optimize performance while leveraging BLAS3 operations for communication-efficient, high-performance applications.

Searching arXiv for recent and foundational papers on shifted Cholesky QR. Search query: shifted CholeskyQR tall-skinny QR ill-conditioned matrices Shifted Cholesky QR is a stabilized CholeskyQR-type method for computing the QR factorization of a tall-skinny matrix by adding a positive diagonal shift to the Gram matrix before the initial Cholesky factorization. In its standard form, CholeskyQR computes

$$A = X^TX,\qquad R=\operatorname{chol}(A),\qquad Q=XR^{-1},$$

which is communication-efficient and attractive on modern architectures, but it is numerically unstable for ill-conditioned inputs and may break down when the Gram matrix becomes numerically indefinite. Shifted Cholesky QR modifies the factorization to

$R=\operatorname{chol}(X^TX+sI),$

with a carefully chosen $s>0$, and then typically applies two further CholeskyQR passes. This three-stage scheme, commonly called Shifted CholeskyQR3, is designed to extend the usable condition-number range from roughly $\mathbf{u}^{-1/2}$ for CholeskyQR2 to $\mathcal{O}(\mathbf{u}^{-1})$, while retaining the BLAS3-heavy, communication-avoiding structure that makes Cholesky-based QR attractive in high-performance and abstract-vector settings (Fukaya et al., 2018, Higgins et al., 2023).

1. Definition and algebraic setting

Shifted Cholesky QR belongs to the class of QR methods derived from Gram-matrix factorizations. In the Euclidean case, the fundamental identity is

$$A = QR \text{ (thin QR)} \quad \Longleftrightarrow \quad R^T R = A^T A,$$

so $R$ is a Cholesky factor of $A^TA$, and $Q=AR^{-1}$ follows formally from the triangular solve (Benner et al., 2020). This algebraic equivalence is the basis of classical CholeskyQR, and Shifted Cholesky QR is a direct stabilization of this construction.

The standard shifted algorithm is defined by the steps

1. form the Gram matrix

$A = X^TX,$

1. choose a positive shift $R=\operatorname{chol}(X^TX+sI),$0,
2. compute

$R=\operatorname{chol}(X^TX+sI),$1

1. form

$R=\operatorname{chol}(X^TX+sI),$2

This differs from standard CholeskyQR only in the shifted Cholesky step (Fukaya et al., 2018).

The role of the shift is narrowly technical but decisive. Plain CholeskyQR can fail if $R=\operatorname{chol}(X^TX+sI),$3 is too close to singular in floating-point arithmetic. The shift makes the matrix more safely positive definite, so that the Cholesky factorization completes numerically (Higgins et al., 2023). In the original stabilized construction, the first shifted pass is not intended to deliver a final orthogonal factor by itself; rather, it acts as a preconditioning stage that reduces the condition number of the computed factor so that the subsequent CholeskyQR2 refinement is safe (Fukaya et al., 2018).

A common misconception is that the shift is merely an ad hoc regularization. The analyses in the literature instead treat it as a mechanism for guaranteeing successful factorization and controlling the condition number of the intermediate factor $R=\operatorname{chol}(X^TX+sI),$4, which then determines whether the later CholeskyQR stages can recover near-machine-precision orthogonality (Fukaya et al., 2018, Fan et al., 2024).

2. Instability of plain CholeskyQR and the rationale for shifting

The numerical difficulty of CholeskyQR arises from the fact that forming the Gram matrix squares the condition number: $R=\operatorname{chol}(X^TX+sI),$5 As a consequence, the roundoff sensitivity of classical CholeskyQR grows rapidly with ill-conditioning, and the Cholesky factorization of $R=\operatorname{chol}(X^TX+sI),$6 may break down when the matrix becomes numerically indefinite (Fan et al., 2021).

CholeskyQR2 improves orthogonality by repeating the factorization, and the general stabilization principle behind repeated Cholesky-based QR is explicitly stated as follows:

Computing $R=\operatorname{chol}(X^TX+sI),$7 is unstable, but the stability can be drastically improved by doing a second repetition, i.e. compute the QR decomposition of $R=\operatorname{chol}(X^TX+sI),$8 (Benner et al., 2020).

However, repetition alone does not remove the need for the first Cholesky factorization to succeed. This is the limitation that Shifted Cholesky QR addresses. The 2018 analysis states that CholeskyQR2 is still limited by the requirement that the Cholesky factorization of the Gram matrix runs to completion, which means it does not always work for matrices $R=\operatorname{chol}(X^TX+sI),$9 with

$s>0$0

The shift extends the applicability to

$s>0$1

by ensuring that

$s>0$2

succeeds numerically (Fukaya et al., 2018).

The mechanism is expressed through a backward-error model. In the shifted analysis,

$s>0$3

The shift is chosen so that it dominates the rounding errors in both Gram formation and Cholesky factorization. Under the stated assumptions,

$s>0$4

which implies

$s>0$5

This is the core argument showing that the shift enforces a safe triangular factor (Fukaya et al., 2018).

This suggests that the shift should be viewed as a controlled perturbation whose size must be balanced. If it is too small, breakdown remains possible; if it is too large, the first-stage factor $s>0$6 becomes less favorable for the subsequent CholeskyQR refinements. Later work makes this tradeoff explicit: too small a shift risks failure, while too large a shift worsens $s>0$7 and can reduce the effectiveness of Shifted CholeskyQR3 (Fan et al., 2024).

3. Shifted CholeskyQR3 and its stability theory

The stabilized method analyzed in the foundational work is effectively a three-pass algorithm. After the first shifted pass,

$s>0$8

two ordinary CholeskyQR refinements are applied: $s>0$9

$\mathbf{u}^{-1/2}$0

This is the procedure commonly referred to as Shifted CholeskyQR3 (Fukaya et al., 2018, Fan et al., 2024, Guan et al., 2024).

The original paper recommends a safe shift of the order

$\mathbf{u}^{-1/2}$1

for the standard inner-product case, with the assumptions

$\mathbf{u}^{-1/2}$2

together with

$\mathbf{u}^{-1/2}$3

Under these conditions, the first shifted step reduces the condition number of the intermediate factor, and the analysis proves that the resulting $\mathbf{u}^{-1/2}$4 satisfies a bound of the form

$\mathbf{u}^{-1/2}$5

which is precisely the regime where CholeskyQR2 can be safely applied (Fukaya et al., 2018).

The paper’s main threshold in the standard inner-product case is

$\mathbf{u}^{-1/2}$6

and the final three-stage factorization then satisfies the stated accuracy guarantees

$\mathbf{u}^{-1/2}$7

$\mathbf{u}^{-1/2}$8

The paper therefore characterizes the method as backward stable up to modest polynomial factors in $\mathbf{u}^{-1/2}$9 and mild dependence on $\mathcal{O}(\mathbf{u}^{-1})$0 (Fukaya et al., 2018).

A concise comparison with related deterministic methods appears repeatedly in later work. CholeskyQR is fast but unstable; CholeskyQR2 improves orthogonality but still fails when the first Cholesky step breaks down; Shifted CholeskyQR3 adds a shift to make the first factorization robust and thereby extends the usable condition-number range to about $\mathcal{O}(\mathbf{u}^{-1})$1 rather than $\mathcal{O}(\mathbf{u}^{-1})$2 (Fukaya et al., 2018, Fan et al., 2021, Higgins et al., 2023).

4. Shift selection: spectral, column-based, probabilistic, and sparse-aware variants

The central design variable in Shifted Cholesky QR is the shifted item $\mathcal{O}(\mathbf{u}^{-1})$3. Later work has focused less on the three-stage structure itself than on deriving less conservative and more structure-aware choices of $\mathcal{O}(\mathbf{u}^{-1})$4.

A first refinement replaces the global spectral norm $\mathcal{O}(\mathbf{u}^{-1})$5 by a column-based quantity

$\mathcal{O}(\mathbf{u}^{-1})$6

which satisfies

$\mathcal{O}(\mathbf{u}^{-1})$7

Using this quantity, the improved algorithm chooses

$\mathcal{O}(\mathbf{u}^{-1})$8

instead of the traditional $\mathcal{O}(\mathbf{u}^{-1})$9 (Fan et al., 2024). The paper introduces

$$A = QR \text{ (thin QR)} \quad \Longleftrightarrow \quad R^T R = A^T A,$$0

so that

$$A = QR \text{ (thin QR)} \quad \Longleftrightarrow \quad R^T R = A^T A,$$1

Because $$A = QR \text{ (thin QR)} \quad \Longleftrightarrow \quad R^T R = A^T A,$$2, the new sufficient condition

$$A = QR \text{ (thin QR)} \quad \Longleftrightarrow \quad R^T R = A^T A,$$3

is always at least as good as the original requirement

$$A = QR \text{ (thin QR)} \quad \Longleftrightarrow \quad R^T R = A^T A,$$4

The same paper proves for the full improved three-stage method that

$$A = QR \text{ (thin QR)} \quad \Longleftrightarrow \quad R^T R = A^T A,$$5

$$A = QR \text{ (thin QR)} \quad \Longleftrightarrow \quad R^T R = A^T A,$$6

This suggests that the main benefit of $$A = QR \text{ (thin QR)} \quad \Longleftrightarrow \quad R^T R = A^T A,$$7 is not a change in algorithmic structure but a sharper estimate of how much stabilization the first pass actually needs (Fan et al., 2024).

A second refinement is probabilistic. Under a randomized rounding-error model, the 2024 probabilistic analysis recommends the shift

$$A = QR \text{ (thin QR)} \quad \Longleftrightarrow \quad R^T R = A^T A,$$8

where

$$A = QR \text{ (thin QR)} \quad \Longleftrightarrow \quad R^T R = A^T A,$$9

This replaces both $R$0 by $R$1 and the size factor $R$2 by $R$3, yielding a smaller shifted item while maintaining numerical stability with high probability (Guan et al., 2024). For Shifted CholeskyQR3, the paper states that if

$R$4

then

$R$5

The same work reports that for $R$6, the randomized shift worked robustly up to $R$7, while the older column-based deterministic shift failed at that level (Guan et al., 2024).

A third line of development specializes the shift to sparse matrices. For sparse $R$8 matrices, with $R$9 dense columns, dense-column sparsity $A^TA$0, sparse-column sparsity $A^TA$1, and largest-magnitude entry

$A^TA$2

the proposed sparse-aware shift is

$A^TA$3

For $A^TA$4 matrices, where all columns are sparse,

$A^TA$5

Under the paper’s element-norm conditions, these shifts are shown to be equivalent to smaller normalized shifts than the generic dense-matrix choices, and the paper concludes that they improve applicability while maintaining numerical stability (Guan et al., 2024).

These developments are best understood as refinements of a single principle: a shift must be large enough to prevent failure in the initial Cholesky factorization, but no larger than needed, because the shifted first pass serves as a preconditioner for the later orthogonalization passes (Fukaya et al., 2018, Fan et al., 2024, Guan et al., 2024, Guan et al., 2024).

5. Relations to repeated, generalized, randomized, and updated Cholesky-based QR

Shifted Cholesky QR is one member of a broader family of stabilized Gram-matrix factorizations. The general pattern emphasized in the literature is: compute a Cholesky-like factorization of a Gram-type matrix, form an approximate orthogonal factor through a triangular solve, and then repeat or otherwise precondition the procedure to recover orthogonality (Benner et al., 2020).

In generalized inner-product settings, the same idea appears beyond the Euclidean QR decomposition. For hyperbolic QR, the analogue of the Cholesky relation is

$A^TA$6

which leads to a scaled $A^TA$7 factorization. For the symplectic case, the corresponding object is a skew-symmetric Cholesky-like factorization. In both settings, the cited work reports that repeated Cholesky-like factorizations can provide better numerical stability than elimination-based approaches, especially for ill-conditioned matrices (Benner et al., 2020). Although that paper does not discuss Shifted Cholesky QR by name, it supports the same stabilization principle: first-pass Cholesky-like factorization may be unstable, but a second pass can repair loss of orthogonality and improve residuals (Benner et al., 2020).

Randomized methods have also been proposed as alternatives to explicit shifting. The paper “Analysis of Randomized Householder-Cholesky QR Factorization with Multisketching” compares CholeskyQR2 and shifted CholeskyQR3 with a randomized method $A^TA$8. It treats shifted CholeskyQR3 as a stable state-of-the-art algorithm for tall-and-skinny matrices, but notes that it has more than $A^TA$9 larger computational and communication cost than CholeskyQR2 (Higgins et al., 2023). The randomized method first computes

$Q=AR^{-1}$0

then applies

$Q=AR^{-1}$1

Its goal is to replace the restrictive condition-number assumptions of classical Cholesky-based QR by first producing a much better-conditioned intermediate factor (Higgins et al., 2023).

Another alternative is randomized XR-based preconditioning. In that framework,

$Q=AR^{-1}$2

where $Q=AR^{-1}$3 is extracted from a sampled submatrix. The key structural identity is

$Q=AR^{-1}$4

which permits a random-matrix analysis of how well the preconditioned matrix behaves. This line of work explicitly presents shifted CholeskyQR3 as the robust but comparatively expensive deterministic baseline (Fan et al., 2021).

More recently, applications in model order reduction have motivated update-friendly and adaptive shifted variants. In pyMOR-oriented work, shifted Cholesky QR is attractive because it is compatible with vector-wise abstractions, naturally updatable, and communication-avoiding relative to Gram–Schmidt (Bindhak et al., 10 Jul 2025). That paper proposes iterated Cholesky QR with shift recomputation, denoted $Q=AR^{-1}$5, in which one repeatedly forms the current Gramian $Q=AR^{-1}$6, attempts a Cholesky factorization, and if needed recomputes

$Q=AR^{-1}$7

It also develops a Cholesky QR update scheme for incrementally appended vectors and a panelized method that applies the update over blocks of columns (Bindhak et al., 10 Jul 2025).

A plausible implication is that the role of Shifted Cholesky QR has broadened from a single deterministic stabilization trick into a reference design pattern for communication-avoiding, refinement-based orthogonalization.

6. Performance characteristics, applications, and limitations

The practical appeal of Shifted Cholesky QR is tied to hardware efficiency. CholeskyQR-type methods are dominated by BLAS-3 operations, are communication-friendly, and are particularly attractive for tall-skinny matrices (Fukaya et al., 2018, Higgins et al., 2023). The 2018 paper reports that shiftedCholeskyQR3 is highly parallelizable and, in experiments, achieves significant speedups over alternative methods, including up to $Q=AR^{-1}$8 speedup in a sparse oblique-inner-product setting (Fukaya et al., 2018).

In direct comparisons against CholeskyQR2, the cost of the extra shifted pass is significant. One recent analysis states that shifted CholeskyQR3 has more than $Q=AR^{-1}$9 larger computational and communication cost than CholeskyQR2 (Higgins et al., 2023). GPU experiments in the same paper report average slowdowns relative to CholeskyQR2 of $A = X^TX,$0 for shifted CholeskyQR3, while multisketch $A = X^TX,$1 is only about $A = X^TX,$2 slower on average and can even be faster in some cases (Higgins et al., 2023). These comparisons are specific to that study, but they explain why shifted CholeskyQR3 is often used as a robustness baseline rather than an unquestioned default.

Application-specific work in model order reduction emphasizes a different tradeoff. In pyMOR-style abstract vector settings, Householder QR is often impractical because it requires elementwise access to entries, while Gram–Schmidt is communication-heavy despite being update-friendly (Bindhak et al., 10 Jul 2025). Shifted Cholesky QR therefore occupies a middle ground: more compatible with abstract vector-array interfaces than Householder QR, more communication-efficient than Gram–Schmidt, and more stable than classical CholeskyQR (Bindhak et al., 10 Jul 2025).

Despite its robustness, Shifted Cholesky QR is not rank revealing. This limitation is especially relevant in model order reduction, where linear dependence among candidate basis vectors may be informative rather than pathological. The pyMOR-oriented paper notes that modified Gram–Schmidt has built-in rank-revealing behavior, whereas the presented Cholesky QR methods do not inherently have that feature, and suggests that pivoted or rank-revealing Cholesky QR would be a promising extension (Bindhak et al., 10 Jul 2025).

Another limitation is that the classical deterministic theory can be conservative. Both the column-based and probabilistic shift papers are motivated by the observation that the usual deterministic bounds often overestimate the rounding-error magnitude and therefore prescribe a shift that is unnecessarily large (Fan et al., 2024, Guan et al., 2024). The resulting recent literature does not reject the shifted framework; instead, it refines it by making the shift data dependent, column aware, probabilistic, or sparsity aware.

In summary, Shifted Cholesky QR is best understood as a stabilized, communication-avoiding QR methodology for ill-conditioned tall-skinny matrices. Its original contribution was to replace the fragile first Cholesky step of CholeskyQR2 by a safely shifted one, thereby extending the admissible condition-number range to $A = X^TX,$3 with orthogonality and residual of order $A = X^TX,$4 (Fukaya et al., 2018). Subsequent work has focused on reducing the shift, sharpening sufficient conditions, exploiting column structure and sparsity, incorporating probabilistic error models, and adapting the method to iterative and update-driven workflows such as model order reduction (Fan et al., 2024, Guan et al., 2024, Guan et al., 2024, Bindhak et al., 10 Jul 2025).