---
title: Generalized Golub-Kahan Bidiagonalization
url: https://www.emergentmind.com/topics/generalized-golub-kahan-bidiagonalization-gkb
type: topic
---

# Generalized Golub-Kahan Bidiagonalization

Generalized Golub–Kahan bidiagonalization (GKB) denotes a family of Krylov subspace processes that extend classical Golub–Kahan bidiagonalization beyond the Euclidean, single-matrix least-squares setting. Across the recent literature, these extensions are used to handle weighted inner products, prior- and noise-weighted inverse problems, generalized singular value decomposition (GSVD) computation, saddle point systems, and tensor equations. A unifying feature is the construction of paired basis sequences and a small projected bidiagonal or Hessenberg problem, so that large-scale structure is accessed through matrix-vector products, weighted normalizations, and low-dimensional regularized subproblems rather than explicit factorization of the full operator [1607.03943].

## 1. Operator viewpoint and defining structure

In its most abstract formulation, generalized GKB replaces the Euclidean geometry of standard bidiagonalization by weighted or Hilbert-space inner products. A representative setting introduces a positive semidefinite matrix \(M\) on the domain and a matrix \(G\) on the data space, so that \(A\) is viewed as an operator
\[
A : (R(M), \langle \cdot,\cdot\rangle_M) \to (\mathbb{R}^m,\langle \cdot,\cdot\rangle_G),
\]
with adjoint
\[
A^*u = M^\dagger A^T G u.
\]
The resulting generalized GKB recursion is
\[
\beta_1 u_1 = b,\qquad
\alpha_i v_i = A^*u_i - \beta_i v_{i-1},\qquad
\beta_{i+1}u_{i+1} = Av_i - \alpha_i u_i,
\]
with \(\|u_i\|_G=\|v_i\|_M=1\) [2404.00655].

This operator formulation is broad enough to cover several distinct algorithmic traditions. In Bayesian inverse problems, \(M\) is induced by the prior covariance and \(G\) by the noise covariance. In GSVD computation, \(M=A^TA+L^TL\) induces the geometry required for matrix pairs \(\{A,L\}\). In saddle point systems, \(M\) and \(N\) encode blockwise energy norms. A plausible implication is that “generalized GKB” is best understood not as a single fixed algorithm, but as a structural template: alternating primal and dual recurrences under non-Euclidean orthogonality constraints.

A concise taxonomy used in the recent literature is as follows.

| Variant | Core modification | Representative use |
|---|---|---|
| Weighted/Hilbert-space gGKB | Replace Euclidean adjoint and orthogonality by \(M\)- and \(G\)-inner products | GSVD components, weighted least squares |
| gen-GK for Bayesian inversion | Use \(Q\)- and \(R^{-1}\)-orthogonality | Prior-conditioned inverse problems |
| Generalized GKB for saddle points | Couple block solves and weighted recurrences | CRAIG/nsCRAIG-type solvers |
| Tensor GKB variants | Replace matrix multiplication by Einstein product or t-product | Image, video, multilinear inverse problems |

## 2. Core bidiagonalization recurrences and projected problems

A central concrete instance appears in Bayesian inverse problems of the form
\[
d = A x + \epsilon,\qquad \epsilon \sim \mathcal{N}(0,R),\qquad x \sim \mathcal{N}(\mu,\lambda^{-2}Q).
\]
To avoid explicit computation of \(Q^{-1}\) or \(Q^{1/2}\), the variable transformation
\[
z = Q^{-1}(x-\mu),\qquad x=\mu+Qz
\]
converts the MAP problem into
\[
\min_z \ \frac12\|AQz-b\|_{R^{-1}}^2 + \frac{\lambda^2}{2}\|z\|_2^2,
\]
with \(b=d-A\mu\). The generalized GKB process then generates
\[
\beta_1 u_1=b,\qquad \alpha_1 v_1=(AQ)^T R^{-1}u_1,
\]
\[
\beta_{k+1}u_{k+1}=AQv_k-\alpha_k u_k,\qquad
\alpha_{k+1}v_{k+1}=(AQ)^T R^{-1}u_{k+1}-\beta_{k+1}v_k,
\]
together with the orthogonality relations
\[
U^TR^{-1}U=I,\qquad V^TQV=I.
\]
The associated right Krylov space is
\[
K_k\!\left((AQ)^TR^{-1}AQ,\ (AQ)^TR^{-1}b\right),
\]
and each iteration requires only matrix-vector products with \(A\), \(A^T\), \(Q\), and solves with \(R\) [1607.03943].

The algorithmic purpose of these recurrences is projection. At step \(k\), one seeks \(z_k \approx V_k y_k\), where \(y_k\) solves the reduced problem
\[
\min_{y_k}\ \frac12\|B_k y_k-\beta_1 e_1\|_2^2+\frac{\lambda^2}{2}\|y_k\|_2^2.
\]
The full-space approximation is then
\[
x_k=\mu+QV_k y_k.
\]
This projected formulation explains why generalized GKB is frequently paired with hybrid regularization: the large-scale operator is compressed into a small bidiagonal model \(B_k\), while regularization is imposed only after projection.

The same projected-least-squares pattern persists in tensor formulations. In the Einstein-product setting, the global tensor GKB process produces a bidiagonal matrix \(C_\ell\) and a reduced Tikhonov problem in \(\mathbb{R}^\ell\), while in the t-product setting the weighted tensor GKB process yields a reduced tensor system with small bidiagonal tensor \(\bar P_k\) [2005.07458; 2109.03912].

## 3. Regularization, hybridization, and inverse-problem interpretation

The strongest concentration of generalized GKB applications is in large-scale inverse problems, where the method is used not only as a Krylov solver but as a regularization mechanism. In the generalized hybrid framework, the projected problem supports adaptive parameter choice by criteria such as GCV and the discrepancy principle, and the approach is reported to inherit the usual advantages of hybrid methods, including avoidance of semi-convergence and automatic estimation of the regularization parameter [1607.03943].

This role becomes clearer when the method is compared with standard LSQR. For the transformed Bayesian problem, gen-LSQR iterates are equivalent to LSQR iterates applied to the prior-conditioned Tikhonov problem
\[
\min_w \ \frac12\|AQ^{1/2}w-d\|_{R^{-1}}^2+\frac{\lambda^2}{2}\|w\|^2,
\]
but the generalized GKB formulation achieves this without forming \(Q^{1/2}\) [1607.03943]. This equivalence is operationally significant in settings where \(Q\) is dense, defined on irregular grids, or only accessible through matrix-vector multiplication, including Matérn-class covariances.

The same projection principle underlies later decomposition methods for mixed-prior inverse problems. In the sdHybr/FGGK framework, generalized GKB is combined with flexible Krylov updates so that a solution \(s=s_1+s_2\) with smooth and sparse components can be approximated by
\[
s_1=\mu_1+QV_k f_k,\qquad s_2=\mu_2+W_k f_k,
\]
and the projected problem incorporates both \(\lambda\) and \(\alpha\):
\[
f_k=\arg\min_f \|M_k f-m_{1,1}e_1\|_2^2+\lambda^2\|f\|_2^2+\alpha^2\|R_{W,k}f\|_2^2.
\]
Within that reduced model, UPRE, DP, and WGCV are used for adaptive parameter selection [2206.06664].

The regularizing effect of GKB-based methods has also been analyzed from the residual side. For LSQR, LSMR, and CRAIG, residuals can be written explicitly in terms of bidiagonalization vectors, and the coefficients in those linear combinations quantify propagated noise. For CRAIG, the residual is a multiple of a single bidiagonalization vector, which makes its norm directly interpretable as a regularization indicator [1612.05551]. Complementary LSQR theory for discrete ill-posed problems with multiple singular values shows that, for severely and moderately ill-posed problems, LSQR attains 2-norm filtering best possible regularized solutions at semi-convergence, and that this behavior is governed by the low-rank approximations and Ritz values generated by Golub–Kahan bidiagonalization [2003.09259].

## 4. GSVD, filtered expansions, and generalized spectral approximation

A second major line of development treats generalized GKB as a computational vehicle for GSVD. For a matrix pair \(\{A,L\}\), the 2024 GSVD study characterizes the GSVD through singular value expansions of two induced operators on finite-dimensional Hilbert spaces and then derives a generalized GKB process tailored to compute nontrivial extreme GSVD components [2404.00655].

In this setting, with
\[
M=A^TA+L^TL,
\]
gGKB generates \(M\)-orthonormal right basis vectors and \(G\)-orthonormal left basis vectors, produces a bidiagonal \(B_k\), and then extracts approximate GSVD triplets from the SVD
\[
B_k = Y_k \Theta_k H_k^T.
\]
The approximations are
\[
(\bar c_i^{(k)},\bar p_{A,i}^{(k)},\bar x_i^{(k)})
=
(\theta_i^{(k)},\,U_{k+1}y_i^{(k)},\,V_k h_i^{(k)}),
\qquad
\bar s_i^{(k)}=\sqrt{1-(\bar c_i^{(k)})^2}.
\]
The paper gives residual-based quality certificates and a computable upper bound involving \(\alpha_{k+1}\beta_{k+1}|e_k^T h_i^{(k)}|\) [2404.00655].

The inverse-problem literature had already connected generalized GKB to GSVD filtering. For the Bayesian MAP estimator, one can write a filtered GSVD expansion
\[
x_\lambda
=
\mu
+
\sum_{j=1}^n
\hat\phi_j
\frac{\bar u_j^T R^{-1} b}{\hat\sigma_j}
\bar v_j,
\qquad
\hat\phi_j=\frac{\hat\sigma_j^2}{\hat\sigma_j^2+\lambda^2},
\]
and the corresponding gen-LSQR iterates act as truncated or polynomial-filtered GSVD approximants [1607.03943].

This spectral perspective clarifies an important point: generalized GKB is not merely a weighted least-squares solver. It is also a structure-preserving reduction mechanism for generalized spectral objects. That includes GSVD components of \(\{A,L\}\), generalized matrix functions of rectangular matrices, and rational-Krylov generalizations in which the projected matrices become quasiseparable rather than bidiagonal [1512.01446; 2107.12074]. In the latter case, the literature describes short recurrences that recover classical GKB when all poles are at infinity, which suggests a continuous transition from polynomial to rational generalized bidiagonalization [2107.12074].

## 5. Saddle point systems and generalized block formulations

Generalized GKB has also become a notable solver framework for saddle point systems. For systems of the form
\[
\begin{bmatrix}
W & A\\
A^T & 0
\end{bmatrix}
\begin{pmatrix}
u\\ p
\end{pmatrix}
=
\begin{pmatrix}
b\\ 0
\end{pmatrix},
\]
the Craig variant of generalized GKB operates in inner products induced by \(W\) and \(N\), and, after an augmented Lagrangian modification
\[
M = W + \eta A N^{-1}A^T,
\]
can exhibit convergence that depends only weakly on model size in structural mechanics applications [1808.07677]. Parallel studies implemented in PETSc show that the resulting inner–outer GKB strategy scales effectively when the inner solves with \(M\) are handled by direct or multigrid-accelerated iterative methods [2001.10424].

Inexactness in the inner solves does not invalidate the method, but it changes the accuracy model. The inner–outer GKB analysis shows that later inner solves may be relaxed because their perturbations are filtered by the decay of the GKB update coefficients \(\zeta_k\), and dynamic relaxation strategies can reduce the total number of inner iterations substantially [2208.01079].

For nonsymmetric positive definite leading blocks, the classical symmetric equivalence with CG breaks down. The nonsymmetric extension constructs a generalized GKB process in which the right basis requires full orthogonalization and the reduced operator becomes upper Hessenberg. In this regime, the method is theoretically equivalent to applying FOM to the Schur complement equation rather than CG, and practical stopping criteria can be based on residual recurrences and energy-norm estimates for the primal variable [2310.06952].

The 2025 extension to generalized saddle point systems with nonzero \((2,2)\) block
\[
\begin{bmatrix}
M & A\\
A^T & -C
\end{bmatrix}
\]
pushes this line further. It develops CRAIG and nsCRAIG analogues for \(C\neq 0\), proves equivalence with Schur-complement reduction combined with inner CG or FOM, and derives residual- and energy-based stopping criteria [2509.02936]. One recurring theme across these block formulations is that generalized GKB offers low memory requirements relative to unrestarted GMRES or full-system Krylov methods, because the bidiagonalization exploits the block structure directly.

## 6. Tensor, multilinear, and application-driven generalizations

A distinct branch of the literature generalizes GKB from matrices to tensors. In the Einstein-product formulation, tensor equations
\[
\mathcal{A}\ast_N \mathcal{X} = \mathcal{C}
\]
are treated by a global Golub–Kahan bidiagonalization process with updates
\[
\widetilde{\mathcal V}=\mathcal A^T\ast_N \mathcal U_j-\sigma_j\mathcal V_{j-1},
\qquad
\rho_j=\|\widetilde{\mathcal V}\|_F,
\]
\[
\widetilde{\mathcal U}=\mathcal A\ast_N \mathcal V_j-\rho_j\mathcal U_j,
\qquad
\sigma_{j+1}=\|\widetilde{\mathcal U}\|_F,
\]
followed by a reduced tensor Tikhonov problem. This construction is aimed at multilinear systems and discrete ill-posed problems, with applications to color image and video restoration [2005.07458].

In the t-product setting, weighted tensor GKB methods introduce SPD tensors \(\mathcal M\) and \(\mathcal L\) to represent colored noise and regularization. The large-scale tensor problem
\[
\min_{\mathcal X}
\left\{
\|\mathcal A * \mathcal X - \mathcal B\|_{\mathcal M^{-1}}^2
+
\mu^{-1}\|\mathcal X\|_{\mathcal L^{-1}}^2
\right\}
\]
is reduced by a weighted bidiagonalization process to a small projected problem, and the discrepancy principle is used to determine both \(\mu\) and the number of bidiagonalization steps [2109.03912]. A related tensor GKB line based on general tensor operators and Tikhonov regularization formulates reduced problems in the small bidiagonal tensor basis and applies discrepancy-principle parameter choice with quadrature support [1907.08811].

Other application-driven generalizations extend the GKB idea rather than only its ambient algebra. In inverse Cauchy problems, a CCBM-GKB method combines a coupled complex boundary reformulation with an infinite-dimensional GKB process and Givens rotations, yielding a discrepancy-principle regularization method faster than classic Landweber in the reported experiments [2204.03416]. In X-ray CT with unmatched forward and back projectors, AB- and BA-GKB are proposed as preconditioned forms of GKB; these preserve a bidiagonal projected structure and are reported to be more resilient to semiconvergence than GMRES-type alternatives in the tested problems [2606.31153]. In image deblurring with Kronecker-product approximations, an enlarged-GKB strategy builds a projected space beyond the desired rank and then truncates, providing low-rank approximations for reordered blur matrices that compete favorably with randomized SVD in the reported tests [2410.00233].

Taken together, these developments show that generalized GKB has become a methodological bridge between classical bidiagonalization and a wider class of weighted, block-structured, generalized-spectral, and tensor-structured problems. The literature does not present a single terminal form of the method; rather, it presents a common bidiagonalization paradigm adapted to the geometry, algebra, and regularization requirements of each problem class.

Source: https://www.emergentmind.com/topics/generalized-golub-kahan-bidiagonalization-gkb