Hybrid CG-Tikhonov as a filtration of the CG Lanczos vectors

Abstract: We consider iterative methods for solving linear ill-posed problems with compact operator and right-hand side only available via noise-polluted measurements. Conjugate gradients (CG) applied to the normal equations with an appropriate stopping rule and CG applied to the system solving for a Tikhonov-regularized solution (CGT) $(A^\ast A + c I_{\mathcal{X}}) x^{(\delta,c)} = A^\ast y^\delta$ are closely related methods as build iterates from the same family of Krylov subspaces. In this work, we show that the CGT iterate can be expressed as $x^{(\delta,c)}_m = \sum_{i=1}^{m} \gamma^{(m)}_i(c) z_i^{(m)}v_i$, where $\left\lbrace \gamma_i^{(m)}(c)\right\rbrace_{i=1}^m$ are functions of the Tikhonov parameter and $x_m = \sum_{i=1}^{m} z_i^{(m)}v_i$ is the $m$-th CG iterate. We call these functions Lanczos filters, and they can be shown to have decay properties as $c\rightarrow\infty$ with the speed of decay increasing with $i$. This has the effect of filtering out the contribution of the later terms of the CG iterate. We demonstrate with numerical experiments that good parameter choices correspond to appropriate damping of the Lanczos vectors corresponding to larger amplifications of the measurement noise. We conclude by noting that analysis of other hybrid regularization schemes via damping of (Krylov) subspace basis vectors from the iteration itself may be a useful avenue for understanding the behavior of these methods for different choice of parameter.