Convergence Analysis of An Alternating Nonlinear GMRES on Linear Systems

Abstract: In this work, we develop an alternating nonlinear Generalized Minimum Residual (NGMRES) algorithm with depth $m$ and periodicity $p$, denoted by aNGMRES($m,p$), applied to linear systems. We provide a theoretical analysis to quantify by how much aNGMRES($m$) can improve the convergence speed of the underlying fixed-point iteration for diagonalizable and symmetric positive definite cases. Our theoretical analysis gives us a better understanding of which factors affect the convergence speed. Moreover, under certain conditions, we prove the periodic equivalence between aNGMRES applied to Richardson iteration and GMRES. Specifically, aNGMRES($\infty,p$) and full GMRES are identical at the iteration index $jp$. aNGMRES($\infty,p$) can be regarded as an alternative to GMRES for solving linear systems. For finite $m$, the iterates of aNGMRES($m,m+1$) and restarted GMRES (GMRES($m+1$)) are the same at the end of each periodic interval of length $p$, i.e, at the iteration index $jp$. The advantages of aNGMRES($m,p$) method are that there is no need to solve a least-squares problem at each iteration which can reduce the computational cost, and it can enhance the robustness against stagnations, which could occur for NGMRES($m$).