GMRES Convergence Bound Analysis

• GMRES Convergence Bound is a metric that quantifies the asymptotic reduction in residual norm for GMRES(1) iterations in solving linear systems.
• The analysis employs nonlinear eigenvalue problems to derive explicit convergence factors, such as the sharp bound for symmetric matrices using spectrum properties.
• Key implications include guiding restart policies and performance predictions in iterative solvers, ensuring efficiency even under worst-case conditions.

The GMRES convergence bound formalizes the rate at which the Generalized Minimal Residual method reduces the residual norm in iterative solutions of linear systems with non-symmetric or nonnormal coefficient matrices. Analytical convergence factors are central to understanding GMRES efficiency in worst-case scenarios, particularly for fixed restart GMRES(m), with $m=1$ (GMRES(1)), and for symmetric or skew-symmetric problems.

1. Foundational Concepts and Definitions

Let $A\in\mathbb{R}^{n\times n}$ be invertible, and consider the sequence $\{r_k\}$ of GMRES(1) residuals generated by the recurrence: $$r_{k+1}= (I-\alpha_kA)\,r_k, \qquad \alpha_k=\frac{r_k^T A^T r_k}{r_k^T A^T A r_k}$$ The asymptotic convergence behavior is quantified as follows:

• Root-convergence factor for the residual sequence:

$$\varrho_k(r_0) = \|r_k\|^{1/k}, \quad \varrho(r_0) = \limsup_{k\to\infty}\varrho_k(r_0), \quad \varrho^* = \max_{r_0} \varrho(r_0)$$

• $q$-linear convergence: $\{r_k\}$ converges $q$-linearly with factor $\sigma\in(0,1)$ if $\|r_k\|\le\sigma\,\|r_{k-1}\|$ for all $k\geq 1$.

The "worst-case root-convergence factor" $\varrho^*$, a key metric, corresponds to the slowest asymptotic contraction induced by the method, maximized over all possible starting residuals.

2. Nonlinear Eigenvalue Problems Governing GMRES(1)

GMRES(1) is governed by a sequence of vector-dependent iterations: $$r_{k+1} = \mathcal{I}(r_k)\,r_k, \qquad \mathcal{I}(v):=I-\alpha(v)A,\qquad \alpha(v)=\frac{\langle v,Av\rangle}{\langle Av,Av\rangle}$$ The analysis of convergence factors is reduced to a nonlinear eigenvalue problem: $$\mathcal{I}_2(u)u = \lambda u, \qquad \mathcal{I}_2(u)=(I-\alpha(u)A^T)(I-\alpha(u)A)$$ The nonzero eigenvalues $\lambda$ of $\mathcal{I}_2$ control the worst-case one-step contraction $$\max_{v\neq 0} \|\mathcal{I}(v)v\|/\|v\| = \max_\lambda \sqrt{|\lambda|}$$. For higher-step contraction analysis (e.g., two-step), a similar machinery involving sequentially composed, vector-dependent operators $\Pi(v)$ is required, with their spectral properties characterizing the attainable asymptotic rates.

3. Explicit Convergence Factor for Symmetric $A$

When $A = A^T$ is real symmetric with eigenvalues $a_1\le \dots \le a_n$, the spectrum of the nonlinear operator reduces to two-eigenvalue interactions: $$\lambda_{ij} = \frac{(a_j - a_i)^2}{(a_j + a_i)^2}, \qquad i\neq j$$ Thus, the maximal possible contraction is

$$\Gamma^* = \max_{i\neq j} \left| \frac{a_j - a_i}{a_j + a_i} \right|$$

This yields the explicit, sharp worst-case bound: $$\varrho^*_{GMRES(1)} = \Gamma^* = \frac{\lambda_{\max}(A) - \lambda_{\min}(A)}{\lambda_{\max}(A) + \lambda_{\min}(A)}$$ The convergence is strictly contractive for positive-definite $A$ ($\Gamma^*<1$), but for indefinite $A$ (i.e., if $a_ib_j<0$ for some $i,j$), $\Gamma^*=1$ and stagnation in residual norm is possible for some initial data, i.e., worst-case no progress.

4. Asymptotic Bound for Skew-Symmetric $(I-A)$

If $M = I-A$ is real skew-symmetric ($M^T=-M$), its eigenvalues are purely imaginary $\pm i m_j$. The contraction is governed by the spectral radius $m^*=\rho(M)=\max_j|m_j|$, yielding

$$\varrho^*_{GMRES(1),ss}= \frac{m^*}{\sqrt{1+(m^*)^2}} < 1$$

This is a uniform and unconditional bound: irrespective of the initial residual, $\|r_k\| \leq (m^*/\sqrt{1+(m^*)^2})^k \|r_0\|$.

5. Relation between $q$-Linear and Root-Convergence Factors

By construction of the nonlinear eigenvalue problems, the maximal $q$-linear contraction coincides with the worst-case root-convergence factor: $$\varrho^* = \limsup_{k\to\infty}\|r_k\|^{1/k} = \Gamma^* \;\; \text{or} \;\; \frac{m^*}{\sqrt{1+(m^*)^2}}$$ Thus, in both the symmetric and skew-symmetric cases, the $q$-linear rate precisely matches the worst-case root factor, and there is no gap between them.

6. Parameter Dependence, Assumptions, and Limiting Cases

• $A$ is assumed invertible ($\det A\ne 0$).
• For the symmetric case: $A$ is real symmetric, and the entire spectrum enters the formula for $\Gamma^*$.
• For the skew-symmetric analysis: $M=I-A$ is skew-symmetric, so $A$ is normal with $\Re\lambda(A)=1$.
• The initial residual $r_0$ is arbitrary (nonzero); the bound is worst-case over all $r_0$.
• If $A$ is positive-definite, contraction is strict and independent of initial condition; if $A$ is indefinite, worst-case stagnation ($\varrho^*=1$) is possible.

7. Numerical Illustrations and Tightness of the Bound

Examples confirm the theoretical predictions:

• For $A=\operatorname{diag}(1,2,3)$, $\varrho^* = (3-1)/(3+1)=0.5$; numerically, $\|r_k\|^{1/k}\to 0.5$ for arbitrary $r_0$.
• For $A=\operatorname{diag}(-1,2,3,4)$ (indefinite), $\varrho^*=1$; numerical simulations produce stagnation for some $r_0$ with asymptotic ratio approaching unity.
• For the basic skew case $M=\begin{bmatrix}0 & 1\-1 & 0\end{bmatrix}$ ($m^*=1$), $\varrho^*_{ss}=1/\sqrt{2}\approx0.7071$ matches the empirical rate $\|r_k\|^{1/k}$ observed in computation.

8. Broader Context and Significance

The closed-form, spectrum-dependent GMRES(1) worst-case convergence bound provides a complete characterization of asymptotic contraction rates for symmetric and skew-symmetric linear systems. The results demonstrate that the spectrum alone, through the explicit expressions for $\Gamma^*$ and $m^*$, suffices to tightly predict the ultimate convergence rate achievable by GMRES(1), including pathological stagnation for indefinite systems. These findings directly inform restart policies, performance predictions, and algorithm selection for practical linear solvers. They also elucidate the precise thresholds for contractivity loss in minimal residual iterations and provide a foundation for further extensions to nonnormal and structure-dependent matrix classes (He, 17 Jan 2025).