--- title: Inertial Krasnoselskii-Mann Method url: https://www.emergentmind.com/topics/inertial-krasnoselskii-mann-ikm type: topic --- # Inertial Krasnoselskii-Mann Method The Inertial Krasnoselskii-Mann (IKM) method generalizes the classical Krasnoselskii-Mann (KM) fixed-point iteration by incorporating a momentum (inertia) term in addition to relaxation. This two-step approach has become a central construct in modern optimization and monotone operator theory, enabling the acceleration of fixed-point solving and monotone splitting algorithms for nonexpansive and quasi-nonexpansive operators on Hilbert spaces. IKM’s convergence theory includes weak, strong, and linear convergence in distinct regimes, robustifies iteration under computational or problem-driven perturbations, and underpins state-of-the-art splitting schemes in imaging, optimization, and large-scale variational problems. ## 1. Formal Description of the Inertial Krasnoselskii-Mann Iteration The IKM iteration generalizes the classical KM update by adding an explicit inertial extrapolation: - **Hilbert space setting:** let $H$ be real Hilbert space, $T : H \to H$ a (quasi-)nonexpansive operator, with ${\rm Fix}(T)\neq\emptyset$. - **Parameters:** Extrapolation (inertia) $(\beta_k)_{k\ge1} \subset [0,1)$ and relaxation $(\alpha_k)_{k\ge1}\subset (0,1)$. - **Initialization:** $x_0, x_1 \in H$. The basic IKM step is, for $k \ge 1$, \[ \begin{aligned} y_k &= x_k + \beta_k (x_k - x_{k-1}) \\ x_{k+1} &= x_k + \alpha_k ( T(y_k) - x_k ) \end{aligned} \] Alternatively: \[ x_{k+1} = (1-\alpha_k)x_k + \alpha_k T( x_k + \beta_k (x_k - x_{k-1}) ) \] This framework covers both time-step and parameter-variant designs, coordinate/parallel extensions, adaptive and random updates, and variants with computational errors or anchoring terms [2210.03791][1403.3330][1908.11029][1704.03563][1604.04845]. ## 2. Convergence Theory and Parameter Conditions The convergence of the IKM method hinges on operator regularity and parameter selection. Two principal settings are quasi-nonexpansive and quasi-contractive maps. ### 2.1. Weak Convergence for Quasi-Nonexpansive Operators - If $T_k$ are quasi-nonexpansive ($\|T_k y - p\| \le \|y-p\|$ for all $p\in F = \bigcap_k \mathrm{Fix}(T_k)$) and parameters satisfy growth and compatibility restrictions (e.g., nondecreasing $\beta_k\in[0,1)$, infimum of $\alpha_k$ bounded away from zero, control on inertia and error terms): - $\sum_{k}\|x_{k+1} - 2x_k + x_{k-1}\|^2 < +\infty,\ \sum_k \| x_k - x_{k-1}\|^2 <+\infty$ - $\sum_k \|T_k(y_k) - y_k\|^2 < +\infty$ - $x_k \rightharpoonup x^*\in F$ - Nonasymptotic residual rate: $\min_{1\leq i \leq n} \|T_i(y_i) - y_i\|^2 \le \mathcal{O}(1/n)$, with $o(1/\sqrt{n})$ for fixed-point residuals [2210.03791][1908.11029][1704.03563]. ### 2.2. Strong and Linear Convergence - For quasi-contractive $T$ ($\|T y - p^*\| \leq q\|y-p^*\|$, $0 < q < 1$) and appropriate $(\alpha_k, \beta_k)$ sequences, IKM achieves: - Strong convergence $x_k \to p^*$ if $\sum_k (1-q_k^2)=\infty$ - Linear convergence rate: $\|x_k - p^*\|^2 = \mathcal{O}(Q^k)$ for an explicit $Q < 1$ - This matches or improves the classical non-inertial regime under the same parameter window [2210.03791]. ### 2.3. Robustness and Perturbation Resilience - Allowance of summable error sequences $e_k$ in the iterative step (the inexact IKM) preserves weak convergence, provided $\sum_k \lambda_k \|e_k\| < \infty$, and inertia is controlled in sum-of-squares or monotonicity [1908.11029]. ### 2.4. Strong Convergence with Anchoring (Halpern/Vicosity Enhancements) - Embedding an external anchor (Halpern iteration) or a contraction (viscosity term) enables strong convergence to the metric projection or the unique viscosity solution, at the expense of step size decays and auxiliary conditions [2001.07346][2411.18574]. ## 3. Algorithmic Variants and Notable Designs IKM serves as the modular core for many acceleration and splitting schemes: | Variant / Paper | Update / Momentum | Notable Features / Context | |:-------------------------------------- |:------------------------------------------------|:-----------------------------------| | Classical IKM [2210.03791][1403.3330] | $y_k$, $\beta_k\in[0,1)$, single operator | Weak/strong/linear convergence | | Stochastic IKM [1604.04845] | Coord.-wise random $T$, inertia | a.s. convergence, block updates | | Chebyshev-inertial [2001.03280] | Periodic $\alpha_k$ (roots of Chebyshev poly) | Locally optimal linear rate | | Adaptive-momentum [2510.24544] | Data-driven $\theta_k$ from $T$, $T^2$ geometry | $o(1/\sqrt{k})$ successively | | Fast KM (Nesterov-style) [2206.09462] | Decaying $\beta_k=k/(k+\alpha)$, relaxed step | $o(1/k)$ rate, weak convergence | | Generalized fast w/ precond. [2411.18574] | Preconditioning, ODE-origin, 2-parameter $(\alpha,\eta)$, anchor | Unified continuous/discrete rates, degenerate metrics | Each variant modulates the momentum step, relaxation/anchoring, selection (full, random, or adaptive), and application of errors or preconditioning, according to problem structure and performance tradeoffs. Theoretical analysis often proceeds via Lyapunov/Energy functionals and generalized Fejér monotonicity. ## 4. Applications and Numerical Performance IKM schemes are widely applied to monotone inclusion and convex optimization: - **Operator splitting:** Inertial Douglas-Rachford, three-operator splitting, primal-dual splitting, often yielding improved iteration/CPU performance over classical schemes [1403.3330][1904.11684][2210.03791]. - **Signal/image processing:** Total-variation denoising, inpainting, and matrix completion, with 12–60% reductions in practical runtime and iteration counts when inertia is employed [2210.03791][2510.24544][1904.11684]. - **Large-scale optimization:** Stochastic coordinate-descent variants accommodate high-dimensional settings, preserving (almost sure) convergence [1604.04845]. - **Optimization geometric problems:** Beckmann optimal transport, geometric medians, clustering—accelerated variants provide $o(k^{-1})$ to $o(k^{-1/2})$ convergence for primal-dual and splitting schemes [2411.18574][1403.3330]. ## 5. Parameter Selection and Practical Guidelines - **Inertial weight $\beta_k$:** Typically $\beta_k\in[0,1)$, sometimes scheduled to increase slowly or tuned adaptively; excessive inertia can degrade worst-case residual bounds but may accelerate practical convergence; best selected with geometric insight or variance control [2210.03791][2510.24544]. - **Relaxation $\alpha_k$:** Fixed or diminishing; in acceleration settings (e.g., Fast KM) $\alpha>2$ and decaying stepsizes deliver the $o(1/k)$ rate [2206.09462][2411.18574]. - **Anchoring/halpern parameter:** $\nu_k\to 0$ with $\sum\nu_k=\infty$ ensures the entire sequence is forced toward the solution as in strong-convergence variants [2001.07346][2411.18574]. - **Perturbations:** Summability or $L^2$-control of errors $e_k$ is sufficient for weak convergence; precise worst-case $O(1/k)$ rates available under further parameter decays [1908.11029]. - **Preconditioning:** Strong or degenerate $M\succcurlyeq 0$ possible, and differences measured in corresponding seminorms; critical in splitting and metric-geometry problems [2411.18574]. ## 6. Broader Impact, Limitations, and Generalizations IKM generalizes standard projection, relaxation, and momentum methods in a unifying two-step framework. Its design unifies Polyak/Heavy-ball, Nesterov acceleration, and Halpern/anchoring into a parameter-rich algorithmic family. While inertia can increase the practical speed of convergence, it may cause oscillatory behavior or slow the worst-case theoretical rates; achieving optimal acceleration remains context-dependent, and parameter selection often requires problem-specific tuning or a posteriori adaptation. Open questions include deriving optimal global step schedules and developing theories for nonmonotone/inexact and nonconvex regimes [2210.03791][2510.24544][2411.18574]. The IKM paradigm is foundational in modern operator theory and optimization, with variations underpinning the fastest first-order methods for large-scale and structured problems across signal processing, computational imaging, and data science. --- **References** - [2210.03791] Inertial Krasnoselskii-Mann Iterations - [1403.3330] Inertial Douglas-Rachford splitting for monotone inclusion problems - [1904.11684] An inertial three-operator splitting algorithm with applications to image inpainting - [2206.09462] Fast Krasnosel'skii-Mann algorithm with a convergence rate of the fixed point iteration of $o\left(\frac{1}{k}\right)$ - [2001.03280] Chebyshev Inertial Iteration for Accelerating Fixed-Point Iterations - [2411.18574] Generalized Fast Krasnoselskii-Mann Method with Preconditioners - [1908.11029] Convergence analysis of an inexact inertial Krasnoselskii-Mann algorithm with applications - [2001.07346] Strong convergence of modified inertial Mann algorithms for nonexpansive mappings - [1704.03563] Quasinonexpansive Iterations on the Affine Hull of Orbits: From Mann's Mean Value Algorithm to Inertial Methods - [1604.04845] A stochastic coordinate descent inertial primal-dual algorithm for large-scale composite optimization - [2510.24544] A Two-step Krasnosel'skii-Mann Algorithm with Adaptive Momentum and Its Applications to Image Denoising and Matrix Completion Source: https://www.emergentmind.com/topics/inertial-krasnoselskii-mann-ikm