Inertial Extrapolation Term

• Inertial Extrapolation Term is a momentum-like correction that leverages past iterates to accelerate computational optimization and variational methods.
• It employs various schemes—constant, vanishing, and adaptive inertia—to improve convergence rates in both convex and non-convex settings.
• Precise parameter tuning is crucial to balance accelerated performance with stability, notably in high-dimensional and complex system applications.

An inertial extrapolation term is a generic mathematical construct used to accelerate iterative algorithms by incorporating a momentum-like correction based on past iterates. In the context of optimization and variational problems, such a term typically takes the form of a weighted difference of recent iterates, and is central to contemporary developments in first-order and operator-splitting methods. The theoretical and algorithmic role of inertial extrapolation is to impart a dynamic analogous to physical inertia or momentum, thereby enhancing convergence speeds and, when properly controlled, preserving or improving convergence guarantees.

1. Mathematical Structure and Variants

At its core, an inertial extrapolation term augments an iterative sequence $\{x_n\}$ using previous displacements. In its archetypal form, this amounts to generating an extrapolated point

$y_n = x_n + \alpha_n (x_n - x_{n-1}),$

where $\alpha_n$ is a (possibly time-varying) inertia parameter. This term, often called the Polyak–heavy-ball or momentum component, is a primary feature in classic algorithms such as inertial proximal methods or accelerated gradient flows (László, 2023, Shehu et al., 2018).

More elaborate variants include double-inertial or multi-term extrapolation schemes, as in

$$y_n = x_n + \theta(x_n - x_{n-1}) + \delta(x_{n-1} - x_{n-2}),$$

with $\theta, \delta$ as independent parameters, yielding greater flexibility and damping capacity in splitting and monotone inclusion algorithms (Iyiola et al., 1 Oct 2024, Wang et al., 2022).

In block-coordinate and majorization-minimization settings, inertial terms can be defined for each block, with possibly parameterized, operator-valued weights (Hien et al., 2020, Hien et al., 2019). In the continuous-time regime, the inertial effect arises through a "second derivative" (acceleration) or via the evaluation point of a dual variable at an inertially extrapolated primal coordinate, as in the controlled primal-dual ODE systems (Zhu et al., 13 Jun 2024).

2. Algorithmic Roles and Motivations

The primary motivation behind incorporating inertial extrapolation is to accelerate the traversal of low-curvature regions in the optimization landscape, leveraging the directionality of previous iterates to "push" the current update further along a productive direction. In convex and monotone regimes, carefully tuned inertia can yield provable acceleration—sometimes up to the optimal $O(1/k^2)$ rate in smooth convex minimization (Long et al., 21 May 2025, László, 2023). In non-convex settings, it can promote escape from shallow or spurious stationary points (Mukkamala et al., 2019).

Multiple works demonstrate that inertial extrapolation terms reduce iteration count and, by extension, wall-clock time—typically 20–40% lower compared to non-inertial methods for comparable precision (Nwakpa et al., 23 Nov 2025, Shehu et al., 2018). The effect is particularly pronounced in high-dimensional or highly structured problems, such as nonnegative matrix factorizations, monotone inclusions, and imaging inverse problems (Hien et al., 2019, Wang et al., 2022). In some advanced algorithms, distinct parameters are used for the "momentum" and for the evaluation point of the linearization or gradient, further increasing adaptivity and theoretical leeway (Hien et al., 2019, Hien et al., 2020).

3. Parameterization and Control Mechanisms

The convergence properties and efficacy of inertial methods are highly sensitive to the choice and scheduling of inertia parameters. Common strategies include:

• Constant Inertia: Fixed $\alpha\in[0,1)$, under constraints established by Lyapunov or energy-based descent analysis to ensure global convergence (Nwakpa et al., 23 Nov 2025).
• Vanishing Inertia: Time- or iteration-varying weights such as $\alpha_k = 1 - \gamma k^{-q}$, with $0<q\leq1$ and $\gamma>0$, to guarantee fast function decay and—if coupled with vanishing regularization or Tikhonov terms—strong convergence to the minimum-norm solution (László, 2023).
• Adaptive Inertia: Terms of the type $$\gamma_k = \mu_k \frac{(m^k)^\top \nabla f(x^k)}{\|m^k\|^2} \beta_k$$ in AIM (Long et al., 21 May 2025), where the direction $m^k$ and scale $\mu_k$ are computed adaptively to ensure descent properties and optimal convergence rates, including quasi-Newton and regularized-Newton regimes.
• Double and Multi-Step Inertia: Incorporation of secondary lag terms (e.g., $\delta_n$) to dampen over-acceleration, proving essential for robust acceleration in splitting methods where one-step inertia can fail or even induce divergence (Iyiola et al., 1 Oct 2024).

Parameter selection must balance aggressive acceleration with stability—excess inertia can cause non-monotonicity or oscillation, while too little inertia forfeits speedup. Several methods employ convex-concave backtracking to adapt both the step size and the inertial parameter at each iteration, thus maintaining global convergence in the non-convex regime (Mukkamala et al., 2019).

4. Theoretical Convergence and Regime Analysis

The inclusion of an inertial extrapolation term enriches the Lyapunov or potential function employed in convergence analysis. Convergence regimes are frequently demarcated by critical thresholds on the inertia parameter and its interplay with regularization, damping, or splitting coefficients.

For instance, in inertial proximal algorithms with Tikhonov regularization, strong convergence to the minimum-norm minimizer is achieved when the regularization parameter decays at a rate slower than inertia, i.e., $p<q+1$ with $C_k = c k^{-p}$ and $\alpha_k = 1-\gamma k^{-q}$, whereas only weak convergence is guaranteed for $p>q+1$ (László, 2023). In the continuous-time primal-dual framework, convergence rates and modes (fast versus moderate regimes) are explicitly determined by the interaction of inertia, damping, regularization, and coupling exponents (Zhu et al., 13 Jun 2024).

The role of inertia in attaining acceleration is confirmed in a variety of contexts, including non-summable yet bounded inertial weights for monotone inclusions, and in multi-step inertial variants that overcome pathologies observed with single-step acceleration (Iyiola et al., 1 Oct 2024, Wang et al., 2022).

5. Implementation Strategies and Applications

Discretization of inertial dynamical systems typically uses forward–Euler-like schemes, translating continuous-time inertia (e.g., via extrapolated gradient or dual evaluation points) into explicit difference-based terms. Parameter schedules are time-varying, with practitioners absorbing step size scalings or employing batch backtracking or linesearch techniques (Zhu et al., 13 Jun 2024, Long et al., 21 May 2025).

Algorithmic templates with inertial extrapolation are widely deployed across:

• Operator splitting for monotone inclusions;
• Block-coordinate minimization in non-convex, non-smooth settings;
• Proximal and projection methods in variational inequalities;
• Accelerated primal–dual flows for constrained optimization;
• Large-scale matrix and tensor factorization, sparse recovery, and inverse problems (Shehu et al., 2018, Hien et al., 2020, Hien et al., 2019, Nwakpa et al., 23 Nov 2025).

Empirical results consistently support reduced iteration complexity and lower computation time. Two-step or adaptively damped inertial variants are preferred in splitting methods, while quasi-Newton-type inertia (as in AIM) achieves near-optimal rates in convex optimization without Hessian inversions (Long et al., 21 May 2025).

6. Inertial Terms in Dynamical Systems and Physics

Outside optimization, the term "inertial term" also denotes fictitious or indirect forces required when equations of motion are formulated in non-inertial (e.g., star-centric) frames in N-body or fluid dynamics (Crida et al., 29 Jun 2025). In this context, an "indirect" inertial term compensates for the acceleration of the reference frame center (e.g., a central massive object), restoring Newtonian dynamics for the remaining bodies. The inclusion or omission of such terms must be closely synchronized with direct interactions to avoid spurious results, and there exist precise recipes for their computation and incorporation in modeling planetary or disc-planet interactions.

7. Summary Table: Common Inertial Extrapolation Schemes

Scheme/Class	Extrapolation Term	Parameter Constraints
Heavy-ball (Polyak)	$y^k = x^k + \gamma(x^k-x^{k-1})$	$0 \leq \gamma < 1$
Nesterov/Accelerated Gradient	$y^k = x^k + \alpha_k(x^k-x^{k-1})$	$\alpha_k \to 1$ suitably
Block Proximal (single)	$\hat x_i = x^k_i + \alpha(x^k_i-y_i)$	Bounded $\alpha$
Block Proximal (double)	$\hat x_i, \grave x_i$ with $\alpha,\gamma$	Decoupled, constraints via decrease lemma
Multi-step inertial splitting	$$y_n = x_n + \theta(x_n-x_{n-1}) + \delta(x_{n-1}-x_{n-2})$$	$0 \leq \theta < 1/2$, $\delta < 0$ (w/ bounds)

Parametric regimes and further technical conditions are dictated by the underlying algorithmic structure and problem class (Wang et al., 2022, Iyiola et al., 1 Oct 2024, László, 2023).

---

• "Strong asymptotic convergence of a slowly damped inertial primal-dual dynamical system controlled by a Tikhonov regularization term" (Zhu et al., 13 Jun 2024)
• "Proximal and Contraction method with Relaxed Inertial and Correction Terms for Solving Mixed Variational Inequality Problems" (Nwakpa et al., 23 Nov 2025)
• "Tseng Splitting Method with Double Inertial Steps for Solving Monotone Inclusion Problems" (Wang et al., 2022)
• "Three-Operator Splitting Method with Two-Step Inertial Extrapolation" (Iyiola et al., 1 Oct 2024)
• "On the convergence of an inertial proximal algorithm with a Tikhonov regularization term" (László, 2023)
• "Adaptive Inertial Method" (Long et al., 21 May 2025)
• "Inertial Block Proximal Methods for Non-Convex Non-Smooth Optimization" (Hien et al., 2019)
• "An Inertial Block Majorization Minimization Framework for Nonsmooth Nonconvex Optimization" (Hien et al., 2020)
• "Convex-Concave Backtracking for Inertial Bregman Proximal Gradient Algorithms in Non-Convex Optimization" (Mukkamala et al., 2019)
• "An inertial extrapolation method for convex simple bilevel optimization" (Shehu et al., 2018)
• For indirect inertial (fictitious) terms in dynamical systems: "On inertial forces (indirect terms) in problems with a central body" (Crida et al., 29 Jun 2025)