Inertial Projection Algorithm

• Inertial Projection Algorithm is an iterative method combining classical projection techniques with momentum-based extrapolation to rapidly solve variational inequalities.
• It employs an inertial term inspired by Polyak heavy-ball and Nesterov strategies, using previous iterates to forecast better search directions before projection.
• The method guarantees weak convergence with an explicit sublinear rate (O(1/n)) and is extendable to include operator splitting, monotone inclusions, and fixed-point problems.

An inertial projection algorithm is a class of iterative methods in nonlinear analysis and optimization that leverages inertial (momentum-based) extrapolation to accelerate the projection-guided approach to fixed points, variational inequalities, or monotone inclusions. The core idea is to supplement classical projection algorithms—where each step consists of a metric projection onto a relevant convex set—with an additional inertial term modeled after the Polyak heavy-ball, Alvarez–Attouch, or Nesterov acceleration strategies. This inertial term uses information from previous iterates to extrapolate new directions before projection, leading to practical improvements in convergence speed without compromising theoretical guarantees when parameters are appropriately selected.

1. Algorithmic Framework and Core Iterative Structure

The archetypal inertial projection algorithm is designed for monotone variational inequalities in a real Hilbert space $H$, over a nonempty closed affine or convex set $C \subset H$, and a monotone, uniformly continuous operator $F:C \to H$: $$\text{Find } x^*\in C \text{ such that } \langle F(x^*), y-x^*\rangle \geq 0 \quad \forall y\in C$$ The algorithm initiates with $x_0 = x_1 \in H$ and proceeds recursively:

1. Inertial Extrapolation: $w_n = x_n + \alpha_n (x_n - x_{n-1})$ with nondecreasing $\{\alpha_n\} \subset (0,1)$, $\alpha_n \leq \alpha < 1$.
2. Projection Step: Compute $z_n = P_C(w_n - F(w_n))$, where $P_C$ is the metric projection onto $C \subset H$0.
3. Residual Computation: $C \subset H$1.
4. Armijo-type Linesearch: Find the minimal integer $C \subset H$2 so that

$C \subset H$3

with backtracking factor $C \subset H$4 and threshold $C \subset H$5.

1. Half-Space Construction: Set $C \subset H$6, $C \subset H$7, and define the half-space

$C \subset H$8

2. Final Projection: $C \subset H$9.

This scheme uniquely combines momentum (via $F:C \to H$0), projection-based contraction, and a problem-adaptive linesearch for robust step selection (Shehu et al., 2021).

2. Parameter Selection and Convergence Guarantees

Rigorous convergence analysis necessitates constraints on extrapolation and backtracking parameters:

• The sequence $F:C \to H$1 must be nondecreasing, bounded above by $F:C \to H$2.
• Parameters $F:C \to H$3 must satisfy:

$F:C \to H$4

• The backtracking factor $F:C \to H$5.

Under these settings, two principal convergence results are established:

• Weak convergence: The sequence $F:C \to H$6 converges weakly to a solution $F:C \to H$7.
• Explicit sublinear rate: For all $F:C \to H$8,

$F:C \to H$9

for an explicit constant $$\text{Find } x^*\in C \text{ such that } \langle F(x^*), y-x^*\rangle \geq 0 \quad \forall y\in C$$0 dependent on all algorithmic parameters, yielding $$\text{Find } x^*\in C \text{ such that } \langle F(x^*), y-x^*\rangle \geq 0 \quad \forall y\in C$$1 decay of the gap measure $$\text{Find } x^*\in C \text{ such that } \langle F(x^*), y-x^*\rangle \geq 0 \quad \forall y\in C$$2.

The proofs rely on forming Fejér-type Lyapunov sequences and telescoping inequalities, leveraging the monotonicity and continuity of $$\text{Find } x^*\in C \text{ such that } \langle F(x^*), y-x^*\rangle \geq 0 \quad \forall y\in C$$3 and firm nonexpansiveness of $$\text{Find } x^*\in C \text{ such that } \langle F(x^*), y-x^*\rangle \geq 0 \quad \forall y\in C$$4 (Shehu et al., 2021).

3. Accelerative Role of the Inertial Extrapolation

The inertial term $$\text{Find } x^*\in C \text{ such that } \langle F(x^*), y-x^*\rangle \geq 0 \quad \forall y\in C$$5 fundamentally impacts both theoretical and empirical convergence:

• Acceleration: Empirical studies in large-scale linear variational inequalities demonstrate dramatic reductions in iteration count and CPU time when $$\text{Find } x^*\in C \text{ such that } \langle F(x^*), y-x^*\rangle \geq 0 \quad \forall y\in C$$6, relative to the base method ($$\text{Find } x^*\in C \text{ such that } \langle F(x^*), y-x^*\rangle \geq 0 \quad \forall y\in C$$7).
• Preservation of monotonicity: Theoretical analysis shows Fejér monotonicity and global convergence are preserved provided $$\text{Find } x^*\in C \text{ such that } \langle F(x^*), y-x^*\rangle \geq 0 \quad \forall y\in C$$8 and descent conditions hold for other parameters.
• Momentum analogy: Inertia functions as a Polyak-heavy-ball or Nesterov-like step, exploiting past trajectory directions for fast progress.

This balancing of acceleration and stability is a hallmark of the inertial paradigm and is confirmed across multiple variants and problem instances (Shehu et al., 2021).

4. Algorithmic Pseudocode and Computational Workflow

A concise description of the classical inertial projection method is as follows:

$x_0 = x_1 \in H$1 This procedure leverages one projection onto $$\text{Find } x^*\in C \text{ such that } \langle F(x^*), y-x^*\rangle \geq 0 \quad \forall y\in C$$9 and one onto a half-space per iteration (Shehu et al., 2021).

5. Variants and Extensions in Recent Research

The inertial projection motif has been extended to several settings:

• Monotone inclusions and operator splitting: Inertial hybrid projection and inertial shrinking projection algorithms for monotone inclusions, often employing further correction steps and additional half-space or hybrid projections for strong convergence (Tan et al., 2020).
• Self-adaptive inertial schemes: Adaptive step-size strategies eliminate the need for explicit operator norm estimations, adjusting inertia and projections according to local geometry and residuals to further boost performance (Zhou et al., 2020).
• Inexact and relaxed algorithms: Allowing inexact projections and relaxed updates (i.e., overrelaxation) produces strong or weak convergence, with nonasymptotic rates given explicit parameter ranges (Alves et al., 2020, Cui et al., 2019).
• Nonexpansive mappings and fixed-point problems: Inertial projection methods are deployed for finding fixed points, with extensions to viscosity methods and multivalued mappings (ALTIPARMAK et al., 2021, Fang et al., 2020).

A broad spectrum of theoretical and applied results demonstrates the effectiveness, flexibility, and robustness of inertial projection paradigms across monotone optimization, equilibrium, inclusion, and fixed-point problems.

6. Empirical Performance and Implementation Considerations

Numerical experiments consistently evidence significant accelerations using inertia in projection algorithms. For linear variational inequalities and split inclusion problems, inertial variants achieve:

• Reduced iteration counts, often by factors of two or more.
• Substantial CPU time improvements.
• High-precision attainment in fewer iterations.
• Acceleration persists across a range of problem scales, from small matrices to high-dimensional VIs.

Parameter selection remains crucial: excessive inertia ($x_0 = x_1 \in H$0 too close to 1) can compromise stability, while moderate to dynamically tuned inertia effectively enhances performance (Shehu et al., 2021).

7. Significance, Directions, and Outlook

The inertial projection algorithm represents an overview of classical projection methods and modern acceleration techniques—achieving nonasymptotic rates under minimal monotonicity and continuity assumptions. Ongoing research explores adaptive parameter scheduling, block and coordinate descent variants, handling inexact or proximal subsolves, and extending inertia to asynchronous and distributed frameworks. The methodology is foundational in convex optimization, monotone operator theory, variational analysis, and computational fixed-point search, and remains a domain of active and fruitful mathematical investigation.