---
title: Inertial Tseng Extragradient Method
url: https://www.emergentmind.com/topics/inertial-tseng-extragradient-method
type: topic
---

# Inertial Tseng Extragradient Method

The inertial Tseng extragradient method is a class of first-order splitting algorithms for monotone inclusion, variational inequality, or equilibrium problems that incorporates inertial (momentum-type) extrapolation into the classical Tseng forward–backward–forward (extragradient) scheme. These methods, typically implemented in real Hilbert spaces, can accommodate both single-valued and set-valued maximally monotone operators, pseudomonotone and quasimonotone maps, and may be adapted to composite, multi-valued, and structured problems. Inertial Tseng methods are motivated by the desire to accelerate convergence and improve practical performance—especially in large-scale and ill-conditioned problems—while preserving rigorous theoretical guarantees such as weak/strong convergence and nonasymptotic complexity rates. This entry presents the foundational problem setups, algorithmic principles, convergence results, parameter selection, and practical applications, as documented in recent literature.

## 1. Foundational Problem Structures

Inertial Tseng extragradient methods address structured monotone inclusion problems of the form:

$$
0 \in A x^* + B x^*,
$$

where $A: H \to H$ is single-valued, monotone, and typically $L$-Lipschitz continuous; $B: H \to 2^H$ is maximally monotone, possibly set-valued. The solution set is denoted as $\Omega = (A+B)^{-1}(0)$, assumed nonempty. This generalizes variational inequalities, fixed-point problems, and equilibrium problems, with extensions covering pseudomonotone or quasimonotone operators, nonconvex objectives, and (multi-)valued mappings [2209.11989][1406.0724][2007.11761][2011.12825][2611.18642].

For variational inequalities, the setting is:
$$
\mathrm{find}\;\; x^* \in C \;\; \text{such that} \;\; \left\langle F(x^*),\, y - x^* \right\rangle \geq 0,\;\; \forall y \in C,
$$
with $C$ closed and convex, and $F$ monotone, pseudomonotone, or quasimonotone.

In equilibrium problems, one works with bifunctions $f: C \times C \rightarrow \mathbb{R}$ such that $f(z,z)=0$, seeking $v^* \in C$ with $f(v^*,z) \geq 0$ for all $z \in C$, again under generalized monotonicity and (sub)continuity assumptions [2511.18642][2011.12825].

## 2. Algorithmic Principles and Core Iterative Schemes

A prototypical inertial Tseng method at iteration $n$ extrapolates from previous iterates to gain momentum:

1. **Inertial Extrapolation** (possibly multi-step):
   - $w_n = x_n + \alpha_n (x_n - x_{n-1})$ (single-step inertia)
   - $w_n = x_n + \alpha_n(x_n - x_{n-1}) + \beta_n(x_{n-1} - x_{n-2})$ (double-step/triple-step inertia) [2209.11989][2601.10370]

2. **(Forward) Backward (Tseng) Step:**
   - $y_n = (I + \lambda_n B)^{-1}(w_n - \lambda_n A w_n)$

3. **Tseng Extragradient Correction:**
   - $x_{n+1} = y_n - \lambda_n (A y_n - A w_n)$

4. **Relaxation/averaged update and possible correction:**
   - $x_{n+1} = (1-\theta_n)z_n + \theta_n(y_n - \lambda_n(A y_n - A w_n))$ for a secondary inertia $z_n$ [2209.11989]
   - More generally, under-relaxation or averaging with other correction mappings.

Parameter selection follows explicit bounds to maintain stability, e.g., $0 \leq \alpha_n < 1$, step-sizes $\lambda_n$ via self-adaptive or Armijo-type rules, and relaxation parameters $\theta_n$ within prescribed intervals [1406.0724][2209.11989][1812.02138][2601.10370].

In set-valued and equilibrium extensions, the forward-backward-forward architecture persists, but explicit subproblems (proximal or variational substeps) and halfspace projections are employed [2511.18642][2011.12825].

## 3. Convergence Properties and Complexity Rates

The key theoretical advances provided by inertial Tseng methodologies include:

- **Weak Convergence:** Under standard monotonicity and Lipschitz assumptions, the sequence $(x_n)$ (possibly along with auxiliary inertial iterates) converges weakly to a solution, typically established via Lyapunov-type Fejér monotonicity and the Opial lemma [2209.11989][1407.0214][1812.02138][1406.0724].

- **Strong Convergence:** Imposing additional strong monotonicity on $A$ or $B$ (or strong pseudomonotonicity in equilibrium/VI settings), one obtains norm convergence to the unique solution [2209.11989][2007.02746][2007.11761][2511.18642][2011.12825].

- **Linear Convergence:** When either $A$ or $B$ is strongly monotone (or the bifunction is strongly pseudomonotone), and the parameters satisfy further contraction conditions, $x_n$ converges Q-linearly to the solution [2209.11989][2511.18642].

- **Nonasymptotic Rates:** For general monotone problems, a $O(1/\sqrt{n})$ pointwise rate for the operator residual is established; $O(1/n)$ ergodic rates hold for averages. For strongly monotone cases, expectation rates of $O(1/n)$ in stochastic variants, and even $O(1/N)$ for saddle-point gaps, are proved [2209.11989][2202.09759][1812.02138].

- **Nonconvex Settings:** If the global objective or Lyapunov function satisfies the Kurdyka–Łojasiewicz (KL) property, strong convergence to limiting critical points holds even in absence of convexity [1406.0724].

- **Quasimonotone and Multi-valued VI/EQ:** The two-step inertial Tseng with a self-adaptive and Armijo-like step-size eliminates the need for a global Lipschitz bound on $F$, and achieves weak convergence for (quasi-)monotone VIs, extending applicability [2601.10370][2011.12825].

## 4. Parameter Selection, Step-Size Adaptivity, and Stabilization

Careful selection of inertia coefficients and step-size is critical:

| Parameter      | Condition/Update                                            | Effect/Role                                         |
|----------------|------------------------------------------------------------|-----------------------------------------------------|
| $\alpha_n$     | $0 \leq \alpha_n < \alpha < 1$; explicit upper bounds      | Controls magnitude of inertial extrapolation         |
| $\lambda_n$    | Self-adaptive: $\min\{\frac{\mu \|w_n-y_n\|}{\|A w_n-A y_n\|}, \lambda_{n-1}\}$, or Armijo/backtracking | Ensures robustness without global Lipschitz         |
| $\theta_n$     | $0 < \theta \leq \theta_n \leq 1/(1+\varepsilon)$          | Adjusts relaxation/averaging, stability/acceleration|
| $(\mu_n,p_n)$  | Decay: $\mu_n\to0, \sum p_n<\infty$                        | Stability in self-adaptive rules                    |

For stochastic and inexact settings, step-sizes and inertia are further regulated by error summability and martingale conditions to maintain almost-sure and expected convergence [2202.09759][1812.02138].

Double- or multi-step inertia (inclusion of $\beta_n$ terms) can empirically accelerate convergence at the cost of tighter stability control and parameter tuning [2209.11989][2601.10370].

## 5. Structural and Practical Algorithmic Enhancements

Inertial Tseng-type schemes can be adapted and generalized in multiple directions:

- **One-projection Schemes:** Several recent algorithms reduce projection complexity by requiring only a single projection per iteration (as opposed to two in classical Korpelevich schemes), lowering per-iteration cost [2007.02746][2006.16615][2011.12825].
- **Step-Size Independence from Global $L$:** Adaptive, local, or backtracking step-size rules circumvent the need for a known global Lipschitz constant, making these methods applicable to problems where such constants are elusive or conservative [2601.10370][2007.02746][2007.11761][2209.11989].
- **Non-Euclidean Geometry:** Replacement of Euclidean projections by Bregman projections for non-Euclidean or mirror-descent frameworks [1406.0724][2011.12825].
- **Stochastic and Inexact Variants:** Accommodate oracle noise, inexact operator evaluations, and relative error via stochastic approximation or hybrid proximal-extragradient approaches [2202.09759][1812.02138].
- **Equilibrium and Multi-Valued Extensions:** Pseudomonotone, quasimonotone, and multi-valued mappings, as in equilibrium theory and generalized (sub)gradient settings, are handled via suitable proximal or correction subroutines [2011.12825][2511.18642].

## 6. Numerical Performance and Practical Impact

Extensive computational experiments have been reported:

- **Compressive Sensing (LASSO):** Incompressive sampling, the double-inertial Tseng method achieves lower iteration count and runtime compared to relaxed/inertial Tseng and alternative acceleration schemes [2209.11989].
- **Large-Scale Variational Inequalities:** With random monotone operators, inertial Tseng-type methods outperform classical/non-inertial extragradient algorithms in both high and infinite dimension settings [2209.11989][2007.02746][2006.16615][2011.12825].
- **Pseudomonotone/Quasimonotone Operators:** Fast and robust performance in generalized monotonicity settings, where classical algorithms may stagnate or fail [2007.11761][2011.12825][2601.10370][2511.18642].
- **Optimal Control:** Application to discretized control problems with box/range constraints, recovering highly nontrivial optimal profiles rapidly [2007.11761].

Empirical results consistently indicate that the inclusion of inertia—especially double/multi-step versions and adaptive step-size control—reduces iteration count, wall-clock time, and increases methodology robustness.

## 7. Theoretical Significance and Methodological Positioning

The inertial Tseng extragradient paradigm unifies and generalizes several important operator splitting methods, including forward-backward, hybrid proximal-extragradient, and Korpelevich extragradient methods. The fundamental theoretical strengths include:

- **Global Weak/Strong/Linear Rates:** Rigorous convergence under minimal assumptions, competitive with the best-in-class splitting algorithms [2209.11989][1812.02138][2511.18642].
- **Parameter Independence from Problem Constants:** Step-size rules that avoid global operator norm estimation enhance practical implementation viability.
- **Extendability:** The architecture accommodates stochastic/inexact oracles, multi-valued operators, and can be adapted for both convex and nonconvex models.
- **Versatility:** Empirically and theoretically validated across disparate domains (signal processing, optimization, PDE-constrained problems, control).

Contemporary developments focus on further acceleration by multi-step inertial terms, increased adaptivity, extension to non-Euclidean geometry and Banach spaces, and removal of restrictive monotonicity/Lipschitz hypotheses [2601.10370][1406.0724][2202.09759]. The method remains an active area for both theoretical investigation and algorithmic development in large-scale, structured, and non-smooth optimization.

Source: https://www.emergentmind.com/topics/inertial-tseng-extragradient-method