Inertial Douglas-Rachford Splitting

• Inertial Douglas-Rachford splitting is an operator-splitting method that integrates history-dependent momentum to accelerate convergence in monotone inclusion and convex optimization problems.
• It extends the classical DR algorithm by incorporating inertial extrapolation techniques, achieving robust convergence under mild conditions for both two- and three-operator frameworks.
• Practical applications, including image restoration and matrix completion, demonstrate its ability to reduce iteration counts and CPU time while enhancing performance.

Inertial Douglas-Rachford splitting is a class of operator-splitting methods that augments the classical Douglas-Rachford (DR) algorithm with inertial extrapolation—history-dependent momentum steps inspired by Nesterov and Polyak—in order to accelerate convergence for monotone inclusions and convex optimization problems. This extension has been formalized both for two-operator and three-operator monotone inclusions in Hilbert spaces, with substantial development in convergence analysis, algorithmic design, and practical applications, as detailed in works including (Iyiola et al., 2024, Bot et al., 2014, Cevher et al., 2019, Yang et al., 2020, Bot et al., 2014), and (Alves et al., 2019).

1. Theoretical Foundations

The inertial Douglas-Rachford framework targets monotone inclusion problems in a real Hilbert space $H$ of the form: $$\text{Find }x\in H\text{ such that }0\in A x + B x \quad \text{(two-operator)}$$ or, in the three-operator variant,

$0\in A x+B x+C x$

where $A$ and $B$ are maximally monotone, and (for the three-operator problem) $C$ is typically assumed to be $\eta$-cocoercive. The classical DR splitting is extended by incorporating extrapolation based on previous iterates, typically of the form $x_n + \alpha_n (x_n - x_{n-1})$ or through a multistep inertial combination.

The emergence of the inertial DR method is connected to efforts in monotone operator theory and convex optimization to achieve convergence acceleration by emulating momentum techniques, while retaining the strong convergence guarantees characteristic of monotone operator splitting methods.

2. Algorithmic Schemes

2.1 Inertial Douglas-Rachford for Two-Operator Problems

The inertial DR iteration for $0\in A x+B x$ takes the form (Bot et al., 2014):

1. $w_n = x_n + \alpha_n (x_n - x_{n-1})$
2. $$\text{Find }x\in H\text{ such that }0\in A x + B x \quad \text{(two-operator)}$$0
3. $$\text{Find }x\in H\text{ such that }0\in A x + B x \quad \text{(two-operator)}$$1
4. $$\text{Find }x\in H\text{ such that }0\in A x + B x \quad \text{(two-operator)}$$2

Here, $$\text{Find }x\in H\text{ such that }0\in A x + B x \quad \text{(two-operator)}$$3 is the resolvent, $$\text{Find }x\in H\text{ such that }0\in A x + B x \quad \text{(two-operator)}$$4 is the inertial parameter (nondecreasing, $$\text{Find }x\in H\text{ such that }0\in A x + B x \quad \text{(two-operator)}$$5), and $$\text{Find }x\in H\text{ such that }0\in A x + B x \quad \text{(two-operator)}$$6 is a relaxation parameter ($$\text{Find }x\in H\text{ such that }0\in A x + B x \quad \text{(two-operator)}$$7). This form generalizes to inexact, relaxed, and multi-step settings (Alves et al., 2019).

2.2 Inertial Three-Operator Splitting

For $$\text{Find }x\in H\text{ such that }0\in A x + B x \quad \text{(two-operator)}$$8 with $$\text{Find }x\in H\text{ such that }0\in A x + B x \quad \text{(two-operator)}$$9 single-valued and cocoercive, multiple inertial extensions exist.

Two-Step Inertial Scheme

(Iyiola et al., 2024) proposes the following update: $0\in A x+B x+C x$0 with specific constraints on $0\in A x+B x+C x$1, and $0\in A x+B x+C x$2 to guarantee convergence.

Forward-Douglas-Rachford (FDR)–Type with Inertia

(Cevher et al., 2019) presents a single-step inertial method: $0\in A x+B x+C x$3 The two-step version of (Iyiola et al., 2024) overcomes notable drawbacks observed with one-step inertia.

3. Convergence Theory

The convergence of inertial Douglas-Rachford methods is established under mild conditions:

• $0\in A x+B x+C x$4 and $0\in A x+B x+C x$5 maximally monotone, $0\in A x+B x+C x$6 cocoercive (in three-operator case)
• Appropriate bounds on step sizes, inertia, and relaxation parameters

For two-operator schemes (Bot et al., 2014, Bot et al., 2014, Alves et al., 2019, Yang et al., 2020):

• The generated sequence converges weakly to a fixed point $0\in A x+B x+C x$7
• If either $0\in A x+B x+C x$8 or $0\in A x+B x+C x$9 is uniformly monotone (or strongly monotone), then strong convergence of iterates to the unique zero is obtained
• The sum $A$0

For three-operator schemes (Iyiola et al., 2024, Cevher et al., 2019):

• The two-step inertial three-operator splitting (with $A$1) ensures weak convergence of iterates to a fixed point of the associated averaged operator $A$2, without requiring summability of inertial parameters
• Strong convergence is guaranteed if uniform monotonicity or demiregularity holds at the solution.

A distinctive advance in (Iyiola et al., 2024) is the use of a Lyapunov-type functional involving the two most recent iterates, extending the classical Opial and Fejér techniques. Notably, the paper eliminates summability requirements previously imposed on inertial parameter sequences, thus broadening the practical design space.

Explicit rates such as $A$3 or linear convergence are not generally established; the established complexity is asymptotic, but square-summability of residuals such as $A$4 is obtained (Iyiola et al., 2024).

4. Special Cases and Connections

Inertial DR encompasses and generalizes several pivotal operator-splitting frameworks:

Case	Inertial Parameters	Operator Structure	Reference
Classical DR (two-operator)	$A$5	$A$6	(Bot et al., 2014)
Relaxed DR	$A$7, $A$8	$A$9	(Bot et al., 2014, Alves et al., 2019)
One-step inertial DR	$B$0	$B$1	(Bot et al., 2014)
Two-step inertial three-operator	$B$2	$B$3	(Iyiola et al., 2024)
Forward–Backward/ISTA/FISTA variants	$B$4 nonsmooth, $B$5	$B$6	(Iyiola et al., 2024)

When $B$7 in the three-operator schemes, the Davis–Yin splitting reduces to Douglas–Rachford splitting, and the inertial framework recovers both one-step and two-step inertial DR algorithms as special cases (Iyiola et al., 2024, Cevher et al., 2019).

5. Parameter Choices and Adaptive Strategies

The choice of inertial and relaxation parameters critically affects convergence. The established theory requires:

• Inertia $B$8, $B$9, $C$0 to satisfy upper bounds strictly less than 1, with explicit inequalities ensuring energy decrease (e.g., (Bot et al., 2014, Iyiola et al., 2024)).
• Relaxation factors $C$1 constrained to $C$2, with explicit coupling to the inertia upper bound (see (Alves et al., 2019)).
• Stepsize $C$3 is bounded above as a function of cocoercivity constants (e.g., $C$4 with $C$5 the cocoercivity of $C$6).

Adaptive inertia (restart) schemes have been investigated (Cevher et al., 2019), where the inertial parameter is recomputed or reset if a monitored objective function stalls, potentially allowing larger inertia in practice.

6. Applications and Numerical Evidence

Inertial Douglas-Rachford methodologies have achieved demonstrable improvements in diverse applications, particularly inverse problems and large-scale convex optimization:

• Image Restoration (LASSO and SCAD): (Iyiola et al., 2024) reports that two-step inertial DR outperforms both non-inertial and one-step inertial methods in signal-to-noise ratio and CPU cost in LASSO image denoising problems. For SCAD regularization, the two-step inertial method converges in fewer iterations and with reduced computation time relative to one-step and classical approaches.
• Matrix Completion, Portfolio Optimization, Cone Projection: (Cevher et al., 2019) demonstrates that IFDR and its restarted variant substantially reduce CPU time versus standard three-operator splitting and interior-point solvers, especially in high-dimensional matrix completion and doubly-nonnegative cone projection.
• Robust Principal Component Pursuit (RPCP): Inertial DR and ADMM reduce iteration count by 20–30% at negligible additional per-iteration cost (Yang et al., 2020).
• Clustering and Location Theory: Empirical results confirm that inertial DR halves iteration counts and CPU times for large-scale nonsmooth convex problems (Bot et al., 2014).

A salient observation is that one-step inertial schemes may sometimes fail to accelerate or can even destabilize convergence, while two-step inertial extrapolation restores or enhances acceleration (Iyiola et al., 2024).

7. Summary and Perspectives

The integration of inertia into Douglas-Rachford splitting constitutes a robust methodological advance in monotone operator theory and convex optimization. The major contributions include:

• Expanding the DR framework to allow inertial acceleration while retaining convergence guarantees
• Removing restrictive summability conditions on inertial parameters (notably in the two-step three-operator case)
• Empirical confirmation that multi-step inertia yields tangible acceleration and improved numerical performance, especially in ill-conditioned or poorly scaled applications

The connection between inertial DR and other operator splitting and primal-dual frameworks (including ADMM and forward-backward methods) highlights the flexibility and broad applicability of these algorithms.

Ongoing research investigates convergence rates, adaptive parameter selection, and extensions to nonconvex problems. The empirical evidence underscores that, for practical large-scale problems, carefully tuned inertial Douglas-Rachford splitting is a preferred first-order strategy, particularly where acceleration is critical and strong monotonicity cannot be assumed (Iyiola et al., 2024, Bot et al., 2014, Cevher et al., 2019, Bot et al., 2014, Alves et al., 2019).