---
title: Inertial Douglas-Rachford Splitting
url: https://www.emergentmind.com/topics/inertial-douglas-rachford-splitting
type: topic
---

# Inertial Douglas-Rachford Splitting

Inertial Douglas-Rachford splitting is a class of operator-splitting methods that augments the classical Douglas-Rachford (DR) algorithm with inertial extrapolation—history-dependent momentum steps inspired by Nesterov and Polyak—in order to accelerate convergence for monotone inclusions and convex optimization problems. This extension has been formalized both for two-operator and three-operator monotone inclusions in Hilbert spaces, with substantial development in convergence analysis, algorithmic design, and practical applications, as detailed in works including [2410.01099], [1404.4582], [1904.12980], [2002.12670], [1403.3330], and [1904.10502].

## 1. Theoretical Foundations

The inertial Douglas-Rachford framework targets monotone inclusion problems in a real Hilbert space $H$ of the form:
\[
\text{Find }x\in H\text{ such that }0\in A x + B x \quad \text{(two-operator)}
\]
or, in the three-operator variant,
\[
0\in A x+B x+C x
\]
where $A$ and $B$ are maximally monotone, and (for the three-operator problem) $C$ is typically assumed to be $\eta$-cocoercive. The classical DR splitting is extended by incorporating extrapolation based on previous iterates, typically of the form $x_n + \alpha_n (x_n - x_{n-1})$ or through a multistep inertial combination.

The emergence of the inertial DR method is connected to efforts in monotone operator theory and convex optimization to achieve convergence acceleration by emulating momentum techniques, while retaining the strong convergence guarantees characteristic of monotone operator splitting methods.

## 2. Algorithmic Schemes

### 2.1 Inertial Douglas-Rachford for Two-Operator Problems

The inertial DR iteration for $0\in A x+B x$ takes the form [1403.3330]:
1. $w_n = x_n + \alpha_n (x_n - x_{n-1})$
2. $y_n = J_{\gamma B}(w_n)$
3. $z_n = J_{\gamma A}(2 y_n - w_n)$
4. $x_{n+1} = w_n + \lambda_n (z_n - y_n)$

Here, $J_{\gamma M}:=(I+\gamma M)^{-1}$ is the resolvent, $\alpha_n$ is the inertial parameter (nondecreasing, $0\leq\alpha_n<1$), and $\lambda_n$ is a relaxation parameter ($0<\lambda_n<2$). This form generalizes to inexact, relaxed, and multi-step settings [1904.10502].

### 2.2 Inertial Three-Operator Splitting

For $0\in A x + B x + C x$ with $C$ single-valued and cocoercive, multiple inertial extensions exist.

#### Two-Step Inertial Scheme
[2410.01099] proposes the following update:
\[
\begin{cases}
y_n = x_n + \theta(x_n - x_{n-1}) + \delta(x_{n-1} - x_{n-2}) \\
w_n = J_{\gamma B}(y_n) \\
z_n = J_{\gamma A}(2w_n - y_n - \gamma C w_n) \\
x_{n+1} = y_n - \rho w_n + \rho z_n
\end{cases}
\]
with specific constraints on $\rho, \theta, \delta$, and $\gamma$ to guarantee convergence.

#### Forward-Douglas-Rachford (FDR)–Type with Inertia
[1904.12980] presents a single-step inertial method:
\[
\begin{align*}
w_n & = \bar{x}_n + \tau_n (\bar{x}_n - \bar{x}_{n-1}) \\
u_n & = J_{\gamma A}(w_n) \\
v_n & = J_{\gamma B}(2u_n - w_n - \gamma C u_n) \\
\bar{x}_{n+1} & = v_n + \lambda_n (v_n-u_n)
\end{align*}
\]
The two-step version of [2410.01099] overcomes notable drawbacks observed with one-step inertia.


## 3. Convergence Theory

The convergence of inertial Douglas-Rachford methods is established under mild conditions:
- $A$ and $B$ maximally monotone, $C$ cocoercive (in three-operator case)
- Appropriate bounds on step sizes, inertia, and relaxation parameters

For two-operator schemes ([1403.3330], [1404.4582], [1904.10502], [2002.12670]):
- The generated sequence converges weakly to a fixed point $x^*\in\operatorname{zer}(A+B)$
- If either $A$ or $B$ is uniformly monotone (or strongly monotone), then strong convergence of iterates to the unique zero is obtained
- The sum $\sum\|x_{n+1}-x_n\|^2<+\infty$

For three-operator schemes ([2410.01099], [1904.12980]):
- The two-step inertial three-operator splitting (with $\theta, \delta$) ensures weak convergence of iterates to a fixed point of the associated averaged operator $T$, without requiring summability of inertial parameters
- Strong convergence is guaranteed if uniform monotonicity or demiregularity holds at the solution.

A distinctive advance in [2410.01099] is the use of a Lyapunov-type functional involving the two most recent iterates, extending the classical Opial and Fejér techniques. Notably, the paper eliminates summability requirements previously imposed on inertial parameter sequences, thus broadening the practical design space.

Explicit rates such as $o(1/n)$ or linear convergence are not generally established; the established complexity is asymptotic, but square-summability of residuals such as $\|x_{n+1}-y_n\|^2$ is obtained [2410.01099].

## 4. Special Cases and Connections

Inertial DR encompasses and generalizes several pivotal operator-splitting frameworks:

| Case                                 | Inertial Parameters      | Operator Structure                     | Reference        |
|---------------------------------------|-------------------------|----------------------------------------|------------------|
| Classical DR (two-operator)           | $\alpha_n\equiv0$       | $0\in A x+B x$                         | [1403.3330]      |
| Relaxed DR                            | $\alpha_n\equiv0$, $\lambda_n\ne1$ | $0\in A x+B x$             | [1403.3330], [1904.10502] |
| One-step inertial DR                  | $\alpha_n>0$            | $0\in A x+B x$                         | [1403.3330]      |
| Two-step inertial three-operator      | $(\theta\ne0, \delta\ne0)$ | $0\in A x+B x+C x$                 | [2410.01099]      |
| Forward–Backward/ISTA/FISTA variants  | $A$ nonsmooth, $B\equiv0$ | $0\in A x + C x$                   | [2410.01099]      |

When $C\equiv 0$ in the three-operator schemes, the Davis–Yin splitting reduces to Douglas–Rachford splitting, and the inertial framework recovers both one-step and two-step inertial DR algorithms as special cases [2410.01099], [1904.12980].

## 5. Parameter Choices and Adaptive Strategies

The choice of inertial and relaxation parameters critically affects convergence. The established theory requires:
- Inertia $\alpha_n$, $\theta$, $\delta$ to satisfy upper bounds strictly less than 1, with explicit inequalities ensuring energy decrease (e.g., [1403.3330], [2410.01099]).
- Relaxation factors $\lambda_n$ constrained to $(0,2)$, with explicit coupling to the inertia upper bound (see [1904.10502]).
- Stepsize $\gamma$ is bounded above as a function of cocoercivity constants (e.g., $0<\gamma<2\eta$ with $\eta$ the cocoercivity of $C$).

Adaptive inertia (restart) schemes have been investigated [1904.12980], where the inertial parameter is recomputed or reset if a monitored objective function stalls, potentially allowing larger inertia in practice.

## 6. Applications and Numerical Evidence

Inertial Douglas-Rachford methodologies have achieved demonstrable improvements in diverse applications, particularly inverse problems and large-scale convex optimization:

- **Image Restoration (LASSO and SCAD):** [2410.01099] reports that two-step inertial DR outperforms both non-inertial and one-step inertial methods in signal-to-noise ratio and CPU cost in LASSO image denoising problems. For SCAD regularization, the two-step inertial method converges in fewer iterations and with reduced computation time relative to one-step and classical approaches.
- **Matrix Completion, Portfolio Optimization, Cone Projection:** [1904.12980] demonstrates that IFDR and its restarted variant substantially reduce CPU time versus standard three-operator splitting and interior-point solvers, especially in high-dimensional matrix completion and doubly-nonnegative cone projection.
- **Robust Principal Component Pursuit (RPCP):** Inertial DR and ADMM reduce iteration count by 20–30% at negligible additional per-iteration cost [2002.12670].
- **Clustering and Location Theory:** Empirical results confirm that inertial DR halves iteration counts and CPU times for large-scale nonsmooth convex problems [1403.3330].

A salient observation is that one-step inertial schemes may sometimes fail to accelerate or can even destabilize convergence, while two-step inertial extrapolation restores or enhances acceleration [2410.01099].

## 7. Summary and Perspectives

The integration of inertia into Douglas-Rachford splitting constitutes a robust methodological advance in monotone operator theory and convex optimization. The major contributions include:
- Expanding the DR framework to allow inertial acceleration while retaining convergence guarantees
- Removing restrictive summability conditions on inertial parameters (notably in the two-step three-operator case)
- Empirical confirmation that multi-step inertia yields tangible acceleration and improved numerical performance, especially in ill-conditioned or poorly scaled applications

The connection between inertial DR and other operator splitting and primal-dual frameworks (including ADMM and forward-backward methods) highlights the flexibility and broad applicability of these algorithms.

Ongoing research investigates convergence rates, adaptive parameter selection, and extensions to nonconvex problems. The empirical evidence underscores that, for practical large-scale problems, carefully tuned inertial Douglas-Rachford splitting is a preferred first-order strategy, particularly where acceleration is critical and strong monotonicity cannot be assumed [2410.01099], [1404.4582], [1904.12980], [1403.3330], [1904.10502].

Source: https://www.emergentmind.com/topics/inertial-douglas-rachford-splitting