Three-Operator Splitting Scheme

• Three-operator splitting is an iterative algorithm that decomposes a complex monotone inclusion into three simpler subproblems, enabling efficient handling of nonlinear components.
• It generalizes classical two-operator methods by incorporating adaptive parameters, inertial variants, and distributed frameworks to improve convergence rates.
• The scheme underpins diverse applications in convex minimization, PDE solvers, and signal processing, uniting methodologies like forward–backward, Douglas–Rachford, and ADMM.

A three-operator splitting scheme is an iterative algorithmic framework designed to solve nonlinear problems where the main operator is expressed as a sum of three components, each possessing specific monotonicity or regularity properties. This scheme generalizes classical two-operator splitting methods, facilitating the decomposition of composite inclusions or minimization problems into subproblems that can be handled via separate, often simpler, substeps. Originally emerging from monotone operator theory and convex optimization, three-operator splitting now underpins numerous advances in large-scale optimization, PDE solvers, control, imaging, and distributed computation.

1. Formulation and Core Algorithmic Principles

Let $\mathcal{H}$ be a real Hilbert space with inner product $\langle \cdot, \cdot \rangle$ and norm $\|\cdot\|$. The canonical monotone inclusion form is

$$\text{find } x \in \mathcal{H} \text{ such that } 0 \in A x + B x + C x,$$

with

• $A: \mathcal{H} \to 2^{\mathcal{H}}$ maximally monotone (set-valued, e.g., subdifferential),
• $B: \mathcal{H} \to 2^{\mathcal{H}}$ maximally monotone (or sometimes single-valued monotone/monotone–Lipschitz),
• $C: \mathcal{H} \to \mathcal{H}$ single-valued and cocoercive ($\exists \beta>0$ such that $$\langle Cx - Cy, x-y \rangle \geq \beta \|C x - C y\|^2$$).

The classical Davis–Yin three-operator splitting (Davis et al., 2015) is as follows. Fix $\gamma \in (0, 2\beta)$ and define for $\langle \cdot, \cdot \rangle$0:

$\|\cdot\|$2 where $\langle \cdot, \cdot \rangle$1 is the resolvent of $\langle \cdot, \cdot \rangle$2.

Convergence is ensured under mild conditions (e.g., for relaxation parameters strictly less than a computable bound, and nonempty solution set). This scheme recovers forward–backward and Douglas–Rachford splitting for appropriate choices of operators, and extends to more general settings in convex and monotone operator theory.

2. Variants, Extensions, and Theoretical Properties

The basic three-operator splitting admits significant algorithmic development:

• Inertial/extrapolated variants: Incorporating momentum terms to accelerate convergence, e.g., two-step inertial extrapolation (Iyiola et al., 2024), or inertial Krasnosel'skii–Mann iteration (1904.11684, Cevher et al., 2019). These schemes demonstrate, under carefully chosen parameters, strict Lyapunov functional decrease and empirically notable acceleration, overcoming certain failures of naive one-step inertial extensions.
• Generalizations to monotone–Lipschitz and nonlinear momentum: Beyond cocoercive $\langle \cdot, \cdot \rangle$3, recent work considers splitting for $\langle \cdot, \cdot \rangle$4 monotone–Lipschitz and general nonlinear preconditioning (Qin et al., 18 Nov 2025), yielding a broad framework with weak and R-linear convergence, depending on strong monotonicity.
• Adaptive parameter and step-size selection: Algorithms with parameters tailored to the local or structural properties of the operators (Dao et al., 2021, Pedregosa et al., 2018), supporting larger step-sizes and facilitating empirical speedup over non-adaptive baselines.
• Asymmetric and primal-dual methods: AFBA (Asymmetric Forward-Backward-Adjoint) (Latafat et al., 2016) and related methods that interpolate between primal-only and primal-dual structures, unifying several recent algorithms.
• Stochastic and distributed splitting: Stochastic three-operator splitting for empirical risk, distributed, or federated optimization, including primal-dual algorithms and stochastic extensions applicable to large-scale and networked settings (Franci et al., 2022, Tang et al., 2022).

Convergence theory for these methods draws on averaged operator fixed-point theory, Lyapunov/energy techniques, and, in some cases, robust control and linear matrix inequality frameworks for tight convergence rate certification (Wang et al., 2019).

3. Convergence Rates and Performance Guarantees

Convergence of three-operator splitting schemes in the monotone or convex setting is now well-characterized:

• Sublinear rates: In general convex settings (one smooth, two nonsmooth terms), non-ergodic $\langle \cdot, \cdot \rangle$5 decay in gradient mapping norm or fixed-point residual is achievable (Pedregosa, 2016, Wang et al., 2019). Algorithms with ergodic (averaged) $\langle \cdot, \cdot \rangle$6 rates for function value or optimality gap are standard (Pedregosa et al., 2018, Davis et al., 2015).
• Linear rates: If any operator (A, B, or C) is strongly monotone (or underlying strong convexity), nonasymptotic geometric convergence can be certified either through direct analysis or via performance estimation (Wang et al., 2019, Sadiev et al., 26 May 2026).
• Acceleration: Recent advances derive accelerated rates for strongly convex–smooth problems, achieving theoretical optimality with complexity proportional to $\langle \cdot, \cdot \rangle$7 under mild assumptions (Sadiev et al., 26 May 2026). Unified primal-dual Bregman frameworks systematically produce such optimal or near-optimal accelerated algorithms for general three-composite objectives.
• Nonconvex settings: Sufficient decrease properties and subsequential convergence to critical points have been shown for relaxed three-operator splitting in structured nonconvex regimes (Alcantara et al., 10 Apr 2025, Yurtsever et al., 2021). Explicit bounds on stationarity/infeasibility decay ($\langle \cdot, \cdot \rangle$8) are available for some classes.

A plausible implication is that, in large-scale optimization with decomposable but nontrivial coupling among three components (e.g., regularizer/data/constraint), operator splitting achieves robust efficiency and convergence without requiring joint proximal calculations.

4. Structurally Adapted Schemes and Applications

Three-operator schemes are tailored to several problem typologies:

• Convex composite minimization: Sum of three terms (f + g + h), with $\langle \cdot, \cdot \rangle$9 proximal-friendly (possibly nonsmooth), $\|\cdot\|$0 smooth with Lipschitz gradient (Davis et al., 2015, Pedregosa et al., 2018). Prototypical in imaging (e.g., joint TV and sparsity regularization plus data term), signal processing, and statistical learning.
• Monotone inclusions and variational inequalities: Linear and nonlinear monotone PDEs, projections, and inclusions naturally decompose into three-maximally monotone parts, enabling domain decomposition or operator splitting for PDE solvers (Gerlach et al., 2013).
• Stochastic and distributed optimization: The structure $\|\cdot\|$1 (smooth+simple+coupled via linear operator) arises in distributed resource allocation and federated learning. Distributed variants support peer-to-peer or networked architectures (Franci et al., 2022).
• Conic programming and ADMM: Several variants serve as natural generalizations of alternating direction methods for multi-block separable convex problems, including convex quadratic semidefinite programming (Chang et al., 2018, Anshika et al., 2024).
• Hamiltonian integration: Splitting flow maps into three analytically tractable subflows can be exploited for symplectic integration of Hamiltonian systems (e.g., in molecular dynamics or nonlinear wave equations) (Gerlach et al., 2013, Spiteri et al., 2023).

5. Algorithmic Enhancements and Practical Considerations

Practical implementation and advancement of three-operator splitting rely on:

• Inertia/momentum and restarts: Properly tuned inertial parameters or multi-step extrapolation (and adaptive restart mechanisms) enhance robustness and empirical efficiency, particularly for ill-conditioned or slow-mixing problems (Iyiola et al., 2024, Cevher et al., 2019).
• Adaptive line-search/step-size control: Leveraging local smoothness via backtracking or line-search often allows orders-of-magnitude larger steps than vanilla conservative theory would permit (Pedregosa et al., 2018), with practical impact evident across benchmark suites.
• Preconditioning and primal-dual embeddings: Variable metric preconditioning, Gauss-Seidel/block updates, and linkages to primal–dual splitting extend the schemes' utility to structured convex-concave objectives, saddle-point problems, and ill-conditioned data (Latafat et al., 2016, Tang et al., 2022).
• Averaging and regularization: Ergodic averaging ensures provable performance without restrictive monotonicity, and is crucial in variance reduction and stochastic optimization contexts.

Empirical studies indicate that properly tuned three-operator splitting outperforms, in wall-clock and iteration count, both simpler and more complex alternatives, especially on problems with true three-component structure and costly joint-prox subproblems.

6. Connections to Classical and Emerging Methodologies

Three-operator splitting unifies and extends much of the operator-splitting and primal–dual literature:

• Encompasses forward–backward, Douglas–Rachford, and various primal-dual methods: As specific cases or limits, the scheme covers two-block and single-block splitting, ADMM, Condat–Vũ, Chambolle–Pock, and their stochastic or coordinate variants (Davis et al., 2015, Sadiev et al., 26 May 2026, Tang et al., 2022).
• Bregman, reflected, and adjoint splitting: Recent frameworks show that asymmetric, reflected, and Bregman-proximal constructions subsume or generalize three-operator splitting, unifying much of recent literature into single variable-metric or primal–dual recipes (Sadiev et al., 26 May 2026, Latafat et al., 2016).
• Specialization to structure yields improved guarantee: In situations where term domains are orthogonal or one component is strongly monotone, additional theoretical and computational advantages emerge, including stability under large step sizes or direct strong convergence (Anshika et al., 2024, Dao et al., 2021).

Three-operator splitting thus serves as a touchstone for the analysis of new algorithms in optimization and inverse problems, both by direct design and through its abstract operator-theoretic ties.

7. Open Challenges and Frontiers

Outstanding frontiers include:

• Nonconvex and nonmonotone regimes: Sufficient-descent envelope constructions and careful parameter relaxation have enabled recent progress on nonconvex three-component objectives, but general complexity and global rate results remain highly nuanced (Alcantara et al., 10 Apr 2025, Yurtsever et al., 2021).
• Optimal acceleration: While unified frameworks now yield optimal rates in strongly convex-smooth frameworks, extending such rates to broad classes of nonsmooth, distributed, or stochastic settings is ongoing (Sadiev et al., 26 May 2026).
• Distributed, asynchronous, and learning-adaptive settings: Incorporating heavy-tailed stochasticity, effective communication, and adaptivity is at the forefront, with practical and theoretical developments evident in distributed three-operator splitting and its stochastic primal–dual analogues (Franci et al., 2022, Tang et al., 2022).
• Empirical tuning and application-specific design: There remains a substantial gap between theoretically optimal parameter regimes and those that deliver best empirical performance; adaptive, data-driven, and meta-learning extensions of splitting schemes are active areas of investigation.

In summary, the three-operator splitting scheme constitutes a fundamental, flexible, and rapidly evolving toolset for decomposable analysis and optimization problems across mathematical analysis, PDEs, optimization, signal processing, and machine learning (Davis et al., 2015, Pedregosa et al., 2018, Iyiola et al., 2024, Qin et al., 18 Nov 2025, Sadiev et al., 26 May 2026).