Variable & Operator Splitting in Optimization

• Variable and Operator Splitting (VOS) is a unified framework that decouples complex convex optimization problems into simpler subproblems through variable-lifting and tailored operator resolvents.
• The framework generalizes classical methods like Douglas–Rachford and forward–backward, supporting decentralized and adaptive schemes with convergence guarantees based on super strong nonexpansivity.
• VOS underpins advanced techniques such as variable-metric preconditioning and accelerated iterations, enabling robust applications in inverse problems, distributed optimization, and PDE simulation.

Variable and Operator Splitting (VOS) is a collective framework for iterative methods in convex optimization, monotone inclusion, and variational analysis. It decouples complex problems into simpler subproblems via systematic introduction of both variable-splitting (lifting into higher-dimensional product spaces or dual formulations) and operator-splitting (composition of maximally monotone, cocoercive, or Lipschitzian operators with tailored resolvent or forward steps). VOS encompasses and generalizes classical schemes such as Douglas–Rachford, Peaceman–Rachford, forward–backward, forward–backward–forward, and their variable-metric and distributed variants, providing a unified platform for algorithmic design, convergence theory, and application-specific adaptation.

1. Uniform Monotonicity and Splitting Operator Foundations

At the core of VOS lies the interplay between uniform monotonicity of set-valued operators and the geometric properties of associated splitting maps. A set-valued operator $A:X \rightrightarrows X$ on a real Hilbert space is called uniformly monotone with modulus $\phi$ if

$$\langle x-y,\,x^*-y^*\rangle \geq \phi(\|x-y\|),\qquad (x,x^*),(y,y^*)\in\operatorname{gra} A,$$

where $\phi:[0,\infty)\to [0,\infty)$ is increasing and vanishes only at 0. The resolvent $J_A = (I+A)^{-1}$ is firmly nonexpansive, and the reflected resolvent $R_A = 2J_A - I$ is nonexpansive if and only if $A$ is maximally monotone.

A key contribution highlights the equivalence between uniform monotonicity of $A$ and super strong nonexpansivity (SNE) of $R_A$, yielding quantitative contraction properties: $$\|x-y\|^2 - \|R_A x - R_A y\|^2 \ge 4\,\phi(\|J_A x - J_A y\|),\qquad \forall x,y \in X.$$ This super-strong nonexpansivity enables sharp convergence rate estimates for composite splitting operators underpinning Douglas–Rachford (DR), Peaceman–Rachford (PR), and forward–backward (FB) methods. The theory is fully self-dual: uniform monotonicity of $\phi$0 or its inverse $\phi$1 suffices for the corresponding contractive behaviour and convergence rates of the associated splitting algorithms (Liu et al., 2022).

2. Generalized Variable–Operator Splitting Schemes

Recent VOS frameworks abstract the iterative process as blockwise updates for multiple operators, often indexed by node sets in decentralized/distributed architectures. Consider the monotone inclusion

$\phi$2

with maximally monotone set-valued $\phi$3 and monotone, possibly Lipschitz, single-valued $\phi$4. The splitting iteration introduces auxiliary variables and coefficient matrices (e.g., $\phi$5), encoding consensus constraints, operator mixing, and topology of networked computation.

The general iteration (for dual variables $\phi$6, primal blocks $\phi$7) has the form: $\phi$8 with specific structure imposed on $\phi$9 to guarantee convergence and enforce block consensus (Dao et al., 21 Apr 2025). Graph-based choices of $$\langle x-y,\,x^*-y^*\rangle \geq \phi(\|x-y\|),\qquad (x,x^*),(y,y^*)\in\operatorname{gra} A,$$0 (incidence, Laplacian, etc.) enable fully distributed VOS algorithms, including parallel and decentralized DR/FBS/FBF/ADMM variants.

Convergence is established through analysis of the conical-averaged nature of the composite iteration, delivering global weak convergence and, under additional regularity, strong convergence results.

3. Adaptive and Three-Operator Splitting

In problems with three (or more) sum operators—especially when only two resolvents are computationally accessible—VOS generalizes to adaptive schemes balancing monotonicity and cocoercivity across blocks. For $$\langle x-y,\,x^*-y^*\rangle \geq \phi(\|x-y\|),\qquad (x,x^*),(y,y^*)\in\operatorname{gra} A,$$1 with $$\langle x-y,\,x^*-y^*\rangle \geq \phi(\|x-y\|),\qquad (x,x^*),(y,y^*)\in\operatorname{gra} A,$$2 monotone and $$\langle x-y,\,x^*-y^*\rangle \geq \phi(\|x-y\|),\qquad (x,x^*),(y,y^*)\in\operatorname{gra} A,$$3 cocoercive, the generic adaptive VOS iteration is: $$\langle x-y,\,x^*-y^*\rangle \geq \phi(\|x-y\|),\qquad (x,x^*),(y,y^*)\in\operatorname{gra} A,$$4 with parameters $$\langle x-y,\,x^*-y^*\rangle \geq \phi(\|x-y\|),\qquad (x,x^*),(y,y^*)\in\operatorname{gra} A,$$5 tuned according to the monotonicity constants. Convergence is established by showing that the composite map is “conically averaged,” and weak or strong convergence of iterates to the solution is guaranteed under mild regularity. The asymptotic regularity rate is $$\langle x-y,\,x^*-y^*\rangle \geq \phi(\|x-y\|),\qquad (x,x^*),(y,y^*)\in\operatorname{gra} A,$$6, and adaptivity enlarges the admissible step-size regime compared to fixed-parameter methods (Dao et al., 2021, Davis et al., 2015).

4. Variable-Metric and Averaged Operator Generalizations

VOS is further extended by introducing variable metrics (time-varying inner products) and relaxation, to enhance algorithmic flexibility and allow for problem-dependent preconditioning. In the variable-metric forward–backward (VM-FBS) and forward–backward–forward (VM-FBF) paradigms, iterations are executed in a sequence of Hilbert spaces $$\langle x-y,\,x^*-y^*\rangle \geq \phi(\|x-y\|),\qquad (x,x^*),(y,y^*)\in\operatorname{gra} A,$$7 determined by positive-definite operators $$\langle x-y,\,x^*-y^*\rangle \geq \phi(\|x-y\|),\qquad (x,x^*),(y,y^*)\in\operatorname{gra} A,$$8. The main convergence theorems assert weak (and sometimes strong) convergence, provided $$\langle x-y,\,x^*-y^*\rangle \geq \phi(\|x-y\|),\qquad (x,x^*),(y,y^*)\in\operatorname{gra} A,$$9 varies slowly and maintains uniform bounds. These frameworks enable more aggressive step-sizes and exploit local curvature, restoring or accelerating convergence under ill-conditioning, composite structure, or large-scale settings (Vũ, 2012, Cui et al., 2018, Glaudin, 2018).

5. Acceleration and Advanced Algorithmic Structures

VOS is foundational in the construction of accelerated first-order schemes via the explicit splitting of variable and operator roles. In continuous-time, variable splitting introduces velocity/momentum-like variables, leading to flows of the form

$\phi:[0,\infty)\to [0,\infty)$0

which, once discretized, yield schemes such as Accelerated Over-Relaxation (AOR), Extrapolation by Predictor–Corrector (EPC), and variants of Nesterov-type methods (Chen et al., 7 May 2025). Lyapunov analysis delivers accelerated linear rates for strongly monotone operators, and sublinear $\phi:[0,\infty)\to [0,\infty)$1 rates in the convex case via dynamic parameter schedules or regularized flows. The broad applicability encompasses composite minimization, saddle-point problems, bilinear games, and mirror descent variants, with clear modularity for incorporating preconditioning, projection, or operator-specific structure (Chen et al., 26 Jan 2026).

6. Applications to Inverse Problems and Distributed Optimization

VOS principles are utilized in large-scale imaging, signal processing, machine learning, and distributed optimization. In plug-and-play image restoration, VOS decouples a linear operator/observation step from an advanced statistical prior (e.g., patchwise GMM denoisers), enabling ADMM-based algorithms with competitive or superior empirical performance on task-specific datasets (Teodoro et al., 2016). In networked settings, VOS generalizations implement consensus and resource allocation tasks by embedding primal–dual variable lifts and graph-structured splitting matrices, yielding scalable and communication-efficient distributed architectures (Dao et al., 21 Apr 2025).

In PDE simulation, VOS enables high-order accurate schemes for kinetic equations by splitting dynamics across physically meaningful bands (e.g., velocity bands in Vlasov–Ampère), preserving integral invariants such as mass, momentum, and energy beyond what is possible with traditional operator-splitting (Rossmanith et al., 22 May 2025).

7. Theoretical Significance and Convergence Analysis

The comprehensive theoretical underpinning of VOS methods rests on fixed-point theory for (super) strongly nonexpansive and conically averaged mappings. The reflected-resolvent formalism and the associated self-duality enable rigorous proof of weak and strong convergence, asymptotic regularity, and even linear convergence under uniform monotonicity or sufficient penalization. Banach–Rakotch contraction analysis, Lyapunov energy dissipation, and quasi-Fejér monotonicity are key tools in demonstrating algorithmic robustness across a spectrum of operator compositions (Liu et al., 2022, Dao et al., 2021, Davis et al., 2015, Cui et al., 2018, Vũ, 2012).

VOS thus serves as a unifying and extensible mathematical scaffold for the design, analysis, and deployment of state-of-the-art convex optimization and monotone inclusion algorithms, integrating geometry-aware, distributed, accelerated, and application-specific features within a common algebraic and analytic framework.