Forward-Backward Splitting PnP Framework

• The FBS-PnP framework is a variable metric method that integrates forward-backward splitting with plug-and-play denoisers for addressing composite monotone inclusion problems.
• It dynamically updates preconditioning metrics to accelerate convergence and enhance stability in ill-conditioned, high-dimensional optimization tasks.
• By extending to primal-dual formulations, the framework flexibly combines classical algorithms with learned priors for robust imaging and signal processing applications.

The Forward-Backward Splitting (FBS) Plug-and-Play (PnP) framework integrates the classical operator splitting methodology—designed to solve composite monotone inclusions or optimization problems—with modern data-driven or otherwise complex priors, often via implicit or explicit denoising steps. This synthesis leverages and extends the classical theory of monotone operator splitting to accommodate variable metrics, preconditioning, and primal-dual formulations, with significant consequences for the analysis and solution of large-scale, structured, and potentially non-smooth or dual problems.

1. Variable Metric Forward-Backward Splitting: Formulation

The classical FBS algorithm is applied to find zeros of the sum of a maximally monotone operator $A: H \rightrightarrows H$ and a cocoercive operator $B: H \to H$ in a real Hilbert space $H$: $0 \in Ax + Bx.$ The variable metric extension generalizes this scheme by updating each iteration with respect to a (potentially changing) sequence of self-adjoint, positive definite linear operators $\{U_n\}_{n\in\mathbb{N}}$ ("preconditioners" or metrics). The generic FBS-PnP iteration is

$$\begin{aligned} y_n &= x_n - \gamma_n U_n (B x_n + b_n), \ x_{n+1} &= x_n + \lambda_n (J_{\gamma_n U_n A}(y_n + \gamma_n U_n a_n) - x_n), \end{aligned}$$

where $J_{\gamma_n U_n A} = (Id + \gamma_n U_n A)^{-1}$ is the $U_n$-resolvent, sequences $\{\gamma_n\}$ and $\{\lambda_n\}$ are step-sizes and relaxation parameters subject to bounds ensuring stability, and $\{a_n\}$, $\{b_n\}$ are absolutely summable error sequences. The control condition for metric updates is

$(1 + \eta_n) U_{n+1} \succeq U_n$

with $\{\eta_n\}$ summable, ensuring that metrics do not vary too abruptly.

This variable metric scheme recovers the standard Euclidean FBS method when $U_n = Id$ for every $n$, but allows for adaptivity and preconditioning—important for ill-conditioned or heterogeneous large-scale problems and for incorporating problem structure in PnP imaging applications.

2. Convergence Properties

The main convergence theorems state:

• If $A$ is maximally monotone, $B$ is $\beta$-cocoercive, the control conditions on $\{U_n\}$ are satisfied, and parameters and error tolerances are chosen appropriately:
  • The iterates $\{x_n\}$ are bounded.
  • Every weak cluster point is a solution to the original monotone inclusion: $x \in zer(A + B)$.
  • Under additional regularity (e.g., $A$ or $B$ is demiregular, or $d_C(x_n) \to 0$ for the solution set $C$), the sequence $\{x_n\}$ converges strongly to a solution.

This guarantees stability and robustness even in the presence of variable preconditioning and perturbations. The weak/strong convergence distinction is essential: many imaging inverse problems or operator splitting schemes involve large-dimensional spaces, where weak convergence is often the default guarantee.

3. Duality and Extension to Composite Monotone Inclusions

A significant methodological innovation is the dualization of the variable metric FBS to address more complex composite monotone inclusions, leading to primal-dual splitting algorithms. The general class is

$$0 \in Ax + \sum_i L_i^* (B_i \circ D_i)(L_i x - r_i) + Cx, \qquad x \in H,$$

with associated dual inclusions. Algorithms emerge that simultaneously update primal ($x$) and dual ($v_1, ..., v_m$) variables, of the form: $$\begin{aligned} x_n &= J_{p^{-1}A}(p^{-1}(z - \textstyle\sum_i L_i^* v_{i,n})), \ v_{i,n+1} &= v_{i,n} + \alpha_n (J_{U_{i,n}^{-1} B_i}(v_{i,n} + U_{i,n}(L_i x_n - r_i)) - v_{i,n}), \end{aligned}$$ with individual metrics for each variable and operator. These allow full operator splitting:

• Each monotone operator is accessed independently (forward for $B_i \circ D_i$; backward or resolvent for $A$).
• The scheme accommodates changing metrics in both primal and dual blocks.
• When $U_n$ are constant, the resulting algorithms are often new even among fixed-metric splitting methods.

In the PnP regime—where classical regularization operators may be replaced by learned or algorithmic denoisers—such splitting is essential for integrating data-fidelity and nontrivial priors.

4. Advantages for the Forward-Backward PnP Framework

Variable metric FBS (and its extensions) possesses three core advantages for FBS-PnP:

• Flexibility: By selecting or adapting the metric, one can optimize for local curvature, accelerate convergence, precondition for ill-conditioning, and tailor the scheme to the specific structure of the underlying inverse problem.
• Adaptivity: Dynamic metric update allows exploitation of local geometric properties or varying smoothness, as encountered in advanced imaging, network flow, and regularization via nontrivial analytically or data-driven priors.
• Extensibility: By formulating the splitting in the dual (or product) space, the framework accommodates composite and hierarchical problems, constraints, and plug-and-play modules, including generalized denoisers and network-based regularizers.

For PnP, this allows consistent integration of complex or operator-valued priors alongside data-fidelity or likelihood terms, either in explicit optimization problems or implicit fixed-point splitting formulations.

5. Application Domains and Implementation Examples

The paper enumerates several applications showcasing the framework's generality:

Application type	Instantiated structure	Notes
Minimization of $f(x) + g(x)$	$A = \partial f$, $B = \nabla g$	Handles $g$ with Lipschitz $\nabla g$
Variational inequalities	$0 \in A(x) + \nabla \Phi(x) + N_C(x)$	For constraints/penalized problems
Primal-dual composite splitting	Block decomposition with multiple monotone operators	E.g., in signal recovery, best approx.
Imaging inverse problems/network flows	Inclusion of additional cocoercive operators	Structured 'block' FBS, plug-in denoisers

For instance, in signal/image processing, $A$ may represent a total variation or wavelet regularizer, $B$ the gradient of a least-squares data-fidelity term, and the PnP component replaces a proximal step with a learned or algorithmic denoiser, all under variable (preconditioned) metrics.

6. Practical Implementation and Tuning

Tuning of step-sizes $\gamma_n$, relaxation $\lambda_n$, and metric update $\{U_n\}$ is essential for performance:

• $\gamma_n$ must be chosen below a bound inversely proportional to $\|U_n\|$ and a cocoercivity parameter.
• $\lambda_n$ in $[\varepsilon, (23-\varepsilon)/\|U_n\|]$ for some $\varepsilon > 0$.
• Adaptation of $\{U_n\}$ typically leverages local approximation to Hessians or curvature, or (in large-scale problems) uses diagonal or block-diagonal preconditioners.

Stop criteria build on norm differences between iterates, violation of inclusion residuals, or the decrease of the objective or proximity functional.

7. Theoretical and Algorithmic Summary

• The variable metric FBS-PnP framework, via iterations

$$\begin{aligned} y_n &= x_n - \gamma_n U_n (B x_n + b_n), \ x_{n+1} &= x_n + \lambda_n (J_{\gamma_n U_n A}(y_n + \gamma_n U_n a_n) - x_n), \end{aligned}$$

generalizes the classical FBS both in metric and in operator composition.

• Convergence to a solution of $A + B = 0$ (weak or strong, dependent on additional regularity) is guaranteed under summability of error, cocoercivity of $B$, and controlled metric evolution.
• The framework extends natively to primal-dual splitting schemes, fully decomposing operators (including in PnP settings with sophisticated or learned priors).
• Applications encompass variational inequalities, hierarchical and composite optimization, network and image reconstruction flows.

This formulation and its analysis furnish a robust scalable architecture for PnP and related monotone inclusion applications, offering both theoretical convergence guarantees and strong empirical performance in large-scale, structured, and data-driven settings (Combettes et al., 2012).