FBHF Splitting Algorithm

• FBHF algorithm is a splitting method for solving monotone inclusion and variational inequality problems by combining backward resolvent steps with forward and half-forward corrections.
• It leverages a composite framework of maximally monotone, Lipschitz monotone, and cocoercive operators to effectively handle structured optimization tasks.
• Extensions include stochastic variance reduction, momentum acceleration, and four-operator/primal-dual variants, enhancing convergence rates and practical performance.

The forward-backward-half forward splitting algorithm (FBHF) is a family of operator splitting algorithms for structured monotone inclusion and variational inequality problems. FBHF exploits the composite nature of problems involving a maximally monotone operator, a monotone (possibly Lipschitz) operator, and a cocoercive operator by interleaving resolvent-based (backward) steps with explicit (forward) evaluations and additional correction steps that leverage Lipschitz or block structure. Recent advances extend the algorithm to nonlinear settings, stochastic/variance-reduced regimes, multivariate/primal-dual forms, and momentum/inertia-augmented algorithms.

1. Mathematical Framework and Operator Structure

FBHF is concerned with monotone inclusions in a Hilbert space $\mathcal{H}$, targeting the structured root-finding problem: $$\text{Find } x \in \mathcal{H} \text{ such that } 0 \in Ax + Bx + Cx,$$ where:

• $A : \mathcal{H} \to 2^{\mathcal{H}}$ is maximally monotone (e.g., subdifferential $\partial G$ or normal cone),
• $B : \mathcal{H} \to \mathcal{H}$ is monotone, typically $L$-Lipschitz, often decomposed as a finite sum $B = \sum_{i=1}^N B_i$ for block or finite-sum structure,
• $C : \mathcal{H} \to \mathcal{H}$ is cocoercive, i.e., there exists $\beta > 0$ so that

$$\langle Cx - Cy, x - y \rangle \geq \beta \| Cx - Cy \|^2, \;\; \forall x,y.$$

This triple-splitting model generalizes classical scenarios such as convex minimization with non-smooth regularization/constraints, composite variational inequalities, and primal-dual saddle-point problems.

FBHF naturally extends to four-operator settings with additional Lipschitz and linear-composite terms, as demonstrated in (Roldán, 2023), and allows nonlinear preconditioning as in (Giselsson, 2019, Qin et al., 28 Oct 2025).

2. Core Algorithmic Structure

The canonical FBHF algorithm performs, per iteration, a maximal monotone resolvent (backward) step, a forward step with the Lipschitz operator, and a half-forward correction to offset lack of cocoercivity in $B$. For deterministic settings, one typical iteration has the update (see (Roldán, 2023, Qin et al., 28 Oct 2025)): $$\begin{aligned} y_k & = J_{\gamma_k A}\left( x_k - \gamma_k( Bx_k + Cx_k ) \right), \ x_{k+1} & = y_k - \gamma_k B y_k + \gamma_k B x_k. \end{aligned}$$ In algorithms with nonlinear or block preconditioning (see (Giselsson, 2019, Qin et al., 28 Oct 2025)), the kernel $M_k$ for the resolvent $J_{\gamma_k M_k+A}$ may be nonlinear or nonsymmetric, enabling further flexibility: $$y_k = (M_k + A)^{-1} \left( M_k x_k - (B + C) x_k + \gamma_k^{-1} u_k \right).$$ Momentum or inertia terms, i.e., variables such as $u_k$ and inertia updates, can be explicitly included for acceleration and improved empirical performance.

For four-operator and primal-dual variants (see (Roldán, 2023)), FBHF can be structured as

$$\begin{aligned} p_{n+1} & = C x_n, \ z_{n+1} & = J_{\tau A} \left( x_n - \tau ( L^* u_n + p_{n+1} + D x_n ) \right), \ q_{n+1} & = \tau ( C z_{n+1} - p_{n+1} ), \ u_{n+1} & = J_{\sigma B^{-1}} \left( u_n + \sigma L (2z_{n+1} - x_n - q_{n+1}) \right), \ x_{n+1} & = z_{n+1} - q_{n+1}, \end{aligned}$$

where $D$ is cocoercive and $L$ is a linear operator.

3. Stochastic and Variance-Reduced FBHF

Recent advances incorporate stochastic and variance-reduced computations into the FBHF framework, critical for large-scale and finite-sum problems (Qin et al., 2023, Qin et al., 28 Oct 2025). In these regimes, the operator $B$ is accessed only through a stochastic oracle $B_\xi$ with unbiasedness and bounded variance: $$\mathbb{E}[ B_\xi(x) ] = B(x), \qquad \mathbb{E} \left[ \| B_\xi(u) - B_\xi(v) \|^2 \right] \leq L^2 \| u-v \|^2.$$

The variance-reduced FBHF (VRFBHF) methodology uses a Polyak-like averaging or momentum on the reference point and employs a control variate decomposed update: $$\begin{aligned} \bar{x}^{k} &= \lambda x^k + (1-\lambda) w^k, \ y^k &= J_{\gamma A}( \bar{x}^k - \gamma ( B + C ) w^k ), \ x^{k+1} &= y^k + \gamma [ B_{\xi_k} w^k - B_{\xi_k} y^k ], \ w^{k+1} &= \begin{cases} x^{k+1}, & \mathrm{w/prob.}\;p,\ w^k, & \mathrm{w/prob.}\;1-p. \end{cases} \end{aligned}$$ Here, $\lambda\in(0,1)$ controls averaging, $p$ the frequency of reference updates. This update is robust to high variance and allows for much smaller $p$ (reference seldom updated) in large-scale settings, substantially reducing total oracle calls compared to standard FBHF.

Stochastic quasi-Fejér monotonicity and Lyapunov function methodologies underpin the convergence proofs and rate guarantees (Qin et al., 2023, Qin et al., 28 Oct 2025, Vũ, 2015).

4. Convergence Properties

Weak and Almost Sure Convergence

Under general monotonicity, cocoercivity, and step-size constraints, almost sure weak convergence to a solution is obtained for both deterministic and stochastic FBHF variants (Qin et al., 2023, Qin et al., 28 Oct 2025, Vũ, 2015). Specifically, with $\gamma$ satisfying

$$\gamma < \frac{4\beta(1-\lambda)}{1+\sqrt{1+16\beta^2 L^2(1-\lambda)}},$$

the VRFBHF iterates satisfy $$\mathbb{P}\left( \lim_{k\to\infty} x^k = x^* \right) = 1$$ for some $x^*\in\operatorname{zer}(A+B+C)$. Lyapunov functions of the type

$$\Phi_k(x) = \lambda \| x^k - x \|^2 + \frac{1-\lambda}{p} \| w^k - x \|^2$$

are shown to contract in expectation, enabling invocation of supermartingale convergence theorems.

Linear Convergence under Strong Monotonicity

When one operator is strongly monotone, explicit $Q$-linear rates are derived. For example, when $B$ is $\mu$-strongly monotone, with parameters $\lambda=1-p$, and stepsize $\gamma = \min\{ \sqrt{p}/(2L), \beta p \}$, the expected error decays as

$$\mathbb{E} \| x^k - x^* \|^2 \leq \left( \frac{1}{1+c/4} \right)^k \frac{2}{1-p} \| x^0 - x^* \|^2,$$

with explicit dependence on $\mu$ and $L$ (Qin et al., 2023, Qin et al., 28 Oct 2025). This result extends to nonlinear preconditioning and momentum-augmented settings and is novel for variance-reduced and stochastic operator-splitting algorithms.

5. Extensions and Generalizations

Nonlinear/Kernerlized and Projection-Corrected FBHF

Recent frameworks generalize FBHF to nonlinear and nonsymmetric kernels (nonlinear $M_k$), allowing for the backward step to be performed in a non-Euclidean or Bregman geometry, or over block and coordinatewise preconditioning. The NOFOB algorithm (Giselsson, 2019) achieves this via a nonlinear forward-backward resolvent and an additional relaxed projection onto a hyperplane that ensures Fejér monotonicity and enables potentially larger step sizes and faster convergence.

Four-Operator and Primal-Dual Extensions

FBHF can be embedded as a special case of generalized four-operator splitting algorithms combining maximally monotone, Lipschitz, cocoercive, and composite linear terms (Roldán, 2023). These generalizations recover algorithms such as Condat–Vu and AFBA and facilitate multivariate splitting, saddle-point problems, and applications with complex regularization.

Momentum and Acceleration

Augmenting FBHF with inertia/momentum, both linear and nonlinear, is shown to be efficient both theoretically and empirically for accelerating convergence and handling ill-conditioning. These momentum terms interact with the splitting framework in nontrivial ways and require tailored Lyapunov-based analysis (Qin et al., 28 Oct 2025).

6. Practical Applications and Numerical Performance

FBHF and its stochastic/variance-reduced variants are applied to broad classes of structured problems:

• Sparse and regularized convex optimization (e.g., LASSO, TV regularization),
• Composite saddle-point and constrained minimax learning,
• Stochastic finite-sum and large-scale machine learning problems,
• Imaging and signal processing (e.g., deblurring, denoising) with primal-dual composite regularizers (Roldán, 2023),
• Portfolio optimization and quadratic programming with blockwise and sparse constraints (Qin et al., 28 Oct 2025).

Empirical studies demonstrate that VRFBHF and momentum-based variants achieve lower iteration counts and wall-clock times, particularly for large $N$ or expensive operator evaluations, and outperform standard forward-backward or extragradient-type schemes. This advantage is pronounced for problems with composite operators, high variance, or ill-conditioned constraint structure.

7. Relation to Other Operator Splitting and Optimization Methods

FBHF encompasses and extends several classic splitting schemes:

• Forward-backward splitting: recovered by omitting the half-forward correction.
• Forward-backward-forward (FBF): a special case of FBHF for two-operator inclusions.
• Extragradient and Mirror-Prox methods: related via the structure of evaluation points and correction steps.
• Nonlinear forward-backward splitting with projection correction: generalizes FBHF to allow for nonlinear, nonsymmetric kernels with relaxed projections (Giselsson, 2019).

FBHF admits integration with stochastic and variance-reduced oracles (SVRG, SAGA, SARAH), allowing tuning for specific data structure and evaluation cost (Qin et al., 2023, Qin et al., 28 Oct 2025). The theoretical foundations leverage stochastic quasi-Fejér convergence and Lyapunov descent arguments, supporting robust convergence—even in the presence of randomness, nonlinearity, or inexactness.

---

In summary, the forward-backward-half forward splitting algorithm and its recent generalizations define a flexible, theoretically principled, and computationally efficient family of methods for large-scale and structured monotone inclusions, offering significant advantages in convergence rate, oracle complexity, and practical implementability over classical splitting schemes, particularly in stochastic, varianced-reduced, and momentum-accelerated regimes (Giselsson, 2019, Roldán, 2023, Qin et al., 2023, Qin et al., 28 Oct 2025).