---
title: FBHF Splitting Algorithm
url: https://www.emergentmind.com/topics/forward-backward-half-forward-splitting-algorithm
type: topic
---

# FBHF Splitting Algorithm

The forward-backward-half forward splitting algorithm (FBHF) is a family of operator splitting algorithms for structured monotone inclusion and variational inequality problems. FBHF exploits the composite nature of problems involving a maximally monotone operator, a monotone (possibly Lipschitz) operator, and a cocoercive operator by interleaving resolvent-based (backward) steps with explicit (forward) evaluations and additional correction steps that leverage Lipschitz or block structure. Recent advances extend the algorithm to nonlinear settings, stochastic/variance-reduced regimes, multivariate/primal-dual forms, and momentum/inertia-augmented algorithms.

## 1. Mathematical Framework and Operator Structure

FBHF is concerned with monotone inclusions in a Hilbert space $\mathcal{H}$, targeting the structured root-finding problem:
\[
\text{Find } x \in \mathcal{H} \text{ such that } 0 \in Ax + Bx + Cx,
\]
where:
- $A : \mathcal{H} \to 2^{\mathcal{H}}$ is maximally monotone (e.g., subdifferential $\partial G$ or normal cone),
- $B : \mathcal{H} \to \mathcal{H}$ is monotone, typically $L$-Lipschitz, often decomposed as a finite sum $B = \sum_{i=1}^N B_i$ for block or finite-sum structure,
- $C : \mathcal{H} \to \mathcal{H}$ is cocoercive, i.e., there exists $\beta > 0$ so that
\[
\langle Cx - Cy, x - y \rangle \geq \beta \| Cx - Cy \|^2, \;\; \forall x,y.
\]
This triple-splitting model generalizes classical scenarios such as convex minimization with non-smooth regularization/constraints, composite variational inequalities, and primal-dual saddle-point problems.

FBHF naturally extends to four-operator settings with additional Lipschitz and linear-composite terms, as demonstrated in [2310.17265], and allows nonlinear preconditioning as in [1908.07449, 2510.24489].

## 2. Core Algorithmic Structure

The canonical FBHF algorithm performs, per iteration, a maximal monotone resolvent (backward) step, a forward step with the Lipschitz operator, and a half-forward correction to offset lack of cocoercivity in $B$. For deterministic settings, one typical iteration has the update (see [2310.17265], [2510.24489]):
\[
\begin{aligned}
y_k & = J_{\gamma_k A}\left( x_k - \gamma_k( Bx_k + Cx_k ) \right), \\
x_{k+1} & = y_k - \gamma_k B y_k + \gamma_k B x_k.
\end{aligned}
\]
In algorithms with nonlinear or block preconditioning (see [1908.07449], [2510.24489]), the kernel $M_k$ for the resolvent $J_{\gamma_k M_k+A}$ may be nonlinear or nonsymmetric, enabling further flexibility:
\[
y_k = (M_k + A)^{-1} \left( M_k x_k - (B + C) x_k + \gamma_k^{-1} u_k \right).
\]
Momentum or inertia terms, i.e., variables such as $u_k$ and inertia updates, can be explicitly included for acceleration and improved empirical performance.

For four-operator and primal-dual variants (see [2310.17265]), FBHF can be structured as
\[
\begin{aligned}
p_{n+1} & = C x_n, \\
z_{n+1} & = J_{\tau A} \left( x_n - \tau ( L^* u_n + p_{n+1} + D x_n ) \right), \\
q_{n+1} & = \tau ( C z_{n+1} - p_{n+1} ), \\
u_{n+1} & = J_{\sigma B^{-1}} \left( u_n + \sigma L (2z_{n+1} - x_n - q_{n+1}) \right), \\
x_{n+1} & = z_{n+1} - q_{n+1},
\end{aligned}
\]
where $D$ is cocoercive and $L$ is a linear operator.

## 3. Stochastic and Variance-Reduced FBHF

Recent advances incorporate stochastic and variance-reduced computations into the FBHF framework, critical for large-scale and finite-sum problems ([2312.00272], [2510.24489]). In these regimes, the operator $B$ is accessed only through a stochastic oracle $B_\xi$ with unbiasedness and bounded variance:
\[
\mathbb{E}[ B_\xi(x) ] = B(x), \qquad \mathbb{E} \left[ \| B_\xi(u) - B_\xi(v) \|^2 \right] \leq L^2 \| u-v \|^2.
\]

The variance-reduced FBHF (VRFBHF) methodology uses a Polyak-like averaging or momentum on the reference point and employs a control variate decomposed update:
\[
\begin{aligned}
\bar{x}^{k} &= \lambda x^k + (1-\lambda) w^k, \\
y^k &= J_{\gamma A}( \bar{x}^k - \gamma ( B + C ) w^k ), \\
x^{k+1} &= y^k + \gamma [ B_{\xi_k} w^k - B_{\xi_k} y^k ], \\
w^{k+1} &=
\begin{cases}
x^{k+1}, & \mathrm{w/prob.}\;p,\\
w^k, & \mathrm{w/prob.}\;1-p.
\end{cases}
\end{aligned}
\]
Here, $\lambda\in(0,1)$ controls averaging, $p$ the frequency of reference updates. This update is robust to high variance and allows for much smaller $p$ (reference seldom updated) in large-scale settings, substantially reducing total oracle calls compared to standard FBHF.

Stochastic quasi-Fejér monotonicity and Lyapunov function methodologies underpin the convergence proofs and rate guarantees ([2312.00272], [2510.24489], [1505.05118]).

## 4. Convergence Properties

### Weak and Almost Sure Convergence

Under general monotonicity, cocoercivity, and step-size constraints, almost sure weak convergence to a solution is obtained for both deterministic and stochastic FBHF variants ([2312.00272], [2510.24489], [1505.05118]). Specifically, with $\gamma$ satisfying
\[
\gamma < \frac{4\beta(1-\lambda)}{1+\sqrt{1+16\beta^2 L^2(1-\lambda)}},
\]
the VRFBHF iterates satisfy $\mathbb{P}\left( \lim_{k\to\infty} x^k = x^* \right) = 1$ for some $x^*\in\operatorname{zer}(A+B+C)$. Lyapunov functions of the type
\[
\Phi_k(x) = \lambda \| x^k - x \|^2 + \frac{1-\lambda}{p} \| w^k - x \|^2
\]
are shown to contract in expectation, enabling invocation of supermartingale convergence theorems.

### Linear Convergence under Strong Monotonicity

When one operator is strongly monotone, explicit $Q$-linear rates are derived. For example, when $B$ is $\mu$-strongly monotone, with parameters $\lambda=1-p$, and stepsize $\gamma = \min\{ \sqrt{p}/(2L), \beta p \}$, the expected error decays as
\[
\mathbb{E} \| x^k - x^* \|^2 \leq \left( \frac{1}{1+c/4} \right)^k \frac{2}{1-p} \| x^0 - x^* \|^2,
\]
with explicit dependence on $\mu$ and $L$ ([2312.00272], [2510.24489]). This result extends to nonlinear preconditioning and momentum-augmented settings and is novel for variance-reduced and stochastic operator-splitting algorithms.

## 5. Extensions and Generalizations

### Nonlinear/Kernerlized and Projection-Corrected FBHF

Recent frameworks generalize FBHF to nonlinear and nonsymmetric kernels (nonlinear $M_k$), allowing for the backward step to be performed in a non-Euclidean or Bregman geometry, or over block and coordinatewise preconditioning. The NOFOB algorithm ([1908.07449]) achieves this via a nonlinear forward-backward resolvent and an additional relaxed projection onto a hyperplane that ensures Fejér monotonicity and enables potentially larger step sizes and faster convergence.

### Four-Operator and Primal-Dual Extensions

FBHF can be embedded as a special case of generalized four-operator splitting algorithms combining maximally monotone, Lipschitz, cocoercive, and composite linear terms ([2310.17265]). These generalizations recover algorithms such as Condat–Vu and AFBA and facilitate multivariate splitting, saddle-point problems, and applications with complex regularization.

### Momentum and Acceleration

Augmenting FBHF with inertia/momentum, both linear and nonlinear, is shown to be efficient both theoretically and empirically for accelerating convergence and handling ill-conditioning. These momentum terms interact with the splitting framework in nontrivial ways and require tailored Lyapunov-based analysis ([2510.24489]).

## 6. Practical Applications and Numerical Performance

FBHF and its stochastic/variance-reduced variants are applied to broad classes of structured problems:
- Sparse and regularized convex optimization (e.g., LASSO, TV regularization),
- Composite saddle-point and constrained minimax learning,
- Stochastic finite-sum and large-scale machine learning problems,
- Imaging and signal processing (e.g., deblurring, denoising) with primal-dual composite regularizers ([2310.17265]),
- Portfolio optimization and quadratic programming with blockwise and sparse constraints ([2510.24489]).

Empirical studies demonstrate that VRFBHF and momentum-based variants achieve lower iteration counts and wall-clock times, particularly for large $N$ or expensive operator evaluations, and outperform standard forward-backward or extragradient-type schemes. This advantage is pronounced for problems with composite operators, high variance, or ill-conditioned constraint structure.

## 7. Relation to Other Operator Splitting and Optimization Methods

FBHF encompasses and extends several classic splitting schemes:
- Forward-backward splitting: recovered by omitting the half-forward correction.
- Forward-backward-forward (FBF): a special case of FBHF for two-operator inclusions.
- Extragradient and Mirror-Prox methods: related via the structure of evaluation points and correction steps.
- Nonlinear forward-backward splitting with projection correction: generalizes FBHF to allow for nonlinear, nonsymmetric kernels with relaxed projections ([1908.07449]).

FBHF admits integration with stochastic and variance-reduced oracles (SVRG, SAGA, SARAH), allowing tuning for specific data structure and evaluation cost ([2312.00272], [2510.24489]). The theoretical foundations leverage stochastic quasi-Fejér convergence and Lyapunov descent arguments, supporting robust convergence—even in the presence of randomness, nonlinearity, or inexactness.

---

In summary, the forward-backward-half forward splitting algorithm and its recent generalizations define a flexible, theoretically principled, and computationally efficient family of methods for large-scale and structured monotone inclusions, offering significant advantages in convergence rate, oracle complexity, and practical implementability over classical splitting schemes, particularly in stochastic, varianced-reduced, and momentum-accelerated regimes [1908.07449, 2310.17265, 2312.00272, 2510.24489].

Source: https://www.emergentmind.com/topics/forward-backward-half-forward-splitting-algorithm