Bregman Monotone Operator Splitting

• Bregman monotone operator splitting is a framework that replaces Euclidean metrics with Bregman divergences to solve monotone inclusions and convex optimization problems.
• It generalizes classical splitting methods, including Douglas–Rachford and ADMM, to non-Euclidean and Banach space settings for improved algorithmic flexibility and convergence rates.
• Practical applications span optimal transport, imaging, and machine learning, offering scalable solutions through tailored geometries and variable-metric methods.

Bregman monotone operator splitting is a broad generalization of classical operator splitting techniques for monotone inclusions and convex optimization. By replacing the standard Euclidean metric with a Bregman divergence induced by a Legendre function, these methods yield algorithms with improved flexibility, scalability, and convergence properties, especially in non-Euclidean geometries. This framework encompasses a family of resolvent-splitting and primal-dual formulations, generalizes proximal algorithms, and underpins scalable methods for structured learning, optimal transport, and imaging.

1. Foundations: Bregman Divergences and Monotone Inclusions

Let $$h: \operatorname{int} \operatorname{dom} h \to \mathbb{R}$$ be a Legendre function—i.e., essentially smooth and strictly convex. The associated Bregman divergence is defined as

$$D_h(x, y) = h(x) - h(y) - \langle \nabla h(y), x - y \rangle.$$

When $h(x) = \frac{1}{2} \|x\|_2^2$, this reduces to the squared Euclidean distance, but other $h$ yield non-Euclidean penalizations (e.g., Boltzmann–Shannon entropy gives Kullback–Leibler divergence).

Given maximally monotone operators $A, B$, the monotone inclusion $0 \in A(x) + B(x)$ defines the canonical splitting context. Classical Douglas–Rachford, Peaceman–Rachford, and Forward–Backward schemes apply only in Hilbert spaces and depend on Euclidean proximation.

Bregman monotone operator splitting generalizes these by employing Bregman resolvents:

$J_T^h := (\nabla h + T)^{-1} \circ \nabla h.$

This formulation allows the splitting of hard-to-proximal operators in tailored geometries or structured product spaces, and extends to Banach spaces and beyond.

2. Bregman Douglas–Rachford Splitting and Generalized Algorithms

The Bregman Douglas–Rachford splitting (BDRS) (Ma et al., 10 Sep 2025, Niwa et al., 2018) defines the core iterative scheme as follows:

• Define Bregman resolvents $J_{T}^h$, Bregman reflections $$R_{T}^h = \nabla h^* \circ (2 \nabla h \circ J_{T}^h - \nabla h)$$, and Mann relaxations.
• The generic BDRS iteration in mirror space is

$$z^{k+1} = M_1^h (R_{\gamma_k A}^h R_{\gamma_k B}^h)(z^k),$$

with $$D_h(x, y) = h(x) - h(y) - \langle \nabla h(y), x - y \rangle.$$0 a Bregman Mann operator.

Equivalently, in primal variables, for each $$D_h(x, y) = h(x) - h(y) - \langle \nabla h(y), x - y \rangle.$$1: $$D_h(x, y) = h(x) - h(y) - \langle \nabla h(y), x - y \rangle.$$2 Each subproblem is a strongly convex minimization with respect to $$D_h(x, y) = h(x) - h(y) - \langle \nabla h(y), x - y \rangle.$$3:

$$D_h(x, y) = h(x) - h(y) - \langle \nabla h(y), x - y \rangle.$$4

Other variants, such as Bregman Peaceman–Rachford splitting (BPRS) and double-backward Bregman splitting, follow from analogous operator compositions, ensuring algorithmic diversity for composite and saddle-point problems (Ma et al., 10 Sep 2025, Jiang et al., 2022, Niwa et al., 2018).

3. Theoretical Guarantees and Convergence Theory

Convergence theory for Bregman monotone operator splitting is grounded in the geometry induced by $$D_h(x, y) = h(x) - h(y) - \langle \nabla h(y), x - y \rangle.$$5, the monotonicity of $$D_h(x, y) = h(x) - h(y) - \langle \nabla h(y), x - y \rangle.$$6, and step-size control. Key results (Ma et al., 10 Sep 2025, Bùi et al., 2019, Nguyen, 2015) include:

• For strongly convex $$D_h(x, y) = h(x) - h(y) - \langle \nabla h(y), x - y \rangle.$$7 and Lipschitz (or relatively smooth) operators:
  • With constant step-size $$D_h(x, y) = h(x) - h(y) - \langle \nabla h(y), x - y \rangle.$$8 (for $$D_h(x, y) = h(x) - h(y) - \langle \nabla h(y), x - y \rangle.$$9 relative smoothness), convergence rates $h(x) = \frac{1}{2} \|x\|_2^2$0 for objective values are obtained.
  • For nonsmooth cases and diminishing step-sizes $h(x) = \frac{1}{2} \|x\|_2^2$1, an ergodic rate $h(x) = \frac{1}{2} \|x\|_2^2$2 is established, with $h(x) = \frac{1}{2} \|x\|_2^2$3 and Bregman gap vanishing.
• The iterates $h(x) = \frac{1}{2} \|x\|_2^2$4 remain in $h(x) = \frac{1}{2} \|x\|_2^2$5 and every cluster point solves $h(x) = \frac{1}{2} \|x\|_2^2$6.
• The class of variable quasi-Bregman monotone sequences provides a Banach-space generalization of Fejér monotonicity, essential for cluster-point analysis and convergence in both weak and strong senses (Nguyen, 2015, Bùi et al., 2019).

Strong convergence and stability obtain even in Banach spaces, under Legendre properties, without requiring global Lipschitz constants or strong monotonicity of all operators (Combettes et al., 2015). This highlights a fundamental advance relative to Hilbert-space-only theory.

4. Equivalence to Bregman ADMM and Exponential Multiplier Methods

Applying BDRS to dual linearly constrained convex problems yields Bregman ADMM:

• For $h(x) = \frac{1}{2} \|x\|_2^2$7, the dual problem's optimality condition $h(x) = \frac{1}{2} \|x\|_2^2$8 with suitable embeddings results in the iteration

$h(x) = \frac{1}{2} \|x\|_2^2$9

Here, $h$0 denotes the convex conjugate.

A notable specialization is when $h$1 is the Boltzmann–Shannon entropy, so $h$2 is the Kullback–Leibler divergence. In this setting, the iteration recovers the classical exponential-multiplier method (EMM) and, when alternating between $h$3 and $h$4, produces the alternating-direction exponential multiplier method (ADEMM)—a Bregman analogue of ADMM. The ADEMM provides closed-form multiplicative updates and, in the discrete OT LP, yields Sinkhorn-like scaling steps without the numerical difficulties associated with $h$5 in classical Sinkhorn regularization (Ma et al., 10 Sep 2025).

5. Design of the Bregman Kernel and Adaptivity

The function $h$6 selects the geometry and preconditioning of the method:

• Quadratic choices (e.g., $h$7) yield variable-metric methods (Ma et al., 10 Sep 2025).
• Entropic, Burg, or IS kernels yield multiplicative updates or address positive constraints and simplex structures (Nguyen, 2015, Niwa et al., 2018).
• Newton-style or adaptive kernels can be set via local Hessians for fast convergence in specific problem instances (Niwa et al., 2018).

Algorithmic flexibility is further enhanced by permitting iteration-dependent kernels $h$8, giving rise to variable-kernel Bregman FB or more sophisticated operator splitting schemes (Bùi et al., 2019).

6. Applications and Implementation Considerations

Bregman monotone operator splitting delivers computational gains across multiple domains (Ma et al., 10 Sep 2025, Lazzaro et al., 2018, Jiang et al., 2022):

• Optimal Transport: ADEMM efficiently solves discrete OT linear programs, stabilizing computation relative to standard entropy-regularized approaches and enabling scaling to large cost matrices.
• Imaging and Inverse Problems: Fast Split Bregman and weighted TV denoising leverage matrix-split Bregman inner solvers, providing accelerated convergence for large-scale imaging problems (Lazzaro et al., 2018).
• Machine Learning: Adaptive Bregman splitting supports composite regularization, relative entropy penalties, and scalable multi-block structures in learning pipelines (Niwa et al., 2018, Bùi et al., 2019).
• Primal-Dual and Three-Operator Splitting: Bregman generalizations of Condat–Vũ and PD3O algorithms extend to fully nonsmooth and constraint-rich settings, while maintaining convergence rates and feasibility (Jiang et al., 2022, Combettes et al., 2015).

Closed-form Bregman projections are available for separable entropic and Fermi–Dirac geometries. However, efficient implementation may demand specialized linear algebra, especially in high-dimensional or block-structured problems.

7. Extensions, Limitations, and Future Directions

The Bregman operator splitting paradigm establishes a unified analytical and algorithmic foundation:

• It applies equally to Hilbert, Banach, and product spaces; seamlessly integrates non-Euclidean and variable-metric geometries; and achieves strong convergence results under mild assumptions, even for multi-block or monotone-inclusion models (Combettes et al., 2015).
• Extensions to three-operator splitting, primal-dual line search, and adaptive stepsizes have been rigorously developed (Jiang et al., 2022).
• Bregman-based splitting is particularly advantageous when Euclidean proximal steps are intractable but Bregman projections are simple or closed-form.

Limitations include the computational burden of non-Euclidean Bregman projections and the necessity of Legendre kernel structures, which may limit applicability to highly nonsmooth or non-strictly-convex geometries (Combettes et al., 2015). Nevertheless, current research continues to expand the theoretical and computational toolkit, fueling advances in scalable convex optimization, structured learning, and large-scale computational mathematics.

---

Key references:

• "Bregman Douglas-Rachford Splitting Method" (Ma et al., 10 Sep 2025)
• "Bregman Forward-Backward Operator Splitting" (Bùi et al., 2019)
• "Bregman Monotone Operator Splitting" (Niwa et al., 2018)
• "Solving Composite Monotone Inclusions... Best Bregman Approximations from their Kuhn-Tucker Set" (Combettes et al., 2015)
• "Bregman three-operator splitting methods" (Jiang et al., 2022)
• "Forward-Backward Splitting with Bregman Distances" (Nguyen, 2015)
• "A Fast Splitting Method for efficient Split Bregman Iterations" (Lazzaro et al., 2018)