Bregman Projection Algorithms

• Bregman Projection Algorithms are iterative methods that use Bregman divergence—a non-Euclidean distance from strictly convex functions—to generalize classic projection techniques.
• They employ advanced schemes such as plug-and-play proximal gradients, split Bregman, and alternating projections to solve composite and feasibility problems with improved performance.
• The algorithms ensure global convergence with linear/sublinear rates and find applications in inverse problems, matrix recovery, optimal transport, and quantum information.

A Bregman projection algorithm refers to a family of iterative optimization and feasibility methods grounded in the concept of Bregman divergence, a class of non-Euclidean “distance” measures derived from strictly convex, differentiable functions. These algorithms generalize classic projection (and proximal) techniques—traditionally based on the Euclidean norm—by leveraging geometries adapted to the structure of the underlying problem, resulting in improved performance and convergence guarantees in diverse settings such as inverse problems, matrix recovery, statistical inference, and large-scale optimization.

1. Definition of Bregman Divergence and Proximal Mapping

Given a strictly convex, continuously differentiable function $\phi:\operatorname{dom}\,\phi \to \mathbb{R}$ (the “potential”), the Bregman divergence (distance) between $x$ and $y$ is

$$D_\phi(x, y) = \phi(x) - \phi(y) - \langle\nabla \phi(y), x - y\rangle$$

This function is always nonnegative and recovers the squared Euclidean distance when $\phi(x) = \frac12\|x\|_2^2$. Other choices for $\phi$ yield divergences more aligned with problem geometry—for example, negative Shannon entropy induces the Kullback–Leibler (KL) divergence.

The Bregman proximal (or projection) operator for a convex function $f$ is defined as

$$\operatorname{prox}^{\phi}_{\lambda f}(y) := \arg\min_x \left\{ \lambda f(x) + D_{\phi}(x, y) \right\}$$

For an indicator function $f = \delta_C$ of a closed convex set $C$, this reduces to the Bregman projection onto $C$: $P_C^\phi(y) := \arg\min_{x \in C} D_\phi(x, y)$ These fundamental definitions unify a wide family of projection and splitting schemes, extending beyond Euclidean geometry to settings such as the simplex and the positive orthant (Al-Shabili et al., 2022).

2. Algorithmic Schemes and Prototypical Frameworks

Bregman projection algorithms materialize in several canonical forms across convex optimization, inverse problems, and machine learning:

2.1. Plug-and-Play Bregman Proximal Gradient (PnP–BPGM)

The PnP–BPGM algorithm tackles composite minimization $f(x)+g(x)$, replacing the proximal step for $g$ with a learned denoiser or other operator $D_\theta$: $$\begin{aligned} \text{Mirror descent:} \quad & z^k = \nabla\phi^*\left( \nabla\phi(x^k) - \gamma\nabla f(x^k) \right) \ \text{Prox/denoising:} \quad & x^{k+1} = D_\theta(z^k) \end{aligned}$$ Equivalently, if $g$ is prox-evaluable, $$x^{k+1} = \operatorname{prox}^{\phi}_{\gamma g}(z^k)$$. Convergence is guaranteed under strong convexity and Lipschitz conditions on $\phi$, $f$, and $D_\theta$ (Al-Shabili et al., 2022).

2.2. Split Bregman for Matrix Recovery

The Split Bregman algorithm for nuclear-norm minimization alternates between:

• Data-fidelity projection (quadratic penalty minimization)
• Proximal mapping for the nonsmooth term (singular value soft-thresholding)
• Dual (Bregman) variable update enforcing constraint consistency This structure yields monotonic descent and asymptotic feasibility, paralleling ADMM with non-Euclidean geometry (Gogna et al., 2013).

2.3. Alternating Bregman Projections

For feasibility over intersections $C\cap D$, iterates alternate: $$y_{k}\;\xrightarrow{P^{\phi}_{C}}\;x_{k+1}\;\xrightarrow{P^{\phi}_{D}}\;y_{k+1}$$ Global convergence is achieved under mild geometric and regularity conditions, and local $R$-linear rates obtain under transversality (Noll, 29 Jul 2025, Bargetz et al., 2019).

2.4. Iterative Bregman Projection in Optimal Transport

Entropic-regularized OT problems reduce to cycles of closed-form KL projections (e.g., Sinkhorn's algorithm) onto constraint sets, or more generally to Bregman–Dykstra iterations for intersections of convex sets (Benamou et al., 2014, Kostic et al., 2021).

2.5. Generalized and Shrinking Bregman-Projections in Banach Spaces

Advanced frameworks support constraints from equilibrium problems, maximal monotone inclusions, and infinite families of fixed-point constraints, typically via nested Bregman projections combined with (potentially) inertial and extragradient steps (Orouji et al., 2022, Sababe et al., 14 May 2025, Ghadampour et al., 2021).

3. Geometric and Variational Properties

Bregman projection algorithms leverage strict convexity to ensure:

• Uniqueness: $P_C^\phi(y)$ is uniquely defined for closed convex $C$ and strictly convex, differentiable $\phi$.
• Optimality: $z = P_C^\phi(y)$ satisfies

$$\forall x \in C,\quad \langle \nabla\phi(y) - \nabla\phi(z), x - z \rangle \leq 0$$

• Monotone Descent: For solutions $x^*\in C$, $$D_\phi(x^*, x_{k+1}) \leq D_\phi(x^*, x_k) - D_\phi(x_{k+1}, x_k)$$ (Fejér-monotonicity). This structure underlies strong and weak convergence guarantees (Bauschke et al., 2013, Kostic et al., 2021).
• Three-Point Identity: For any $x,y,z$,

$$D_\phi(x, y) = D_\phi(x, z) - D_\phi(y, z) + \langle \nabla\phi(z) - \nabla\phi(y), x - y \rangle$$

This quantitative geometry is central to convergence analysis in both deterministic and stochastic variants (Noll, 29 Jul 2025, Yuan et al., 2021).

4. Convergence Theory and Rates

The convergence analysis of Bregman projection algorithms is well-developed:

• Linear and Sublinear Rates: For affine feasibility and favorable geometric conditions (e.g., linear regularity, transversality, uniform convexity/smoothness), global $Q$-linear convergence rates are achieved. In Banach and Hilbert settings, the rate depends on modulus of convexity, smoothness, and regularity constants (Kostic et al., 2021, Bargetz et al., 2019).
• Alternating Projections: Alternating Bregman projections between two (possibly non-convex) sets converge either to a point in the intersection or, in the infeasible case, to the closest pair realizing a gap; sublinear rates are generic, local linear rates are tied to geometric “angle”/transversality (Noll, 29 Jul 2025).
• Stochastic/Adaptive Variants: In randomized and sketching schemes, contraction per step is quantified via spectral constants of sketched losses or averaged projectors (Yuan et al., 2021).
• Fejér Monotonicity: Monotonicity in Bregman distance yields boundedness and summability of distances to the solution set, supporting strong and weak convergence (Bauschke et al., 2013, Orouji et al., 2022, Ghadampour et al., 2021).

5. Applications and Extensions

Bregman projection algorithms underpin a broad range of applications:

• Inverse Problems and Denoising: PnP–BPGM and RED–Bregman Steepest Descent deploy Bregman geometry to integrate learned denoisers in non-Euclidean Proximal Gradient frameworks, broadening the expressivity and convergence robustness of plug-and-play regularization (Al-Shabili et al., 2022).
• Matrix Recovery and Compressed Sensing: The Split Bregman method for low-rank matrix recovery exhibits superior convergence and accuracy compared to classical methods via alternated Bregman projections and dual variable correction (Gogna et al., 2013).
• Optimal Transport and Barycenters: Entropic regularization renders OT constraints convex and tractable by Bregman projection (KL-divergence); iterative proportional fitting procedures (Sinkhorn/Greenkhorn) are direct instances (Benamou et al., 2014, Kostic et al., 2021).
• Nonlinear Equations and Sparse Recovery: Bregman–Kaczmarz and linearized Bregman methods generalize classic projection for (sparse) solution of (non)linear systems, with adaptivity to problem geometry for improved conditioning and rate (Gower et al., 2023, Lorenz et al., 2013).
• Composite Problems in Banach Spaces: Shrinking Bregman projection algorithms and their variants address equilibrium, monotone inclusions, and (infinite) families of fixed-point problems, with strong convergence in uniform convexity (Orouji et al., 2022, Sababe et al., 14 May 2025).
• Quantum Information and Noncommutative Optimization: Matrix Legendre–Bregman projections extend the framework to matrix settings (e.g., maximum entropy inference in quantum systems), and quantum-accelerated algorithms exploit the structure for speedup (Ji, 2022).
• Self-Supervised Learning Stability: Stochastic Bregman projection processes in distribution space explain model collapse in self-referential learning, and entropy-reservoir variants provide theoretical quantification and unification of stabilization heuristics (Chen, 16 Dec 2025).

6. Canonical Examples and Implementation Details

Table: Bregman Divergence and Associated Projections

$\phi(x)$	$D_\phi(x, y)$	Application/Projection
$\frac12\|x\|_2^2$	$\frac12\|x - y\|_2^2$	Euclidean, standard prox
$\sum_i x_i \log x_i$	$\sum_i x_i \log(x_i/y_i) - x_i + y_i$ (KL)	Simplex, Sinkhorn, OT
$-\sum_i \log x_i$	$\sum_i(\frac{x_i}{y_i} - \log \frac{x_i}{y_i} - 1)$	Positive orthant, Itakura–Saito
$\lambda \|x\|_1 + \frac12\|x\|_2^2$	$\lambda (\|x\|_1 - \|y\|_1) + \frac12\|x-y\|_2^2$	Sparse prox, linearized Bregman

Implementation remarks include direct closed-forms for projection onto affine/simplex sets under KL divergence, multiplicative updates for nonnegativity constraints, and globalized Newton/bisection for 1D convex solves in Bregman–Kaczmarz (Gower et al., 2023, Benamou et al., 2014, Al-Shabili et al., 2022).

7. Theoretical Unity and Research Directions

Recent research elucidates a unified perspective in which classical, stochastic, and deep-learning-motivated projection methods are realized as Bregman projection iterations under suitable geometry:

• Broad convergence theory, encompassing nonconvex set projections given tame geometry (e.g., definable sets, prox-regularity) (Noll, 29 Jul 2025).
• Quantitative entropy contractivity and stabilization in closed-loop learning systems derives analytically from the Bregman framework (Chen, 16 Dec 2025).
• Quantum generalizations and noncommutative divergences extend applicability to operator-theoretic and information-theoretic domains (Ji, 2022).

The versatility and generality of Bregman projections continue to fuel research in algorithmic design for high-dimensional, structured, and non-Euclidean optimization domains.