---
title: Alternating Bregman Projections
url: https://www.emergentmind.com/topics/alternating-bregman-projections
type: topic
---

# Alternating Bregman Projections

Alternating Bregman Projections (ABP) are a fundamental generalization of the classical method of alternating projections, replacing the Euclidean metric with divergences generated by strictly convex (Legendre-type) functions. This framework subsumes numerous important algorithms, including projection methods in convex optimization, iterative regularization schemes, and the Expectation-Maximization (EM) algorithm in statistics, notably when using Kullback–Leibler (KL) divergence as the underlying geometry. ABP methods have a geometric foundation that enables convergence and rate guarantees well beyond convex settings, including nonconvex, tame, and semi-algebraic constraints [2507.21840].

## 1. Mathematical Foundation and Definitions

Central to ABP is the Bregman divergence, $D_\phi(x, y) = \phi(x) - \phi(y) - \langle \nabla \phi(y), x - y \rangle$, where $\phi : \mathbb{R}^d \to \mathbb{R} \cup \{+\infty\}$ is a Legendre function—proper, strictly convex, and twice continuously differentiable on the interior of its domain [2507.21840]. For $y$ in the interior of $\dom\phi$, Bregman divergence quantifies non-symmetric, non-metric "distance" between $x$ and $y$. 

Given a closed set $C$ in $\dom\phi$, two projections are defined:
- Left-projection: $P_C^\ell(a) = \arg\min_{z \in C} D_\phi(z, a)$ for $a \in \Int(\dom\phi)$.
- Right-projection: $P_A^r(b) = \arg\min_{z \in A} D_\phi(b, z)$ for $b \in \dom\phi$.

Alternating projections correspond to sequential minimization of Bregman divergences over two sets $A, B \subset \Int(\dom\phi)$.

## 2. General Algorithmic Structure

Let $A, B \subset \Int(\dom\phi)$ be closed sets and $a_0 \in A$ an initial point. The ABP method generates sequences $(a_k) \subset A$ and $(b_k) \subset B$ according to:

1. $b_k \in P_B^\ell(a_k)$,
2. $a_{k+1} \in P_A^r(b_k)$,

with termination when $\|a_{k+1} - a_k\| + \|b_k - b_{k-1}\|$ falls below a given tolerance [2507.21840]. The update contractivity is monitored by the monotonicity $D_\phi(b_k, a_k) \ge D_\phi(b_k, a_{k+1}) \ge D_\phi(b_{k+1}, a_{k+1})$, ensuring the Bregman distances form a non-increasing sequence.

In convex settings, each projection is well-defined and single-valued, but ABP has been generalized to handle nonconvex but tame (definable, prox-regular) sets, provided certain geometric and interiority conditions [2507.21840].

## 3. Convergence Theory and Geometric Conditions

The global convergence of ABP is secured by two key geometric hypotheses:

- **Angle condition**: A type of Kurdyka–Łojasiewicz (KL) inequality, specifying that for any subsequential accumulation point $(b^*, a^*)$, the angular separation of update directions does not degenerate too fast, i.e.,
  $$
  \frac{1-\cos\alpha}{\sigma(D_\phi(b^+, a^+) - \frac{1}{2} r^{*2})} \ge \gamma > 0
  $$
  for shrinking function $\sigma$ and parameter $\gamma$. When $\sigma(s) = s^\theta$, the rate exponent $\theta \in [1/2, 1)$ captures transversality or tangentiality [2507.21840, 1605.08290].

- **Three-point inequality**: Near any gap pair, it is required that
  $$
  D_\phi(b, a^+) \ge D_\phi(b^+, a^+) + \ell D_\phi(b, b^+),\quad \ell \in (0, 1].
  $$
  This controls local contraction. Convex sets automatically satisfy this with $\ell=1$ [2507.21840].

**Main result**: Under these two geometric conditions, $(b_k)$ converges to a unique $b^* \in B$, and if $A \cap B \neq \emptyset$ ($r^*=0$), then $(a_k)$ and $(b_k)$ converge to a common point in the intersection [2507.21840]. Feasible (zero-gap) and infeasible (positive-gap) cases are distinguished: in the latter, ABP converges to a gap-realizing pair that minimizes Bregman distance between $A$ and $B$.

## 4. Rates of Convergence

The rate at which $(b_k)$ and $(a_k)$ converge depends on the exponent $\theta$ from the angle (KL) condition:

- **General sublinear rate**: If $\phi'(s) = s^{-\theta}$, $\theta \in (1/2, 1)$, then
  $$
  \|b_k - b^*\| = O(k^{-\rho}), \quad \rho = \frac{1-\theta}{2\theta - 1} > 0.
  $$
- **R-linear (geometric) convergence**: If intersection is transversal, so that $\theta = 1/2$, the algorithm enjoys $O(q^k)$ convergence for some $q \in (0, 1)$ [2507.21840, 1605.08290, 1905.00605].

In Banach spaces that are uniformly convex and smooth of power type, explicit linear convergence rates are established for alternating Bregman projections onto closed linear subspaces, provided a suitable linear Bregman regularity (modulus) condition holds [1905.00605].

## 5. Connections to Optimization and Statistical Algorithms

The ABP method unifies several classical iterative schemes:

- **Expectation-Maximization (EM) Algorithm**: When the Bregman divergence is chosen as Kullback–Leibler (KL), the E-step is a left-Bregman projection onto the data-constraint set, and the M-step is a right-Bregman projection onto the model manifold. Under definability assumptions on the model and data sets, ABP guarantees convergence of EM-type algorithms, even for nonconvex parameter spaces, with the same rate theory as above [2507.21840].
- **Alternating minimization (AM, AAM, PALM)**: ABP encapsulates block-wise minimization frameworks with Bregman-proximal regularization, enabling generalization of convergence theory beyond the Euclidean case [1605.08290].
- **Sinkhorn/Greenhorn algorithms**: ABP formalizes the approach in entropic optimal transport, where iterated KL-Bregman projections yield Q-linear or robust $O(1/k)$ convergence rates for the dual objective in multi-marginal OT and related LP relaxations [2101.01704, 2602.01372].
- **Split feasibility and Kaczmarz-type methods**: ABP extends to high-dimensional and infinite-dimensional settings, including nonlinear split feasibility, through integration with proximal/inertial schemes and strong convergence to the unique Bregman projection [2505.11537, 1309.2094].

## 6. Special Cases, Extensions, and Applications

Selected structured regimes and applications include:

| Application Area           | Underlying Divergence          | Convergence Guarantee             |
|---------------------------|-------------------------------|-----------------------------------|
| Convex Feasibility        | General Legendre/Bregman      | Q-linear (deterministic/random)   |
| Entropic Optimal Transport| KL-divergence                 | Q-linear/sublinear [2602.01372]   |
| EM and Statistical Models | KL-divergence                 | Sublinear/linear (tame geometry)  |
| Sparse Kaczmarz Methods   | $\ell_1$/$\ell_2$ hybrid      | As for Bregman [1309.2094]        |
| Infinite-Dim Feasibility  | Strongly convex $\varphi$     | Strong convergence [2505.11537]   |

The framework handles nonconvex but prox-regular, semi-algebraic, or subanalytic sets, with local convergence in zero-gap cases of slowly vanishing regularity (e.g., the "cubic-root'' curve) [2507.21840]. For high-dimensional or nonlinear inverse problems, inertial and hybrid Bregman methods yield strong convergence guarantees and outperform classical metric projection schemes [2505.11537].

## 7. Open Questions and Future Directions

Several open directions are highlighted:

- **Quantitative KL Exponents**: Determining explicit values for desingularizing exponents $\theta$ to provide sharper complexity bounds [2507.21840].
- **Relaxations Beyond Tameness**: Extending ABP convergence theory to broader classes, such as constraints that fail definability or prox-regularity.
- **Stochastic and Projection-Free Variants**: Development of stochastic ABP algorithms and variants that circumvent direct projection computations [2507.21840].
- **Implementation in Infinite Dimensions**: Sophisticated algorithms based on ABP principles continue to be developed for infinite-dimensional function spaces and nonlinear operators [2505.11537].

Alternating Bregman projections represent a unifying methodology that clarifies and extends a wide array of iterative algorithms in analysis, optimization, signal processing, and statistics via a transparent geometric principle, establishing sharp convergence criteria and rates under minimal or structural assumptions [2507.21840, 1605.08290, 2101.01704, 2602.01372].

Source: https://www.emergentmind.com/topics/alternating-bregman-projections