Alternating Bregman Projections

• Alternating Bregman Projections are iterative methods that replace the Euclidean metric with Bregman divergences generated by strictly convex functions, extending classical projection techniques.
• The algorithm alternates between left and right projections under specific geometric conditions, ensuring convergence in both convex and certain nonconvex settings.
• This framework unifies various algorithms in convex optimization and statistics, such as the EM and Sinkhorn methods, by leveraging tailored divergence measures for improved convergence rates.

Alternating Bregman Projections (ABP) are a fundamental generalization of the classical method of alternating projections, replacing the Euclidean metric with divergences generated by strictly convex (Legendre-type) functions. This framework subsumes numerous important algorithms, including projection methods in convex optimization, iterative regularization schemes, and the Expectation-Maximization (EM) algorithm in statistics, notably when using Kullback–Leibler (KL) divergence as the underlying geometry. ABP methods have a geometric foundation that enables convergence and rate guarantees well beyond convex settings, including nonconvex, tame, and semi-algebraic constraints (Noll, 29 Jul 2025).

1. Mathematical Foundation and Definitions

Central to ABP is the Bregman divergence, $$D_\phi(x, y) = \phi(x) - \phi(y) - \langle \nabla \phi(y), x - y \rangle$$, where $$\phi : \mathbb{R}^d \to \mathbb{R} \cup \{+\infty\}$$ is a Legendre function—proper, strictly convex, and twice continuously differentiable on the interior of its domain (Noll, 29 Jul 2025). For $y$ in the interior of $\dom\phi$, Bregman divergence quantifies non-symmetric, non-metric "distance" between $x$ and $y$.

Given a closed set $C$ in $\dom\phi$, two projections are defined:

• Left-projection: $P_C^\ell(a) = \arg\min_{z \in C} D_\phi(z, a)$ for $a \in \Int(\dom\phi)$.
• Right-projection: $$\phi : \mathbb{R}^d \to \mathbb{R} \cup \{+\infty\}$$0 for $$\phi : \mathbb{R}^d \to \mathbb{R} \cup \{+\infty\}$$1.

Alternating projections correspond to sequential minimization of Bregman divergences over two sets $$\phi : \mathbb{R}^d \to \mathbb{R} \cup \{+\infty\}$$2.

2. General Algorithmic Structure

Let $$\phi : \mathbb{R}^d \to \mathbb{R} \cup \{+\infty\}$$3 be closed sets and $$\phi : \mathbb{R}^d \to \mathbb{R} \cup \{+\infty\}$$4 an initial point. The ABP method generates sequences $$\phi : \mathbb{R}^d \to \mathbb{R} \cup \{+\infty\}$$5 and $$\phi : \mathbb{R}^d \to \mathbb{R} \cup \{+\infty\}$$6 according to:

1. $$\phi : \mathbb{R}^d \to \mathbb{R} \cup \{+\infty\}$$7,
2. $$\phi : \mathbb{R}^d \to \mathbb{R} \cup \{+\infty\}$$8,

with termination when $$\phi : \mathbb{R}^d \to \mathbb{R} \cup \{+\infty\}$$9 falls below a given tolerance (Noll, 29 Jul 2025). The update contractivity is monitored by the monotonicity $y$0, ensuring the Bregman distances form a non-increasing sequence.

In convex settings, each projection is well-defined and single-valued, but ABP has been generalized to handle nonconvex but tame (definable, prox-regular) sets, provided certain geometric and interiority conditions (Noll, 29 Jul 2025).

3. Convergence Theory and Geometric Conditions

The global convergence of ABP is secured by two key geometric hypotheses:

• Angle condition: A type of Kurdyka–Łojasiewicz (KL) inequality, specifying that for any subsequential accumulation point $y$1, the angular separation of update directions does not degenerate too fast, i.e.,

$y$2

for shrinking function $y$3 and parameter $y$4. When $y$5, the rate exponent $y$6 captures transversality or tangentiality (Noll, 29 Jul 2025, Sun et al., 2016).

• Three-point inequality: Near any gap pair, it is required that

$y$7

This controls local contraction. Convex sets automatically satisfy this with $y$8 (Noll, 29 Jul 2025).

Main result: Under these two geometric conditions, $y$9 converges to a unique $\dom\phi$0, and if $\dom\phi$1 ($\dom\phi$2), then $\dom\phi$3 and $\dom\phi$4 converge to a common point in the intersection (Noll, 29 Jul 2025). Feasible (zero-gap) and infeasible (positive-gap) cases are distinguished: in the latter, ABP converges to a gap-realizing pair that minimizes Bregman distance between $\dom\phi$5 and $\dom\phi$6.

4. Rates of Convergence

The rate at which $\dom\phi$7 and $\dom\phi$8 converge depends on the exponent $\dom\phi$9 from the angle (KL) condition:

• General sublinear rate: If $x$0, $x$1, then

$x$2

• R-linear (geometric) convergence: If intersection is transversal, so that $x$3, the algorithm enjoys $x$4 convergence for some $x$5 (Noll, 29 Jul 2025, Sun et al., 2016, Bargetz et al., 2019).

In Banach spaces that are uniformly convex and smooth of power type, explicit linear convergence rates are established for alternating Bregman projections onto closed linear subspaces, provided a suitable linear Bregman regularity (modulus) condition holds (Bargetz et al., 2019).

5. Connections to Optimization and Statistical Algorithms

The ABP method unifies several classical iterative schemes:

• Expectation-Maximization (EM) Algorithm: When the Bregman divergence is chosen as Kullback–Leibler (KL), the E-step is a left-Bregman projection onto the data-constraint set, and the M-step is a right-Bregman projection onto the model manifold. Under definability assumptions on the model and data sets, ABP guarantees convergence of EM-type algorithms, even for nonconvex parameter spaces, with the same rate theory as above (Noll, 29 Jul 2025).
• Alternating minimization (AM, AAM, PALM): ABP encapsulates block-wise minimization frameworks with Bregman-proximal regularization, enabling generalization of convergence theory beyond the Euclidean case (Sun et al., 2016).
• Sinkhorn/Greenhorn algorithms: ABP formalizes the approach in entropic optimal transport, where iterated KL-Bregman projections yield Q-linear or robust $x$6 convergence rates for the dual objective in multi-marginal OT and related LP relaxations (Kostic et al., 2021, Peyré, 1 Feb 2026).
• Split feasibility and Kaczmarz-type methods: ABP extends to high-dimensional and infinite-dimensional settings, including nonlinear split feasibility, through integration with proximal/inertial schemes and strong convergence to the unique Bregman projection (Sababe et al., 14 May 2025, Lorenz et al., 2013).

6. Special Cases, Extensions, and Applications

Selected structured regimes and applications include:

Application Area	Underlying Divergence	Convergence Guarantee
Convex Feasibility	General Legendre/Bregman	Q-linear (deterministic/random)
Entropic Optimal Transport	KL-divergence	Q-linear/sublinear (Peyré, 1 Feb 2026)
EM and Statistical Models	KL-divergence	Sublinear/linear (tame geometry)
Sparse Kaczmarz Methods	$x$7/$x$8 hybrid	As for Bregman (Lorenz et al., 2013)
Infinite-Dim Feasibility	Strongly convex $x$9	Strong convergence (Sababe et al., 14 May 2025)

The framework handles nonconvex but prox-regular, semi-algebraic, or subanalytic sets, with local convergence in zero-gap cases of slowly vanishing regularity (e.g., the "cubic-root'' curve) (Noll, 29 Jul 2025). For high-dimensional or nonlinear inverse problems, inertial and hybrid Bregman methods yield strong convergence guarantees and outperform classical metric projection schemes (Sababe et al., 14 May 2025).

7. Open Questions and Future Directions

Several open directions are highlighted:

• Quantitative KL Exponents: Determining explicit values for desingularizing exponents $y$0 to provide sharper complexity bounds (Noll, 29 Jul 2025).
• Relaxations Beyond Tameness: Extending ABP convergence theory to broader classes, such as constraints that fail definability or prox-regularity.
• Stochastic and Projection-Free Variants: Development of stochastic ABP algorithms and variants that circumvent direct projection computations (Noll, 29 Jul 2025).
• Implementation in Infinite Dimensions: Sophisticated algorithms based on ABP principles continue to be developed for infinite-dimensional function spaces and nonlinear operators (Sababe et al., 14 May 2025).

Alternating Bregman projections represent a unifying methodology that clarifies and extends a wide array of iterative algorithms in analysis, optimization, signal processing, and statistics via a transparent geometric principle, establishing sharp convergence criteria and rates under minimal or structural assumptions (Noll, 29 Jul 2025, Sun et al., 2016, Kostic et al., 2021, Peyré, 1 Feb 2026).