Bregman–Fejér Monotonicity in Optimization

• Bregman–Fejér monotonicity is a generalization of classical Fejér monotonicity that uses Bregman distances to analyze convergence in non-Euclidean spaces.
• It enables convergence analysis for diverse iterative methods such as Mann–proximal and stochastic mirror descent algorithms in Banach and Hilbert spaces.
• The framework provides quantitative convergence rates and stability guarantees under error summability, supporting advances in optimization and machine learning.

Bregman–Fejér monotonicity generalizes classical Fejér monotonicity to non-Euclidean geometries by employing Bregman distances in place of metrics. This framework encompasses and extends monotonicity concepts critical for the convergence theory of fixed-point and variational algorithms in Banach and Hilbert spaces, with substantial implications for modern optimization, stochastic approximation, convex feasibility, and large-scale machine learning.

1. Abstract Definition and Variants

Let $(X, d)$ be a metric space, $F \subseteq X$ nonempty, and $(\phi_n)$ a sequence of distance-type functions $\phi_n: X \times X \to \mathbb{R}_+$. Given comparison functions $G, H: \mathbb{R}_+ \rightarrow \mathbb{R}_+$ and an error sequence $(\varepsilon_n)$ with $\sum \varepsilon_n < \infty$, $(x_n) \subset X$ is called $(\phi_n)$-$\!(G,H)$–quasi-Fejér monotone w.r.t.\ $F \subseteq X$0 if

$F \subseteq X$1

Specializing to Bregman distances, $F \subseteq X$2, with $F \subseteq X$3 convex and Gâteaux differentiable, yields Bregman–Fejér monotonicity. In the exact case ($F \subseteq X$4, $F \subseteq X$5),

$F \subseteq X$6

Partial or alternating variants require only selected subsequences (e.g., even-indexed) to satisfy the full Bregman–Fejér inequality, while the rest satisfy weaker forms, facilitating analysis of inertial or block-coordinate schemes (Pischke, 2023).

2. Extensions to Banach Geometry and Stochastic Settings

The Banach–Bregman framework replaces Euclidean geometry with non-Euclidean Bregman divergences defined for a Legendre function $F \subseteq X$7, as $F \subseteq X$8. The stochastic Bregman–Fejér property is encoded as

$F \subseteq X$9

for a filtration $(\phi_n)$0, descent terms $(\phi_n)$1, and error terms $(\phi_n)$2 (Zhang et al., 17 Sep 2025). Randomized and adaptive algorithms, including stochastic mirror descent, natural gradient, and KL-regularized policy iteration, all fit this abstract monotonicity template. Notably, the framework supports relaxation coefficients $(\phi_n)$3 ("super-relaxation"), unique to non-Hilbertian settings, with empirical acceleration in simplex and natural-gradient geometries.

3. Quantitative Convergence and Metastability

Strong quantitative results are established for Bregman–Fejér monotone sequences. Under total boundedness, weak triangle inequalities, and summability of errors, explicit rates of metastability are derived: $(\phi_n)$4 with concrete dependencies on modulus functions for boundedness, continuity, and summability (Pischke, 2023). When a modulus of regularity is available (as in zero-sets of suitable functions), monotone, explicit rates of convergence in the Bregman (or a dominated) metric are obtained. These statements are pivotal for proof mining and constructive algorithm analysis.

4. Applications in Optimization Algorithms

The Bregman–Fejér notion systematically underpins the convergence of diverse iterative methods:

• Mann–Proximal–Point in Banach Spaces: For $(\phi_n)$5 uniformly smooth/convex, the Mann–proximal–point iteration

$(\phi_n)$6

is uniformly Bregman–Fejér monotone with respect to $(\phi_n)$7, admitting computable moduli for metastability, fixed-point approximation, and full convergence rates (Pischke, 2023).

• Bregman Variational Learning Dynamics (BVLD): The operator

$(\phi_n)$8

in Hilbert space, with strictly convex $(\phi_n)$9, exhibits Fejér monotonicity:

$\phi_n: X \times X \to \mathbb{R}_+$0

for $\phi_n: X \times X \to \mathbb{R}_+$1, yielding explicit geometric convergence and continuous-time analogues via evolution variational inequalities (EVI) (CHA et al., 23 Oct 2025).

• Variable-Bregman Proximal Algorithms: For dynamically changing Bregman divergences, variable quasi-Bregman monotonicity is central to guarantees of boundedness, weak/strong convergence, and asymptotic regularity in Banach spaces (Nguyen, 2015).

5. Generalized Bregman–Fejér Frameworks

Variable-metric and quasi-Bregman monotonicity extend the fixed Bregman setting. For a sequence of differentiable, Legendre-type functions $\phi_n: X \times X \to \mathbb{R}_+$2, $\phi_n: X \times X \to \mathbb{R}_+$3 is variable quasi-Bregman monotone w.r.t.\ a closed set $\phi_n: X \times X \to \mathbb{R}_+$4 if for all $\phi_n: X \times X \to \mathbb{R}_+$5 and $\phi_n: X \times X \to \mathbb{R}_+$6,

$\phi_n: X \times X \to \mathbb{R}_+$7

with $\phi_n: X \times X \to \mathbb{R}_+$8 (Nguyen, 2015). This unifies Bregman-monotone sequences, variable-metric quasi-Fejér frameworks, and iterative projection methods. Convergence follows from iterative descent and strong convexity/coercivity.

6. Further Generalizations and Operator Geometry

Recent advancements include:

• Forward Bregman Monotonicity: $\phi_n: X \times X \to \mathbb{R}_+$9 is forward Bregman monotone with respect to $G, H: \mathbb{R}_+ \rightarrow \mathbb{R}_+$0 if $G, H: \mathbb{R}_+ \rightarrow \mathbb{R}_+$1 for all $G, H: \mathbb{R}_+ \rightarrow \mathbb{R}_+$2, enabling boundedness and, under essential strict convexity, weak convergence to a point in $G, H: \mathbb{R}_+ \rightarrow \mathbb{R}_+$3 (Ouyang, 2021).
• Bregman Circumcenter Iterations: Iterations defined via the Bregman circumcenter of operator images result in sequences that are forward Bregman monotone under mild conditions (Legendre functions, Bregman isometries, single-valuedness), converging weakly to a joint fixed point of the operators involved (Ouyang, 2021).
• Stochastic and Drift-Aware Algorithms: In time-varying or stochastic environments, Bregman–Fejér monotonicity ensures stability and convergence rates, subject to cumulative drift bounds and regularity of the underlying operator sequence (CHA et al., 23 Oct 2025Zhang et al., 17 Sep 2025).

7. Impact and Unified Perspective

The Bregman–Fejér monotonicity principle provides a mathematically rigorous, unifying foundation for a broad class of optimization and fixed-point algorithms. It accommodates time-varying or heterogeneous distance-generating functions, incorporates stochastic or deterministic errors, and subsumes classical (metric) Fejér monotonicity as a special case. The framework guarantees, under mild geometric and regularity conditions, both qualitative and fully quantitative convergence phenomena, supporting proof mining, algorithm design, and performance guarantees from nonlinear analysis to contemporary machine learning and large-scale optimization (Pischke, 2023Zhang et al., 17 Sep 2025CHA et al., 23 Oct 2025Nguyen, 2015Ouyang, 2021).

---

References

• "Generalized Fejér monotone sequences and their finitary content" (Pischke, 2023)
• "A Universal Banach--Bregman Framework for Stochastic Iterations: Unifying Stochastic Mirror Descent, Learning and LLM Training" (Zhang et al., 17 Sep 2025)
• "Optimization of Bregman Variational Learning Dynamics" (CHA et al., 23 Oct 2025)
• "Variable Quasi-Bregman Monotone Sequences" (Nguyen, 2015)
• "Bregman Circumcenters: Monotonicity and Forward Weak Convergence" (Ouyang, 2021)