Bregman and Mirror FRB Methods

• Bregman and Mirror FRB methods are optimization frameworks that integrate generalized Bregman distances with mirror descent to operate in both Euclidean and non-Euclidean geometries.
• They incorporate inertial forward-reflection steps to extend traditional splitting methods, achieving robust convergence under convexity and Kurdyka–Łojasiewicz conditions.
• The approaches connect convex duality, natural gradient techniques, and information geometry, with applications in stochastic approximation, parameter estimation, and opinion dynamics.

Bregman and Mirror Forward-Reflection Bregman (FRB) methods are optimization frameworks that use generalized Bregman distances and mirror descent to facilitate scalable, structure-adaptive algorithms in both Euclidean and non-Euclidean geometries. These approaches bridge convex-analytic duality, information geometry, and variational optimization, and admit inertial extensions via forward-reflection splitting. The interplay between Bregman divergences, mirror maps, and Riemannian information geometry leads to a unified analysis of first-order methods, with applications ranging from stochastic approximation to social-power dynamics on simplices.

1. Foundations: Bregman Distances and Mirror Descent

Let $J: \mathcal{X} \to \mathbb{R} \cup \{+\infty\}$ be a proper, convex, lower semicontinuous function. The (generalized) Bregman distance between $x, y \in \mathrm{dom}(J)$ with $\xi\in\partial J(y)$ is

$$D_J^\xi(x, y) = J(x) - J(y) - \langle \xi, x - y \rangle.$$

When $J$ is differentiable, $\xi = \nabla J(y)$, so $$D_J(x, y) = J(x) - J(y) - \langle \nabla J(y), x - y \rangle$$ (Benning et al., 2016). These divergences serve as non-Euclidean proximities, generalizing squared-Euclidean distance.

Mirror descent, developed by Nemirovski and Yudin, operates as follows: for a loss function $L$ and mirror potential $\phi$, the update at step $t$ is

$x, y \in \mathrm{dom}(J)$0

In dual coordinates, this yields $x, y \in \mathrm{dom}(J)$1, moving the iterate in the direction of steepest descent under the geometry induced by $x, y \in \mathrm{dom}(J)$2 (Raskutti et al., 2013).

2. Natural Gradient, Information Geometry, and Bregman Equivalence

Every twice-differentiable, strictly convex potential $x, y \in \mathrm{dom}(J)$3 defines a Legendre–Fenchel conjugate $x, y \in \mathrm{dom}(J)$4 and Riemannian metric $x, y \in \mathrm{dom}(J)$5, which may correspond, for example, to the Fisher information in exponential families.

The mirror descent update is precisely equivalent to the natural gradient method along the Riemannian manifold $x, y \in \mathrm{dom}(J)$6: $x, y \in \mathrm{dom}(J)$7 This equivalence establishes that mirror descent, when defined via Bregman divergence induced by $x, y \in \mathrm{dom}(J)$8, enacts first-order Riemannian steepest descent—mirroring geodesic flows to leading order approximation (Raskutti et al., 2013).

3. Forward-Reflection Bregman (FRB) Splitting Methods

Forward-reflection Bregman (FRB) methods generalize classical forward-backward splitting to Bregman distances with an inertial, or "reflection", step (Benning et al., 2016). The unified FRB iteration is: $x, y \in \mathrm{dom}(J)$9 where $\xi\in\partial J(y)$0 is a smooth functional and $\xi\in\partial J(y)$1 a (possibly nonsmooth) convex regularizer. This framework subsumes linearized Bregman, mirror descent, and entropic mirror updates as special cases.

Theoretical convergence of FRB-type methods can be established under the Kurdyka–Łojasiewicz property, given strong convexity of the Bregman generator and its Fenchel dual, and appropriate descent estimates for $\xi\in\partial J(y)$2 (Benning et al., 2016).

4. Entropic Mirror Descent and Opinion Dynamics

Entropic mirror descent utilizes the negative entropy $\xi\in\partial J(y)$3 as the mirror map, yielding the Kullback–Leibler divergence as the Bregman distance. For $\xi\in\partial J(y)$4, the entropic mirror-descent update admits the closed form

$\xi\in\partial J(y)$5

with normalization.

A notable variational interpretation appears in the DeGroot–Friedkin map for opinion dynamics, which is realized as entropic mirror descent on the simplex against the cost $\xi\in\partial J(y)$6, where $\xi\in\partial J(y)$7 is the Perron vector of the influence matrix and $\xi\in\partial J(y)$8 is the "extropy" (Halder, 2018). This approach yields a convex, globally-attractive dynamics with the interpretation of steepest descent with respect to log-likelihood geometry on the simplex.

5. Statistical Optimality: Cramér–Rao Bound and Efficiency

For parameter estimation in regular exponential families, where $\xi\in\partial J(y)$9 is the log-partition function and the Riemannian metric $$D_J^\xi(x, y) = J(x) - J(y) - \langle \xi, x - y \rangle.$$0 equals the Fisher information, mirror descent with a log-likelihood loss coincides with the natural-gradient approach. Using diminishing step sizes $$D_J^\xi(x, y) = J(x) - J(y) - \langle \xi, x - y \rangle.$$1, the covariance of the estimator sequence $$D_J^\xi(x, y) = J(x) - J(y) - \langle \xi, x - y \rangle.$$2 satisfies

$$D_J^\xi(x, y) = J(x) - J(y) - \langle \xi, x - y \rangle.$$3

attaining the Cramér–Rao lower bound asymptotically and verifying asymptotic Fisher efficiency (Raskutti et al., 2013).

6. Practical Implementation and Computational Aspects

Efficient implementation of Bregman and mirror FRB methods relies on several strategies:

• Choice of step-size: Diminishing schedules $$D_J^\xi(x, y) = J(x) - J(y) - \langle \xi, x - y \rangle.$$4 balance bias and variance.
• Computational forms: For exponential families, calculations use closed forms for $$D_J^\xi(x, y) = J(x) - J(y) - \langle \xi, x - y \rangle.$$5, $$D_J^\xi(x, y) = J(x) - J(y) - \langle \xi, x - y \rangle.$$6 (mean and natural parameter correspondences).
• Approximations: Block-diagonal, diagonal, or quasi-Newton approximations to $$D_J^\xi(x, y) = J(x) - J(y) - \langle \xi, x - y \rangle.$$7 permit scaling to large dimensions.
• Algorithmic templates: All such methods can be expressed as minimizing a local linearization plus a Bregman penalty, directly relating to their underlying information geometry (Raskutti et al., 2013, Benning et al., 2016).

7. Assumptions, Convergence, and Unified Analysis

Rigorous convergence and statistical guarantees for Bregman and mirror FRB methods require:

• Strict convexity and differentiability of the potential function on an open convex domain.
• Step-size sequences $$D_J^\xi(x, y) = J(x) - J(y) - \langle \xi, x - y \rangle.$$8 ensuring $$D_J^\xi(x, y) = J(x) - J(y) - \langle \xi, x - y \rangle.$$9, $J$0 for almost-sure convergence; $J$1 for Fisher efficiency.
• Smoothness, gradient Lipschitzness, and (in stochastic regimes) regularity conditions for loss functions.
• (For nonconvex or nonsmooth settings) Global convergence under the Kurdyka–Łojasiewicz property, leveraging sufficient decrease, gradient bounds, and vanishing increments (Benning et al., 2016).

This theoretical unity demonstrates that Bregman and mirror-FRB methods provide comprehensive algorithmic and geometric frameworks for diverse optimization and statistical inference problems, directly linking non-Euclidean descent with statistical efficiency and manifold geometry (Raskutti et al., 2013, Benning et al., 2016, Halder, 2018).