Transport Information Bregman Divergences

• Transport Information Bregman Divergences are a family of measures that interpolate between classical divergence metrics and optimal transport costs, unifying geometry, statistics, and variational analysis.
• They extend traditional divergences like KL and α-divergences by incorporating transport-based cost functions and curved geometric structures to enhance inference and optimization.
• Efficient computational methods, including the Sinkhorn algorithm and Bregman-Dykstra iterations, enable scalable applications in generative modeling, clustering, and robust distributional optimization.

Transport Information Bregman Divergences are a broad family of divergences interpolating between Bregman divergences from information geometry and cost-minimizing transport discrepancies from optimal transport theory. They enable the systematic extension of classical information-theoretic divergences—such as the Kullback–Leibler (KL), Bregman, and α-divergences—to the space of probability measures equipped with optimal transport, thereby unifying geometry, statistics, and variational analysis. This framework generalizes notions of dissimilarity in statistical inference, supports curved geometric structures, and underpins state-of-the-art computational solvers now fundamental in generative modeling, distributional optimization, and geometric statistics.

1. Foundations: Bregman Divergences and Optimal Transport

A classical Bregman divergence is generated by a strictly convex, smooth potential $\psi$ on a convex domain $\Omega \subset \mathbb{R}^n$, defined by

$$D_\psi(x\|x') = \psi(x) - \psi(x') - \langle \nabla\psi(x'), x - x' \rangle.$$

Bregman divergences are nonnegative, vanish only at $x=x'$, and induce a dually flat statistical manifold structure $(g, \nabla, \nabla^*)$, with metric $g = \nabla^2 \psi$ and flat affine connections conjugate with respect to $g$ (Khan et al., 2022).

Optimal transport considers the infimum of cost integrals over couplings $\pi \in \Pi(P, Q)$ of probability measures $P, Q$ on spaces $X, Y$ with cost $\Omega \subset \mathbb{R}^n$0. The Kantorovich problem is

$\Omega \subset \mathbb{R}^n$1

Quadratic cost $\Omega \subset \mathbb{R}^n$2 recovers the geometry underlying classical Wasserstein distances and, via the Brenier map, relations to convex analysis and Bregman forms.

Transport information Bregman divergences generalize both these constructions, providing a spectrum of divergences sensitive to both geometric (transport-based) and informational (potential-based) structures (Khan et al., 2022, Wong et al., 2019, Li, 2021).

2. Transport Bregman Divergences: Definition and Variants

The essential transport Bregman divergence for measures $\Omega \subset \mathbb{R}^n$3 over spaces $\Omega \subset \mathbb{R}^n$4 with cost $\Omega \subset \mathbb{R}^n$5 is given by

$\Omega \subset \mathbb{R}^n$6

where $\Omega \subset \mathbb{R}^n$7 are potentials solving the dual Kantorovich problem and $\Omega \subset \mathbb{R}^n$8 is the optimal coupling (Khan et al., 2022, Li, 2021).

This construction—sometimes called 'transport–Bregman divergence'—extends the Bregman formula by replacing the linear inner product with an arbitrary cost $\Omega \subset \mathbb{R}^n$9, and the usual gradient/dual structure with $$D_\psi(x\|x') = \psi(x) - \psi(x') - \langle \nabla\psi(x'), x - x' \rangle.$$0-conjugate potentials.

Notable cases:

• $$D_\psi(x\|x') = \psi(x) - \psi(x') - \langle \nabla\psi(x'), x - x' \rangle.$$1: $$D_\psi(x\|x') = \psi(x) - \psi(x') - \langle \nabla\psi(x'), x - x' \rangle.$$2 reduces to the standard Bregman divergence between means (Khan et al., 2022).
• $$D_\psi(x\|x') = \psi(x) - \psi(x') - \langle \nabla\psi(x'), x - x' \rangle.$$3: Interpolates between squared Euclidean divergences and the Wasserstein-2 distance; yields explicit formulas for Gaussian distributions (the Bures/Hellinger–Wasserstein metric) (Khan et al., 2022).
• Entropy regularization: Adding $$D_\psi(x\|x') = \psi(x) - \psi(x') - \langle \nabla\psi(x'), x - x' \rangle.$$4 yields the Sinkhorn divergence $$D_\psi(x\|x') = \psi(x) - \psi(x') - \langle \nabla\psi(x'), x - x' \rangle.$$5, which is smooth in $$D_\psi(x\|x') = \psi(x) - \psi(x') - \langle \nabla\psi(x'), x - x' \rangle.$$6 and converges to $$D_\psi(x\|x') = \psi(x) - \psi(x') - \langle \nabla\psi(x'), x - x' \rangle.$$7 as $$D_\psi(x\|x') = \psi(x) - \psi(x') - \langle \nabla\psi(x'), x - x' \rangle.$$8 (Benamou et al., 2014, Khan et al., 2022).

In the Wasserstein geometry, defining $$D_\psi(x\|x') = \psi(x) - \psi(x') - \langle \nabla\psi(x'), x - x' \rangle.$$9 as a strictly displacement-convex functional, the transport–Bregman divergence is (Li, 2021): $x=x'$0 with $x=x'$1 the optimal map sending $x=x'$2 to $x=x'$3.

3. Curved Information Geometries and $x=x'$4-Transport Divergences

A central advance is the realization of canonical divergences for non-flat (curved) information geometries, driven by non-Euclidean transport costs. The logarithmic $x=x'$5-divergence, induced by $x=x'$6, extrapolates the Bregman divergence ($x=x'$7) and yields a manifold of constant sectional curvature $x=x'$8 (Wong, 2017, Wong et al., 2019):

$x=x'$9

This divergence admits a self-dual transport-representation: $(g, \nabla, \nabla^*)$0 For $(g, \nabla, \nabla^*)$1, this is the $(g, \nabla, \nabla^*)$2-divergence; as $(g, \nabla, \nabla^*)$3, the classical Bregman form is recovered. Such divergences appear as canonical on dually projectively flat statistical manifolds of constant curvature (Wong, 2017, Wong et al., 2019, Wong et al., 2019).

4. Regularized and Iterative Computational Methods

Transport Bregman divergences are at the core of computationally viable algorithms for regularized optimal transport. When regularized with an entropic (KL) or general Bregman penalty, the minimization admits efficient iterative matrix-scaling and projection-based schemes (Benamou et al., 2014, Takatsu, 2021):

• Sinkhorn algorithm: Alternating KL/Bregman projections onto marginal constraint sets, yielding the unique entropic OT minimizer as a scaling of a Gibbs kernel.
• Bregman-Dykstra iterations: For inclusion of affine and convex (potentially inequality) constraints, with proven convergence in both affine (Sinkhorn–Knopp theorem) and convex settings.
• Complexity: Each iteration is $(g, \nabla, \nabla^*)$4 (for $(g, \nabla, \nabla^*)$5 couplings); empirical convergence is $(g, \nabla, \nabla^*)$6 for entropic regularization parameter $(g, \nabla, \nabla^*)$7, yielding $(g, \nabla, \nabla^*)$8 total cost (Benamou et al., 2014, Morikuni et al., 2023).

On the statistical manifold side, gradient-based approaches generalize the matrix-scaling ideas, allowing convex descent in spaces of couplings (Takatsu, 2021). Non-asymptotic error bounds for Bregman-regularized OT display super-exponential convergence rates for strictly convex (singular at zero) generators, substantially outperforming squared-Euclidean regularization (Morikuni et al., 2023).

5. Statistical and Geometric Properties

Transport information Bregman divergences possess key analytical and statistical attributes (Khan et al., 2022, Li, 2021, Kainth et al., 2023):

• Non-negativity: $(g, \nabla, \nabla^*)$9, with equality if and only if $g = \nabla^2 \psi$0 and $g = \nabla^2 \psi$1 are $g = \nabla^2 \psi$2-equivalent.
• Convexity: Convex in the first argument for fixed reference measure, and (often) joint in both arguments for suitable costs.
• Dualistic structure: Induce a pair of affine charts and dually coupled connections $g = \nabla^2 \psi$3, generalizing the dually flat geometry of classical information geometry to para-Kähler or curved Kähler geometries depending on $g = \nabla^2 \psi$4 (Wong et al., 2019, Khan et al., 2022).
• Curvature: $g = \nabla^2 \psi$5 divergences realize constant curvature geometries; the Ma–Trudinger–Wang tensor and higher-order expansions of the divergence yield intrinsic curvature invariants (Wong et al., 2019, Wong, 2017, Wong et al., 2019).
• Generalized Pythagorean theorem: Holds in the infinite-dimensional manifold context, with adjustments for curvature.

6. One-Dimensional and Special Families: Analytical Formulas

For one-dimensional densities, transport information Bregman divergences admit explicit formulas in terms of quantile functions and quantile densities (Li, 2021, Li, 18 Apr 2025): $g = \nabla^2 \psi$6 with $g = \nabla^2 \psi$7, $g = \nabla^2 \psi$8 the quantile densities. More generally, selecting a convex generator $g = \nabla^2 \psi$9, one defines the transport $g$0-divergence

$g$1

recovering KL and Hessian divergences for $g$2 (Li, 18 Apr 2025). For special classes (e.g., scale families like Gaussians), the divergence collapses to closed-form expressions.

7. Applications and Impact

Transport information Bregman divergences appear in a variety of domains:

• Generative modeling: Sinkhorn and transport–Bregman divergences provide tractable, geometrically meaningful loss functions for probabilistic generative models, including GANs, flows, and robust risk modeling (Benamou et al., 2014, Khan et al., 2022, Baptista et al., 17 May 2025).
• Robust optimization: The divergences act as shape constraints and provide tight, efficiently computable bounds in distributionally robust optimization (Pesenti et al., 2023).
• Clustering, barycenters, and statistics: Unique barycenters, clustering robust to support mismatch or unbalanced mass, and minimax procedures for distortion risk are formulated naturally via Bregman-Wasserstein geometry (Benamou et al., 2014, Kainth et al., 2023, Pesenti et al., 2023).
• Theoretical statistics and geometry: Allow a seamless generalization of fundamental inequalities (transport–information, transport–entropy), canonically extend exponential families, and enrich the theory of statistical manifolds to include non-flat curvature (Khan et al., 2022, Wong, 2017).
• Regularized/debiased OT: Proximal OT divergences interpolate between information (KL, $g$3-divergence) and Wasserstein distances, with efficient primal-dual and mean-field PDE formulations (Baptista et al., 17 May 2025).

Transport information Bregman divergences thus establish a unified framework for modeling, computation, and theory at the intersection of information geometry, optimal transport, and statistical science. They offer both the fine-grained statistical structure of information divergences and the geometric flexibility of optimal transport metrics, underpinning modern developments in machine learning and geometric statistics.