---
title: Transport Information Bregman Divergences
url: https://www.emergentmind.com/topics/transport-information-bregman-divergences
type: topic
---

# Transport Information Bregman Divergences

Transport Information Bregman Divergences are a broad family of divergences interpolating between Bregman divergences from information geometry and cost-minimizing transport discrepancies from optimal transport theory. They enable the systematic extension of classical information-theoretic divergences—such as the Kullback–Leibler (KL), Bregman, and α-divergences—to the space of probability measures equipped with optimal transport, thereby unifying geometry, statistics, and variational analysis. This framework generalizes notions of dissimilarity in statistical inference, supports curved geometric structures, and underpins state-of-the-art computational solvers now fundamental in generative modeling, distributional optimization, and geometric statistics.

## 1. Foundations: Bregman Divergences and Optimal Transport

A classical Bregman divergence is generated by a strictly convex, smooth potential $\psi$ on a convex domain $\Omega \subset \mathbb{R}^n$, defined by
\[
D_\psi(x\|x') = \psi(x) - \psi(x') - \langle \nabla\psi(x'), x - x' \rangle.
\]
Bregman divergences are nonnegative, vanish only at $x=x'$, and induce a dually flat statistical manifold structure $(g, \nabla, \nabla^*)$, with metric $g = \nabla^2 \psi$ and flat affine connections conjugate with respect to $g$ [2206.14791]. 

Optimal transport considers the infimum of cost integrals over couplings $\pi \in \Pi(P, Q)$ of probability measures $P, Q$ on spaces $X, Y$ with cost $c(x, y)$. The Kantorovich problem is
\[
\inf_{\pi \in \Pi(P, Q)} \int c(x, y)\, d\pi(x, y).
\]
Quadratic cost $c(x, y)=\frac12\|x-y\|^2$ recovers the geometry underlying classical Wasserstein distances and, via the Brenier map, relations to convex analysis and Bregman forms.

Transport information Bregman divergences generalize both these constructions, providing a spectrum of divergences sensitive to both geometric (transport-based) and informational (potential-based) structures [2206.14791, 1906.00030, 2101.01162].

## 2. Transport Bregman Divergences: Definition and Variants

The essential transport Bregman divergence for measures $P,Q$ over spaces $X, Y$ with cost $c$ is given by
\[
D_c(P\|Q) = \int_X \varphi(x) dP(x) + \int_Y \varphi^c(y) dQ(y) - \iint_{X \times Y} c(x, y) d\pi^*(x, y),
\]
where $(\varphi, \varphi^c)$ are potentials solving the dual Kantorovich problem and $\pi^*$ is the optimal coupling [2206.14791, 2101.01162].

This construction—sometimes called 'transport–Bregman divergence'—extends the Bregman formula by replacing the linear inner product with an arbitrary cost $c$, and the usual gradient/dual structure with $c$-conjugate potentials.

Notable cases:
- **$c(x, y) = -\langle x, y \rangle$**: $D_c(P\|Q)$ reduces to the standard Bregman divergence between means [2206.14791].
- **$c(x, y) = \|x - y\|^2$**: Interpolates between squared Euclidean divergences and the Wasserstein-2 distance; yields explicit formulas for Gaussian distributions (the Bures/Hellinger–Wasserstein metric) [2206.14791].
- **Entropy regularization**: Adding $-\varepsilon H(\pi)$ yields the Sinkhorn divergence $D_c^\varepsilon(P\|Q)$, which is smooth in $\varepsilon$ and converges to $D_c$ as $\varepsilon \to 0$ [1412.5154, 2206.14791].

In the Wasserstein geometry, defining $\mathcal{F} : \mathcal{P}(\Omega) \to \mathbb{R}$ as a strictly displacement-convex functional, the transport–Bregman divergence is [2101.01162]:
\[
D_{T, \mathcal{F}}(p\|q) = \mathcal{F}(p) - \mathcal{F}(q) - \int_\Omega \left\langle \nabla_x \frac{\delta \mathcal{F}}{\delta q}(q), T_{q \to p}(x) - x\right\rangle q(x) dx,
\]
with $T_{q \to p}$ the optimal map sending $q$ to $p$.

## 3. Curved Information Geometries and $L^{(\alpha)}$-Transport Divergences

A central advance is the realization of canonical divergences for non-flat (curved) information geometries, driven by non-Euclidean transport costs. The logarithmic $L^{(\alpha)}$-divergence, induced by $c^{(\alpha)}(x, y) = \frac{1}{\alpha} \log(1 + \alpha x \cdot y)$, extrapolates the Bregman divergence ($\alpha \to 0$) and yields a manifold of constant sectional curvature $\mp \alpha$ [1712.03610, 1906.09103]:

\[
\mathbf{D}^{(\alpha)}[\xi:\xi'] = \frac{1}{\alpha} \log(1 + \alpha D\varphi(\xi') \cdot (\xi - \xi')) - (\varphi(\xi) - \varphi(\xi')).
\]

This divergence admits a self-dual transport-representation:
\[
\mathbf{D}^{(\alpha)}[\xi:\xi'] = c^{(\alpha)}(\xi, \eta') - \varphi(\xi) - \psi(\eta'),\qquad \eta' = D^{(\alpha)}\varphi(\xi').
\]
For $\alpha=1$, this is the $L$-divergence; as $\alpha \to 0^+$, the classical Bregman form is recovered. Such divergences appear as canonical on dually projectively flat statistical manifolds of constant curvature [1712.03610, 1906.00030, 1906.09103].

## 4. Regularized and Iterative Computational Methods

Transport Bregman divergences are at the core of computationally viable algorithms for regularized optimal transport. When regularized with an entropic (KL) or general Bregman penalty, the minimization admits efficient iterative matrix-scaling and projection-based schemes [1412.5154, 2102.07336]:

- **Sinkhorn algorithm**: Alternating KL/Bregman projections onto marginal constraint sets, yielding the unique entropic OT minimizer as a scaling of a Gibbs kernel.
- **Bregman-Dykstra iterations**: For inclusion of affine and convex (potentially inequality) constraints, with proven convergence in both affine (Sinkhorn–Knopp theorem) and convex settings.
- **Complexity**: Each iteration is $O(N^2)$ (for $N\times N$ couplings); empirical convergence is $O(1/\epsilon)$ for entropic regularization parameter $\epsilon$, yielding $O(N^2/\epsilon)$ total cost [1412.5154, 2309.11666].

On the statistical manifold side, gradient-based approaches generalize the matrix-scaling ideas, allowing convex descent in spaces of couplings [2102.07336]. Non-asymptotic error bounds for Bregman-regularized OT display super-exponential convergence rates for strictly convex (singular at zero) generators, substantially outperforming squared-Euclidean regularization [2309.11666].

## 5. Statistical and Geometric Properties

Transport information Bregman divergences possess key analytical and statistical attributes [2206.14791, 2101.01162, 2302.05833]:

- **Non-negativity**: $D_c(P\|Q) \geq 0$, with equality if and only if $P$ and $Q$ are $c$-equivalent.
- **Convexity**: Convex in the first argument for fixed reference measure, and (often) joint in both arguments for suitable costs.
- **Dualistic structure**: Induce a pair of affine charts and dually coupled connections $(g, \nabla, \nabla^*)$, generalizing the dually flat geometry of classical information geometry to para-Kähler or curved Kähler geometries depending on $c$ [1906.00030, 2206.14791].
- **Curvature**: $L^{(\alpha)}$ divergences realize constant curvature geometries; the Ma–Trudinger–Wang tensor and higher-order expansions of the divergence yield intrinsic curvature invariants [1906.00030, 1712.03610, 1906.09103].
- **Generalized Pythagorean theorem**: Holds in the infinite-dimensional manifold context, with adjustments for curvature.

## 6. One-Dimensional and Special Families: Analytical Formulas

For one-dimensional densities, transport information Bregman divergences admit explicit formulas in terms of quantile functions and quantile densities [2101.01162, 2504.14084]:
\[
D_{TKL}(p\|q) = \int_0^1 \left( \frac{q_p(u)}{q_q(u)} - \log \frac{q_p(u)}{q_q(u)} - 1 \right) du,
\]
with $q_p(u)$, $q_q(u)$ the quantile densities. More generally, selecting a convex generator $F_\alpha$, one defines the transport $\alpha$-divergence
\[
D_{T, \alpha}(p\|q) = F_\alpha(q_p) - F_\alpha(q_q) - \int_0^1 \frac{\delta F_\alpha}{\delta q_p}\Big|_{q_q}(u) \cdot (q_p(u) - q_q(u)) du,
\]
recovering KL and Hessian divergences for $\alpha=1, 0$ [2504.14084]. For special classes (e.g., scale families like Gaussians), the divergence collapses to closed-form expressions.

## 7. Applications and Impact

Transport information Bregman divergences appear in a variety of domains:

- **Generative modeling**: Sinkhorn and transport–Bregman divergences provide tractable, geometrically meaningful loss functions for probabilistic generative models, including GANs, flows, and robust risk modeling [1412.5154, 2206.14791, 2505.12097].
- **Robust optimization**: The divergences act as shape constraints and provide tight, efficiently computable bounds in distributionally robust optimization [2311.12183].
- **Clustering, barycenters, and statistics**: Unique barycenters, clustering robust to support mismatch or unbalanced mass, and minimax procedures for distortion risk are formulated naturally via Bregman-Wasserstein geometry [1412.5154, 2302.05833, 2311.12183].
- **Theoretical statistics and geometry**: Allow a seamless generalization of fundamental inequalities (transport–information, transport–entropy), canonically extend exponential families, and enrich the theory of statistical manifolds to include non-flat curvature [2206.14791, 1712.03610].
- **Regularized/debiased OT**: Proximal OT divergences interpolate between information (KL, $f$-divergence) and Wasserstein distances, with efficient primal-dual and mean-field PDE formulations [2505.12097].

Transport information Bregman divergences thus establish a unified framework for modeling, computation, and theory at the intersection of information geometry, optimal transport, and statistical science. They offer both the fine-grained statistical structure of information divergences and the geometric flexibility of optimal transport metrics, underpinning modern developments in machine learning and geometric statistics.

Source: https://www.emergentmind.com/topics/transport-information-bregman-divergences