---
title: Divergence-Regularized Optimal Transport
url: https://www.emergentmind.com/topics/divergence-regularized-optimal-transport-c87e7b14-3385-45e5-bc4a-c35661d3f1f5
type: topic
---

# Divergence-Regularized Optimal Transport

Divergence-Regularized Optimal Transport is a broad framework for convex relaxations of the classical optimal transport (OT) problem, in which the original linear programming problem is smoothed or "regularized" by a convex divergence term acting on the space of couplings. This methodology encompasses entropic regularization (based on Kullback-Leibler (KL) divergence), Tsallis, Rényi, $\beta$-divergence, Bregman, and other $f$-divergence regularizations, as well as kernel methods such as MMD-regularized OT. It enables scalable numerical algorithms, admits rigorous statistical theory, improves robustness and sample complexity, and provides a unifying interface between geometry, information, convex analysis, and applications, especially in high-dimensional data science, statistical inference, and generative modeling.

## 1. Mathematical Formulation of Divergence-Regularized OT

Let $(X_1,\mathcal{B}_1,\mu)$ and $(X_2,\mathcal{B}_2,\nu)$ be Polish probability spaces and $c : X_1 \times X_2 \to [0, +\infty)$ a lower-semicontinuous cost. The set of couplings is
$$
\Pi(\mu, \nu) = \{ \pi \in \mathcal P(X_1 \times X_2) : \pi_{X_1} = \mu,\, \pi_{X_2} = \nu \}.
$$
The unregularized Monge–Kantorovich problem is
$$
\mathrm{OT}(\mu, \nu) = \inf_{\pi \in \Pi(\mu, \nu)} \int_{X_1 \times X_2} c(x, y)\,d\pi(x, y).
$$

Divergence regularization introduces an additive penalty on $\pi$ with respect to a reference, usually $\mu\otimes\nu$, via a convex divergence $D_f$:
$$
\mathrm{OT}_{f, \varepsilon}(\mu, \nu) = \inf_{\pi \in \Pi(\mu, \nu)} \left\{ \int c\,d\pi + \varepsilon\,D_f(\pi\|\mu\otimes\nu) \right\}
$$
with
$$
D_f(\pi\|\mu\otimes\nu) = \int_{X_1 \times X_2} f\left( \frac{d\pi}{d(\mu\otimes\nu)} \right)\,d(\mu\otimes\nu)
$$
for convex $f$ satisfying $f(1) = 0$. Notable cases:
- KL (entropic): $f(u) = u \log u$
- Tsallis: $f(u) = (u^q - u)/(q-1)$ for $q > 1$
- $\beta$-divergence: $f_\beta$ as in [2212.13251]
- Bregman, Rényi, MMD, and others

Dual formulations are given via Fenchel–Rockafellar conjugacy using the convex dual $f^*$. For Tsallis-regularized OT [2304.06616], e.g.,
$$
\mathrm{OT}_{q,\varepsilon}(\mu,\nu) = \sup_{(h_1, h_2) \in L^1(\mu) \times L^1(\nu)} \left\{ \int h_1 d\mu + \int h_2 d\nu - \varepsilon \int f_q^*\left( \frac{h_1(x) + h_2(y) - c(x, y)}{\varepsilon} \right) d\mu d\nu \right\}.
$$

## 2. Classes of Divergences, Interpolation, and Limiting Behavior

Divergence regularizations span a wide spectrum. KL, Tsallis, $\beta$-divergence, Bregman divergences, and Rényi divergences (for $\alpha \in (0, 1)$) are prominent:
- **KL ($q \to 1$, $\beta \to 1$):** recovers the entropic regularizer and Sinkhorn algorithm.
- **Tsallis ($q > 1$):** allows polynomial penalty structure, fusing sparsity with smoothness, converges to KL as $q \to 1$; as $q \to 0$, recovers classic OT [2304.06616].
- **$\beta$-divergence [2212.13251]:** interpolates entropic (KL) and robust hard-thresholding regimes. Robust to outliers for $\beta > 1$.
- **Rényi divergence [2404.18834]:** for $\alpha \nearrow 1$ recovers KL; for $\alpha \searrow 0$ recovers unregularized OT. Not an $f$-divergence/Bregman distance, but admits strict convexity, metrization, and symmetry.

This interpolation property allows one to tune regularization parameters (e.g. $\alpha$ in Rényi or $q$ in Tsallis) to squeeze the regularized solution toward the true OT plan or, inversely, to maximize smoothness for tractable Sinkhorn-like computation.

## 3. Convergence Rates and $\Gamma$-Convergence

Central theoretical results concern the vanishing regularization limit $\varepsilon \to 0$:
- The functional $\mathrm{OT}_{f,\varepsilon}$ $\Gamma$-converges to the unregularized $\mathrm{OT}$ [2304.06616], [2208.14391].
- Quantization and shadow coupling arguments yield explicit convergence rates: in the entropic case (KL), the gap $\mathrm{OT}_{\rm KL, \varepsilon} - \mathrm{OT} = O(\varepsilon\log(1/\varepsilon))$ for typical costs and dimension [2304.06616], [2208.14391].
- For Tsallis regularization $(q > 1)$, rates slow to polynomial decay in $\varepsilon$: $\mathrm{OT}_{q, \varepsilon} - \mathrm{OT} = O\big(\varepsilon^{1/(q-1)+1}\big)$, and the KL case is provably optimal among all $q \ge 1$ [2304.06616].

The limits in other regularization families parallel this pattern—$\alpha$-Rényi divergence regularization recovers the hard OT plan for $\alpha \to 0$ without the numerical instability attendant to $\varepsilon \to 0$ in classical entropic regularization [2404.18834].

## 4. Numerical Algorithms and Computation

Divergence regularization transforms the OT problem into a strictly convex optimization, admitting scalable algorithms:
- **Sinkhorn Algorithm:** Classical for KL/entropic regularization; multiplicative row-column scaling [1412.5154], [1609.04495], [1711.08947].
- **Generalized Sinkhorn and IPFP:** For Tsallis/$f$-divergences, projections via Newton-type updates or mirror descent, or via alternation of marginal constraints [2212.13251], [2105.14337], [2007.00976].
- **Mirror Descent for Rényi-regularized OT:** Uses negative Shannon entropy map, explicit gradient computation, and marginal-projection steps [2404.18834].
- **Bregman Projection Methods:** For general smooth convex regularizers, using Dykstra-style alternation, Newton–Raphson for separable cases, sparse extension for high dimension [1610.06447].
- **Accelerated Projected Gradient Descent:** For kernelized (MMD) and other IPM regularized OT [2011.05001].
- **Infinite-dimensional settings:** Dynamic (Benamou–Brenier, mean-field game) representations for proximal OT divergences, blending divergence and OT via infimal convolution [2505.12097], [2309.03791].

Strict convexity and superlinear growth of divergence ensure unique minimizers, strong duality, and global convergence, subject to compactness and regularity.

## 5. Regularized OT: Statistical Theory and Sample Complexity

Divergence regularization fundamentally shapes the statistical behavior of empirical OT estimators:
- **Parametric rates ($n^{-1/2}$) for all $f$-divergences:** Provided the cost is bounded and $f^*$ is $C^1$, empirical regularized OT achieves the parametric rate for sample complexity, in sharp contrast with the curse of dimensionality intrinsic to unregularized OT [2510.02489], [2505.04721], [2212.00367].
- **Central Limit Theorems:** Limiting Gaussian distributions for the regularized OT cost, plan, and dual potentials, in both one- and two-sample regimes, with explicit covariance formulas [1810.09880], [1711.08947], [2510.02489], [2505.04721].
- **Bootstrap Consistency:** Ordinary $n$-out-of-$n$ bootstrap is valid for empirical divergence-regularized OT, enabling statistical inference and confidence bands [1711.08947], [1810.09880].
- **Stability and Regularity:** Quantitative bounds on the change in optimizers under marginal perturbations, strengthening robustness.
- **Intrinsic Dimension and Smoothness:** Fast rates are achievable depending on the regularity of cost/divergence and the “intrinsic dimension” of the data [2212.00367].

## 6. Applications: Robustness, Inference, and Geometry

Divergence-regularized OT is used extensively across disciplines:
- **Robust Learning:** $\beta$-potential regularization prevents mass transport to outliers, achieving statistical performance superior to entropic OT in contaminated data [2212.13251].
- **Ecological inference:** Tsallis-regularized OT offers state-of-the-art marginal reconstruction for political science datasets [1609.04495].
- **Kernel and RKHS-based OT:** MMD regularization interpolates classical OT and kernel distances, combining sample efficiency and ground-metric geometry [2011.05001].
- **Generative modeling:** Proximal OT divergences provide tractable interpolants between GAN-style $f$-divergences and OT distances, governing flows in probability space [2505.12097], [2309.03791].
- **Distributionally Robust Optimization:** Infimal-convolution (OT-regularized divergence) sets define ambiguity sets that generalize Wasserstein and $f$-divergence DRO schemes [2309.03791], [2505.12097].

Empirical results show that, for moderate regularization, sparser couplings (Tsallis, Rényi, $\beta$) yield OT plans closer to the ground truth than KL-regularized schemes, particularly in real-world inference tasks [2404.18834].

## 7. Comparisons, Limitations, and Generalizations

- **KL regularization:** Fast convergence, maximal smoothness, and computational simplicity via the Sinkhorn algorithm; cost bias vanishes logarithmically as $\varepsilon \to 0$.
- **Tsallis/$L^p$ regularization:** Allows sparse couplings and variable convergence rates polynomial in $\varepsilon$; bias decays slower than KL [2304.06616], [1609.04495], [2505.04721].
- **Rényi regularization:** Enables interpolation from OT to KL without numerical instability, yielding plans that empirically outperform both [2404.18834].
- **General $f$-divergence/Bregman regularization:** All known convex regularizers with suitable smoothness yield strict convexity, well-behaved limits, statistical efficiency, and strong convergence theory.
- **Extensions:** Multi-marginal divergence-regularized OT, barycenters, unbalanced OT, and models with spatially varying divergences (e.g., homogeneous UROT/OT with boundary [2201.02082]), and optimal transport-regularized divergences (infimal convolution) [2505.12097].

Sharpness of rates and uniform statistical bounds depend critically on the specific divergence, cost regularity, and data geometry. KL regularization remains optimal in terms of fastest vanishing bias, but non-entropy divergences enable practical gains in robustness and inference accuracy.

---

### References

- [2304.06616] Convergence rate of Tsallis entropic regularized optimal transport
- [2505.04721] Sparse Regularized Optimal Transport without Curse of Dimensionality 
- [2510.02489] General Divergence Regularized Optimal Transport: Sample Complexity and Central Limit Theorems
- [1609.04495] Tsallis Regularized Optimal Transport and Ecological Inference
- [2212.13251] Robust computation of optimal transport by $\beta$-potential regularization
- [2404.18834] Interpolating between Optimal Transport and KL regularized Optimal Transport using Rényi Divergences
- [2011.05001] MMD-Regularized Unbalanced Optimal Transport
- [2208.14391] Convergence Rates for Regularized Optimal Transport via Quantization
- [2105.14337] Optimal transport with $f$-divergence regularization and generalized Sinkhorn algorithm
- [2007.00976] Optimal Transport losses and Sinkhorn algorithm with general convex regularization 
- [1610.06447] Regularized Optimal Transport and the Rot Mover's Distance
- [2212.00367] Stability and Sample Complexity of Divergence Regularized Optimal Transport
- [1810.09880] Empirical Regularized Optimal Transport: Statistical Theory and Applications
- [1711.08947] Central limit theorems for entropy-regularized optimal transport on finite spaces and statistical applications
- [2309.03791] Adversarially Robust Learning with Optimal Transport Regularized Divergences
- [2505.12097] Proximal optimal transport divergences
- [2201.02082] An Homogeneous Unbalanced Regularized Optimal Transport model with applications to Optimal Transport with Boundary
- [2309.11666] Error estimate for regularized optimal transport problems via Bregman divergence
- [1412.5154] Iterative Bregman Projections for Regularized Transportation Problems

Source: https://www.emergentmind.com/topics/divergence-regularized-optimal-transport-c87e7b14-3385-45e5-bc4a-c35661d3f1f5