---
title: 'Sinkhorn Divergence: OT, Geometry & Applications'
url: https://www.emergentmind.com/topics/sinkhorn-divergence
type: topic
---

# Sinkhorn Divergence: OT, Geometry & Applications

The Sinkhorn divergence is a geometric functional for comparing probability measures, derived by debiasing the entropic-regularized optimal transport (OT) cost. It interpolates between the Wasserstein metric and kernel-based discrepancies such as Maximum Mean Discrepancy (MMD), and is central in modern computational optimal transport, statistical inference, and machine learning. The Sinkhorn divergence admits a rigorous Riemannian differential structure, explicit geometric and metric properties, efficient computational algorithms, and deep connections to RKHS theory, convex analysis, and gradient flows.

## 1. Mathematical Definition and Debiasing Principle

Given a compact metric space $X$, probability measures $\mu,\nu\in\mathcal P(X)$, a continuous cost $c:X\times X\to\mathbb{R}_+$, and entropic regularization parameter $\varepsilon>0$, the classical entropic OT cost is
\[
\mathrm{OT}_\varepsilon(\mu,\nu) = \inf_{\pi\in\Pi(\mu,\nu)} \int_{X\times X} c(x,y)\, d\pi(x,y) + \varepsilon\, \mathrm{KL}(\pi\|\mu\otimes\nu)
\]
where $\Pi(\mu,\nu)$ is the set of couplings with marginals $\mu,\nu$ and $\mathrm{KL}(\pi\|\rho)=\int \frac{d\pi}{d\rho}\log\frac{d\pi}{d\rho} - 1\,d\rho$.

This regularized cost does not vanish on the diagonal: $\mathrm{OT}_\varepsilon(\mu,\mu)>0$, so it is not a distance. The Sinkhorn divergence is obtained via debiasing:
\[
S_\varepsilon(\mu,\nu) := \mathrm{OT}_\varepsilon(\mu,\nu) - \tfrac{1}{2} \mathrm{OT}_\varepsilon(\mu,\mu) - \tfrac{1}{2} \mathrm{OT}_\varepsilon(\nu,\nu)
\]
This correction ensures $S_\varepsilon(\mu,\mu)=0$ and $S_\varepsilon(\mu,\nu) \geq 0$, with equality if and only if $\mu=\nu$ under universal kernel assumptions [2405.04987].

## 2. Geometric Structure and Riemannian Geometry

The Sinkhorn divergence induces a Riemannian metric structure on the space of measures, obtained via differentiating $S_\varepsilon$ with respect to mass perturbations. For a path $\mu_t = \mu + t\,\delta\mu$ with $\delta\mu\in M_0(X)$ (zero integral), the metric tensor at $\mu$ is:
\[
g_\mu(\delta\mu_1, \delta\mu_2) = \frac{\varepsilon}{2} \langle \delta\mu_1, (I - K_\mu^2)^{-1} H_\mu[\delta\mu_2] \rangle
\]
Here, $K_\mu$ and $H_\mu$ are integral operators defined via the Schrödinger dual potential and the entropic kernel $k_\mu(x,y) = \exp((f_{\mu,\mu}(x) + f_{\mu,\mu}(y) - c(x,y))/\varepsilon)$. This metric "lifts" the geometry of $X$ to the measure space and canonically identifies the tangent space at $\mu$ with an RKHS dual space [2405.04987].

By charting the space via a fixed RKHS $\mathcal H_c$ (associated to $k_c(x,y)=e^{-c(x,y)/\varepsilon}$), the geometry becomes tractable and all tangent spaces embed smoothly into $\mathcal H_c$. The geodesic equations can be explicitly written in $\mathcal H_c$ coordinates.

## 3. Metric, Topological, and Convexity Properties

**Distance and topology:** The metric induced by the energy minimization of $g_\mu$ (denoted $d_E$) fulfills symmetry, positivity, triangle inequality, and metrizes the weak-* topology on $\mathcal P(X)$. Explicitly, there exists
\[
\sqrt{\frac{\varepsilon}{2}}\|\beta_1 - \beta_0\|_{\mathcal H_c} \le d_E(\mu_0,\mu_1) \le \sqrt{\frac{\varepsilon}{2}}\pi C\|\beta_1 - \beta_0\|_{\mathcal H_c}
\]
with constants and map $\mu \mapsto \beta \in \mathcal H_c$ [2405.04987].

**Non-metricity of $\sqrt{S_\varepsilon}$:** The square root $\sqrt{S_\varepsilon}$ does not satisfy the triangle inequality. Explicit counterexamples (e.g., mixtures of Diracs or Gaussians) demonstrate failure of the metric property for any power $S_\varepsilon^\alpha,\,\alpha\ge1/2$ [2405.04987].

**Convexity:** $S_\varepsilon$ is not jointly convex in $(\mu,\nu)$: its Hessian can be indefinite, as established via two-point examples and asymptotic arguments. This differentiates it from the classical Wasserstein distance, which is jointly convex [2405.04987].

**Smoothness:** The divergence is infinitely differentiable inside the probability simplex, benefiting optimization and statistical procedures [2206.13948][2405.04987].

## 4. Computation, Algorithmics, and Scaling

**Sinkhorn-Knopp Iterations:** The functional is computed efficiently using the Sinkhorn-Knopp scaling algorithm. For discrete measures and Gibbs kernel $K_{ij} = \exp(-c_{ij}/\varepsilon)$, the dual potentials $u,v$ are obtained via alternating updates:
\[
u \leftarrow a/(K v),\qquad v \leftarrow b/(K^\top u)
\]
and the optimal coupling is $\pi^*_{ij} = u_i K_{ij} v_j$. The computational complexity per iteration is $O(nm)$, but can be reduced using positive feature decompositions, RKHS approaches, or hierarchical matrix representations for structured problems [2006.07057][2004.12511].

**Automatic Differentiation:** The smooth dependence of $S_\varepsilon$ on input measures permits efficient gradient computation using backpropagation through fixed-point iterations [1706.00292].

**Statistical Efficiency:** The sample complexity and convergence rates interpolate between $n^{-1/2}$ for MMD and $n^{-1/d}$ for classical OT, depending on $\varepsilon$ [1706.00292][2006.08172].

**Large-scale/linear time approximations:** For large $n$, positive feature maps enable $O(n r)$ per iteration scaling without forming explicit full kernel matrices [2006.07057]. Hierarchical low-rank methods can further reduce runtimes to $O(n\log^3 n)$ under Kronecker or block-structure cost assumptions [2004.12511].

## 5. Applications and Practical Implementations

Sinkhorn divergence is used as a loss in generative modeling (GAN training), domain adaptation, deep feature alignment, and statistical registration problems.

- **Generative Models:** Sinkhorn-based losses are integrated in adversarial training to enable robust optimization over distributions. The Sinkhorn Natural Gradient (SiNG) algorithm uses the explicit Riemannian structure for efficient optimization over parameterized measure families [2011.04162].
- **Statistical Registration:** In diffeomorphic image or shape registration, Sinkhorn divergences offer smooth, fewer-minima losses than classical OT, and admit automatic differentiation and statistical guarantees on consistency [2206.13948].
- **Deep Representation Learning:** The divergence allows stack-wise alignment in deep regression architectures, e.g., N-BEATS, and provides thermostat-like regularization in feature space across domains [2305.15196].
- **Coreset Compression:** Convexly-weighted coresets for Sinkhorn divergence can be constructed via second-order functional expansions, reducing the problem to MMD minimization in the RKHS induced by the divergence’s Hessian. This enables selection of $O(\log n)$ representative points, critical for scaling up Sinkhorn computations [2504.20194].

## 6. Generalizations, Gradient Flows, and Theoretical Extensions

**Gradient Flows:** Substituting the Wasserstein metric with the Sinkhorn divergence in the JKO (minimizing movement) scheme defines a new gradient flow in measure space. The evolution is best understood in a variable change to the RKHS $\mathcal H_c$, where it becomes a monotone operator evolution. This flow is well-posed, dissipative, non-expansive in RKHS norm, and converges globally to minimizers of the potential energy. Notably, it regularizes mass-splitting singularities and allows overcoming potential barriers that would trap classical Wasserstein flows [2511.14278].

**Drift Field Interpretation:** In generative dynamics, the Sinkhorn divergence induces a drift vector field decomposed into cross (model-to-target) and self-repulsion terms, both computed using entropic couplings via two-sided Sinkhorn scaling. This structure robustifies parameter updates, ensures identifiability (zero drift iff measures agree), and improves empirical stability over one-sided "drifting" dynamics [2603.12366].

**Unbalanced and $f$-Divergence Regularized OT:** The Sinkhorn divergence framework extends to unbalanced transport (measures with varying mass) by relaxing marginal constraints via Csiszár $f$-divergences [1910.12958]. The generalized Sinkhorn divergence incorporates arbitrary smooth convex penalty functions, leading to a family of divergences and dual algorithms tailored to application constraints on sparsity and statistical robustness [2105.14337].

**Asymptotic Distributions and Bootstrap Theory:** The Sinkhorn divergence admits Hadamard differentiability with respect to the input measures, enabling a precise central limit theorem for empirical estimators. Limit distributions (under the null and under alternatives), bootstrap consistency, and asymptotic minimax efficiency are derived, providing a full statistical toolkit for hypothesis testing, confidence intervals, and sample complexity controls [2206.13948][2207.08683][1812.09150].

## 7. Limitations and Negative Results

Although the Sinkhorn divergence is strictly positive definite and metrizes weak-* convergence, it lacks true joint convexity and its square-root is not a metric, failing the triangle inequality [2405.04987]. These features distinguish it structurally from classical Wasserstein distances and require caution when interpreting $S_\varepsilon$ as a metric or deploying it in convex optimization frameworks.

---

The Sinkhorn divergence serves as a foundational object in computational optimal transport and geometric statistics, combining computational tractability, rich geometric structure, and broad applicability in high-dimensional data science tasks [2405.04987][1810.08278][1706.00292][2011.04162].

Source: https://www.emergentmind.com/topics/sinkhorn-divergence