---
title: Sinkhorn Iterations in Optimal Transport
url: https://www.emergentmind.com/topics/sinkhorn-iterations
type: topic
---

# Sinkhorn Iterations in Optimal Transport

Sinkhorn iterations refer to a class of alternating matrix (or operator) scaling algorithms that compute entropy-regularized couplings in optimal transport (OT), as well as projections onto polytopes of matrices/tensors with prescribed marginals. This iterative proportional fitting paradigm realizes efficient solutions to entropy-regularized OT, is fundamental in computational and statistical optimal transport, and finds deep connections to mirror descent, stochastic processes, matrix analysis, PDEs, and scalable regularized linear algebra.

## 1. Mathematical Formulation and Canonical Algorithm

Given two discrete probability vectors $\mu \in \Delta_n$, $\nu \in \Delta_m$, and a cost matrix $C \in \mathbb{R}^{n \times m}$, the entropy-regularized OT problem is
\[
\min_{\pi \in \Pi(\mu, \nu)} \sum_{i,j} C_{ij}\,\pi_{ij} + \varepsilon \sum_{i,j} \pi_{ij} (\log \pi_{ij} - 1),
\]
where $\Pi(\mu, \nu) = \{\pi \geq 0 : \pi \mathbf{1} = \mu,\, \pi^T \mathbf{1} = \nu\}$ is the polytope of couplings [2311.16706][1612.02273][1706.00292].

The scaling fixed point characterization follows: define the Gibbs kernel $K = \exp(-C/\varepsilon)$ elementwise. The optimal plan $\pi^* = \operatorname{diag}(u)\,K\,\operatorname{diag}(v)$ for some vectors $u > 0$, $v > 0$ such that $u$ and $v$ solve
\[
u = \mu / (K v), \qquad v = \nu / (K^T u).
\]
The Sinkhorn iterations then alternate these updates:
\[
u^{(k+1)} = \mu / (K v^{(k)}), \qquad v^{(k+1)} = \nu / (K^T u^{(k+1)}),
\]
where division is elementwise [2311.16706][1706.00292].

## 2. Mirror Descent and Optimization-Theoretic Perspective

Sinkhorn iterations arise as alternating Bregman projections or mirror descent in the Kullback–Leibler geometry [2311.16706][2307.13370]. The function $F(\pi) = \mathrm{KL}(\pi\|\mu \otimes \nu)$ is convex over couplings, and the classical iteration may be interpreted as block coordinate mirror descent with step size $\gamma=1$ alternating projection onto row- and column-marginal sets. Discrete-time updates for the dual potentials (logarithms of $u$ and $v$) yield explicit forms:
\[
f^{n+1}(x) = -\log \int \exp\bigl(g^n(y) - C(x, y)/\varepsilon \bigr) \nu(dy),
\quad 
g^{n+1}(y) = g^n(y) - \log \frac{(\pi^n)_Y(y)}{\nu(y)},
\]
with the primal-dual structure making the iteration amenable to convex analysis and stochastic approximation, and allowing for flexible discretizations and generalizations [2311.16706].

## 3. Continuous-Time Sinkhorn Flow, PDE Limit, and Links to Wasserstein Dynamics

As the step size vanishes, time-rescaled Sinkhorn iterations converge to a continuous-time flow. The “Sinkhorn flow” satisfies:
\[
\frac{d}{dt} \log \pi(t; x, y)
= -\log \frac{(\pi(t))_Y(y)}{\nu(y)} + \mathbb{E}_{z \sim \pi(t)|x} \left[ \log \frac{(\pi(t))_Y(z)}{\nu(z)} \right].
\]
A dual potential flow perspective (in terms of Schrödinger potentials) gives:
\[
\dot{g}(t, y) = -\log \frac{(\pi(t))_Y(y)}{\nu(y)},
\]
with the coupling reconstructed as $\pi(t) = \operatorname{diag}(\exp(f(t)))\,K\,\operatorname{diag}(\exp(g(t)))$ [2311.16706][2307.16421].

This continuous-time limit has been rigorously analyzed as a Wasserstein mirror gradient flow, interpolating between relative entropy (KL) and quadratic cost via the mirror functional $U(\rho) = \frac{1}{2} W_2^2(\rho, \nu)$. The dynamics can be recast as a parabolic Monge--Ampère PDE in the convex potential $u_t$:
\[
\partial_t u_t(x) = f(x) - g(\nabla u_t(x)) + \log \det (\nabla^2 u_t(x)),
\]
again highlighting the synergy between discrete iteration and transport PDEs [2307.16421].

## 4. Convergence, Complexity, and Phase Transition Phenomena

In the classical square matrix scaling (“Sinkhorn–Knopp”) context, alternating normalization converges linearly in the Hilbert projective metric, and $O(\log n - \log \varepsilon)$ iterations suffice for dense matrices to achieve error $\varepsilon$ [2507.09711][1706.00292]. However, for matrices with subcritical density $\gamma < 1/2$, the required iteration count can degrade to $\Omega(\sqrt{n}/\varepsilon)$, revealing a sharp phase transition [2507.09711].

For entropy-regularized OT, non-asymptotic rates show geometric convergence in the potentials and in Wasserstein and relative entropy metrics. These exponential rates extend to non-compact and weakly convex settings under log-concavity assumptions, and adapt to log domain implementations for numerical stability [2304.04451][2304.06549][2304.04451].

Complexity per iteration is dominated by two matrix–vector multiplies, $O(nm)$ for dense kernels, but can be reduced to $O(n)$ using structured kernels and positive feature approximations [2006.07057][2206.09049]. Damped and stochastic variants further enhance robustness and numerical behavior in practice [2307.13370][2311.16706].

## 5. Generalizations and Connections to Broader Mathematical Structures

Sinkhorn-type iterations generalize to a wide class of scaling and constraint imposition problems:
- **Generalized Sinkhorn**: Extensions compute the proximal operator of regularized transport and appear in proximal splitting for inverse problems [1612.02273].
- **Operator Scaling and Sinkhorn**: Non-commutative (operator) versions, e.g., “Operator Sinkhorn Iteration,” enable scaling for completely positive maps, with applications to quantum information and combinatorial optimization [2603.12955][2207.08311].
- **Continuous, Torus, and Gaussian settings**: The algorithm adapts to $\mathbb{R}^d$ with unbounded cost, the torus $\mathbb{T}_L^d$ (with spectral and HJB techniques), and closed-form Riccati flows in Gaussian models [2412.18432][2304.04451][2304.06549].

Table: Key Sinkhorn Variants and Domains
| Domain/Problem            | Features                          | Source          |
|--------------------------|-----------------------------------|-----------------|
| Discrete OT (classical)  | Alternating scaling, matrix form  | [1706.00292][2311.16706]   |
| Operator scaling         | Hilbert-metric geodesics          | [2603.12955][2207.08311]   |
| Continuous/Quadratic OT  | PDE, parabolic Monge-Ampère       | [2304.04451][2307.16421]   |
| Gaussian models          | Riccati/Kalman recursions         | [2412.18432]    |


## 6. Algorithms, Stabilization, and Accelerated Schemes

The canonical (classical) implementation is numerically stable for moderate regularization but may require stabilization in the small-$\varepsilon$ regime due to floating point under/overflow. Log-domain Sinkhorn and periodic centering of dual potentials are standard [1612.02273][1706.00292][2502.03749]. Accelerated and hybrid algorithms blend Sinkhorn with sparse Newton steps, yielding super-exponential convergence when approaching the OT vertex [2401.12253][2502.03749], and variants—such as damped Sinkhorn, inexact Sinkhorn, and projected-gradient generalizations—enhance stability and extend applicability, e.g., to vector quantile regression [2311.16706][2603.21554].

## 7. Applications and Extensions

Sinkhorn iterations underpin scalable computation in numerous domains:
- **Machine Learning**: As differentiable “Sinkhorn layers” within deep architectures for generative models, with full support for back-propagation and GPU parallelism [1706.00292][2006.07057].
- **Inverse Problems**: As a black-box proximal operator in splitting schemes for inverse tomography and image reconstruction [1612.02273].
- **Adversarial Robustness**: Projection onto Wasserstein balls is efficiently solved using projected Sinkhorn variants [1902.07906].
- **Stochastic Control and Schrödinger Bridges**: Iterated Sinkhorn yields discrete-time approximations to Schrödinger bridges, with finite-sample error bounds combining statistical and algorithmic error [2510.22560].
- **Barycenters and Vector Quantile Regression**: Integration with barycentric OT and high-dimensional regression, with rigorous convergence under non-classical constraints [2307.13370][2603.21554].

A plausible implication is that the versatility and scalability of Sinkhorn-type iterations make them a central primitive for regularized transport in modern statistics, generative modeling, theoretical computer science, and applied mathematics.

## References

- [2311.16706] Sinkhorn Flow: A Continuous-Time Framework for Understanding and Generalizing the Sinkhorn Algorithm
- [2507.09711] Phase transition of the Sinkhorn-Knopp algorithm
- [1706.00292] Learning Generative Models with Sinkhorn Divergences
- [1612.02273] Generalized Sinkhorn iterations for regularizing inverse problems using optimal mass transport
- [2304.04451] Quantitative contraction rates for Sinkhorn algorithm: beyond bounded costs and compact marginals
- [2307.13370] Computational Guarantees for Doubly Entropic Wasserstein Barycenters via Damped Sinkhorn Iterations
- [2307.16421] Wasserstein Mirror Gradient Flow as the limit of the Sinkhorn Algorithm
- [2412.18432] Gaussian entropic optimal transport: Schrödinger bridges and the Sinkhorn algorithm
- [2401.12253] Accelerating Sinkhorn Algorithm with Sparse Newton Iterations
- [2006.07057] Linear Time Sinkhorn Divergences using Positive Features
- [2502.03749] PINS: Proximal Iterations with Sparse Newton and Sinkhorn for Optimal Transport
- [2603.21554] Sinkhorn algorithms for entropic vector quantile regression
- [2206.09049] Fast Sinkhorn II: Collinear Triangular Matrix and Linear Time Accurate Computation of Optimal Transport
- [2603.12955] Numerically stable variants of overrelaxation for operator Sinkhorn iteration
- [2207.08311] Shrunk subspaces via operator Sinkhorn iteration
- [1902.07906] Wasserstein Adversarial Examples via Projected Sinkhorn Iterations
- [1903.06778] Matrix scaling limits in finitely many iterations

Source: https://www.emergentmind.com/topics/sinkhorn-iterations