Online Sinkhorn Algorithms

• Online Sinkhorn is a scalable iterative method that computes entropic-regularized optimal transport distances by updating dual potentials directly from data streams.
• It employs stochastic approximation and mini-batch updates to achieve robust convergence and reduced memory usage compared to batch methods.
• Extensions such as Mirror and Compressed Online Sinkhorn enhance efficiency by controlling computational complexity while preserving convergence rates.

Online Sinkhorn algorithms constitute a family of scalable iterative methods for computing entropic-regularized optimal transport (OT) distances and more general convex objectives over transport polytopes, leveraging sample streams and stochastic approximation. Unlike classical batch Sinkhorn methods that operate on static, finite measures, online Sinkhorn approaches update nonparametric representations of dual potentials or transport plans directly from incoming sample streams. These methods are robust to noise, require only one pass through data, and have favorable convergence guarantees, making them suitable for large-scale, streaming, or online machine learning applications.

1. Foundational Problem Formulations

The core objective is to compute the entropic-regularized OT cost between two probability measures $\alpha$ and $\beta$ on a compact space $\mathcal{X}$ with cost function $C : \mathcal{X} \times \mathcal{X} \to \mathbb{R}$,

$$\min_{\pi \in \Pi(\alpha, \beta)} \int_{\mathcal{X} \times \mathcal{X}} C(x, y) \, d\pi(x, y) + \varepsilon \, KL(\pi \| \alpha \otimes \beta),$$

where $\varepsilon > 0$, and $\Pi(\alpha, \beta)$ denotes the set of couplings. The dual formulation is smooth and strictly convex,

$$\max_{f, g \in C(\mathcal{X})} \langle f, \alpha \rangle + \langle g, \beta \rangle - \varepsilon \int \exp\left(\frac{f(x)+g(y)-C(x,y)}{\varepsilon}\right) d\alpha(x) d\beta(y).$$

For discrete measures $\mu \in \Delta_m$, $\nu \in \Delta_n$, optimization is over the transport polytope

$$T(\mu, \nu) = \left\{\gamma \in \mathbb{R}_+^{m \times n} : \gamma \mathbf{1}_n = \mu, \, \gamma^\top \mathbf{1}_m = \nu\right\}$$

for general convex $f$, with special cases including linear OT and entropic-regularized OT (Mensch et al., 2020, Wang et al., 2023, Ballu et al., 2022).

2. Stochastic and Online Sinkhorn Algorithms

The classic Sinkhorn algorithm alternates soft $C$-transforms to update dual potentials on discretized measures, requiring full batch access. Mensch and Peyré introduced a stochastic framework for streaming data:

• Dual potentials are represented by growing kernel expansions:

$$e^{-f_t(x)} = \sum_{i=1}^{N_t} w^{(f)}_{i, t} \exp(-C(x, y_i)/\varepsilon),$$

and similarly for $g_t(y)$ in terms of past samples $\{x_j, y_i\}$.

• Update steps use mini-batches $(X_t, Y_t)$:

$$u_{t+1} = (1-\eta_t)u_t + \eta_t \widehat{T}_{\beta_t}(v_t), \quad v_{t+1} = (1-\eta_t)v_t + \eta_t \widehat{T}_{\alpha_t}(u_{t+1})$$

with $\eta_t$ vanishing, and unbiased Monte Carlo approximations of soft $C$-transforms.

• Per-iteration cost is $O(n N_t)$ for batch size $n$ and kernel centers $N_t$ (Mensch et al., 2020, Wang et al., 2023).

For the discrete convex setup, the Mirror Sinkhorn algorithm executes a single-loop of mirror descent with a negative-entropy mirror map, alternating Sinkhorn-style marginal projections:

1. Mirror step: $\gamma_t' = \gamma_t \odot \exp(-\eta_t g_t)$,
2. Normalizations alternate row- or column-scaling via Diag operators,
3. Averaging: Optional running average $\bar{\gamma}_t$ in stochastic settings,
4. Output: Last or averaged iterate, depending on the regime (Ballu et al., 2022).

3. Convergence Analysis and Theoretical Guarantees

Online Sinkhorn algorithms admit strong convergence properties:

• For streaming settings with growing batch sizes $b_t = t^{2a}$, decreasing step sizes $\eta_t = t^b$, and $-1 < b < -1/2$, the expected variation-norm error for the duals decays as $O(N^{-a/(2a+1)})$, where $N$ is the total sample count (Wang et al., 2023).
• Previous results only established $O(N^{-1+\epsilon})$ rates; refined analyses lead to sharper algebraic exponents tied to $a, b$ (Mensch et al., 2020, Wang et al., 2023).
• In the discrete finite case, Mirror Sinkhorn yields, for $B$-Lipschitz convex $f_t$, online regret bounds $R_T \leq (9B/8)\sqrt{T}(2+\log T)$ and constraint violation sums $\leq (3/2)\sqrt{T}(2+\log T)$ (Ballu et al., 2022).
• When $f$ is strongly convex and smooth in KL divergence, the method obtains $O(1/T)$ aggregate error up to logarithmic factors.
• The Bregman decomposition underpins the proofs, leveraging contraction and Pinsker’s inequalities.

4. Algorithmic Extensions: Compression and Mirror Descent

To counteract the unbounded memory and kernel-evaluation cost, Compressed Online Sinkhorn periodically reduces the support of the dual potentials using moment-preserving compression:

• Compression by moment-matching: Replace the kernel expansion with a reduced atomic measure $\hat{\mu}$ matching selected functional moments (e.g., polynomials in Gaussian Quadrature for $d=1$, Fourier moments for any $d$). For Fourier-based compression, error bounds are $O((\log m)^d/m^\zeta)$ for $\zeta < 1$.
• Theoretical result: If $\hat{f}_t$ (compressed) satisfies $\sup_x |f_t(x)-\hat{f}_t(x)| = O(m_t^{-\zeta})$, and $m_t = t^{(a-b)/\zeta}$, the compressed algorithm preserves the $O(N^{-a/(2a+1)})$ convergence rate for an appropriate compression scheme (Wang et al., 2023).

Mirror Sinkhorn is further extensible to non-linear, stochastic, or multi-marginal objectives and to dual-space variants.

5. Empirical Performance and Implementation Considerations

Empirical studies validate that:

• Standard Online Sinkhorn matches batch Sinkhorn objectives with substantially less memory and enables online warm-starting of batch algorithms, yielding speed-ups (e.g., $2-4\times$ for 3D shape matching).
• Compressed Online Sinkhorn reduces wall-clock times by $2.5-4\times$ (e.g., in 5D GMMs), maintaining matching error decay envelopes with reduced evaluation and storage costs.
• In convex discrete settings, Mirror Sinkhorn achieves zero-bias convergence and outperforms entropic-regularized Sinkhorn in accuracy, especially under stochastic-noise settings (Ballu et al., 2022).
• Typical choices for regularization ($\varepsilon = 0.3-0.5$), step-sizes ($b \approx -0.6$), batching ($a \approx 1.2-1.7$), and compression levels are empirically justified (Wang et al., 2023).

6. Comparisons, Advantages, and Limitations

Algorithm	Per-iteration Cost	Memory (w.r.t. N)	Constraint Consistency	Rate
Batch Sinkhorn	$O(N^2)$	$O(N^2)$	Finite sample bias	$O(N^{-2/3})$ (dual error)
Online Sinkhorn	$O(n N_t)$	$O(N_t)$	Consistent	$O(N^{-a/(2a+1)})$, $N_t$ growing
Mirror Sinkhorn (discrete)	$O(m n)$	$O(m n)$	Marginal error $O(1/\sqrt{t})$	$O(1/\sqrt{T})$, $O(1/T)$ (strong convex.)
Compressed Online Sinkhorn	$O(m n_t)$	$O(m_t)$	Preserved rate	Same as Online, lower asymp. cost

• Online Sinkhorn is single-pass and streaming-friendly, in contrast to two-phase batch approaches (Mensch et al., 2020, Wang et al., 2023).
• Mirror Sinkhorn avoids nested loops and the need to know a target precision $\varepsilon$ in advance (Ballu et al., 2022).
• Compression schemes further control memory and arithmetic complexity, yielding asymptotic computational exponents as low as $2 + 1/a$ (Wang et al., 2023).
• A limitation is that marginal constraint satisfaction in stochastic methods is typically $O(1/\sqrt{t})$, slower than exponentially fast convergence in highly entropic batch Sinkhorn. No $O(1/t)$ rate is guaranteed in the non-strongly convex case for Mirror Sinkhorn (Ballu et al., 2022).

7. Applications and Extensions

Online Sinkhorn methods are applicable across OT-based tasks in machine learning, including domain adaptation, WGAN training, barycenter computation, and matching in high-dimensional or streaming-data settings:

• These algorithms extend naturally to unbalanced OT, multi-marginal OT, and semi-discrete OT, and to robust optimization under gradient noise.
• Possible future directions include adaptive annealed scaling of projections, more efficient dual-space stochastic mirror-descent variants, and applications to nonconvex or measure-valued settings (Ballu et al., 2022, Wang et al., 2023, Mensch et al., 2020).