Entropy-Regularized Optimal Transport

• Entropy-regularized optimal transport is defined by incorporating an entropic penalty into classical OT to improve computational tractability and guarantee uniqueness.
• The Sinkhorn divergence method yields smooth dual solutions and scalable algorithms, facilitating practical applications in high-dimensional settings.
• Central limit theorems and bootstrap techniques validate the statistical behavior of the Sinkhorn loss, enabling reliable hypothesis testing between distributions.

Entropy-regularized optimal transport, frequently referenced as the Sinkhorn divergence in computational settings, is a modification of the classical optimal transport (OT) problem that incorporates an entropic penalty into the transport objective. This regularization confers fundamental advantages: computational tractability via smooth dual solutions, strict convexity and uniqueness in the OT functional, and a direct connection to scalable algorithms such as Sinkhorn's method. Recently, the statistical behavior, inferential applications, and convergence properties of entropy-regularized OT on finite spaces have been rigorously analyzed, enabling principled statistical hypothesis testing between multivariate distributions and quantifying the trade-offs in convergence as the regularization parameter vanishes.

1. Mathematical Formulation and Regularization Principle

Given two probability measures $a$ and $b$ supported on a finite space, the entropy-regularized Wasserstein-$p$ cost is

$$W_{p,\varepsilon}^p(a, b) = \min_{T \in \mathbb{R}_{+}^{n \times n}} \left\{ \langle T, C \rangle - \varepsilon E(T) \;:\; T\mathbf{1} = a, \; T^{\top}\mathbf{1} = b \right\}$$

where $C$ is the cost matrix (e.g., induced by a metric on the ground space), and the entropy $E(T) = -\sum_{i,j} T_{ij} \log T_{ij}$. The entropic penalty parameter $\varepsilon > 0$ interpolates between the classic OT problem ($\varepsilon\to 0$) and a maximal entropy (uniform) plan ($\varepsilon\to\infty$).

A bias-corrected version, the "Sinkhorn loss," is defined by centering: $$\text{Sinkhorn loss}(a, b) = W_{p,\varepsilon}^p(a, b) - \frac12 W_{p,\varepsilon}^p(a, a) - \frac12 W_{p,\varepsilon}^p(b, b)$$ ensuring $\text{Sinkhorn loss}(a,a) = 0$ and improved metric behavior.

2. Statistical Theory: Central Limit Theorems

Let $a$ be estimated from multinomial sampling (empirical measure $\hat{a}_n$), and $b$ be either known or itself estimated by an independent empirical measure $\hat{b}_m$. For the empirical Sinkhorn divergence,

$$\sqrt{n}\left( W_{p,\varepsilon}^p (\hat{a}_n, b) - W_{p,\varepsilon}^p (a, b) \right) \to \langle G, u_\varepsilon \rangle$$

in distribution as $n \to \infty$, where $G$ is a Gaussian random vector with covariance given by the multinomial distribution and $u_\varepsilon$ the dual variable at the true measure. In the two-sample case ($\hat{a}_n, \hat{b}_m$),

$$\rho_{n,m}\left( W_{p,\varepsilon}^p (\hat{a}_n, \hat{b}_m) - W_{p,\varepsilon}^p (a, b) \right) \to \sqrt{\gamma}\langle G, u_\varepsilon \rangle + \sqrt{1-\gamma}\langle H, v_\varepsilon \rangle$$

with $\rho_{n,m} = \sqrt{mn/(m+n)}$, $\gamma = m/(m+n)$, and $H$ an independent Gaussian with corresponding covariance.

When testing for distributional equality, the Sinkhorn loss statistic exhibits further nuances:

• Under $a \neq b$, a classical first-order CLT applies.
• Under $a = b$ (null hypothesis), the first derivative vanishes, requiring a second-order (quadratic) delta-method analysis. The limiting law is then a mixture of chi-squared random variables,

$$n W_{p,\varepsilon}^p (\hat{a}_n, a) \to \frac{1}{2} \sum_i \lambda_i \chi^2_{(1)}$$

where $\lambda_i$ are eigenvalues of a matrix constructed from the Hessian of the entropy-regularized functional and the covariance of the multinomial process.

3. Inferential Methods: Statistical Testing and Bootstrap

These CLTs render empirical Sinkhorn divergences suitable for hypothesis testing between multivariate distributions. The test is constructed by comparing the empirical Sinkhorn divergence (or loss) to its sampling distribution under the null.

To address finite-sample regimes and enable accurate inference, a bootstrap procedure is devised:

• Empirical measures are resampled (from multinomial counts).
• Test statistics are re-centered, leveraging the delta-method.
• In null cases where the first derivative is degenerate, a Babu-corrected bootstrap incorporates second-order effects.

This yields bootstrap approximations of variance and quantile estimates for the test statistic, facilitating valid hypothesis tests in practical multivariate problems.

4. Relation to Classical (Unregularized) Optimal Transport

By letting the regularization $\varepsilon \to 0$ at a suitable rate with respect to the sample size (specifically, $\sqrt{n} \varepsilon \log(1/\varepsilon) \to 0$ in one sample), the empirical Sinkhorn divergence converges (in law) to the same limit as the classical unregularized OT cost established in prior work (Sommerfeld and Munk 2016). Thus, the Sinkhorn divergence is statistically consistent with the classical Wasserstein distance in the small-$\varepsilon$ regime, while being computationally advantageous for moderate $\varepsilon$.

5. Practical Performance and Data-Driven Illustrations

Extensive simulations were conducted:

• Discrete measures supported on grids (e.g., $5\times5$ and $20\times20$ in $\mathbb{R}^2$) were used to demonstrate convergence of the empirical Sinkhorn loss to the predicted Gaussian or chi-squared mixture limiting distributions.
• Bootstrap-based law approximations were shown to be accurate in two-sample settings.
• Power analysis (probability of correctly rejecting the null under alternatives) revealed dependence on both the regularization parameter $\varepsilon$ and grid size: discrimination power may decrease as $\varepsilon$ increases or grid size decreases.

Real-data applications were presented using color histograms from digital images (RGB space, grid-based histogram): empirical Sinkhorn losses and the associated bootstrap test lead to strong rejection of the hypothesis that image color distributions (e.g., Autumn vs. Winter) are equal; homogeneity testing within a season (splitting a set of Autumn images) did not detect significant difference.

These findings illustrate robust practical performance for high-dimensional, multi-modal, and real-world statistical inference tasks where unregularized OT would be computationally infeasible.

6. Future Directions and Open Problems

Major avenues for further research include:

• Generalizing multi-sample and high-dimensional comparison tests analogous to MANOVA, potentially via Wasserstein barycenter constructions.
• Developing analytical tools for the eigenvalues of the Sinkhorn loss Hessian, giving explicit control of limiting distributions under the null and sharpening efficiency/power understanding.
• Providing principled data-driven methodologies for selecting the regularization parameter $\varepsilon$, in view of its dual role: computational stability and test sensitivity.
• Expanding applications in image analysis, generative modeling, and unsupervised learning, where entropy-regularized OT balances accuracy with algorithmic speed.

7. Summary of Methodological and Conceptual Advances

Topic	Key Results or Insights
Central limit theorems	CLTs for empirical Sinkhorn divergence and loss, covering both one-sample and two-sample settings, with Gaussian and chi-squared mixture limits depending on null/alternative.
Statistical testing and bootstrap	Empirical Sinkhorn loss enables robust tests for distributional equality; bootstrap procedures (with Babu correction where needed) provide calibrated finite-sample inference.
Comparison to unregularized OT	As $\varepsilon \to 0$, the empirical regularized OT matches classical Wasserstein limit laws; thus, entropy-regularized OT is consistent both computationally and statistically.
Practical application and verification	Theoretical results are confirmed empirically on grid-based and color histogram data; test efficacy depends on regularization, grid resolution, and is effective in real scenarios.
Future research and open problems	Open topics include: high-dimensional inference, eigenvalue analytics for null limit laws, $\varepsilon$ selection methods, and broader unsupervised learning applications.

These analytic, algorithmic, and practical results substantially deepen the foundation and practical utility of entropy-regularized optimal transport in empirical statistical settings, highlighting its dual role as a computational tool and as a theoretically robust metric for inference on probability measures (Bigot et al., 2017).