Statistical optimal transport (2407.18163v2)

Abstract: We present an introduction to the field of statistical optimal transport, based on lectures given at \'Ecole d'\'Et\'e de Probabilit\'es de Saint-Flour XLIX.

• The paper demonstrates how to estimate Wasserstein distances and transport maps from empirical data, addressing challenges like the curse of dimensionality.
• It introduces regularization methods such as entropic regularization and sliced Wasserstein distances to enable robust and efficient computations.
• The study highlights practical applications in machine learning, including domain adaptation and generative modeling, supported by strong theoretical guarantees.

Statistical Optimal Transport: An Overview

The concept of optimal transport (OT) dates back to a foundational problem proposed by Gaspard Monge in 1781, which seeks the most efficient way to redistribute material from one configuration to another. The field has since matured significantly, incorporating key developments such as Kantorovich’s relaxation, Brenier's theorem for maps, and computational algorithms like the Sinkhorn iteration. With broad connections to diverse areas ranging from differential geometry to machine learning, recent advances have increasingly focused on statistical OT, exploring its applications and theoretical underpinnings in a statistical framework.

Historical Context and Mathematical Foundations

Optimal transport’s evolution began with Monge’s problem, seeking a transport map that minimizes the cost of moving mass. The problem's complexity arises from its non-convex constraints, making it challenging to find solutions universally. Kantorovich’s relaxation in the mid-20th century provided a pivotal reformulation. By allowing for probabilistic couplings rather than deterministic mappings, Kantorovich introduced a linear programming framework, broadening OT’s applicability and solvability.

The development of OT has been deeply intertwined with measure theory and functional analysis, with the Wasserstein metric emerging as a standard for measuring distances between probability measures. The metric benefits from rich geometric interpretations, most notably through the lens of Riemannian geometry, thanks to the influential work of Otto and others, who formalized the differential structure on the space of probability measures.

Statistical Optimal Transport: Concepts and Tools

Statistical optimal transport is concerned with estimating transport maps and distances from empirical data, which introduces statistical noise and uncertainty. Specifically, two main tasks emerge: estimating Wasserstein distances and transport maps between probability measures derived from sample data.

Wasserstein Law of Large Numbers: A fundamental result is the Wasserstein law of large numbers, which guarantees that empirical measures converge to true population measures in Wasserstein distance as sample size increases. This convergence is slower than traditional parametric rates, often exhibiting a dimension-dependent rate of $n^{-1/d}$, evidencing the curse of dimensionality in high-dimensional spaces.

Dyadic Partitioning and Chaining: To mitigate these computational challenges, techniques like dyadic partitioning and chaining are employed. These methods systematically refine the analysis of point-cloud data to improve computation and estimation, focusing on partition strategies and covering numbers.

Estimating Transport Maps

Estimating transport maps (e.g., mappings that transform one probability distribution into another) is central to applications such as domain adaptation and generative models. The semidual problem formulation offers a promising framework by casting map estimation as an optimization problem over potential functions. Strong duality results help link solutions of the primal and dual problems, facilitating efficient computations and statistical analysis.

The statistical analysis often involves examining the empirical distribution’s deviation from the true distribution in function space, and bounding these deviations using the covering numbers of function spaces. Such semidual approaches benefit from rich geometric insights, providing a robust foundation for understanding statistical OT’s statistical properties.

Regularization Techniques in Statistical OT

The curse of dimensionality, a critical issue, is alleviated through regularization techniques. Entropic Regularization introduces entropy terms into the optimization problem, producing smooth solutions that are computationally tractable, thanks to the celebrated Sinkhorn algorithm. Sliced Wasserstein Distances consider marginal distributions over random projections, significantly reducing computational burden while preserving key metric properties. These techniques demonstrate that regularized OT converges rapidly, often achieving parametric rates of convergence and robust to high-dimensional noise.

Theoretical and Practical Implications

The implications of statistical OT are vast, both theoretically and practically:

1. Practical Applications: Machine learning tasks involving distributions, such as clustering and domain adaptation, heavily benefit from statistical OT. The Wasserstein distance provides a meaningful, semantically rich metric for comparing distributions of complex data types like images and texts.
2. Further Theoretical Developments: The interplay between OT’s geometric insights and statistical learning principles continues to inspire theoretical advances. Theoretical work is branching into the understanding of geodesic structures induced by Wasserstein spaces and their implications on the design of learning algorithms.
3. Computational Advancements: The establishment of techniques like entropic regularization positions OT as a computational tool that leverages modern hardware, offering scalable solutions for real-world data problems.

Conclusion

In summary, statistical optimal transport integrates the mathematical elegance and rigor of classical OT with the practical necessities of modern data science. By introducing regularization and focusing on empirical processes, it successfully translates theoretical constructs into a statistically viable toolkit for diverse applications. As computational power grows and datasets grow more complex, statistical OT stands as a pillar at the intersection of probability theory, statistics, and machine learning, promising continued innovation and application across myriad scientific domains.