Synthesis and Analysis of Data as Probability Measures with Entropy-Regularized Optimal Transport (2501.07446v3)

Abstract: We consider synthesis and analysis of probability measures using the entropy-regularized Wasserstein-2 cost and its unbiased version, the Sinkhorn divergence. The synthesis problem consists of computing the barycenter, with respect to these costs, of reference measures given a set of coefficients belonging to the simplex. The analysis problem consists of finding the coefficients for the closest barycenter in the Wasserstein-2 distance to a given measure. Under the weakest assumptions on the measures thus far in the literature, we compute the derivative of the entropy-regularized Wasserstein-2 cost. We leverage this to establish a characterization of barycenters with respect to the entropy-regularized Wasserstein-2 cost as solutions that correspond to a fixed point of an average of the entropy-regularized displacement maps. This characterization yields a finite-dimensional, convex, quadratic program for solving the analysis problem when the measure being analyzed is a barycenter with respect to the entropy-regularized Wasserstein-2 cost. We show that these coefficients, as well as the value of the barycenter functional, can be estimated from samples with dimension-independent rates of convergence, and that barycentric coefficients are stable with respect to perturbations in the Wasserstein-2 metric. We employ the barycentric coefficients as features for classification of corrupted point cloud data, and show that compared to neural network baselines, our approach is more efficient in small training data regimes.

• The paper introduces an entropy-regularized optimal transport method to efficiently compute barycenters and analyze probability measures from data.
• A key finding is the method's dimension-independent convergence rate and its stability against data perturbations, verified through experiments.
• The approach demonstrates superior performance in classifying corrupted point cloud data compared to neural networks, especially with limited training data.

Entropy-Regularized Optimal Transport for Probability Measures: Synthesis and Analysis

The paper "Synthesis and Analysis of Data as Probability Measures with Entropy-Regularized Optimal Transport" investigates the synthesis and analysis of probability measures via entropy-regularized optimal transport (OT). The research focuses on solving two primary problems: the synthesis problem, which involves finding the barycenter of multiple reference measures; and the analysis problem, which aims to determine the coefficients for the closest barycenter in Wasserstein-2 space to a given measure.

Key Contributions and Methodology

1. Entropy-Regularized Barycenter Computation: The paper employs the entropy-regularized Wasserstein-2 cost, a variant that allows the computation of barycenters more efficiently than traditional OT methods. The inclusion of the Sinkhorn divergence offers unbiased regularization, which is used to compute minimal barycenters under relatively weak assumptions on the measures.
2. Solution Characterization: A significant contribution is the characterization of regularized barycenters through fixed-point equations involving average entropic maps from the barycenter to the reference measures. This results in a well-defined convex optimization problem, essential for solving the synthesis problem.
3. Sample Complexity and Stability: A notable finding is the dimension-independent rate of convergence when estimating these barycenters from samples. The paper provides experimental verification of these rates, affirming the robustness of entropy-regularized OT methods against perturbations in the analyzed measure using Wasserstein-2 metric.
4. Applications in Classification: By leveraging barycentric coordinates as characteristic features, the research demonstrates the efficacy of the proposed approach in classifying corrupted point cloud data. Remarkably, when compared to neural network baselines like PointNet, this method showed superior performance, especially in scenarios with limited training data.

Discussion and Implications

The findings have important implications for both theoretical and practical applications. Practically, the method provides an efficient and robust framework for dealing with high-dimensional data and corrupted datasets in machine learning tasks. Theoretically, the entropy-regularized approach offers insights into the convergence properties of OT-based methods, particularly in settings where traditional computation could be problematic due to high dimensionality.

Future Directions

The research opens several avenues for further exploration:

• Enhanced Algorithms: Developing algorithms that can further exploit the dimensional independence in entropy-regularized OT, potentially improving efficiency and accuracy.
• Broader Applications: Extending the application capacity of barycenters, particularly in real-time and dynamic data settings like streaming data and signal processing.
• Theoretical Insights: Further analysis on the stability bounds of entropy-regularized OT to develop more robust applications that can handle extreme corruptions in data.

In summary, the paper establishes significant progress in understanding and implementing entropy-regularized OT for probability measure synthesis and analysis, balancing computational tractability with accuracy and stability. This has substantial implications for AI development, providing a foundation for implementing more adaptable, efficient, and robust models in data-rich domains.