On Wasserstein Two Sample Testing and Related Families of Nonparametric Tests (1509.02237v2)

Abstract: Nonparametric two sample or homogeneity testing is a decision theoretic problem that involves identifying differences between two random variables without making parametric assumptions about their underlying distributions. The literature is old and rich, with a wide variety of statistics having being intelligently designed and analyzed, both for the unidimensional and the multivariate setting. Our contribution is to tie together many of these tests, drawing connections between seemingly very different statistics. In this work, our central object is the Wasserstein distance, as we form a chain of connections from univariate methods like the Kolmogorov-Smirnov test, PP/QQ plots and ROC/ODC curves, to multivariate tests involving energy statistics and kernel based maximum mean discrepancy. Some connections proceed through the construction of a \textit{smoothed} Wasserstein distance, and others through the pursuit of a "distribution-free" Wasserstein test. Some observations in this chain are implicit in the literature, while others seem to have not been noticed thus far. Given nonparametric two sample testing's classical and continued importance, we aim to provide useful connections for theorists and practitioners familiar with one subset of methods but not others.

• The paper demonstrates that entropy-smoothing of the Wasserstein distance can interpolate between traditional Wasserstein and Energy Distance, enhancing test power and robustness.
• The paper bridges multivariate analysis techniques by linking distance-based tests with kernel methods like MMD, clarifying their relative strengths.
• The paper proposes a distribution-free Wasserstein test that mitigates dependency on specific distribution forms, improving practical applicability in diverse fields.

Insights into Wasserstein Two Sample Testing and Nonparametric Test Relationships

The paper, "On Wasserstein Two Sample Testing and Related Families of Nonparametric Tests," authored by Aaditya Ramdas, Nicol Garcia Trillos, and Marco Cuturi, explores the intricacies of nonparametric two-sample testing, emphasizing the role of the Wasserstein distance. This comprehensive paper elucidates the connections among various nonparametric tests, providing a valuable resource for both theorists and practitioners.

Nonparametric two-sample testing, also known as homogeneity testing, seeks to detect differences between two distributions based on sample data without assuming specific parametric forms. The literature is rich with different approaches, particularly distinguishing between tests designed for univariate (d=1) and multivariate (d>1) contexts. This paper attempts to unify these approaches via the lens of the Wasserstein distance, offering a deeper understanding of how these tests interconnect.

Core Contributions

1. Wasserstein Distance & Its Extensions:
   • The Wasserstein distance, or earth-mover's distance, is central to this work. Traditionally used to measure distances between probability distributions, the paper examines its application in two-sample testing scenarios. The authors explore the potential of an entropy-smoothed Wasserstein distance, demonstrating that varying the entropy parameter can interpolate between the traditional Wasserstein distance and the multivariate Energy Distance.
2. Multivariate Test Connections:
   • By introducing the notion of a smoothed Wasserstein distance, the authors reveal a direct link to the Energy Distance and the Kernel Maximum Mean Discrepancy (MMD). This finding is vital as it bridges two different schools of thought in multivariate analysis, thereby offering a more comprehensive understanding of the strengths and weaknesses of using distance-based versus kernel-based methods.
3. Univariate Test Analysis:
   • The paper extends the discussion to univariate tests, comparing the Wasserstein distance with popular graphical tools like QQ plots. It critiques traditional methods such as the Kolmogorov-Smirnov test, demonstrating situations where the Wasserstein metric offers clearer insights, particularly in statistical power and robustness.
4. Distribution-Free Wasserstein Tests:
   • Addressing a common limitation in statistical tests, the authors propose a distribution-free Wasserstein test by connecting it to concepts from ROC and ODC curves. This innovation is significant because it offers a practical solution to the dependency problem on underlying distribution forms, making it robust for various real-world applications.

Theoretical and Practical Implications

The theoretical implications of the paper are profound, offering a unified framework that connects distinct families of nonparametric tests under a single mathematical construct – the Wasserstein metric. Beyond theoretical elegance, the practical implications are considerable, providing a toolkit that is both versatile and adaptable across multiple domains, from bioinformatics to machine learning.

Potential Future Directions

This work opens up numerous avenues for further exploration:

• Robustness Against High Dimensions: Evaluating the efficacy of proposed methods in high-dimensional data scenarios where traditional tests struggle.
• Computational Efficiency: Given the computational intensity of calculating Wasserstein distances, further refinements in algorithmic efficiency could accelerate practical adoption.
• Empirical Evaluations: Systematic empirical studies to corroborate theoretical claims and explore the real-world impact across various datasets and use cases.

In conclusion, this paper not only enhances the understanding of how different nonparametric tests relate through the Wasserstein distance but also offers a practical framework for statistical testing that can be readily applied in diverse fields. Its insights are poised to significantly impact the field of data science, offering both a deeper theoretical comprehension and a robust practical foundation.