A New ECDF Two-Sample Test Statistic (2007.01360v1)

Abstract: Empirical cumulative distribution functions (ECDFs) have been used to test the hypothesis that two samples come from the same distribution since the seminal contribution by Kolmogorov and Smirnov. This paper describes a statistic which is usable under the same conditions as Kolmogorov-Smirnov, but provides more power than other extant tests in that vein. I demonstrate a valid (conservative) procedure for producing finite-sample p-values. I outline the close relationship between this statistic and its two main predecessors. I also provide a public R package (CRAN: twosamples [2018]) implementing the testing procedure in $O(N\log(N))$ time with $O(N)$ memory. Using the package's functions, I perform several simulation studies showing the power improvements.