Biased bootstrap sampling for efficient two-sample testing

Abstract: The so-called 'energy test' is a frequentist technique used in experimental particle physics to decide whether two samples are drawn from the same distribution. Its usage requires a good understanding of the distribution of the test statistic, T, under the null hypothesis. We propose a technique which allows the extreme tails of the T-distribution to be determined more efficiently than possible with present methods. This allows quick evaluation of (for example) 5-sigma confidence intervals that otherwise would have required prohibitively costly computation times or approximations to have been made. Furthermore, we comment on other ways that T computations could be sped up using established results from the statistics community. Beyond two-sample testing, the proposed biased bootstrap method may provide benefit anywhere extreme values are currently obtained with bootstrap sampling.