Goodness of fit by Neyman-Pearson testing (2305.14137v2)
Abstract: The Neyman-Pearson strategy for hypothesis testing can be employed for goodness of fit if the alternative hypothesis is selected from data by exploring a rich parametrised family of models, while controlling the impact of statistical fluctuations. The New Physics Learning Machine (NPLM) methodology has been developed as a concrete implementation of this idea, to target the detection of new physical effects in the context of high energy physics collider experiments. In this paper we conduct a comparison of this approach to goodness of fit with others, in particular with classifier-based strategies that share strong similarities with NPLM. From our comparison, NPLM emerges as the more sensitive test to small departures of the data from the expected distribution and not biased towards detecting specific types of anomalies. These features make it suited for agnostic searches for new physics at collider experiments. Its deployment in other scientific and industrial scenarios should be investigated.
- John Wiley & Sons, Ltd, ISBN 9783527677320, https://doi.org/10.1002/9783527677320.ch3 (2013), https://onlinelibrary.wiley.com/doi/pdf/10.1002/9783527677320.ch3.
- R. D. Cousins, Lectures on Statistics in Theory: Prelude to Statistics in Practice (2018), 1807.05996.
- J. H. Friedman, On multivariate goodness of fit and two sample testing, eConf C030908, THPD002 (2003).
- M. Williams, How good are your fits? Unbinned multivariate goodness-of-fit tests in high energy physics, JINST 5, P09004 (2010), 10.1088/1748-0221/5/09/P09004, 1006.3019.
- C. Weisser and M. Williams, Machine learning and multivariate goodness of fit (2016), 1612.07186.
- G. Claeskens and N. L. Hjort, Goodness of fit via non-parametric likelihood ratios, Scandinavian Journal of Statistics 31(4), 487 (2004).
- Model-Independent Detection of New Physics Signals Using Interpretable Semi-Supervised Classifier Tests (2021), 2102.07679.
- D. Lopez-Paz and M. Oquab, Revisiting classifier two-sample tests, In International Conference on Learning Representations (2017), 1610.06545.
- J. Neyman and E. S. Pearson, On the Problem of the Most Efficient Tests of Statistical Hypotheses, Phil. Trans. Roy. Soc. Lond. A 231(694-706), 289 (1933), 10.1098/rsta.1933.0009.
- S. Baker and R. D. Cousins, Clarification of the Use of Chi Square and Likelihood Functions in Fits to Histograms, Nucl. Instrum. Meth. 221, 437 (1984), 10.1016/0167-5087(84)90016-4.
- Model-Independent Jets plus Missing Energy Searches, Phys. Rev. D 79, 015005 (2009), 10.1103/PhysRevD.79.015005, 0809.3264.
- R. T. D’Agnolo and A. Wulzer, Learning New Physics from a Machine, Phys. Rev. D 99(1), 015014 (2019), 10.1103/PhysRevD.99.015014, 1806.02350.
- Learning multivariate new physics, Eur. Phys. J. C 81(1), 89 (2021), 10.1140/epjc/s10052-021-08853-y, 1912.12155.
- Learning new physics from an imperfect machine, Eur. Phys. J. C 82(3), 275 (2022), 10.1140/epjc/s10052-022-10226-y, 2111.13633.
- Learning new physics efficiently with nonparametric methods, Eur. Phys. J. C 82(10), 879 (2022), 10.1140/epjc/s10052-022-10830-y, 2204.02317.
- Fast kernel methods for Data Quality Monitoring as a goodness-of-fit test (2023), 2303.05413.
- C. Krause and D. Shih, Fast and accurate simulations of calorimeter showers with normalizing flows, Phys. Rev. D 107(11), 113003 (2023), 10.1103/PhysRevD.107.113003, 2106.05285.
- Evaluating generative models in high energy physics, Phys. Rev. D 107(7), 076017 (2023), 10.1103/PhysRevD.107.076017, 2211.10295.
- Nplm: Learning multivariate new physics, 10.5281/zenodo.4442665 (2021).
- Anonymous, Dqm for drift tube chambers, 10.5281/zenodo.7128223 (2022).
- P. Eller and L. Shtembari, A goodness-of-fit test based on a recursive product of spacings, JINST 18(03), P03048 (2023), 10.1088/1748-0221/18/03/P03048, 2111.02252.
Collections
Sign up for free to add this paper to one or more collections.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.