Asymptotic formulae for likelihood-based tests of new physics (1007.1727v3)

Abstract: We describe likelihood-based statistical tests for use in high energy physics for the discovery of new phenomena and for construction of confidence intervals on model parameters. We focus on the properties of the test procedures that allow one to account for systematic uncertainties. Explicit formulae for the asymptotic distributions of test statistics are derived using results of Wilks and Wald. We motivate and justify the use of a representative data set, called the "Asimov data set", which provides a simple method to obtain the median experimental sensitivity of a search or measurement as well as fluctuations about this expectation.

• The paper develops asymptotic formulae that reduce reliance on Monte Carlo simulations by using profile likelihood methods.
• The paper introduces the 'Asimov data set' to efficiently estimate median sensitivity and incorporate systematic uncertainties.
• The paper demonstrates applications in particle physics, enabling rapid computation of discovery probabilities and confidence intervals.

Asymptotic Formulae for Likelihood-Based Tests of New Physics

The paper "Asymptotic formulae for likelihood-based tests of new physics" by Cowan et al. presents a comprehensive paper of likelihood-based statistical tests aimed at high energy physics applications. The authors focus on methodologies for both discovering new phenomena, such as Higgs boson production, and constructing confidence intervals for model parameters. Their paper emphasizes frequentist methods, profile likelihood ratios, and asymptotic methods, offering explicit formulae derived using the foundational works of Wilks and Wald.

At the core of their research is the ability to quantitatively account for systematic uncertainties within the framework of statistical tests. By employing asymptotic expansions, the authors substantially reduce the computational costs typically associated with Monte Carlo methods when estimating the significance of experimental findings, especially for sizable discovery thresholds, such as those seen when $Z \geq 5$ for significant discoveries in particle physics.

The authors introduce the concept of the "Asimov data set." This innovative approach leverages a hypothetical data set meant to reflect the expected experimental outcomes without statistical fluctuations. By doing so, it provides a straightforward means of estimating the median sensitivity — a measure critical for planning and interpreting experimental particle physics results. This technique allows for quick and efficient sensitivity analyses across extensive parameter spaces, which would otherwise demand prohibitive computational resources.

Beyond theoretical formulations, Cowan et al. extend their methodology to a variety of test statistics including $t_{\mu}$, $\tilde{t}_{\mu}$, $q_{0}$, and $q_{\mu}$, which they employed to test hypotheses regarding parameters of interest. An essential aspect of their approach is that it remains robust in the presence of nuisance parameters, ensuring that models can include systematic uncertainties through appropriate parameterizations.

Furthermore, the distribution of these test statistics is shown to follow non-central and central chi-square distributions in the asymptotic regime. This insight is critical for establishing thresholds for hypothesis testing and enables the derivation of $p$-values and significance levels essential for experimental results' interpretation.

The paper also explores the implementation of these methods in practical scenarios. Through detailed examples such as counting experiments and shape analyses, the authors illustrate how their methods can readily be applied to real-world experimental setups. Notably, by leveraging these advancements, researchers can efficiently compute discovery probabilities and exclusion limits, and thereby gain meaningful insights into experimental data without extensive simulation-based approaches.

In terms of practical implications, the methods outlined facilitate a faster evaluation of the potential reach of future experiments – a significant advantage during the experimental planning phases of large projects like the CERN Large Hadron Collider. The Asimov data set, in particular, allows researchers to derive error bands and expected statistical variation, granting a clearer understanding of experimental sensitivity.

The theoretical implications are extensive, primarily due to the advanced treatment of systematic uncertainties and the enhancements in computational efficiency. This work opens the door for broader application of similar asymptotic approaches beyond particle physics, potentially influencing statistics-heavy fields such as cosmology and genetics.

Future developments may see these techniques applied to analyzing LHC data, particularly as researchers continue to seek rare processes with subtle signatures. Additionally, the integration of these methods into software platforms like RooStats further signifies the potential for extended adoption in the scientific community.

In conclusion, Cowan et al.'s paper delivers critical advancements in the field of statistical methods for hypothesis testing in high energy physics, providing both a robust theoretical foundation and practical tools for contemporary and future research endeavors.