Quantify the scaling threshold needed to recover asymptotic properties with finite MC samples

Determine the minimal scaling of data and/or Monte Carlo sample sizes required to restore the asymptotic validity of maximum-likelihood estimators and the profile-likelihood ratio—yielding correct coverage—when using the full Barlow–Beeston likelihood for high-statistics binned analyses with nuisance parameters and finite-size Monte Carlo templates. The goal is to ascertain quantitatively how large the scaling of data and/or simulation must be so that Wilks’ theorem applies and Hessian- or PLR-based confidence intervals achieve the nominal coverage.

Background

The paper demonstrates, via a paradigmatic toy model, that in high-statistics binned analyses where the model is built from finite Monte Carlo samples and includes nuisance parameters, confidence intervals based on asymptotic properties (Hessian or profile-likelihood ratio) can systematically under-cover. The authors show numerically that increasing the size of data and/or simulation eventually restores the expected asymptotic behavior, but the required scale is not straightforward to determine.

This uncertainty arises despite large event counts per bin and many bins, indicating that the conventional indicators of asymptotics (e.g., large N) are insufficient when MC template fluctuations and nuisance parameter profiling are present. A clear quantitative criterion for the necessary scaling would guide practitioners in ensuring correct coverage without resorting to impractically large simulations.