Scaling threshold needed to recover asymptotic coverage with finite MC templates

Determine the minimal scaling of data and/or Monte Carlo sample sizes in binned Poisson-likelihood analyses with nuisance parameters and finite Monte Carlo–derived templates (treated via the full Barlow–Beeston likelihood) that is sufficient to restore the asymptotic validity of Wilks’ theorem and yield correct coverage for confidence intervals constructed from the profile-likelihood ratio or Hessian methods. Specify quantitative conditions on the total number of events N, the number of bins n, and the data-to-MC statistical power ratio k under which these asymptotic confidence intervals attain nominal coverage in the presence of Monte Carlo statistical fluctuations.

Background

The paper demonstrates with a high-statistics toy model that confidence intervals based on asymptotic properties (Hessian or profile-likelihood ratio) can systematically under-cover when model templates are derived from finite Monte Carlo samples and nuisance parameters are profiled. Even with seemingly large event counts and many bins, Wilks’ theorem can be invalidated due to Monte Carlo–induced fluctuations in both the gradient with respect to the parameter of interest and the Jacobian with respect to nuisance parameters.

The authors show that increasing the size of the data and/or simulation eventually recovers correct coverage for asymptotic methods, but they do not identify a principled quantitative threshold for how large the scaling must be. This leaves open the task of characterizing, in terms of N, n, and k, the conditions under which asymptotic confidence intervals regain their nominal coverage in the presence of finite Monte Carlo uncertainties.