Bias, variance, and confidence intervals for efficiency estimators in particle physics experiments

Abstract: We compute bias, variance, and approximate confidence intervals for the efficiency of a random selection process under various special conditions that occur in practical data analysis. We consider the following cases: a) the number of trials is not constant but drawn from a Poisson distribution, b) the samples are weighted, c) the numbers of successes and failures have a variance which exceeds that of a Poisson process, which is the case, for example, when these numbers are obtained from a fit to mixture of signal and background events. Generalized Wilson intervals based on these variances are computed, and their coverage probability is studied. The efficiency estimators are unbiased in all considered cases, except when the samples are weighted. The standard Wilson interval is also suitable for case a). For most of the other cases, generalized Wilson intervals can be computed with closed-form expressions.