Analysis of multichannel measurements of rare processes with uncertain expected background and acceptance (1309.0531v5)

Abstract: A typical experiment in high energy physics is considered. The result of the experiment is assumed to be a histogram consisting of bins or channels with numbers of corresponding registered events. The expected background and expected signal shape or acceptance are measured in separate auxiliary experiments, or calculated by the Monte Carlo method with finite sample size, and hence with finite precision. An especially complex situation occurs when the expected background in some of the channels happens to be zero due to either a fluctuation of the auxiliary measurement (or simulation) or because it is truly zero. Different statistical methods give different confidence intervals for the full signal rate and different significances of the signal+background hypothesis versus the pure background hypothesis. Detailed analysis and numerical tests are presented.