Analytic derivation of Type I and Type II error curves

Derive analytic expressions for the Type I and Type II error curves of the Bayesian and Li–Ma criteria in the Poisson On/Off problem, rather than obtaining them solely through Monte Carlo simulations.

Background

The paper compares Bayesian criteria based on flat, Jeffreys, and scale-invariant priors with the frequentist Li–Ma criterion for testing whether a Poisson signal is present in an On region when the background is estimated from an independent Off region. Type I and Type II error rates are evaluated numerically through Monte Carlo simulations under different background levels, signal strengths, exposure ratios, and overdispersion parameters. Although these simulations establish comparative behavior of the criteria, the authors explicitly identify the unresolved question of whether the resulting error curves can be obtained analytically. A solution would provide theoretical rather than simulation-based characterizations of the criteria’s false-positive and false-negative behavior.

References

Besides, it is intriguing if it is possible to obtain these Type I and Type II curves on analytic grounds.

— Bayesian Superiority in On/Off analysis  (2609.11683 - Babenov et al., 10 Sep 2026) in Conclusion, immediately before the paragraph beginning “The main physical results are as follows.”

And the claim that the missed signals do not depend on the OFF panels, while the false alarms do, follows from the input design but was not tested against a detector that reads all six panels.

— Missed Detections and False Alarms in a CNN Technosignature Detector on Breakthrough Listen Data  (2610.22050 - Imran, 15 Oct 2026) in Section 6, “Limitations”