Characterize sample complexity in the weak detection regime for general priors

Characterize the sample complexity n_B(p, q, alpha, alpha(1 − gamma)) for Bayesian simple binary hypothesis testing in the weak detection regime (delta = alpha(1 − gamma)) for arbitrary discrete distributions p and q and priors alpha ∈ (0, 1/2]. Establish tight, constant-factor expressions that capture the dependence on alpha, gamma, and the divergences between p and q.

Background

Beyond the uniform prior, the weak detection regime presents additional subtleties: for non-uniform priors, the sample complexity may exhibit behavior that is independent of gamma for small gamma, and the known upper and lower bounds differ by a factor of gamma. The paper provides bounds and illustrative examples but emphasizes that a complete characterization remains unresolved.

The authors’ main results address regimes where delta is at most alpha/4 and reduce certain sublinear error regimes to linear ones, but they explicitly leave the weak detection regime (delta approaching alpha from below) open for future resolution.

References

Characterizing the sample complexity in the weak detection regime (cf. \Cref{sub:results-large-failure}) remains an open problem.

— The Sample Complexity of Simple Binary Hypothesis Testing  (2403.16981 - Pensia et al., 2024) in Section Discussion

On the theory side, the remaining open problems are spurious-inclusion control (assumption \Cref{ass:c6}) for nonparametric priors, an explicit second-order term in the detection threshold, and conditions under which training provably achieves uniform amortization.

— SelectPFN: Amortized Bayesian Variable Selection for Sparse Nonlinear Regression with Prior-Data Fitted Networks  (2610.21950 - Luo, 15 Oct 2026) in Section 4.5, Discussion, paragraph “Extensions”