Asymptotically optimal sequential FDR and pFDR control with (or without) prior information on the number of signals (1905.00177v3)

Abstract: We investigate asymptotically optimal multiple testing procedures for streams of sequential data in the context of prior information on the number of false null hypotheses ("signals"). We show that the "gap" and "gap-intersection" procedures, recently proposed and shown by Song and Fellouris (2017, Electron. J. Statist.) to be asymptotically optimal for controlling type 1 and 2 familywise error rates (FWEs), are also asymptotically optimal for controlling FDR/FNR when their critical values are appropriately adjusted. Generalizing this result, we show that these procedures, again with appropriately adjusted critical values, are asymptotically optimal for controlling any multiple testing error metric that is bounded between multiples of FWE in a certain sense. This class of metrics includes FDR/FNR but also pFDR/pFNR, the per-comparison and per-family error rates, and the false positive rate. Our analysis includes asymptotic regimes in which the number of null hypotheses approaches $\infty$ as the type 1 and 2 error metrics approach $0$.