Dynamic adaptive multiple tests with finite sample FDR control

Abstract: The present paper introduces new adaptive multiple tests which rely on the estimation of the number of true null hypotheses and which control the false discovery rate (FDR) at level alpha for finite sample size. We derive exact formulas for the FDR for a large class of adaptive multiple tests which apply to a new class of testing procedures. In the following, generalized Storey estimators and weighted versions are introduced and it turns out that the corresponding adaptive step up and step down tests control the FDR. The present results also include particular dynamic adaptive step wise tests which use a data dependent weighting of the new generalized Storey estimators. In addition, a converse of the Benjamini Hochberg (1995) theorem is given. The Benjamini Hochberg (1995) test is the only "distribution free" step up test with FDR independent of the distribution of the p-values of false null hypotheses.