Comparison of Bayesian and Frequentist Multiplicity Correction For Testing Mutually Exclusive Hypotheses Under Data Dependence (1611.05909v1)

Abstract: The problem of testing mutually exclusive hypotheses with dependent test statistics is considered. Bayesian and frequentist approaches to multiplicity control are studied and compared to help gain understanding as to the effect of test statistic dependence on each approach. The Bayesian approach is shown to have excellent frequentist properties and is argued to be the most effective way of obtaining frequentist multiplicity control, without sacrificing power, when there is considerable test statistic dependence.