Blending Bayesian and frequentist methods according to the precision of prior information with an application to hypothesis testing (1107.2353v1)

Abstract: The following zero-sum game between nature and a statistician blends Bayesian methods with frequentist methods such as p-values and confidence intervals. Nature chooses a posterior distribution consistent with a set of possible priors. At the same time, the statistician selects a parameter distribution for inference with the goal of maximizing the minimum Kullback-Leibler information gained over a confidence distribution or other benchmark distribution. An application to testing a simple null hypothesis leads the statistician to report a posterior probability of the hypothesis that is informed by both Bayesian and frequentist methodology, each weighted according how well the prior is known. Since neither the Bayesian approach nor the frequentist approach is entirely satisfactory in situations involving partial knowledge of the prior distribution, the proposed procedure reduces to a Bayesian method given complete knowledge of the prior, to a frequentist method given complete ignorance about the prior, and to a blend between the two methods given partial knowledge of the prior. The blended approach resembles the Bayesian method rather than the frequentist method to the precise extent that the prior is known. The problem of testing a point null hypothesis illustrates the proposed framework. The blended probability that the null hypothesis is true is equal to the p-value or a lower bound of an unknown Bayesian posterior probability, whichever is greater. Thus, given total ignorance represented by a lower bound of 0, the p-value is used instead of any Bayesian posterior probability. At the opposite extreme of a known prior, the p-value is ignored. In the intermediate case, the possible Bayesian posterior probability that is closest to the p-value is used for inference. Thus, both the Bayesian method and the frequentist method influence the inferences made.