The Lindley paradox: The loss of resolution in Bayesian inference

Abstract: There are three principle paradigms of statistics: Bayesian, frequentist and information-based inference. Although these paradigms are in agreement in some contexts, the Lindley paradox describes a class of problems, models of unknown dimension, where conflicting conclusions are generated by frequentist and Bayesian inference. This conflict can materially affect the scientific conclusions. Understanding the Lindley paradox---where it applies, why it occurs, and how it can be avoided---is therefore essential to the understanding of statistical analysis. In this paper, we revisit the Lindley paradox in the context of a simple biophysical application. We describe how predictive and postdictive measures of model performance provide a natural framework for understanding the Lindley paradox. We then identify methods which result in optimal experimental resolution for discovery.