Perspective from the Literature on the Role of Expert Judgment in Scientific and Statistical Research and Practice

Abstract: This article, produced as a result of the Symposium on Statistical Inference, is an introduction to the literature on the function of expertise, judgment, and choice in the practice of statistics and scientific research. In particular, expert judgment plays a critical role in conducting Frequentist hypothesis tests and Bayesian models, especially in selection of appropriate prior distributions for model parameters. The subtlety of interpreting results is also discussed. Finally, external recommendations are collected for how to more effectively encourage proper use of judgment in statistics. The paper synthesizes the literature for the purpose of creating a single reference and inciting more productive discussions on how to improve the future of statistics and science.