The Letter Pi : Bayesian interpretation of p-values, Reproducibility and Considerations for Replication in the Generalized Linear Model

Abstract: Significance testing based on p-values has been implicated in the reproducibility crisis in scientific research, with one of the proposals being to eliminate them in favor of Bayesian analyses. Defenders of the p-values have countered that it is the improper use and errors in interpretation, rather than the p-values themselves that are to blame. Similar exchanges about the role of p-values have occurred with some regularity every 10 to 15 years since their formal introduction in statistical practice. The apparent contradiction between the repeated failures in interpretation and continuous use of p-values suggest that there is an inferential value in the computation of these values. In this work we propose to attach a radical Bayesian interpretation to the number computed and reported as a p-value for the Generalized Linear Model, which has been the workhorse of applied statistical work. We introduce a decision analytic framework for thresholding posterior tail areas (pi-values) which for any given Bayesian analysis will have a direct correspondence to p-values in non-Bayesian approaches. Pi-values are non-controversial, posterior probability summaries of treatment effects. A predictive probability argument is made to justify the exploration of the stochastic variation (replication probability) of p and pi-values and culminates into a concrete proposal for the synthesis of Likelihood and Bayesian approaches to data analyses that aim for reproducibility. We illustrate these concepts using the results of recent randomized controlled trials in cardiometabolic and kidney diseases and provide R code for the implementation of the proposed methodology.