Why bother with Bayesian t-tests? (2211.02613v1)

Abstract: Given the well-known and fundamental problems with hypothesis testing via classical (point-form) significance tests, there has been a general move to alternative approaches, often focused on the Bayesian t-test. We show that the Bayesian t-test approach does not address the observed problems with classical significance testing, that Bayesian and classical t-tests are mathematically equivalent and linearly related in order of magnitude (so that the Bayesian t-test providing no further information beyond that given by point-form significance tests), and that Bayesian t-tests are subject to serious risks of misinterpretation, in some cases more problematic than seen for classical tests (with, for example, a negative sample mean in an experiment giving strong Bayesian t-test evidence in favour of a positive population mean). We do not suggest a return to the classical, point-form significance approach to hypothesis testing. Instead we argue for an alternative distributional approach to significance testing, which addresses the observed problems with classical hypothesis testing and provides a natural link between the Bayesian and frequentist approaches.