Confidences in Hypotheses (2111.10715v4)

Abstract: This article outlines a broadly-applicable new method of statistical analysis for situations involving two competing hypotheses. Hypotheses assessment is a frequentist procedure designed to answer the question: Given the sample evidence (and assumed model), what is the relative plausibility of each hypothesis? Our aim is to determine frequentist confidences in the hypotheses that are relevant to the data at hand and are as powerful as the particular application allows. Hypotheses assessments complement significance tests because providing confidences in the hypotheses in addition to test results can better inform applied researchers about the strength of evidence provided by the data. For simple hypotheses, the method produces minimum and maximum confidences in each hypothesis. The composite case is more complex, and we introduce two conventions to aid with understanding the strength of evidence. Assessments are qualitatively different from significance test and confidence interval outcomes, and thus fill a gap in the statistician's toolkit.