Principled selection of the p-value significance threshold α

Determine a principled criterion for selecting the significance threshold α that distinguishes significant from non-significant p-values in classical null hypothesis significance testing, thereby demarcating unlikely versus likely null hypotheses.

Background

The paper reviews the classical logic of significance testing and highlights a fundamental ambiguity: while p-values quantify the extremeness of observed data under a null hypothesis, there is no agreed-upon rule for where to draw the boundary between “unlikely” and “likely” hypotheses based on a fixed threshold α.

This ambiguity is central to widespread misinterpretations, as noted in the ASA statement and related literature. The authors’ proposed fuzzy-set perspective aims to avoid the arbitrary choice of α by interpreting the p-value as a membership function over parameter values, but the underlying question of establishing a principled α selection criterion remains explicitly stated as unresolved.