REACT to NHST: Sensible conclusions to meaningful hypotheses

Abstract: While Null Hypothesis Significance Testing (NHST) remains a widely used statistical tool, it suffers from several shortcomings, such as conflating statistical and practical significance, sensitivity to sample size, and the inability to distinguish between accepting the null hypothesis and failing to reject it. Recent efforts have focused on developing alternatives to NHST to address these issues. Despite these efforts, conventional NHST remains dominant in scientific research due to its simplicity and perceived ease of interpretation. Our work presents a novel alternative to NHST that is just as accessible and intuitive: REACT. It not only tackles the shortcomings of NHST but also offers additional advantages over existing alternatives. For instance, REACT is easily applicable to multiparametric hypotheses and does not require stringent significance-level corrections when conducting multiple tests. We illustrate the practical utility of REACT through real-world data examples, using criteria aligned with common research practices to distinguish between the absence of evidence and evidence of absence.