Dice Question Streamline Icon: https://streamlinehq.com

Operational criteria for context-dependent choice of statistical school

Determine operational decision criteria for selecting an appropriate normative statistical framework under ambiguous research contexts, including whether to elicit and use expert priors for subjective Bayesian analysis when expert competence is uncertain, and whether a given number of replications (e.g., 100) suffices to justify adopting frequentist methods.

Information Square Streamline Icon: https://streamlinehq.com

Background

The essay advocates a context-dependent approach to choosing between statistical schools but acknowledges unresolved practicalities in ambiguous cases. Specifically, when expert quality is uncertain, it is unclear whether subjective Bayesian analysis via prior elicitation is warranted.

The authors also highlight uncertainty about sample-size thresholds that justify frequentist methods in domains where replications are typically scarce, underscoring the need for concrete, context-sensitive decision rules.

References

It is, for example, not entirely clear what to do when the context is ambiguous. Let us suppose that we have an expert in our research team, but it is not clear whether he truly is mastering the subject. Should we elicit his prior and perform a Bayesian analysis? Or, in a setting where we have 100 replications and experimenters usually have few (e.g., western blotting), is 100 enough to warrant Frequentism?

My Statistics is Better than Yours (2412.10296 - Benhaïem, 13 Dec 2024) in Section 5, Conclusion