The Experimental Unit Information Index: Balancing Evidentiary Value and Sample Size of Adaptive Designs

Abstract: Reducing the number of experimental units is one of the three pillars of the 3R principles (Replace, Reduce, Refine) in animal research. At the same time, statistical error rates need to be controlled to enable reliable inferences and decisions. This paper proposes a novel measure to quantify the evidentiary value of one experimental unit for a given study design. The experimental unit information index (EUII) is based on power, Type-I error and sample size, and has attractive interpretations both in terms of frequentist error rates and Bayesian posterior odds. We introduce the EUII in simple statistical test settings and show that its asymptotic value depends only on the assumed relative effect size under the alternative. We then extend the definition to adaptive designs where early stopping for efficacy or futility may cause reductions in sample size. Applications to group-sequential designs and a recently proposed adaptive statistical test procedure show the usefulness of the approach when the goal is to maximize the evidentiary value of one experimental unit.