Experimental Unit Information Index (EUII)

• Experimental Unit Information Index (EUII) is a metric that quantifies evidentiary value per experimental unit by normalizing the diagnostic odds ratio in hypothesis testing.
• It applies to both fixed-sample and adaptive designs, offering interpretation from frequentist and Bayesian perspectives to optimize study parameters.
• Numerical and asymptotic analyses reveal that increasing power and employing early-stopping rules enhance EUII, leading to more efficient and ethically sound experiments.

The Experimental Unit Information Index (EUII) quantifies the evidentiary value contributed by a single experimental unit in the context of hypothesis testing. Designed to enable rigorous trade-offs between statistical power, Type I error, and sample size, the EUII provides a single, unit-normalized metric that characterizes the per-unit accumulation of evidentiary value in both fixed-sample and adaptive designs. It is interpretable from both frequentist and Bayesian perspectives and offers guidance for optimizing study designs, particularly in fields such as animal research where reduction in experimental units is ethically mandated (Held et al., 21 Nov 2025).

1. Definition in Fixed-Sample Designs

The EUII for a fixed-sample design derives directly from the diagnostic odds ratio (DOR), which combines the likelihood ratios for significant and non-significant outcomes under both null ($H_0$) and alternative ($H_1$) hypotheses. For a test with per-sample Type I error $\alpha = P_0(\mathrm{reject}\ H_0)$ and power $1-\beta = P_1(\mathrm{reject}\ H_0)$, define:

• Positive likelihood ratio: $\mathrm{LR}^+ = \frac{\mathrm{Power}}{\alpha}$
• Negative likelihood ratio: $\mathrm{LR}^- = \frac{1-\mathrm{Power}}{1-\alpha}$

The diagnostic odds ratio:

$$\mathrm{DOR} = \frac{\mathrm{LR}^+}{\mathrm{LR}^-} = \frac{\mathrm{Power}/\alpha}{(1-\mathrm{Power})/(1-\alpha)} = \frac{\mathrm{Power}/(1-\mathrm{Power})}{\alpha/(1-\alpha)}$$

If $n$ independent units are used, each unit is attributed with the $n$th root of the DOR:

$$\mathrm{EUII}_n = \mathrm{DOR}^{1/n} = \left(\frac{\mathrm{Power}/(1-\mathrm{Power})}{\alpha/(1-\alpha)}\right)^{1/n}$$

A test is considered evidentially useful if $\mathrm{EUII}_n > 1$. This metric is agnostic to the specific values of $\alpha$, $1-\beta$, or $n$ and is readily computed using design specifications.

2. Asymptotic Properties

The behavior of EUII as $n \to \infty$ elucidates its theoretical bounds. For a one-sided, one-sample $z$-test with effect size $\delta > 0$:

$$\mathrm{Power} = \Phi(\delta \sqrt{n} - z_{1-\alpha})$$

where $\Phi$ is the standard normal CDF. Setting $x_n = \delta \sqrt{n} - z_{1-\alpha}$ yields:

$$\mathrm{Power\ Odds} = \frac{\Phi(x_n)}{1 - \Phi(x_n)}$$

Since $\alpha/(1-\alpha)$ is asymptotically constant, $\mathrm{EUII}_n$ converges as:

$$\lim_{n \to \infty} \mathrm{EUII}_n = \exp\left(\frac{\delta^2}{2}\right)$$

For a two-sample $z$-test of mean difference $\delta$, the corresponding limit is:

$$\lim_{n \to \infty} \mathrm{EUII}_n = \exp\left(\frac{\delta^2}{8}\right)$$

This establishes that the per-unit informational gain exhibits asymptotic saturation, reflecting diminishing per-unit returns as sample size increases.

3. Interpretations: Frequentist and Bayesian Perspectives

Frequentist Interpretation: EUII represents the geometric mean increase in odds of a significant result under $H_1$ compared to $H_0$:

$$\mathrm{DOR} = \frac{\mathrm{Power\ Odds}}{\mathrm{T1E\ Odds}}$$

Exponentiation by $1/n$ interprets EUII as the per-unit multiplicative increase in evidentiary odds.

Bayesian Interpretation: By Bayes’ theorem, observing a significant (or non-significant) result modifies the posterior odds for $H_1$:

• Significant: $$\mathrm{Odds}(H_1 | \text{sig}) = \mathrm{LR}^+ \cdot \mathrm{Odds}(H_1)$$
• Non-significant: $$\mathrm{Odds}(H_1 | \text{nonsig}) = \mathrm{LR}^- \cdot \mathrm{Odds}(H_1)$$

The DOR quantifies the ratio of posterior odds between significant and non-significant outcomes. Therefore, $\mathrm{EUII}_n$ is the per-unit geometric average change in Bayes-factor-equivalent posterior odds distinguishing significant from non-significant results.

4. Extension to Adaptive and Group-Sequential Designs

In adaptive or group-sequential studies, the sample size $N$ is a random variable contingent on interim stopping for efficacy or futility, differing under $H_0$ and $H_1$. Define:

• $\mathbb{E}(N_+)$: Expected sample size when stopping with significance (“sig”)
• $\mathbb{E}(N_-)$: Expected sample size when stopping with non-significance (“nonsig”)

The generalized EUII becomes:

$$\mathrm{EUII} = (\mathrm{LR}^+)^{1/\mathbb{E}(N_+)} \cdot (\mathrm{LR}^-)^{-1/\mathbb{E}(N_-)}$$

Variability in $N$ can be accommodated by a second-order Taylor expansion:

$$\widetilde{\mathrm{EUII}} = (\mathrm{LR}^+)^{(1 + CV_+^2)/\mathbb{E}(N_+)} \cdot (\mathrm{LR}^-)^{-(1 + CV_-^2)/\mathbb{E}(N_-)}$$

where $CV_+ = \mathrm{SD}(N_+)/\mathbb{E}(N_+)$ and similarly for $CV_-$.

Analytic or simulation-based estimation of $\mathbb{E}(N_+),\ \mathbb{E}(N_-),\ CV_+,\ CV_-$ enables concrete EUII calculation for varied adaptive designs, allowing precise evaluation of how early-stopping rules impact per-unit evidentiary value.

5. Numerical Examples

A two-arm, fixed-sample design with $\alpha = 0.05$, power $0.80$, and effect size $\delta = 0.50$:

• $n \approx 126$,
• $\mathrm{LR}^+ = 0.80 / 0.05 = 16$
• $\mathrm{LR}^- = 0.20 / 0.95 \approx 0.2105$
• $\mathrm{DOR} = 16 / 0.2105 \approx 76$
• $$\mathrm{EUII}_{126} \approx 76^{1/126} \approx 1.035$$

Each unit increases DOR by approximately $3.5\%$.

A constant-bound Pocock group-sequential design (four looks), with $\alpha=0.05$, $\delta=0.50$, and $n_{\max} \approx 136$:

• $\mathbb{E}(N_+)\approx90$, $\mathbb{E}(N_-)\approx136$
• $$\mathrm{EUII} = 16^{1/90} \cdot 0.2105^{-1/136} \approx 1.038$$

Early stopping confers an additional per-unit evidentiary value of $\approx0.3\%$ over the fixed-sample design.

6. Implications for Design Optimization

Maximizing EUII entails:

• Recognizing that for a fixed effect size $\delta$, the asymptotic bound $\exp(\delta^2 / 2)$ (or $\exp(\delta^2 / 8)$ for two-sample tests) is unattainable by further increasing $n$; per-unit information exhibits diminishing returns.
• Tuning $\alpha$ at finite $n$ trades off between $\mathrm{LR}^+$ and $\mathrm{LR}^-$. Standard choices for $\alpha$ are typically near-optimal.
• Lower $\beta$ (higher power) always increases $\mathrm{LR}^+$ and EUII but necessitates larger $n$.
• Maximizing power for fixed $n$ yields the uniformly most powerful test and, hence, the highest EUII.
• Adaptive designs, notably those with effective early-stopping rules for efficacy or futility, reduce $\mathbb{E}(N_+)$ or $\mathbb{E}(N_-)$, increasing EUII substantially. Two to four well-selected interim analyses and application of predictive-power futility boundaries (e.g., stop if predictive power $<0.2$–$0.3$) capture a significant fraction of possible evidentiary gains.
• Unbalanced randomization reduces power at fixed $n$ and thus modestly lowers EUII; equal allocation optimizes EUII if this metric is prioritized.

In summary, the EUII provides a rigorous, unified measure of evidence efficiency per experimental unit in both frequentist and Bayesian contexts. It is maximized by adopting most powerful critical values at fixed $\alpha$, minimizing $\alpha$ for given power, and employing adaptive early-stopping rules where feasible to reduce expected sample size while preserving evidentiary value (Held et al., 21 Nov 2025).