When correcting for regression to the mean is worse than no correction at all

Abstract: The ubiquitous regression to the mean (RTM) effect complicates statistical inference in biological studies of change. We demonstrate that common RTM correction methods are flawed: the Berry et al. method popularized by Kelly & Price in The American Naturalist is unreliable for hypothesis testing, leading to both false positives and negatives, while the theoretically unbiased Blomqvist method has poor efficiency in limited sample sizes. Our findings show that the most robust approach to handling RTM is not to correct the data but to use the crude slope in conjunction with an assessment of the experiment's repeatability. Ultimately, we argue that any conclusion about a differential treatment effect is statistically unfounded without a clear understanding of the experiment's repeatability.