Local Randomization Regression Discontinuity Designs when Test Scores are the Running Variable

Abstract: Explanations of the internal validity of regression discontinuity designs (RDD) generally appeal to the idea that RDDs are ``as good as" random near the treatment cut point. Cattaneo, Frandsen, and Titiunik (2015) are the first to take this justification to its full conclusion and propose estimating the RDD local average treatment effect (LATE) the same as one would a randomized experiment. This paper explores the implications of analyzing an RDD as a local random experiment when the running variable is a test score. I derive a formula for the bias in the LATE estimate estimated using the local randomization method, $a\rho\Delta$. Where $a$ is the relationship between latent proficiency and the potential outcome absent treatment, $\rho$ is the test reliability, and $\Delta$ is the distance between the treatment and control running variable value. I use this quantification of the bias to demonstrate that local randomization and related design based methods for estimating RDDs are problematic when the running variable is test score or other human developed measure (e.g., medical tests).