Design-based effect sizes and their synthesis in randomized multiple-baseline experiments

Abstract: Effect sizes for single-case experimental designs usually rely on models for the baseline trend and the serial dependence, which are challenging to check with few sessions. We develop design-based effect sizes for multiple-baseline experiments with a randomized intervention onset. We define case-level and pooled standardized effects with potential outcomes and propose weighted estimators that are unbiased or asymptotically unbiased without an outcome model. The usual difference between phase means is generally biased when the untreated outcome changes over sessions. For research synthesis, we provide variance estimators and asymptotic confidence intervals that require no model for serial dependence, and variance formulas for planning. Simulations and three randomized studies of reading and writing interventions illustrate the methods.