Randomization-based Inference for Bernoulli-Trial Experiments and Implications for Observational Studies

Abstract: We present a randomization-based inferential framework for experiments characterized by a strongly ignorable assignment mechanism where units have independent probabilities of receiving treatment. Previous works on randomization tests often assume these probabilities are equal within blocks of units. We consider the general case where they differ across units and show how to perform randomization tests and obtain point estimates and confidence intervals. Furthermore, we develop a rejection-sampling algorithm to conduct randomization-based inference conditional on ancillary statistics, covariate balance, or other statistics of interest. Through simulation we demonstrate how our algorithm can yield powerful randomization tests and thus precise inference. Our work also has implications for observational studies, which commonly assume a strongly ignorable assignment mechanism. Most methodologies for observational studies make additional modeling or asymptotic assumptions, while our framework only assumes the strongly ignorable assignment mechanism, and thus can be considered a minimal-assumption approach.