A Modified Randomization Test for the Level of Clustering

Abstract: Suppose a researcher observes individuals within a county within a state. Given concerns about correlation across individuals, it is common to group observations into clusters and conduct inference treating observations across clusters as roughly independent. However, a researcher that has chosen to cluster at the county level may be unsure of their decision, given knowledge that observations are independent across states. This paper proposes a modified randomization test as a robustness check for the chosen level of clustering in a linear regression setting. Existing tests require either the number of states or number of counties to be large. Our method is designed for settings with few states and few counties. While the method is conservative, it has competitive power in settings that may be relevant to empirical work.