Conditional Randomization Tests for Behavioral and Neural Time Series

Abstract: Randomization tests allow simple and unambiguous tests of null hypotheses, by comparing observed data to a null ensemble in which experimentally-controlled variables are randomly resampled. In behavioral and neuroscience experiments, however, the stimuli presented often depend on the subject's previous actions, so simple randomization tests are not possible. We describe how conditional randomization can be used to perform exact hypothesis tests in this situation, and illustrate it with two examples. We contrast conditional randomization with a related approach of tangent randomization, in which stimuli are resampled based only on events occurring in the past, which is not valid for all choices of test statistic. We discuss how to design experiments that allow conditional randomization tests to be used.