Sensitivity Analysis for Quantiles of Hidden Biases in Matched Observational Studies

Abstract: Causal conclusions from observational studies may be sensitive to unmeasured confounding. In such cases, a sensitivity analysis is often conducted, which tries to infer the minimum amount of hidden biases or the minimum strength of unmeasured confounding needed in order to explain away the observed association between treatment and outcome. If the needed bias is large, then the treatment is likely to have significant effects. The Rosenbaum sensitivity analysis is a modern approach for conducting sensitivity analysis in matched observational studies. It investigates what magnitude the maximum of hidden biases from all matched sets needs to be in order to explain away the observed association. However, such a sensitivity analysis can be overly conservative and pessimistic, especially when investigators suspect that some matched sets may have exceptionally large hidden biases. In this paper, we generalize Rosenbaum's framework to conduct sensitivity analysis on quantiles of hidden biases from all matched sets, which are more robust than the maximum. Moreover, the proposed sensitivity analysis is simultaneously valid across all quantiles of hidden biases and is thus a free lunch added to the conventional sensitivity analysis. The proposed approach works for general outcomes, general matched studies and general test statistics. In addition, we demonstrate that the proposed sensitivity analysis also works for bounded null hypotheses when the test statistic satisfies certain properties. An R package implementing the proposed approach is available online.