Sensitivity Analysis when Generalizing Causal Effects from Multiple Studies to a Target Population: Motivation from the ECHO Program

Abstract: Unobserved effect modifiers can induce bias when generalizing causal effect estimates to target populations. In this work, we extend a sensitivity analysis framework assessing the robustness of study results to unobserved effect modification that adapts to various generalizability scenarios, including multiple (conditionally) randomized trials, observational studies, or combinations thereof. This framework is interpretable and does not rely on distributional or functional assumptions about unknown parameters. We demonstrate how to leverage the multi-study setting to detect violation of the generalizability assumption through hypothesis testing, showing with simulations that the proposed test achieves high power under real-world sample sizes. Finally, we apply our sensitivity analysis framework to analyze the generalized effect estimate of secondhand smoke exposure on birth weight using cohort sites from the Environmental influences on Child Health Outcomes (ECHO) study.