Difference-in-Differences with Unpoolable Data
Abstract: Difference-in-differences (DID) is commonly used to estimate treatment effects but is infeasible in settings where data are unpoolable due to privacy concerns or legal restrictions on data sharing, particularly across jurisdictions. In this study, we identify and relax the assumption of data poolability in DID estimation. We propose an innovative approach to estimate DID with unpoolable data (UN-DID) which can accommodate covariates, multiple groups, and staggered adoption. Through analytical proofs and Monte Carlo simulations, we show that UN-DID and conventional DID estimates of the average treatment effect and standard errors are equal and unbiased in settings without covariates. With covariates, both methods produce estimates that are unbiased, equivalent, and converge to the true value. The estimates differ slightly but the statistical inference and substantive conclusions remain the same. Two empirical examples with real-world data further underscore UN-DID's utility. The UN-DID method allows the estimation of cross-jurisdictional treatment effects with unpoolable data, enabling better counterfactuals to be used and new research questions to be answered.
- Abadie, A. (2005) “Semiparametric difference-in-differences estimators,” The review of economic studies 72(1), 1–19
- Bernal, J. L., S. Cummins, and A. Gasparrini (2017) “Interrupted time series regression for the evaluation of public health interventions: a tutorial,” International journal of epidemiology 46(1), 348–355
- Bertrand, M., E. Duflo, and S. Mullainathan (2004) “How much should we trust differences-in-differences estimates?,” The Quarterly journal of economics 119(1), 249–275
- Caetano, C., B. Callaway, S. Payne, and H. S. Rodrigues (2022) “Difference in differences with time-varying covariates,” arXiv preprint arXiv:2202.02903
- Callaway, B., and P. H. Sant’Anna (2021) “Difference-in-differences with multiple time periods,” Journal of Econometrics 225(2), 200–230
- Card, D., and A. B. Krueger (1993) “Minimum wages and employment: A case study of the fast food industry in new jersey and pennsylvania,”
- Conley, T. G., and C. R. Taber (2011) “Inference with “difference in differences” with a small number of policy changes,” The Review of Economics and Statistics 93(1), 113–125
- De Chaisemartin, C., and X. d’Haultfoeuille (2020a) “Two-way fixed effects estimators with heterogeneous treatment effects,” American Economic Review 110(9), 2964–2996
- Deming, D., and S. Dynarski (2010) “College aid,” in Targeting investments in children: Fighting poverty when resources are limited, pp. 283–302, University of Chicago Press
- Hallock, H., S. E. Marshall, P. A. t Hoen, J. F. Nygård, B. Hoorne, C. Fox, and S. Alagaratnam (2021) “Federated networks for distributed analysis of health data,” Frontiers in Public Health 9, 712569
- Heckman, J. J., H. Ichimura, and P. E. Todd (1997) “Matching as an econometric evaluation estimator: Evidence from evaluating a job training programme,” The review of economic studies 64(4), 605–654
- Li, Q., Y. Diao, Q. Chen, and B. He (2022) “Federated learning on non-iid data silos: An experimental study,” in 2022 IEEE 38th International Conference on Data Engineering (ICDE), pp. 965–978, IEEE
- Masten, M. A., and A. Poirier (2018) “Identification of treatment effects under conditional partial independence,” Econometrica 86(1), 317–351
- McCullough, J. S. (2008) “The adoption of hospital information systems,” Health economics 17(5), 649–664
- Miller, A. R., and C. Tucker (2014) “Health information exchange, system size and information silos,” Journal of health economics 33, 28–42
- Roth, J., P. H. Sant’Anna, A. Bilinski, and J. Poe (2022) “What’s trending in difference-in-differences? a synthesis of the recent econometrics literature,” arXiv preprint arXiv:2201.01194
- Sabia, J. J., J. Swigert, and T. Young (2017) “The effect of medical marijuana laws on body weight,” Health economics 26(1), 6–34
- Sant’Anna, P. H., and J. Zhao (2020) “Doubly robust difference-in-differences estimators,” Journal of Econometrics 219(1), 101–122
- Wang, L., and C. A. Alexander (2020) “Big data analytics in medical engineering and healthcare: methods, advances and challenges,” Journal of medical engineering & technology 44(6), 267–283
- Xiong, R., A. Koenecke, M. Powell, Z. Shen, J. T. Vogelstein, and S. Athey (2023) “Federated causal inference in heterogeneous observational data,” Statistics in Medicine 42(24), 4418–4439
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.
