Incentive-Aware Synthetic Control: Accurate Counterfactual Estimation via Incentivized Exploration (2312.16307v2)
Abstract: We consider the setting of synthetic control methods (SCMs), a canonical approach used to estimate the treatment effect on the treated in a panel data setting. We shed light on a frequently overlooked but ubiquitous assumption made in SCMs of "overlap": a treated unit can be written as some combination -- typically, convex or linear combination -- of the units that remain under control. We show that if units select their own interventions, and there is sufficiently large heterogeneity between units that prefer different interventions, overlap will not hold. We address this issue by proposing a framework which incentivizes units with different preferences to take interventions they would not normally consider. Specifically, leveraging tools from information design and online learning, we propose a SCM that incentivizes exploration in panel data settings by providing incentive-compatible intervention recommendations to units. We establish this estimator obtains valid counterfactual estimates without the need for an a priori overlap assumption. We extend our results to the setting of synthetic interventions, where the goal is to produce counterfactual outcomes under all interventions, not just control. Finally, we provide two hypothesis tests for determining whether unit overlap holds for a given panel dataset.
- The economic costs of conflict: A case study of the basque country. American economic review, 93(1):113–132, 2003.
- Synthetic control methods for comparative case studies: Estimating the effect of california’s tobacco control program. Journal of the American statistical Association, 105(490):493–505, 2010.
- On model identification and out-of-sample prediction of principal component regression: Applications to synthetic controls. arXiv preprint arXiv:2010.14449, 2020a.
- Synthetic interventions. arXiv preprint arXiv:2006.07691, 2020b.
- Adaptive Principal Component Regression with Applications to Panel Data. Papers 2307.01357, arXiv.org, July 2023. URL https://ideas.repec.org/p/arx/papers/2307.01357.html.
- Robust synthetic control. The Journal of Machine Learning Research, 19(1):802–852, 2018.
- mrsc: Multi-dimensional robust synthetic control. Proceedings of the ACM on Measurement and Analysis of Computing Systems, 3(2):1–27, 2019.
- Mostly Harmless Econometrics: An Empiricist’s Companion. Princeton University Press, 2009. ISBN 9780691120348.
- Manuel Arellano and Bo Honore. Panel data models: Some recent developments. Handbook of Econometrics, 02 2000.
- Using the longitudinal structure of earnings to estimate the effect of training programs, 1984.
- Information design: A unified perspective. Journal of Economic Literature, 57(1):44–95, March 2019. doi: 10.1257/jel.20181489. URL https://www.aeaweb.org/articles?id=10.1257/jel.20181489.
- How much should we trust differences-in-differences estimates? The Quarterly journal of economics, 119(1):249–275, 2004.
- Gary Chamberlain. Panel data. In Z. Griliches† and M. D. Intriligator, editors, Handbook of Econometrics, volume 2, chapter 22, pages 1247–1318. Elsevier, 1 edition, 1984. URL https://EconPapers.repec.org/RePEc:eee:ecochp:2-22.
- Jiafeng Chen. Synthetic control as online linear regression. Econometrica, 91(2):465–491, 2023.
- Inference with difference-in-differences and other panel data. The review of Economics and Statistics, 89(2):221–233, 2007.
- Counterfactual inference for sequential experiments. arXiv preprint arXiv:2202.06891, 2022.
- EBay, Jun 2022. URL https://partnerhelp.ebay.com/helpcenter/s/article/Incrementality-Testing-by-Location-IP?language=en_US.
- Synthetically controlled bandits. arXiv preprint arXiv:2202.07079, 2022.
- Danilo Freire. Evaluating the effect of homicide prevention strategies in são paulo, brazil: A synthetic control approach. Latin American Research Review, 53(2):231–249, 2018.
- Susan Groth. Honorarium or coercion: Use of incentives for participants in clinical research. The Journal of the New York State Nurses’ Association, 41:11–3; quiz 22, 03 2010.
- Strategyproof decision-making in panel data settings and beyond. arXiv preprint arXiv:2211.14236, 2022.
- Incentivizing exploration with selective data disclosure, 2023.
- Ian T Jolliffe. A note on the use of principal components in regression. Journal of the Royal Statistical Society Series C: Applied Statistics, 31(3):300–303, 1982.
- Emir Kamenica. Bayesian persuasion and information design. Annual Review of Economics, 11:249–272, 2019.
- Bayesian persuasion. American Economic Review, 101(6):2590–2615, October 2011. doi: 10.1257/aer.101.6.2590. URL https://www.aeaweb.org/articles?id=10.1257/aer.101.6.2590.
- Implementing the “wisdom of the crowd”. Journal of Political Economy, 122(5):988–1012, 2014. ISSN 00223808, 1537534X. URL http://www.jstor.org/stable/10.1086/676597.
- Bandit algorithms. Cambridge University Press, 2020.
- Incentivizing participation in clinical trials. arXiv preprint arXiv:2202.06191, 2022.
- Longitudinal data analysis using generalized linear models. Biometrika, 73(1):13–22, 04 1986. ISSN 0006-3444. doi: 10.1093/biomet/73.1.13. URL https://doi.org/10.1093/biomet/73.1.13.
- Alexander R Luedtke and Mark J van der Laan. Optimal individualized treatments in resource-limited settings. The international journal of biostatistics, 12(1):283–303, 2016.
- Bayesian incentive-compatible bandit exploration. In Proceedings of the Sixteenth ACM Conference on Economics and Computation, pages 565–582, 2015.
- Bayesian incentive-compatible bandit exploration, 2019.
- Bayesian incentive-compatible bandit exploration. Operations Research, 68(4):1132–1161, 2020.
- Bayesian exploration: Incentivizing exploration in bayesian games, 2021.
- William F Massy. Principal components regression in exploratory statistical research. Journal of the American Statistical Association, 60(309):234–256, 1965.
- Elevated police turnover following the summer of george floyd protests: A synthetic control study. Criminology & Public Policy, 21(1):9–33, 2022.
- Optimal individualized decision rules using instrumental variable methods. Journal of the American Statistical Association, 116(533):174–191, 2021.
- Individualized treatment rules under stochastic treatment cost constraints. Journal of Causal Inference, 10(1):480–493, 2022.
- Mark Sellke. Incentivizing exploration with linear contexts and combinatorial actions, 2023.
- The price of incentivizing exploration: A characterization via thompson sampling and sample complexity, 2022.
- Aleksandrs Slivkins. Exploration and persuasion, 2021.
- Aleksandrs Slivkins et al. Introduction to multi-armed bandits. Foundations and Trends® in Machine Learning, 12(1-2):1–286, 2019.
- Treatment allocation under uncertain costs. arXiv preprint arXiv:2103.11066, 2021.
- Uber, Jun 2019. URL https://www.uber.com/blog/causal-inference-at-uber/.
- R. Vershynin. High-Dimensional Probability: An Introduction with Applications in Data Science. Cambridge Series in Statistical and Probabilistic Mathematics. Cambridge University Press, 2018. ISBN 9781108415194. URL https://books.google.com/books?id=J-VjswEACAAJ.
- Quantile-regression-based clustering for panel data. Journal of Econometrics, 213(1):54–67, 2019.