Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
167 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Causal Effect Estimation from Observational and Interventional Data Through Matrix Weighted Linear Estimators (2306.06002v1)

Published 9 Jun 2023 in stat.ME and cs.AI

Abstract: We study causal effect estimation from a mixture of observational and interventional data in a confounded linear regression model with multivariate treatments. We show that the statistical efficiency in terms of expected squared error can be improved by combining estimators arising from both the observational and interventional setting. To this end, we derive methods based on matrix weighted linear estimators and prove that our methods are asymptotically unbiased in the infinite sample limit. This is an important improvement compared to the pooled estimator using the union of interventional and observational data, for which the bias only vanishes if the ratio of observational to interventional data tends to zero. Studies on synthetic data confirm our theoretical findings. In settings where confounding is substantial and the ratio of observational to interventional data is large, our estimators outperform a Stein-type estimator and various other baselines.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (58)
  1. Mostly harmless econometrics: An empiricist’s companion. Princeton University Press, 2009.
  2. Identification of causal effects using instrumental variables. Journal of the American Statistical Association, 91(434):444–455, 1996.
  3. H. Bang and J. M. Robins. Doubly robust estimation in missing data and causal inference models. Biometrics, 61(4):962–973, 2005.
  4. E. Bareinboim and J. Pearl. Causal Inference by Surrogate Experiments: z-Identifiability. In Proceedings of the 28th Conference on Uncertainty in Artificial Intelligence, pages 113–120, 2012.
  5. E. Bareinboim and J. Pearl. Causal inference and the data-fusion problem. Proceedings of the National Academy of Sciences, 113(27):7345–7352, 2016.
  6. R. Caruana. Multitask Learning. Machine Learning, 28(1):41–75, 1997.
  7. Spectral Deconfounding via Perturbed Sparse Linear Models. The Journal of Machine Learning Research, 21(1):9442–9482, 2020.
  8. D. Cheng and T. Cai. Adaptive Combination of Randomized and Observational Data. arXiv:2111.15012, 2021.
  9. Double/debiased machine learning for treatment and structural parameters: Double/debiased machine learning. The Econometrics Journal, 21(1), 2018.
  10. Causal inference methods for combining randomized trials and observational studies: a review. arXiv:2011.08047, 2020.
  11. J. Correa and E. Bareinboim. General transportability of soft interventions: Completeness results. Advances in Neural Information Processing Systems, 33:10902–10912, 2020.
  12. F. Eberhardt and R. Scheines. Interventions and causal inference. Philosophy of science, 74(5):981–995, 2007.
  13. B. Efron. Large-scale inference: empirical Bayes methods for estimation, testing, and prediction, volume 1. Cambridge University Press, 2012.
  14. B. Efron and C. Morris. Stein’s estimation rule and its competitors—an empirical Bayes approach. Journal of the American Statistical Association, 68(341):117–130, 1973.
  15. Least Angle Regression. The Annals of Statistics, 32(2), 2004.
  16. R. A. Fisher. Design of experiments. British Medical Journal, 1(3923):554, 1936.
  17. Domain adaptation with conditional transferable components. In International Conference on Machine Learning, pages 2839–2848, 2016.
  18. A James-Stein Type Estimator for Combining Unbiased and Possibly Biased Estimators. Journal of the American Statistical Association, 86(416):1001–1006, 1991.
  19. Improved Estimation for Multiple Means with Heterogeneous Variances. Forest Science, 51(1):1–6, 2005.
  20. The Elements of Statistical Learning. Springer, 2009.
  21. Combining observational and randomized data for estimating heterogeneous treatment effects. arXiv:2202.12891, 2022.
  22. Graphical criteria for efficient total effect estimation via adjustment in causal linear models. Journal of the Royal Statistical Society Series B, 84(2):579–599, 2022.
  23. Causal inference: What if. Boca Raton: Chapman & Hall/CRC, 2020.
  24. A. E. Hoerl. Ridge Regression: Biased Estimation for Nonorthogonal Problems. Technometrics, 12(1):55–67, 1970.
  25. Y. Huang and M. Valtorta. Identifiability in Causal Bayesian Networks: A Sound and Complete Algorithm. In Proceedings of the National Conference on Artificial Intelligence, volume 21, pages 1149–1154, 2006.
  26. Combining Interventional and Observational Data Using Causal Reductions. arXiv:2103.04786, pages 1–42, 2021.
  27. K. Imai and D. A. V. Dyk. Causal Inference with General Treatment Regimes: Generalizing the Propensity Score. Journal of the American Statistical Association, 99(467):854–866, 2004.
  28. Causal inference in statistics, social, and biomedical sciences. Cambridge University Press, 2015.
  29. An Introduction to Statistical Learning. Springer, 2013.
  30. W. James and C. Stein. Estimation with Quadratic Loss. In Proceedings of the 4th Berkeley Symposium on Probability and Statistics. Berkeley, CA: University of California Press, 1961.
  31. Removing hidden confounding by experimental grounding. Advances in Neural Information Processing Systems, 31, 2018.
  32. General Identifiability with Arbitrary Surrogate Experiments. In Proceedings of the 35th Uncertainty in Artificial Intelligence Conference, pages 389–398, 2020.
  33. S. L. Morgan and C. Winship. Counterfactuals and Causal Inference: Methods and Principles for Social Research. Cambridge University Press, 2014.
  34. J. Neyman. On the application of probability theory to agricultural experiments: essay on principles. Statistical Science, 5:465–480, 1923.
  35. J. Pearl. Causal diagrams for empirical research. Biometrika, 82(4):669–688, 1995.
  36. J. Pearl. Causality: models, reasoning, and inference. Cambridge University Press, 2nd edition, 2009.
  37. J. Pearl and E. Bareinboim. External Validity: From Do-Calculus to Transportability Across Populations. Statistical Science, 29(4):579–595, 2014.
  38. Elements of Causal Inference: Foundations and Learning Algorithms. MIT Press, 2017.
  39. Dataset Shift in Machine Learning. MIT Press, 2008.
  40. H. Reichenbach. The Direction of Time, volume 65. University of California Press, 1956.
  41. H. Robbins. The empirical Bayes approach to statistical decision problems. The Annals of Mathematical Statistics, 35(1):1–20, 1964.
  42. J. M. Robins and A. Rotnitzky. Semiparametric efficiency in multivariate regression models with missing data. Journal of the American Statistical Association, 90(429):122–129, 1995.
  43. Invariant Models for Causal Transfer Learning. The Journal of Machine Learning Research, 19(1):1309–1342, 2018.
  44. Combining observational and experimental datasets using shrinkage estimators. Biometrics, 2020.
  45. Propensity score methods for merging observational and experimental datasets. Statistics in Medicine, 41(1):65–86, 2022.
  46. D. B. Rubin. Estimating causal effects of treatments in randomized and nonrandomized studies. Journal of educational Psychology, 66(5):688–701, 1974.
  47. C. Schaffer. Selecting a Classification Method by Cross-Validation. Machine Learning, 13:135–143, 1993.
  48. On Causal and Anticausal Learning. In International Conference on Machine Learning, 2012.
  49. I. Shpitser and J. Pearl. Identification of Joint Interventional Distributions in Recursive Semi-Markovian Causal Models. In Proceedings of the National Conference on Artificial Intelligence, volume 21, pages 1219–1226, 2006.
  50. Causation, Prediction, and Search. MIT Press, 2000.
  51. C. Stein. Inadmissibility of the Usual Estimator for the Mean of a Multivariate Normal Distribution. In Proceedings of the third Berkeley Symposium on Mathematical Statistics and Probability, volume 3, pages 197–207. University of California Press, 1956.
  52. S. Thrun. Is Learning The n-th Thing Any Easier Than Learning The First? Advances in Neural Information Processing Systems, 8, 1995.
  53. J. Tian and J. Pearl. A general identification condition for causal effects. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 567–573, 2002.
  54. R. Tibshirani. Regression Shrinkage and Selection via the Lasso. Journal of the Royal Statistical Society: Series B (Methodological), 58(1):267–288, 1996.
  55. Permutation-based Causal Inference Algorithms with Interventions. Advances in Neural Information Processing Systems, 30, 2017.
  56. L. Wasserman. All of Nonparametric Statistics. Springer, 2006.
  57. S. Yang and P. Ding. Combining Multiple Observational Data Sources to Estimate Causal Effects. Journal of the American Statistical Association, 115(531):1540–1554, 2020.
  58. Domain Adaptation under Target and Conditional Shift. In International Conference on Machine Learning, pages 819–827, 2013.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now