Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
162 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Longitudinal Targeted Minimum Loss-based Estimation with Temporal-Difference Heterogeneous Transformer (2404.04399v2)

Published 5 Apr 2024 in stat.ML, cs.AI, cs.LG, stat.AP, and stat.ME

Abstract: We propose Deep Longitudinal Targeted Minimum Loss-based Estimation (Deep LTMLE), a novel approach to estimate the counterfactual mean of outcome under dynamic treatment policies in longitudinal problem settings. Our approach utilizes a transformer architecture with heterogeneous type embedding trained using temporal-difference learning. After obtaining an initial estimate using the transformer, following the targeted minimum loss-based likelihood estimation (TMLE) framework, we statistically corrected for the bias commonly associated with machine learning algorithms. Furthermore, our method also facilitates statistical inference by enabling the provision of 95% confidence intervals grounded in asymptotic statistical theory. Simulation results demonstrate our method's superior performance over existing approaches, particularly in complex, long time-horizon scenarios. It remains effective in small-sample, short-duration contexts, matching the performance of asymptotically efficient estimators. To demonstrate our method in practice, we applied our method to estimate counterfactual mean outcomes for standard versus intensive blood pressure management strategies in a real-world cardiovascular epidemiology cohort study.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (38)
  1. Doubly robust estimation in missing data and causal inference models. Biometrics, 61(4):962–973, 2005.
  2. Estimating counterfactual treatment outcomes over time through adversarially balanced representations. In International Conference on Learning Representations, 2020.
  3. Efficient and Adaptive Estimation for Semiparametric Models. Johns Hopkins Series in the Mathematical Sciences. Springer New York, 1993. ISBN 978-0-387-98473-5.
  4. Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901, 2020.
  5. xgboost: Extreme Gradient Boosting, 2022. URL https://CRAN.R-project.org/package=xgboost. R package version 1.7.6.1.
  6. Double/debiased machine learning for treatment and structural parameters. The Econometrics Journal, 21(1):C1–C68, 2018.
  7. Locally robust semiparametric estimation. Econometrica, 90(4):1501–1535, 2022.
  8. A causal roadmap for generating high-quality real-world evidence. Journal of Clinical and Translational Science, 7(1):e212, 2023.
  9. More robust doubly robust off-policy evaluation. In Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Research, pp.  1447–1456. PMLR, 10–15 Jul 2018.
  10. Estimating average causal effects from patient trajectories. Proceedings of the AAAI Conference on Artificial Intelligence, 37(6):7586–7594, 2023.
  11. A targeted maximum likelihood estimator of a causal effect on a bounded continuous outcome. The International Journal of Biostatistics, 6(1):Article 26, 2010. ISSN 1557-4679. doi: 10.2202/1557-4679.1260.
  12. Targeted minimum loss based estimation of a causal effect on an outcome with known conditional bounds. The international journal of biostatistics, 8(1):21–21, 2012. ISSN 1557-4679.
  13. Doubly robust off-policy value evaluation for reinforcement learning. In Proceedings of The 33rd International Conference on Machine Learning, volume 48 of Proceedings of Machine Learning Research, pp.  652–661, New York, New York, USA, 20–22 Jun 2016. PMLR.
  14. Double reinforcement learning for efficient and robust off-policy evaluation. In Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research, pp.  5078–5088. PMLR, 13–18 Jul 2020.
  15. Kennedy, E. H. Semiparametric doubly robust targeted double machine learning: A review. arXiv preprint arXiv:2203.06469, 2022.
  16. Klaassen, C. A. J. Consistent estimation of the influence function of locally asymptotically linear estimators. The Annals of Statistics, 15(4):1548–1562, 1987.
  17. ltmle: An R package implementing targeted minimum loss-based estimation for longitudinal data. Journal of Statistical Software, 81(1):1–21, 2017. doi: 10.18637/jss.v081.i01.
  18. Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems. arXiv preprint arXiv:2005.01643, 2020.
  19. G-net: A recurrent network approach to G-computation for counterfactual prediction under a dynamic treatment regime. In Proceedings of Machine Learning for Health, volume 158 of Proceedings of Machine Learning Research, pp.  282–299. PMLR, 2021.
  20. Causal transformer for estimating counterfactual outcomes. In Chaudhuri, K., Jegelka, S., Song, L., Szepesvari, C., Niu, G., and Sabato, S. (eds.), Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedings of Machine Learning Research, pp.  15293–15329. PMLR, 2022.
  21. Milborrow, S. earth: Multivariate Adaptive Regression Splines, 2023. URL https://CRAN.R-project.org/package=earth. R package version 5.3.2.
  22. Playing atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602, 2013.
  23. Debiased off-policy evaluation for recommendation systems. In Proceedings of the 15th ACM Conference on Recommender Systems, RecSys ’21, pp.  372–379, New York, NY, USA, 2021. Association for Computing Machinery. ISBN 9781450384582.
  24. Causal models and learning from data: Integrating causal modeling and statistical estimation. Epidemiology (Cambridge, Mass.), 25(3):418–426, 2014.
  25. SuperLearner: Super Learner Prediction, 2021. URL https://CRAN.R-project.org/package=SuperLearner. R package version 2.0-28.1.
  26. Robins, J. A new approach to causal inference in mortality studies with a sustained exposure period—application to control of the healthy worker survivor effect. Mathematical modelling, 7(9-12):1393–1512, 1986.
  27. Estimation of Regression Coefficients When Some Regressors Are Not Always Observed. Journal of the American Statistical Association, 89(427):846–866, 1994.
  28. High-dimensional survival analysis: Methods and applications. Annual review of statistics and its application, 10:25–49, 2023.
  29. Sutton, R. S. Learning to predict by the methods of temporal differences. Machine learning, 3:9–44, 1988.
  30. Targeted Maximum Likelihood Learning. The International Journal of Biostatistics, 2(1), 2006.
  31. Targeted Minimum Loss Based Estimation of Causal Effects of Multiple Time Point Interventions. The International Journal of Biostatistics, 8(1), 2012.
  32. Unified Methods for Censored Longitudinal Data and Causality. Springer Series in Statistics. Springer New York, 2003. ISBN 978-0-387-21700-0.
  33. Targeted Learning: Causal Inference for Observational and Experimental Data. Springer Series in Statistics. Springer, 2011. ISBN 978-1-4419-9781-4 978-1-4419-9782-1.
  34. Targeted Learning in Data Science: Causal Inference for Complex Longitudinal Studies. Springer Series in Statistics. Springer International Publishing, 2018.
  35. Super learner. Statistical Applications in Genetics and Molecular Biology, 6(1):1309–1309, 2007. ISSN 1544-6115.
  36. Attention is all you need. In Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017.
  37. Machine learning for improving high‐dimensional proxy confounder adjustment in healthcare database studies: An overview of the current literature. Pharmacoepidemiology and drug safety, 31(9):932–943, 2022. ISSN 1053-8569.
  38. The Circulatory Risk in Communities Study (CIRCS): A Long-Term Epidemiological Study for Lifestyle-Related Disease Among Japanese Men and Women Living in Communities. Journal of Epidemiology, 29(3):83–91, 2019.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets