Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
194 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
46 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Pareto-Optimal Estimation and Policy Learning on Short-term and Long-term Treatment Effects (2403.02624v2)

Published 5 Mar 2024 in cs.LG and cs.AI

Abstract: This paper focuses on developing Pareto-optimal estimation and policy learning to identify the most effective treatment that maximizes the total reward from both short-term and long-term effects, which might conflict with each other. For example, a higher dosage of medication might increase the speed of a patient's recovery (short-term) but could also result in severe long-term side effects. Although recent works have investigated the problems about short-term or long-term effects or the both, how to trade-off between them to achieve optimal treatment remains an open challenge. Moreover, when multiple objectives are directly estimated using conventional causal representation learning, the optimization directions among various tasks can conflict as well. In this paper, we systematically investigate these issues and introduce a Pareto-Efficient algorithm, comprising Pareto-Optimal Estimation (POE) and Pareto-Optimal Policy Learning (POPL), to tackle them. POE incorporates a continuous Pareto module with representation balancing, enhancing estimation efficiency across multiple tasks. As for POPL, it involves deriving short-term and long-term outcomes linked with various treatment levels, facilitating an exploration of the Pareto frontier emanating from these outcomes. Results on both the synthetic and real-world datasets demonstrate the superiority of our method.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (62)
  1. W. Hu, X. Zhou, and P. Wu, “Identification and estimation of treatment effects on long-term outcomes in clinical trials with external observational data,” 2023.
  2. R. Chetty, J. N. Friedman, N. Hilger, E. Saez, D. W. Schanzenbach, and D. Yagan, “How does your kindergarten classroom affect your earnings? evidence from project star,” The Quarterly Journal of Economics, vol. 126, no. 4, pp. 1593–1660, 2011.
  3. J. Yang, D. Eckles, P. S. Dhillon, and S. Aral, “Targeting for long-term outcomes,” CoRR, vol. abs/2010.15835, 2020.
  4. D. B. Rubin, “Estimating causal effects of treatments in randomized and nonrandomized studies.” Journal of educational Psychology, vol. 66, no. 5, p. 688, 1974.
  5. P. R. Rosenbaum and D. B. Rubin, “Reducing bias in observational studies using subclassification on the propensity score,” Journal of the American statistical Association, vol. 79, no. 387, pp. 516–524, 1984.
  6. P. R. Rosenbaum, “Model-based direct adjustment,” Journal of the American statistical Association, vol. 82, no. 398, pp. 387–394, 1987.
  7. R. H. Dehejia and S. Wahba, “Propensity score-matching methods for nonexperimental causal studies,” Review of Economics and statistics, vol. 84, no. 1, pp. 151–161, 2002.
  8. J. Hainmueller, “Entropy balancing for causal effects: A multivariate reweighting method to produce balanced samples in observational studies,” Political analysis, vol. 20, no. 1, pp. 25–46, 2012.
  9. S. Athey, G. Imbens, and S. Wager, “Approximate residual balancing: debiased inference of average treatment effects in high dimensions,” Journal of the Royal Statistical Society: Series B (Statistical Methodology), vol. 80, 2016.
  10. F. Johansson, U. Shalit, and D. Sontag, “Learning representations for counterfactual inference,” in International conference on machine learning.   PMLR, 2016, pp. 3020–3029.
  11. U. Shalit, F. D. Johansson, and D. Sontag, “Estimating individual treatment effect: generalization bounds and algorithms,” in International Conference on Machine Learning.   PMLR, 2017, pp. 3076–3085.
  12. J. Yoon, J. Jordon, and M. van der Schaar, “GANITE: estimation of individualized treatment effects using generative adversarial nets,” in 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings.   OpenReview.net, 2018.
  13. C. Louizos, U. Shalit, J. M. Mooij, D. A. Sontag, R. S. Zemel, and M. Welling, “Causal effect inference with deep latent-variable models,” in Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA, 2017, pp. 6446–6456.
  14. S. Athey, R. Chetty, G. W. Imbens, and H. Kang, “The Surrogate Index: Combining Short-Term Proxies to Estimate Long-Term Treatment Effects More Rapidly and Precisely,” National Bureau of Economic Research, Inc, NBER Working Papers 26463, Nov. 2019.
  15. S. Athey, R. Chetty, and G. Imbens, “Combining experimental and observational data to estimate treatment effects on long term outcomes,” 2020.
  16. J. Chen and D. M. Ritzwoller, “Semiparametric estimation of long-term treatment effects,” 2023.
  17. N. Kallus and X. Mao, “On the role of surrogates in the efficient estimation of treatment effects with limited outcome data,” 2022.
  18. L. S. Freedman, B. I. Graubard, and A. Schatzkin, “Statistical validation of intermediate endpoints for chronic diseases,” Statistics in medicine, vol. 11, no. 2, pp. 167–178, 1992.
  19. C. E. Frangakis and D. B. Rubin, “Principal stratification in causal inference,” Biometrics, vol. 58, no. 1, pp. 21–29, 2002.
  20. M. M. Joffe and T. Greene, “Related causal frameworks for surrogate outcomes,” Biometrics, vol. 65, no. 2, pp. 530–538, 2009.
  21. G. Imbens, N. Kallus, X. Mao, and Y. Wang, “Long-term causal inference under persistent confounding via data combination,” 2023.
  22. A. Ghassami, A. Yang, D. Richardson, I. Shpitser, and E. T. Tchetgen, “Combining experimental and observational data for identification and estimation of long-term causal effects,” 2022.
  23. A. Norcliffe, B. Cebere, F. Imrie, P. Lio, and M. van der Schaar, “Survivalgan: Generating time-to-event data for survival analysis,” 2023.
  24. I. J. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. C. Courville, and Y. Bengio, “Generative adversarial nets,” in Advances in Neural Information Processing Systems 27: Annual Conference on Neural Information Processing Systems 2014, December 8-13 2014, Montreal, Quebec, Canada, 2014, pp. 2672–2680.
  25. R. Kohavi, A. Deng, B. Frasca, R. Longbotham, T. Walker, and Y. Xu, “Trustworthy online controlled experiments: five puzzling outcomes explained,” in The 18th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’12, Beijing, China, August 12-16, 2012.   ACM, 2012, pp. 786–794.
  26. L. Cheng, R. Guo, and H. Liu, “Long-term effect estimation with surrogate representation,” in WSDM ’21, The Fourteenth ACM International Conference on Web Search and Data Mining, Virtual Event, Israel, March 8-12, 2021.   ACM, 2021, pp. 274–282.
  27. Z. Chu, R. Li, S. Rathbun, and S. Li, “Continual causal inference with incremental observational data,” in 2023 IEEE 39th International Conference on Data Engineering (ICDE), 2023, pp. 3430–3439.
  28. F. Zhu, M. Zhong, X. Yang, L. Li, L. Yu, T. Zhang, J. Zhou, C. Chen, F. Wu, G. Liu, and Y. Wang, “Dcmt: A direct entire-space causal multi-task framework for post-click conversion estimation,” in 2023 IEEE 39th International Conference on Data Engineering (ICDE), 2023, pp. 3113–3125.
  29. Z. Wang, X. Chen, R. Zhou, Q. Dai, Z. Dong, and J.-R. Wen, “Sequential recommendation with user causal behavior discovery,” in 2023 IEEE 39th International Conference on Data Engineering (ICDE).   IEEE, 2023, pp. 28–40.
  30. F. Shen, K. Heravi, O. Gomez, S. Galhotra, A. Gilad, S. Roy, and B. Salimi, “Causal what-if and how-to analysis using hyper,” in IEEE International Conference on Data Engineering (ICDE 2023), Demonstration Track, 2023.
  31. B. Youngmann, M. Cafarella, Y. Moskovitch, and B. Salimi, “On explaining confounding bias,” in 2023 IEEE 39th International Conference on Data Engineering (ICDE).   IEEE, 2023, pp. 1846–1859.
  32. Y. Zhang and Q. Yang, “A survey on multi-task learning,” IEEE Transactions on Knowledge and Data Engineering, vol. 34, no. 12, pp. 5586–5609, 2022.
  33. H. Davoudi, Z. Rashidi, A. An, M. Zihayat, and G. Edall, “Paywall policy learning in digital news media,” IEEE Transactions on Knowledge and Data Engineering, vol. 33, no. 10, pp. 3394–3409, 2021.
  34. S. Zhao, M. K. Chen, C. Borcea, and Y. Chen, “Personalized dynamic counter ad-blocking using deep learning,” IEEE Transactions on Knowledge and Data Engineering, vol. 35, no. 8, pp. 8358–8371, 2023.
  35. X.-H. Chen, B. He, Y. Yu, Q. Li, Z. Qin, W. Shang, J. Ye, and C. Ma, “Sim2rec: A simulator-based decision-making approach to optimize real-world long-term user engagement in sequential recommender systems,” in 2023 IEEE 39th International Conference on Data Engineering (ICDE), 2023, pp. 3389–3402.
  36. P. R. Rosenbaum and D. B. Rubin, “The central role of the propensity score in observational studies for causal effects,” Biometrika, vol. 70, no. 1, pp. 41–55, 1983.
  37. R. L. Prentice, “Surrogate endpoints in clinical trials: definition and operational criteria,” Statistics in medicine, vol. 8, no. 4, pp. 431–440, 1989.
  38. S. L. Lauritzen, O. O. Aalen, D. B. Rubin, and E. Arjas, “Discussion on causality [with reply],” Scandinavian Journal of Statistics, vol. 31, no. 2, pp. 189–201, 2004.
  39. H. Chen, Z. Geng, and J. Jia, “Criteria for surrogate end points,” Journal of the Royal Statistical Society Series B: Statistical Methodology, vol. 69, no. 5, pp. 919–932, 2007.
  40. C. Ju and Z. Geng, “Criteria for surrogate end points based on causal distributions,” Journal of the Royal Statistical Society Series B: Statistical Methodology, vol. 72, no. 1, pp. 129–142, 2010.
  41. O. Sener and V. Koltun, “Multi-task learning as multi-objective optimization,” in Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, NeurIPS 2018, December 3-8, 2018, Montréal, Canada, 2018, pp. 525–536.
  42. X. Lin, H. Zhen, Z. Li, Q. Zhang, and S. Kwong, “Pareto multi-task learning,” in Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada, 2019, pp. 12 037–12 047.
  43. P. Ma, T. Du, and W. Matusik, “Efficient continuous pareto exploration in multi-task learning,” in Proceedings of the 37th International Conference on Machine Learning, ICML 2020, 13-18 July 2020, Virtual Event, ser. Proceedings of Machine Learning Research, vol. 119.   PMLR, 2020, pp. 6522–6531.
  44. M. Momma, C. Dong, and J. Liu, “A multi-objective / multi-task learning framework induced by pareto stationarity,” in International Conference on Machine Learning, ICML 2022, 17-23 July 2022, Baltimore, Maryland, USA, ser. Proceedings of Machine Learning Research, vol. 162.   PMLR, 2022, pp. 15 895–15 907.
  45. A. R. Luedtke and M. J. Van Der Laan, “Statistical inference for the mean outcome under a possibly non-unique optimal treatment strategy,” Annals of statistics, vol. 44, no. 2, p. 713, 2016.
  46. M. Qian and S. A. Murphy, “Performance guarantees for individualized treatment rules,” Annals of statistics, vol. 39, no. 2, p. 1180, 2011.
  47. B. Zhang, A. A. Tsiatis, M. Davidian, M. Zhang, and E. Laber, “Estimating optimal treatment regimes from a classification perspective,” Stat, vol. 1, no. 1, pp. 103–114, 2012.
  48. A. Beygelzimer and J. Langford, “The offset tree for learning with partial labels,” in Proceedings of the 15th ACM SIGKDD international conference on Knowledge discovery and data mining, 2009, pp. 129–138.
  49. A. Swaminathan and T. Joachims, “Batch learning from logged bandit feedback through counterfactual risk minimization,” The Journal of Machine Learning Research, vol. 16, no. 1, pp. 1731–1755, 2015.
  50. N. Kallus, “Balanced policy evaluation and learning,” Advances in neural information processing systems, vol. 31, 2018.
  51. E. Zitzler and L. Thiele, “Multiobjective evolutionary algorithms: a comparative case study and the strength pareto approach,” IEEE Trans. Evol. Comput., vol. 3, no. 4, pp. 257–271, 1999.
  52. F. Lv, J. Liang, K. Gong, S. Li, C. H. Liu, H. Li, D. Liu, and G. Wang, “Pareto domain adaptation,” in Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, NeurIPS 2021, December 6-14, 2021, virtual, 2021, pp. 12 917–12 929.
  53. J. Yuan, A. Wu, K. Kuang, B. Li, R. Wu, F. Wu, and L. Lin, “Auto IV: counterfactual prediction via automatic instrumental variable decomposition,” ACM Trans. Knowl. Discov. Data, vol. 16, no. 4, pp. 74:1–74:20, 2022.
  54. A. Wu, J. Yuan, K. Kuang, B. Li, R. Wu, Q. Zhu, Y. Zhuang, and F. Wu, “Learning decomposed representations for treatment effect estimation,” IEEE Transactions on Knowledge and Data Engineering, vol. 35, no. 5, pp. 4989–5001, 2022.
  55. P. Cheng, W. Hao, S. Dai, J. Liu, Z. Gan, and L. Carin, “Club: A contrastive log-ratio upper bound of mutual information,” in International conference on machine learning.   PMLR, 2020, pp. 1779–1788.
  56. A. Zhou, A. Koo, N. Kallus, R. Ropac, R. Peterson, S. Koppel, and T. Bergin, “Synthetic control analysis of the short-term impact of new york state’s bail elimination act on aggregate crime,” Statistics and Public Policy, no. just-accepted, pp. 1–26, 2023.
  57. J. Brooks-Gunn, F. Liaw, and P. K. Klebanov, “Effects of early intervention on cognitive function of low birth weight preterm infants,” The Journal of pediatrics, vol. 120, no. 3, pp. 350–359, 1992.
  58. R. J. LaLonde, “Evaluating the econometric evaluations of training programs with experimental data,” The American Economic Review, vol. 76, no. 4, pp. 604–620, 1986.
  59. D. Almond, K. Y. Chay, and D. S. Lee, “The costs of low birth weight,” The Quarterly Journal of Economics, vol. 120, no. 3, pp. 1031–1083, 2005.
  60. P. Schwab, L. Linhardt, S. Bauer, J. M. Buhmann, and W. Karlen, “Learning counterfactual representations for estimating individual dose-response curves,” in The Thirty-Fourth AAAI Conference on Artificial Intelligence, AAAI 2020, The Thirty-Second Innovative Applications of Artificial Intelligence Conference, IAAI 2020, The Tenth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2020, New York, NY, USA, February 7-12, 2020.   AAAI Press, 2020, pp. 5612–5619.
  61. L. Nie, M. Ye, Q. Liu, and D. Nicolae, “Vcnet and functional targeted regularization for learning causal effects of continuous treatments,” in 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021.   OpenReview.net, 2021.
  62. C. Shi, D. M. Blei, and V. Veitch, “Adapting neural networks for the estimation of treatment effects,” in Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada, H. M. Wallach, H. Larochelle, A. Beygelzimer, F. d’Alché-Buc, E. B. Fox, and R. Garnett, Eds., 2019, pp. 2503–2513.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now