Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash 90 tok/s
Gemini 2.5 Pro 57 tok/s Pro
GPT-5 Medium 27 tok/s
GPT-5 High 22 tok/s Pro
GPT-4o 101 tok/s
GPT OSS 120B 467 tok/s Pro
Kimi K2 163 tok/s Pro
2000 character limit reached

Estimating counterfactual treatment outcomes over time in complex multiagent scenarios (2206.01900v4)

Published 4 Jun 2022 in cs.AI, cs.LG, cs.MA, stat.ME, and stat.ML

Abstract: Evaluation of intervention in a multiagent system, e.g., when humans should intervene in autonomous driving systems and when a player should pass to teammates for a good shot, is challenging in various engineering and scientific fields. Estimating the individual treatment effect (ITE) using counterfactual long-term prediction is practical to evaluate such interventions. However, most of the conventional frameworks did not consider the time-varying complex structure of multiagent relationships and covariate counterfactual prediction. This may lead to erroneous assessments of ITE and difficulty in interpretation. Here we propose an interpretable, counterfactual recurrent network in multiagent systems to estimate the effect of the intervention. Our model leverages graph variational recurrent neural networks and theory-based computation with domain knowledge for the ITE estimation framework based on long-term prediction of multiagent covariates and outcomes, which can confirm the circumstances under which the intervention is effective. On simulated models of an automated vehicle and biological agents with time-varying confounders, we show that our methods achieved lower estimation errors in counterfactual covariates and the most effective treatment timing than the baselines. Furthermore, using real basketball data, our methods performed realistic counterfactual predictions and evaluated the counterfactual passes in shot scenarios.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (80)
  1. T. A. Glass, S. N. Goodman, M. A. Hernán, and J. M. Samet, “Causal inference in public health,” Annual Review of Public Health, vol. 34, pp. 61–75, 2013.
  2. N. Baum-Snow and F. Ferreira, “Causal inference in urban and regional economics,” in Handbook of regional and urban economics.   Elsevier, 2015, vol. 5, pp. 3–68.
  3. P. Wang, W. Sun, D. Yin, J. Yang, and Y. Chang, “Robust tree-based causal inference for complex ad effectiveness analysis,” in Proceedings of the Eighth ACM International Conference on Web Search and Data Mining, 2015, pp. 67–76.
  4. B. Lim, A. M. Alaa, and M. van der Schaar, “Forecasting treatment responses over time using recurrent marginal structural networks.” Advances in Neural Information Processing Systems, vol. 18, pp. 7483–7493, 2018.
  5. I. Bica, A. M. Alaa, J. Jordon, and M. van der Schaar, “Estimating counterfactual treatment outcomes over time through adversarially balanced representations,” in International Conference on Learning Representations, 2020.
  6. I. Bica, A. Alaa, and M. Van Der Schaar, “Time series deconfounder: Estimating treatment effects over time in the presence of hidden confounders,” in International Conference on Machine Learning.   PMLR, 2020, pp. 884–895.
  7. R. Liu, C. Yin, and P. Zhang, “Estimating individual treatment effects with time-varying confounders,” in 2020 IEEE International Conference on Data Mining (ICDM).   IEEE, 2020, pp. 382–391.
  8. J. Ma, R. Guo, C. Chen, A. Zhang, and J. Li, “Deconfounding with networked observational data in a dynamic environment,” in Proceedings of the 14th ACM International Conference on Web Search and Data Mining, 2021, pp. 166–174.
  9. T. Vicsek, A. Czirók, E. Ben-Jacob, I. Cohen, and O. Shochet, “Novel type of phase transition in a system of self-driven particles,” Physical Review Letters, vol. 75, no. 6, pp. 1226–1229, 1995.
  10. I. D. Couzin, J. Krause, R. James, G. D. Ruxton, and N. R. Franks, “Collective memory and spatial sorting in animal groups,” Journal of Theoretical Biology, vol. 218, no. 1, pp. 1–11, 2002.
  11. K. Fujii, T. Kawasaki, Y. Inaba, and Y. Kawahara, “Prediction and classification in equation-free collective motion dynamics,” PLoS Computational Biology, vol. 14, no. 11, p. e1006545, 2018.
  12. K. Fujii, N. Takeishi, M. Hojo, Y. Inaba, and Y. Kawahara, “Physically-interpretable classification of network dynamics for complex collective motions,” Scientific Reports, vol. 10, no. 3005, 2020.
  13. R. A. Yeh, A. G. Schwing, J. Huang, and K. Murphy, “Diverse generation for multi-agent sports games,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2019, pp. 4610–4619.
  14. K. Fujii, K. Takeuchi, A. Kuribayashi, N. Takeishi, Y. Kawahara, and K. Takeda, “Estimating counterfactual treatment outcomes over time in multi-vehicle simulation,” in Proceedings of the 30th International Conference on Advances in Geographic Information Systems (SIGSPATIAL’22), 2022.
  15. J. Chung, K. Kastner, L. Dinh, K. Goel, A. C. Courville, and Y. Bengio, “A recurrent latent variable model for sequential data,” in Advances in Neural Information Processing Systems 28, 2015, pp. 2980–2988.
  16. T. Kipf, E. Fetaya, K.-C. Wang, M. Welling, and R. Zemel, “Neural relational inference for interacting systems,” in International Conference on Machine Learning, 2018, pp. 2688–2697.
  17. D. B. Rubin, “Bayesian inference for causal effects: The role of randomization,” The Annals of statistics, pp. 34–58, 1978.
  18. J. M. Robins and M. A. Hernán, “Estimation of the causal effects of time-varying exposures,” in Longitudinal Data Analysis, G. Fitzmaurice, M. Davidian, G. Verbeke et al., Eds.   New York, NY: Chapman & Hall/CRC Press, 2009, pp. 553–597.
  19. J. M. Robins, M. A. Hernan, and B. Brumback, “Marginal structural models and causal inference in epidemiology,” 2000.
  20. E. Zhan, S. Zheng, Y. Yue, L. Sha, and P. Lucey, “Generating multi-agent trajectories using programmatic weak supervision,” in International Conference on Learning Representations, 2019.
  21. M. Fraccaro, S. K. Sønderby, U. Paquet, and O. Winther, “Sequential neural models with stochastic layers,” in Advances in Neural Information Processing Systems 29, 2016, pp. 2199–2207.
  22. A. G. A. P. Goyal, A. Sordoni, M.-A. Côté, N. R. Ke, and Y. Bengio, “Z-forcing: Training stochastic recurrent networks,” in Advances in Neural Information Processing Systems 30, 2017, pp. 6713–6723.
  23. M. Zaheer, S. Kottur, S. Ravanbakhsh, B. Poczos, R. R. Salakhutdinov, and A. J. Smola, “Deep sets,” Advances in Neural Information Processing Systems, vol. 30, pp. 3394––3404, 2017.
  24. N. Takeishi and A. Kalousis, “Physics-integrated variational autoencoders for robust and interpretable generative modeling,” Advances in Neural Information Processing Systems, vol. 34, pp. 14 809–14 821, 2021.
  25. K. Fujii, N. Takeishi, Y. Kawahara, and K. Takeda, “Decentralized policy learning with partial observation and mechanical constraints for multiperson modeling,” Neural Networks, vol. 171, pp. 40–52, 2024.
  26. P. R. Rosenbaum and D. B. Rubin, “The central role of the propensity score in observational studies for causal effects,” Biometrika, vol. 70, no. 1, pp. 41–55, 1983.
  27. U. Shalit, F. D. Johansson, and D. Sontag, “Estimating individual treatment effect: generalization bounds and algorithms,” in International Conference on Machine Learning.   PMLR, 2017, pp. 3076–3085.
  28. Y. Ganin, E. Ustinova, H. Ajakan, P. Germain, H. Larochelle, F. Laviolette, M. Marchand, and V. Lempitsky, “Domain-adversarial training of neural networks,” The Journal of Machine Learning Research, vol. 17, no. 1, pp. 2096–2030, 2016.
  29. J. Robins, “A new approach to causal inference in mortality studies with a sustained exposure period—application to control of the healthy worker survivor effect,” Mathematical Modelling, vol. 7, no. 9-12, pp. 1393–1512, 1986.
  30. J. M. Robins, “Correcting for non-compliance in randomized trials using structural nested mean models,” Communications in Statistics-Theory and methods, vol. 23, no. 8, pp. 2379–2412, 1994.
  31. Y. Xu, Y. Xu, and S. Saria, “A bayesian nonparametric approach for estimating individualized treatment-response curves,” in Machine Learning for Healthcare Conference.   PMLR, 2016, pp. 282–300.
  32. P. Schulam and S. Saria, “Reliable decision support using counterfactual models,” Advances in Neural Information Processing Systems, vol. 30, pp. 1697–1708, 2017.
  33. H. Soleimani, A. Subbaswamy, and S. Saria, “Treatment-response models for counterfactual reasoning with continuous-time, continuous-valued interventions,” in 33rd Conference on Uncertainty in Artificial Intelligence, UAI 2017.   AUAI Press Corvallis, OR, 2017.
  34. J. Roy, K. J. Lum, and M. J. Daniels, “A bayesian nonparametric approach to marginal structural models for point treatments and a continuous or survival outcome,” Biostatistics, vol. 18, no. 1, pp. 32–47, 2017.
  35. J. L. Hill, “Bayesian nonparametric modeling for causal inference,” Journal of Computational and Graphical Statistics, vol. 20, no. 1, pp. 217–240, 2011.
  36. S. Wager and S. Athey, “Estimation and inference of heterogeneous treatment effects using random forests,” Journal of the American Statistical Association, vol. 113, no. 523, pp. 1228–1242, 2018.
  37. A. Alaa and M. Schaar, “Limits of estimating heterogeneous treatment effects: Guidelines for practical algorithm design,” in International Conference on Machine Learning.   PMLR, 2018, pp. 129–138.
  38. F. Johansson, U. Shalit, and D. Sontag, “Learning representations for counterfactual inference,” in International Conference on Machine Learning.   PMLR, 2016, pp. 3020–3029.
  39. L. Yao, S. Li, Y. Li, M. Huai, J. Gao, and A. Zhang, “Representation learning for treatment effect estimation from observational data,” Advances in Neural Information Processing Systems, vol. 31, 2018.
  40. J. Yoon, J. Jordon, and M. Van Der Schaar, “Ganite: Estimation of individualized treatment effects using generative adversarial nets,” in International Conference on Learning Representations, 2018.
  41. C. Shi, D. M. Blei, and V. Veitch, “Adapting neural networks for the estimation of treatment effects,” in Proceedings of the 33rd International Conference on Neural Information Processing Systems, 2019, pp. 2507–2517.
  42. P. Grecov, A. N. Prasanna, K. Ackermann, S. Campbell, D. Scott, D. I. Lubman, and C. Bergmeir, “Probabilistic causal effect estimation with global neural network forecasting models,” IEEE Transactions on Neural Networks and Learning Systems, pp. 1–15, July 2022.
  43. M. Abroshan, K. H. Yip, C. Tekin, and M. van der Schaar, “Conservative policy construction using variational autoencoders for logged data with missing values,” IEEE Transactions on Neural Networks and Learning Systems, pp. 1–11, 2022.
  44. Q. Li, Z. Wang, S. Liu, G. Li, and G. Xu, “Causal optimal transport for treatment effect estimation,” IEEE Transactions on Neural Networks and Learning Systems, pp. 1–13, 2021.
  45. Z. Wu, S. Pan, G. Long, J. Jiang, and C. Zhang, “Graph wavenet for deep spatial-temporal graph modeling,” in Proceedings of the 28th International Joint Conference on Artificial Intelligence, 2019, pp. 1907–1913.
  46. Q. Zhang, J. Chang, G. Meng, S. Xiang, and C. Pan, “Spatio-temporal graph structure learning for traffic forecasting,” in Proceedings of the AAAI conference on artificial intelligence, vol. 34, no. 01, 2020, pp. 1177–1185.
  47. K. Takeuchi, R. Nishida, H. Kashima, and M. Onishi, “Causal effect estimation on hierarchical spatial graph data,” in Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2023, pp. 2145–2154.
  48. Z. Guo, T. Xiao, C. Aggarwal, H. Liu, and S. Wang, “Counterfactual learning on graphs: A survey,” arXiv preprint arXiv:2304.01391, 2023.
  49. N. Seedat, F. Imrie, A. Bellot, Z. Qian, and M. van der Schaar, “Continuous-time modeling of counterfactual outcomes using neural controlled differential equations,” in International Conference on Machine Learning.   PMLR, 2022, pp. 19 497–19 521.
  50. S. Jiang, Z. Huang, X. Luo, and Y. Sun, “Cf-gode: Continuous-time causal inference for multi-agent dynamical systems,” arXiv preprint arXiv:2306.11216, 2023.
  51. G. Chen, J. Li, J. Lu, and J. Zhou, “Human trajectory prediction via counterfactual analysis,” arXiv preprint arXiv:2107.14202, 2021.
  52. K. Fujii, N. Takeishi, K. Tsutsui, E. Fujioka, N. Nishiumi, R. Tanaka, M. Fukushiro, K. Ide, H. Kohno, K. Yoda, S. Takahashi, S. Hiryu, and Y. Kawahara, “Learning interaction rules from multi-animal trajectories via augmented behavioral models,” Advances in Neural Information Processing Systems, vol. 34, pp. 11 108–11 122, 2021.
  53. H. Nakahara, K. Takeda, and K. Fujii, “Estimating the effect of hitting strategies in baseball using counterfactual virtual simulation with deep learning,” International Journal of Computer Science in Sport, vol. 22, no. 1, pp. 1–12, January 2022.
  54. M. Teranishi, K. Tsutsui, K. Takeda, and K. Fujii, “Evaluation of creating scoring opportunities for teammates in soccer via trajectory prediction,” in International Workshop on Machine Learning and Data Mining for Sports Analytics.   NY: Springer, 2022, pp. 53–73.
  55. A. Lerer, S. Gross, and R. Fergus, “Learning physical intuition of block towers by example,” in International conference on machine learning.   PMLR, 2016, pp. 430–438.
  56. G. Liu and O. Schulte, “Deep reinforcement learning in ice hockey for context-aware player evaluation,” in Proceedings of the 27th International Joint Conference on Artificial Intelligence, 2018, pp. 3442–3448.
  57. G. Liu, Y. Luo, O. Schulte, and T. Kharrat, “Deep soccer analytics: learning an action-value function for evaluating soccer players,” Data Mining and Knowledge Discovery, vol. 34, no. 5, pp. 1531–1559, 2020.
  58. P. Rahimian, J. Van Haaren, T. Abzhanova, and L. Toka, “Beyond action valuation: A deep reinforcement learning framework for optimizing player decisions in soccer,” in 16th Annual MIT Sloan Sports Analytics Conference. Boston, MA, USA: MIT, 2022, p. 25.
  59. H. Nakahara, K. Tsutsui, K. Takeda, and K. Fujii, “Action valuation of on-and off-ball soccer players based on multi-agent deep reinforcement learning,” IEEE Access, vol. 11, pp. 131 237–131 244, 2023.
  60. V. Ramanishka, Y.-T. Chen, T. Misu, and K. Saenko, “Toward driving scene understanding: A dataset for learning driver behavior and causal reasoning,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 7699–7707.
  61. T. You and B. Han, “Traffic accident benchmark for causality recognition,” in European Conference on Computer Vision.   NY: Springer, 2020, pp. 540–556.
  62. D. McDuff, Y. Song, J. Lee, V. Vineet, S. Vemprala, N. Gyde, H. Salman, S. Ma, K. Sohn, and A. Kapoor, “Causalcity: Complex simulations with agency for causal discovery and reasoning,” arXiv preprint arXiv:2106.13364, 2021.
  63. K. Takeuchi, R. Nishida, H. Kashima, and M. Onishi, “Grab the reins of crowds: Estimating the effects of crowd movement guidance using causal inference,” in Proceedings of the 20th International Conference on Autonomous Agents and MultiAgent Systems, 2021, pp. 1290–1298.
  64. D. R. Yam and M. J. Lopez, “What was lost? a causal estimate of fourth down behavior in the national football league,” Journal of Sports Analytics, vol. 5, no. 3, pp. 153–167, 2019.
  65. A. Toumi and M. Lopez, “From grapes and prunes to apples and apples: Using matched methods to estimate optimal zone entry decision-making in the national hockey league,” in Carnegie Mellon Sports Analytics Conference 2019, 2019.
  66. C. Gibbs, R. Elmore, and B. Fosdick, “The causal effect of a timeout at stopping an opposing run in the nba,” arXiv preprint arXiv:2011.11691, 2020.
  67. H. Nakahara, K. Takeda, and K. Fujii, “Pitching strategy evaluation via stratified analysis using propensity score,” Journal of Quantitative Analysis in Sports, vol. 19, no. 2, pp. 91–102, 2023.
  68. D. M. Vock and L. F. B. Vock, “Estimating the effect of plate discipline using a causal inference framework: an application of the g-computation algorithm,” Journal of Quantitative Analysis in Sports, vol. 14, no. 2, pp. 37–56, 2018.
  69. A. Dosovitskiy, G. Ros, F. Codevilla, A. Lopez, and V. Koltun, “Carla: An open urban driving simulator,” in Conference on Robot Learning.   PMLR, 2017, pp. 1–16.
  70. K. Cho, B. Van Merriënboer, D. Bahdanau, and Y. Bengio, “On the properties of neural machine translation: Encoder-decoder approaches,” arXiv preprint arXiv:1409.1259, 2014.
  71. C. J. Willmott and K. Matsuura, “Advantages of the mean absolute error (mae) over the root mean square error (rmse) in assessing average model performance,” Climate Research, vol. 30, no. 1, pp. 79–82, 2005.
  72. S. Kato, E. Takeuchi, Y. Ishiguro, Y. Ninomiya, K. Takeda, and T. Hamada, “An open approach to autonomous vehicles,” IEEE Micro, vol. 35, no. 6, pp. 60–68, 2015.
  73. C. W. Reynolds, “Flocks, herds and schools: A distributed behavioral model,” in Proceedings of the 14th annual Conference on Computer Graphics and Interactive Techniques, 1987, pp. 25–34.
  74. K. Fujii, Y. Inaba, and Y. Kawahara, “Koopman spectral kernels for comparing complex dynamics: Application to multiagent sport plays,” in European Conference on Machine Learning and Knowledge Discovery in Databases (ECML-PKDD’17).   NY: Springer, 2017, pp. 127–139.
  75. D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” in International Conference on Learning Representations, 2015.
  76. M. Quigley, K. Conley, B. Gerkey, J. Faust, T. Foote, J. Leibs, R. Wheeler, A. Y. Ng et al., “Ros: an open-source robot operating system,” in ICRA workshop on open source software, vol. 3, no. 3.2.   Kobe, Japan, 2009, p. 5.
  77. K. Fujii, K. Yokoyama, T. Koyama, A. Rikukawa, H. Yamada, and Y. Yamamoto, “Resilient help to switch and overlap hierarchical subsystems in a small human group,” Scientific Reports, vol. 6, 2016.
  78. T. Decroos, L. Bransen, J. Van Haaren, and J. Davis, “Actions speak louder than goals: Valuing player actions in soccer,” in KDD, 2019, pp. 1851–1861.
  79. K. Toda, M. Teranishi, K. Kushiro, and K. Fujii, “Evaluation of soccer team defense based on prediction models of ball recovery and being attacked,” PLoS One, vol. 17, no. 1, p. e0263051, 2022.
  80. G. Ke, Q. Meng, T. Finley, T. Wang, W. Chen, W. Ma, Q. Ye, and T.-Y. Liu, “Lightgbm: A highly efficient gradient boosting decision tree,” Advances in Neural Information Processing Systems, vol. 30, pp. 3146–3154, 2017.
Citations (12)
View on Semantic Scholar
List To Do Tasks Checklist Streamline Icon: https://streamlinehq.com

Collections

Sign up for free to add this paper to one or more collections.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
Ai Generate Text Spark Streamline Icon: https://streamlinehq.com

Paper Prompts

Sign up for free to create and run prompts on this paper using GPT-5.

List Streamline Icon: https://streamlinehq.com Explore 10 Community Prompts
Dice Question Streamline Icon: https://streamlinehq.com

Follow-up Questions

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now