SportsNGEN: Sustained Generation of Realistic Multi-player Sports Gameplay (2403.12977v3)
Abstract: We present a transformer decoder based sports simulation engine, SportsNGEN, trained on sports player and ball tracking sequences, that is capable of generating sustained gameplay and accurately mimicking the decision making of real players. By training on a large database of professional tennis tracking data, we demonstrate that simulations produced by SportsNGEN can be used to predict the outcomes of rallies, determine the best shot choices at any point, and evaluate counterfactual or what if scenarios to inform coaching decisions and elevate broadcast coverage. By combining the generated simulations with a shot classifier and logic to start and end rallies, the system is capable of simulating an entire tennis match. We evaluate SportsNGEN by comparing statistics of the simulations with those of real matches between the same players. We show that the model output sampling parameters are crucial to simulation realism and that SportsNGEN is probabilistically well-calibrated to real data. In addition, a generic version of SportsNGEN can be customized to a specific player by fine-tuning on the subset of match data that includes that player. Finally, we show qualitative results indicating the same approach works for football.
- baller2vec: A multi-entity transformer for multi-agent spatiotemporal modeling. arXiv preprint arXiv:2102.03291, 2021.
- rsoccer: A framework for studying reinforcement learning in small and very small size robot soccer. RoboCup 2021: Robot World Cup XXIV, 13132:165, 2022.
- Professional basketball player behavior synthesis via planning with diffusion. arXiv preprint arXiv:2306.04090, 2023.
- Group activity recognition in basketball tracking data–neural embeddings in team sports (nets). arXiv preprint arXiv:2209.00451, 2022.
- The curious case of neural text degeneration. In International Conference on Learning Representations, 2020.
- Google research football: A novel reinforcement learning environment. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pp. 4501–4510, 2020.
- Data-driven ghosting using deep imitation learning. 2017a.
- Coordinated multi-agent imitation learning. In International Conference on Machine Learning, pp. 1995–2003. PMLR, 2017b.
- Grin: Generative relation and intention network for multi-agent trajectory prediction. Advances in Neural Information Processing Systems, 34:27107–27118, 2021.
- From motor control to team play in simulated humanoid football. arXiv preprint arXiv:2105.12196, 2021.
- Factorized point process intensities: A spatial analysis of professional basketball. In Xing, E. P. and Jebara, T. (eds.), Proceedings of the 31st International Conference on Machine Learning, volume 32 of Proceedings of Machine Learning Research, pp. 235–243, Bejing, China, 22–24 Jun 2014. PMLR. URL https://proceedings.mlr.press/v32/miller14.html.
- Multiagent off-screen behavior prediction in football. Scientific reports, 12(1):8638, 2022.
- Collaborative uncertainty in multi-agent trajectory forecasting. Advances in Neural Information Processing Systems, 34:6328–6340, 2021.
- Evaluation of creating scoring opportunities for teammates in soccer via trajectory prediction. In International Workshop on Machine Learning and Data Mining for Sports Analytics, pp. 53–73. Springer, 2022.
- Attention is all you need. Advances in neural information processing systems, 30, 2017.
- Tacticai: an ai assistant for football tactics. arXiv preprint arXiv:2310.10553, 2023.
- Basketball gan: Sportingly acceptable trajectory prediction. 2021.
- Asynchronous multi-agent reinforcement learning for efficient real-time multi-robot cooperative exploration. In Proceedings of the 2023 International Conference on Autonomous Agents and Multiagent Systems, AAMAS ’23, pp. 1107–1115, Richland, SC, 2023. International Foundation for Autonomous Agents and Multiagent Systems. ISBN 9781450394321.
- Learning fine-grained spatial models for dynamic sports play prediction. In 2014 IEEE international conference on data mining, pp. 670–679. IEEE, 2014.
- Generating multi-agent trajectories using programmatic weak supervision. In International Conference on Learning Representations, 2019.
- A survey of deep learning in sports applications: Perception, comprehension, and decision. arXiv preprint arXiv:2307.03353, 2023.
- Generating long-term trajectories using deep hierarchical networks. Advances in Neural Information Processing Systems, 29, 2016.