Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
184 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

PyTAG: Tabletop Games for Multi-Agent Reinforcement Learning (2405.18123v1)

Published 28 May 2024 in cs.AI

Abstract: Modern Tabletop Games present various interesting challenges for Multi-agent Reinforcement Learning. In this paper, we introduce PyTAG, a new framework that supports interacting with a large collection of games implemented in the Tabletop Games framework. In this work we highlight the challenges tabletop games provide, from a game-playing agent perspective, along with the opportunities they provide for future research. Additionally, we highlight the technical challenges that involve training Reinforcement Learning agents on these games. To explore the Multi-agent setting provided by PyTAG we train the popular Proximal Policy Optimisation Reinforcement Learning algorithm using self-play on a subset of games and evaluate the trained policies against some simple agents and Monte-Carlo Tree Search implemented in the Tabletop Games framework.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (33)
  1. D. Silver et al., “Mastering the game of go with deep neural networks and tree search,” Nature, vol. 529, pp. 484–489, 01 2016.
  2. O. Vinyals et al., “Grandmaster level in starcraft ii using multi-agent reinforcement learning,” Nature, vol. 575:7782, pp. 350–354, 2019.
  3. C. Berner et al., “Dota 2 with large scale deep reinforcement learning,” preprint arXiv:1912.06680, 2019.
  4. I. Bravi, D. Perez-Liebana, S. M. Lucas, and J. Liu, “Rinascimento: Optimising statistical forward planning agents for playing splendor,” in 2019 IEEE Conference on Games (CoG), 2019, pp. 1–8.
  5. R. Gaina, M. Balla, A. Dockhorn, R. Montoliu Colás, and D. Perez, “TAG: A tabletop games framework.”   CEUR Workshop Proc., 2020.
  6. C. Browne et al., “A survey of monte carlo tree search methods,” IEEE Transactions on Computational Intelligence and AI in games, vol. 4:1, pp. 1–43, 2012.
  7. K. Zhang, Z. Yang, and T. Başar, “Multi-agent reinforcement learning: A selective overview of theories and algorithms,” Handbook of reinforcement learning and control, pp. 321–384, 2021.
  8. “Verified market research 2023,” Mar 2023. [Online]. Available: https://www.verifiedmarketresearch.com/product/board-games-market/
  9. M. Balla, G. E. Long, D. Jeurissen, J. Goodman, R. D. Gaina, and D. Perez-Liebana, “Pytag: Challenges and opportunities for reinforcement learning in tabletop games,” pp. 1–8, 2023.
  10. J. Terry et al., “Pettingzoo: Gym for multi-agent reinforcement learning,” Advances in Neural Information Processing Systems, vol. 34, pp. 15 032–15 043, 2021.
  11. S. Huang, S. Ontañón, C. Bamford, and L. Grela, “Gym-µrts: Toward affordable full game real-time strategy games research with deep reinforcement learning,” in 2021 IEEE Conference on Games, pp. 1–8.
  12. S. Whiteson et al., “The starcraft multi-agent challenge,” in Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS, 2019, pp. 2186–2188.
  13. N. Bard et al., “The hanabi challenge: A new frontier for ai research,” Artificial Intelligence, vol. 280, p. 103216, 2020.
  14. M. Johnson et al., “The Malmo Platform for Artificial Intelligence Experimentation,” in IJCAI, 2016, pp. 4246–4247.
  15. D. Zha, K.-H. Lai, Y. Cao, S. Huang, R. Wei, J. Guo, and X. Hu, “Rlcard: A toolkit for reinforcement learning in card games,” arXiv preprint arXiv:1910.04376, 2019.
  16. M. Lanctot et al., “Openspiel: A framework for reinforcement learning in games,” arXiv preprint arXiv:1908.09453, 2019.
  17. E. Piette, D. J. Soemers, M. Stephenson, C. F. Sironi, M. H. Winands, and C. Browne, “Ludii–the ludemic general game system,” arXiv preprint arXiv:1905.05013, 2019.
  18. D. Hernandez, K. Denamganaï, Y. Gao, P. York, S. Devlin, S. Samothrakis, and J. A. Walker, “A generalized framework for self-play training,” in 2019 IEEE Conference on Games (CoG).   IEEE, 2019, pp. 1–8.
  19. J. Heinrich, M. Lanctot, and D. Silver, “Fictitious self-play in extensive-form games,” in International conference on machine learning.   PMLR, 2015, pp. 805–813.
  20. L. S. Shapley, “Stochastic games,” Proceedings of the national academy of sciences, vol. 39, no. 10, pp. 1095–1100, 1953.
  21. G. Brockman, V. Cheung, L. Pettersson, J. Schneider, J. Schulman, J. Tang, and W. Zaremba, “Openai gym,” arXiv preprint arXiv:1606.01540, 2016.
  22. C. Bamford and A. Ovalle, “Generalising discrete action spaces with conditional action trees,” in 2021 IEEE Conference on Games (CoG).   IEEE, 2021, pp. 1–8.
  23. D. Anderson, M. Stephenson, J. Togelius, C. Salge, J. Levine, and J. Renz, “Deceptive games,” in Applications of Evolutionary Computation: 21st International Conference, EvoApplications 2018, Parma, Italy, April 4-6, 2018, Proceedings 21.   Springer, 2018, pp. 376–391.
  24. J. Goodman, D. Perez-Liebana, and S. Lucas, “Following the leader in multiplayer tabletop games,” in Proceedings of the 18th International Conference on the Foundations of Digital Games, ser. FDG ’23.   New York, NY, USA: Association for Computing Machinery, 2023.
  25. S. Huang, R. F. J. Dossa et al., “Cleanrl: High-quality single-file implementations of deep reinforcement learning algorithms,” Journal of Machine Learning Research, vol. 23, no. 274, pp. 1–18, 2022.
  26. J. Barker and R. Korf, “Solving dots-and-boxes,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 26:1, 2012, pp. 414–419.
  27. T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell et al., “Language models are few-shot learners,” Advances in neural information processing systems, vol. 33, pp. 1877–1901, 2020.
  28. G. Wang, Y. Xie, Y. Jiang, A. Mandlekar, C. Xiao, Y. Zhu, L. Fan, and A. Anandkumar, “Voyager: An open-ended embodied agent with large language models,” arXiv preprint arXiv:2305.16291, 2023.
  29. M. Klissarov, P. D’Oro, S. Sodhani, R. Raileanu, P.-L. Bacon, P. Vincent, A. Zhang, and M. Henaff, “Motif: Intrinsic motivation from artificial intelligence feedback,” in NeurIPS 2023 Foundation Models for Decision Making Workshop, 2023.
  30. F. Toriumi, H. Osawa, M. Inaba, D. Katagami, K. Shinoda, and H. Matsubara, “Ai wolf contest—development of game ai using collective intelligence—,” in 5th Workshop on Computer Games, and 5th Workshop on General Intelligence in Game-Playing Agents, IJCAI 2016.   Springer, 2017, pp. 101–115.
  31. M. F. A. R. D. T. (FAIR)† et al., “Human-level play in the game of ¡i¿diplomacy¡/i¿ by combining language models with strategic reasoning,” Science, vol. 378, no. 6624, pp. 1067–1074, 2022.
  32. M. Zhou, Z. Wan, H. Wang, M. Wen, R. Wu, Y. Wen, Y. Yang, Y. Yu, J. Wang, and W. Zhang, “Malib: A parallel framework for population-based multi-agent reinforcement learning,” Journal of Machine Learning Research, vol. 24, no. 150, pp. 1–12, 2023.
  33. R. Herbrich, T. Minka, and T. Graepel, “Trueskill™: a bayesian skill rating system,” Advances in NIPS, vol. 19, 2006.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets