Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
156 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Proof Number Based Monte-Carlo Tree Search (2303.09449v4)

Published 16 Mar 2023 in cs.AI

Abstract: This paper proposes a new game-search algorithm, PN-MCTS, which combines Monte-Carlo Tree Search (MCTS) and Proof-Number Search (PNS). These two algorithms have been successfully applied for decision making in a range of domains. We define three areas where the additional knowledge provided by the proof and disproof numbers gathered in MCTS trees might be used: final move selection, solving subtrees, and the UCB1 selection mechanism. We test all possible combinations on different time settings, playing against vanilla UCT on several games: Lines of Action ($7$$\times$$7$ and $8$$\times$$8$ board sizes), MiniShogi, Knightthrough, and Awari. Furthermore, we extend this new algorithm to properly address games with draws, like Awari, by adding an additional layer of PNS on top of the MCTS tree. The experiments show that PN-MCTS is able to outperform MCTS in all tested game domains, achieving win rates up to 96.2% for Lines of Action.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (47)
  1. R. Coulom, “Efficient selectivity and backup operators in Monte-Carlo Tree Search,” in Computers and Games (CG 2006), ser. Lecture Notes in Computer Science, H. J. van den Herik, P. Ciancarini, and H. H. L. M. Donkers, Eds., vol. 4630.   Berlin Heidelberg, Germany: Springer-Verlag, 2007, pp. 72–83.
  2. L. Kocsis and C. Szepesvári, “Bandit Based Monte-Carlo Planning,” in Machine Learning: ECML 2006, ser. Lecture Notes in Artificial Intelligence, J. Fürnkranz, T. Scheffer, and M. Spiliopoulou, Eds., vol. 4212, 2006, pp. 282–293.
  3. C. B. Browne, E. Powley, D. Whitehouse, S. M. Lucas, P. I. Cowling, P. Rohlfshagen, S. Tavener, D. Perez, S. Samothrakis, and S. Colton, “A survey of Monte Carlo Tree Search methods,” IEEE Transactions on Computational Intelligence and AI in Games, vol. 4, no. 1, pp. 1–43, 2012.
  4. H. Finnsson and Y. Björnsson, “Simulation-based approach to general game playing.” in AAAI, vol. 8, 2008, pp. 259–264.
  5. S. Gelly and D. Silver, “Monte-carlo tree search and rapid action value estimation in computer go,” Artificial Intelligence, vol. 175, no. 11, pp. 1856–1875, 2011.
  6. M. Świechowski, K. Godlewski, B. Sawicki, and J. Mańdziuk, “Monte carlo tree search: A review of recent modifications and applications,” Artificial Intelligence Review, vol. 56, no. 3, pp. 2497–2562, 2023.
  7. J. Kowalski, M. Mika, W. Pawlik, J. Sutowicz, M. Szykuła, and M. H. M. Winands, “Split Moves for Monte-Carlo Tree Search,” AAAI Conference on Artificial Intelligence, vol. 36, no. 9, pp. 10 247–10 255, 2022.
  8. D. Silver, J. Schrittwieser, K. Simonyan, I. Antonoglou, A. Huang, A. Guez, T. Hubert, L. Baker, M. Lai, A. Bolton, Y. Chen, T. Lillicrap, F. Hui, L. Sifre, G. van den Driessche, T. Graepel, and D. Hassabis, “Mastering the game of Go without human knowledge,” Nature, vol. 550, pp. 354–359, 2017.
  9. R. J. Lorentz, “Amazons discover Monte-Carlo,” in Computers and Games (CG 2008), ser. Lecture Notes in Computer Science, H. J. van den Herik, X. Xu, Z. Ma, and M. H. M. Winands, Eds., vol. 5131.   Berlin Heidelberg, Germany: Springer, 2008, pp. 13–24.
  10. B. Arneson, R. B. Hayward, and P. Henderson, “Monte Carlo Tree Search in Hex,” IEEE Transactions on Computational Intelligence and AI in Games, vol. 2, no. 4, pp. 251–258, 2010.
  11. M. H. M. Winands, Y. Björnsson, and J.-T. Saito, “Monte Carlo Tree Search in Lines of Action,” IEEE Transactions on Computational Intelligence and AI in Games, vol. 2, no. 4, pp. 239–250, 2010.
  12. Y. Björnsson and H. Finnsson, “CadiaPlayer: A simulation-based General Game Player,” IEEE Transactions on Computational Intelligence and AI in Games, vol. 1, no. 1, pp. 4–15, 2009.
  13. N. R. Sturtevant, “An analysis of UCT in multi-player games,” ICGA Journal, vol. 31, no. 4, pp. 195–208, 2008.
  14. P. Ciancarini and G. P. Favini, “Monte Carlo Tree Search in Kriegspiel,” AI Journal, vol. 174, no. 11, pp. 670––684, 2010.
  15. J. A. M. Nijssen and M. H. M. Winands, “Monte-Carlo Tree Search for the hide-and-seek game Scotland Yard,” Transactions on Computational Intelligence and AI in Games, vol. 4, no. 4, pp. 282–294, 2012.
  16. M. H. M. Winands, Y. Björnsson, and J.-T. Saito, “Monte-Carlo Tree Search Solver,” in Computers and Games (CG 2008), ser. Lecture Notes in Computer Science (LNCS), H. J. van den Herik, X. Xu, Z. Ma, and M. H. M. Winands, Eds., vol. 5131.   Berlin Heidelberg, Germany: Springer, 2008, pp. 25–36.
  17. M. H. M. Winands and Y. Björnsson, “α⁢β𝛼𝛽\alpha\betaitalic_α italic_β-based play-outs in Monte-Carlo Tree Search,” in 2011 IEEE Conference on Computational Intelligence and Games (CIG 2011), S.-B. Cho, S. M. Lucas, and P. Hingston, Eds.   IEEE, 2011, pp. 110–117.
  18. M. Lanctot, M. H. M. Winands, T. Pepels, and N. R. Sturtevant, “Monte carlo tree search with heuristic evaluations using implicit minimax backups,” in 2014 IEEE Conference on Computational Intelligence and Games, CIG 2014, 2014, pp. 341–348.
  19. H. Baier and M. H. M. Winands, “MCTS-minimax hybrids,” IEEE Transactions on Computational Intelligence and AI in Games, vol. 7, no. 2, pp. 167–179, 2015.
  20. L. V. Allis, M. van der Meulen, and H. J. van den Herik, “Proof-number search,” Artificial Intelligence, vol. 66, no. 1, pp. 91–123, 1994.
  21. H. J. van den Herik and M. H. M. Winands, “Proof-number search and its variants,” in Oppositional Concepts in Computational Intelligence, H. R. Tizhoosh and M. Ventresca, Eds.   Berlin, Heidelberg: Springer Berlin Heidelberg, 2008, pp. 91–118.
  22. D. M. Breuker, “Memory versus search in games,” Ph.D. dissertation, Maastricht University, Maastricht, The Netherlands, 1998.
  23. A. Nagai, “Df-pn algorithm for searching AND/OR trees and its applications,” Ph.D. dissertation, The University of Tokyo, Tokyo, Japan, 2002.
  24. M. H. M. Winands, J. W. H. M. Uiterwijk, and H. J. van den Herik, “An effective two-level proof-number search algorithm,” Theoretical Computer Science, vol. 313, no. 3, pp. 511–525, 2004.
  25. A. Kishimoto and M. Müller, “Search versus knowledge for solving life and death problems in Go,” in Proceedings of the 20th National Conference on Artificial Intelligence (AAAI’05), M. M. Veloso and S. Kambhampati, Eds.   Menlo Park, CA, USA: AAAI Press / MIT Press, 2005, pp. 1374–1379.
  26. J. Schaeffer, N. Burch, Y. Björnsson, A. Kishimoto, M. Müller, R. Lake, P. Lu, and S. Sutphen, “Checkers is solved,” Science, vol. 317, no. 5844, pp. 1518–1522, 2007.
  27. I.-C. Wu, H.-H. Lin, P.-H. Lin, D.-J. Sun, Y.-C. Chan, and B.-T. Chen, “Job-level proof-number search for connect6,” in Computers and Games (CG’10), ser. Lecture Notes in Computer Science, H. J. van den Herik, H. Iida, and A. Plaat, Eds., vol. 6515.   Springer-Verlag, Berlin, Germany, 2011, pp. 11–22.
  28. J.-T. Saito and M. H. M. Winands, “Paranoid proof-number search,” in In Proceedings of the Computational Intelligence and Games Conference (CIG’10), G. N. Yannakakis and J. Togelius, Eds.   IEEE Press, 2010, pp. 203–210.
  29. E. Doe, M. H. M. Winands, D. J. N. J. Soemers, and C. Browne, “Combining Monte-Carlo Tree Search with Proof-Number Search,” in IEEE Conference on Games (CoG), 2022, pp. 206–212.
  30. G. M. J.-B. Chaslot, M. H. M. Winands, H. J. van den Herik, J. W. H. M. Uiterwijk, and B. Bouzy, “Progressive strategies for Monte-Carlo Tree Search,” New Mathematics and Natural Computation, vol. 4, no. 3, pp. 343–357, 2008.
  31. P. Auer, N. Cesa-Bianchi, and P. Fischer, “Finite-time analysis of the multiarmed bandit problem,” Machine Learning, vol. 47, no. 2–3, pp. 235–256, 2002.
  32. T. Cazenave and A. Saffidine, “Score Bounded Monte-Carlo Tree Search,” in Computers and Games, ser. LNCS, vol. 6515, 2011, pp. 93––104.
  33. É. Piette, D. J. N. J. Soemers, M. Stephenson, C. F. Sironi, M. H. M. Winands, and C. Browne, “Ludii – the ludemic general game system,” in Proceedings of the 24th European Conference on Artificial Intelligence (ECAI 2020), ser. Frontiers in Artificial Intelligence and Applications, G. D. Giacomo, A. Catala, B. Dilkina, M. Milano, S. Barro, A. Bugarín, and J. Lang, Eds., vol. 325.   IOS Press, 2020, pp. 411–418.
  34. M. Genesereth, N. Love, and B. Pell, “General Game Playing: Overview of the AAAI Competition,” AI Magazine, vol. 26, pp. 62–72, 2005.
  35. J. Kowalski, M. Mika, J. Sutowicz, and M. Szykuła, “Regular Boardgames,” AAAI, vol. 33, no. 1, pp. 1699–1706, 2019.
  36. N. Love, T. Hinrichs, D. Haley, E. Schkufza, and M. Genesereth, “General game playing: Game description language specification,” Stanford Logic Group, Tech. Rep. LG-2006-01, 2008.
  37. M. J. Herskovits, “Wari in the new world.” The Journal of the Royal Anthropological Institute of Great Britain and Ireland, vol. 62, p. 23–37, 1932.
  38. A. Kishimoto, M. H. M. Winands, M. Müller, and J.-T. Saito, “Game-tree search using proof numbers: The first twenty years,” ICGA Journal, vol. 35, no. 3, pp. 131–156, 2012.
  39. D. Silver, T. Hubert, J. Schrittwieser, I. Antonoglou, M. Lai, A. Guez, M. Lanctot, L. Sifre, D. Kumaran, T. Graepel, T. Lillicrap, K. Simonyan, and D. Hassabis, “A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play,” Science, vol. 362, no. 6419, pp. 1140–1144, 2018.
  40. K. Handscomb, “8×8\times8 ×8 game design competition: The winning game: Breakthrough … and two other favorites,” Abstract Games, vol. 7, pp. 8–9, 2001.
  41. C. F. Sironi, J. Liu, and M. H. M. Winands, “Self-adaptive Monte Carlo Tree Search in general game playing,” IEEE Transactions on Games, vol. 12, no. 2, pp. 132–144, 2020.
  42. A. Saffidine, N. Jouandeau, and T. Cazenave, “Solving Breakthrough with race patterns and job-level proof number search,” in Advances in Computer Games, ser. Lecture Notes in Computer Science, vol. 7168.   Springer, 2011, pp. 196–207.
  43. L. V. Allis, H. J. van den Herik, and M. P. H. Huntjens, “Go-moku solved by new search techniques,” Computational Intelligence, vol. 12, pp. 7–23, 1996.
  44. Z. Tang, D. Zhao, K. Shao, and L. Lv, “ADP with MCTS algorithm for Gomoku,” in 2016 IEEE Symposium Series on Computational Intelligence, SSCI 2016, Athens, Greece, December 6-9, 2016.   IEEE, 2016, pp. 1–7.
  45. A. Saffidine and T. Cazenave, “Multiple-outcome proof number search,” in ECAI, 2012, pp. 708–713.
  46. J. R. Slagle and P. Bursky, “Experiments With a Multipurpose, Theorem-Proving Heuristic Program,” J. ACM, vol. 15, no. 1, p. 85–99, 1968.
  47. A. Saffidine and T. Cazenave, “Developments on product propagation,” in Computers and Games, 2014, pp. 100–109.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now