2000 character limit reached
Empirical Analysis of Fictitious Play for Nash Equilibrium Computation in Multiplayer Games (2001.11165v10)
Published 30 Jan 2020 in cs.GT, cs.AI, cs.MA, and econ.TH
Abstract: While fictitious play is guaranteed to converge to Nash equilibrium in certain game classes, such as two-player zero-sum games, it is not guaranteed to converge in non-zero-sum and multiplayer games. We show that fictitious play in fact leads to improved Nash equilibrium approximation over a variety of game classes and sizes than (counterfactual) regret minimization, which has recently produced superhuman play for multiplayer poker. We also show that when fictitious play is run several times using random initializations it is able to solve several known challenge problems in which the standard version is known to not converge, including Shapley's classic counterexample. These provide some of the first positive results for fictitious play in these settings, despite the fact that worst-case theoretical results are negative.
- Nick Abou Risk and Duane Szafron. Using counterfactual regret minimization to create competitive multiplayer poker agents. In Proceedings of the International Conference on Autonomous Agents and Multi-Agent Systems (AAMAS), pages 159–166, 2010.
- Exclusion method for finding Nash equilibrium in multiplayer games. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI), pages 383–389, 2017.
- Ulrich Berger. Fictitious play in 2xn games. Journal of Economic Theory, 120:139–154, 2005.
- Heads-up limit hold’em poker is solved. Science, 347(6218):145–149, January 2015.
- George W. Brown. Iterative solutions of games by fictitious play. In Tjalling C. Koopmans, editor, Activity Analysis of Production and Allocation, pages 374–376. John Wiley & Sons, 1951.
- Deep counterfactual regret minimization. In Proceedings of the International Conference on Machine Learning (ICML), 2019.
- Superhuman AI for heads-up no-limit poker: Libratus beats top professionals. Science, 359:418–424, 2017.
- Superhuman AI for multiplayer poker. Science, 365:885–890, 2019.
- Depth-limited solving for imperfect-information games. In Proceedings of the Annual Conference on Neural Information Processing Systems (NIPS), 2018.
- Xi Chen and Xiaotie Deng. 3-Nash is PPAD-complete. Electronic Colloquium on Computational Complexity, Report No. 134:1–12, 2005.
- The complexity of computing a Nash equilibrium. SIAM Journal on Computing, 1(39):195–259, 2009.
- On the nonconvergence of fictitious play in coordination games. Games and Economic Behavior, 25:79–94, 1998.
- The Theory of Learning in Games. MIT Press, 1998.
- Sam Ganzfried. Mistakes in games. In Proceedings of the International Conference on Distributed Artificial Intelligence (DAI), 2019.
- Sam Ganzfried. Fictitious play with maximin initialization. In IEEE Conference on Decision and Control (CDC), 2022.
- Parallel algorithm for Nash equilibrium in multiplayer stochastic games with application to naval strategic planning. In Proceedings of the International Conference on Distributed Artificial Intelligence (DAI), 2020.
- Successful Nash equilibrium agent for a 3-player imperfect-information game. Games, 9(33), 2018.
- Computing an approximate jam/fold equilibrium for 3-player no-limit Texas hold ’em tournaments. In Proceedings of the International Conference on Autonomous Agents and Multi-Agent Systems (AAMAS), 2008.
- Computing equilibria in multiplayer stochastic games of imperfect information. In Proceedings of the 21st International Joint Conference on Artificial Intelligence (IJCAI), 2009.
- Endgame solving in large imperfect-information games. In Proceedings of the International Conference on Autonomous Agents and Multi-Agent Systems (AAMAS), 2015.
- Richard Gibson. Regret Minimization in Games and the Development of Champion Multiplayer Computer Poker-Playing Agents. PhD thesis, University of Alberta, 2014.
- First-order algorithm with 𝒪(ln(1/ϵ))𝒪ln1italic-ϵ\mathcal{O}(\mathrm{ln}(1/\epsilon))caligraphic_O ( roman_ln ( 1 / italic_ϵ ) ) convergence for ϵitalic-ϵ\epsilonitalic_ϵ-equilibrium in two-person zero-sum games. Mathematical Programming, 133(1–2):279–298, 2012.
- Fictitious self-play in extensive-form games. In Proceedings of the International Conference on Machine Learning (ICML), 2015.
- Deep Reinforcement Learning from Self-Play in Imperfect-Information Games. arXiv e-prints, page arXiv:1603.01121, Mar 2016.
- Midgame solving: A new weapon for efficient large-scale equilibrium approximation. In IEEE International Conference on Tools with Artificial Intelligence, 2017.
- Monte Carlo sampling for regret minimization in extensive games. In Proceedings of the Annual Conference on Neural Information Processing Systems (NIPS), 2009.
- Deepstack: Expert-level artificial intelligence in heads-up no-limit poker. Science, 356:508–513, 2017.
- John Nash. Non-cooperative games. PhD thesis, Princeton University, 1950.
- Run the GAMUT: A comprehensive approach to evaluating game-theoretic algorithms. In Proceedings of the International Conference on Autonomous Agents and Multi-Agent Systems (AAMAS), 2004.
- Generalised fictitious play for a continuum of anonymous players. In Proceedings of the 21st International Joint Conference on Artificial Intelligence (IJCAI), 2009.
- Computing pure Bayesian-Nash equilibria in games with finite actions and continuous types. Artificial Intelligence, 195:106–139, February 2013.
- Julia Robinson. An iterative method of solving a game. Annals of Mathematics, 54:296–301, 1951.
- Lloyd S Shapley. Some topics in two-person games. In M. Drescher, L. S. Shapley, and A. W. Tucker, editors, Advances in Game Theory. Princeton University Press, 1964.
- Wikipedia contributors. Fictitious play — Wikipedia, the free encyclopedia, 2022. [Online; accessed 1-September-2022].
- Regret minimization in games with incomplete information. In Proceedings of the Annual Conference on Neural Information Processing Systems (NIPS), 2007.
- Sam Ganzfried (26 papers)