PokéChamp: an Expert-level Minimax Language Agent (2503.04094v1)

Abstract: We introduce Pok\'eChamp, a minimax agent powered by LLMs for Pok\'emon battles. Built on a general framework for two-player competitive games, Pok\'eChamp leverages the generalist capabilities of LLMs to enhance minimax tree search. Specifically, LLMs replace three key modules: (1) player action sampling, (2) opponent modeling, and (3) value function estimation, enabling the agent to effectively utilize gameplay history and human knowledge to reduce the search space and address partial observability. Notably, our framework requires no additional LLM training. We evaluate Pok\'eChamp in the popular Gen 9 OU format. When powered by GPT-4o, it achieves a win rate of 76% against the best existing LLM-based bot and 84% against the strongest rule-based bot, demonstrating its superior performance. Even with an open-source 8-billion-parameter Llama 3.1 model, Pok\'eChamp consistently outperforms the previous best LLM-based bot, Pok\'ellmon powered by GPT-4o, with a 64% win rate. Pok\'eChamp attains a projected Elo of 1300-1500 on the Pok\'emon Showdown online ladder, placing it among the top 30%-10% of human players. In addition, this work compiles the largest real-player Pok\'emon battle dataset, featuring over 3 million games, including more than 500k high-Elo matches. Based on this dataset, we establish a series of battle benchmarks and puzzles to evaluate specific battling skills. We further provide key updates to the local game engine. We hope this work fosters further research that leverage Pok\'emon battle as benchmark to integrate LLM technologies with game-theoretic algorithms addressing general multiagent problems. Videos, code, and dataset available at https://sites.google.com/view/pokechamp-LLM.

• The paper introduces PokéChamp, an expert minimax language agent that integrates LLMs to enhance traditional search methods for complex Pokémon battles without task-specific training.
• PokéChamp leverages LLMs to sample plausible actions and predict opponent strategies, effectively managing the large decision space and hidden information inherent in Pokémon battles.
• Experimental evaluation shows PokéChamp significantly outperforms existing bots (76% win rate vs LLM bots) and attains an Elo rating placing it among the top 30%-10% of human competitors on Pokémon Showdown.

The paper introduces PokéChamp, an expert-level minimax language agent designed to play Pokémon battles using the capabilities of LLM. PokéChamp enhances traditional minimax tree search methods with LLMs to address three core components of decision-making: player action sampling, opponent modeling, and value function estimation. The integration of LLMs allows the agent to effectively handle the expansive state space and partial observability inherent in Pokémon battles without requiring additional task-specific LLM training.

Key Contributions:

1. Framework Utilization: PokéChamp leverages LLMs to sample plausible actions based on current game states and to predict opponent strategies, thus limiting the extensive search space typically involved in minimax approaches. By embedding LLM power, it circumvents the need for exhaustive search trials while intelligently managing the large variety of decisions and hidden information characteristic of Pokémon battles.
2. World Model Development: The agent integrates statistical data from Pokémon battles to simulate game transitions and predict potential scenarios. This involves creating prompts that estimate the likelihood of move effectiveness, anticipating the opponent's strategic options, and calculating the expected knockout outcomes, thereby refining the decision-making process.
3. Evaluation: Experimental evaluations demonstrate PokéChamp's efficacy in various competitive formats, including the Generation 9 OverUsed (OU) format, where it significantly outperforms existing bots (LLM-based and heuristic). With GPT-4o, PokéChamp achieves a 76% win rate against leading LLM-based systems and an 84% win rate against the strongest heuristic bot, highlighting its tactical superiority. In real human competition on the Pokémon Showdown ladder, PokéChamp attains an Elo rating of 1300-1500, placing it among the top 30%-10% of competitors.

Technical Insights:

• Minimax Tree Search Augmentation: By integrating LLMs into minimax tree search, PokéChamp refines traditional algorithms, replacing deep inspections with probabilistic predictions and inferred game value estimations.
• Opponent Strategy Prediction: Through rigorous LLM prompting, PokéChamp advances in opponent strategy prediction even with limited visibility, enriching gameplay altering decisions dynamically.
• Effective Against Humans: In user-conducted battles, the agent's real-time decision-making leads to notable win rates, demonstrating its capacity to anticipate human strategies effectively.

Implications and Future Directions:

The success of PokéChamp hints at broader applications for LLM integration into game-theoretic algorithms across various multi-agent competitive environments without incurring computationally expensive reinforcement learning or agent-specific training. While the approach provides robust performance, future research can further improve its adaptability against sophisticated opponent switches and long-term strategies frequently applied by human players. Additionally, enhancements in opponent modeling and increased search tree breadth could further elevate its strategic dexterity. Overall, PokéChamp's development sets a precedent for the syntheses of LLM and game-theoretic search methods, illuminating paths for advanced AI applications in complex, competitive settings devoid of perfect information.