A Multi-Agent Pokemon Tournament for Evaluating Strategic Reasoning of Large Language Models (2508.01623v1)

Abstract: This research presents LLM Pokemon League, a competitive tournament system that leverages LLMs as intelligent agents to simulate strategic decision-making in Pok\'emon battles. The platform is designed to analyze and compare the reasoning, adaptability, and tactical depth exhibited by different LLMs in a type-based, turn-based combat environment. By structuring the competition as a single-elimination tournament involving diverse AI trainers, the system captures detailed decision logs, including team-building rationale, action selection strategies, and switching decisions. The project enables rich exploration into comparative AI behavior, battle psychology, and meta-strategy development in constrained, rule-based game environments. Through this system, we investigate how modern LLMs understand, adapt, and optimize decisions under uncertainty, making Pok\'emon League a novel benchmark for AI research in strategic reasoning and competitive learning.