Poker Arena: Multi-Axis Profiling of Strategic Reasoning and Memory in LLMs

Abstract: Strategic reasoning under uncertainty underpins consequential decisions in negotiation, finance, and policy, but prevailing game-play benchmarks collapse heterogeneous reasoning dimensions into a single scalar, leaving the capability structure of frontier LLMs unexamined. We introduce Poker Arena, a no-limit Texas Hold'em tournament platform that couples a three-layer memory architecture (within-hand, session, and cross-session) with a nine-axis cognitive profile decomposing strategic reasoning into interpretable dimensions such as bet-sizing calibration and positional awareness. We evaluate seven frontier models across 50 sessions of 1,000 hands and a controlled memory ablation; tournament chips and aggregate axis score order the field differently: Claude Opus 4.6 wins +$15,730 chips with 14 first-place finishes, yet ranks only fifth of seven on mean axis score, while persistent memory helps some models and hurts others. These findings show that multi-axis evaluation surfaces capability structure that scalar leaderboards systematically misrank, with cross-dimensional consistency outweighing peak performance on any single axis.

• The paper introduces a three-layer memory system and a nine-axis profiling framework to dissect strategic reasoning in LLMs during poker tournaments.
• The methodology benchmarks 7 leading LLMs over 50 tournaments, revealing detailed trade-offs between financial performance and cognitive metrics through controlled ablation studies.
• Results highlight that multi-axis evaluation uncovers nuanced deficiencies and adaptive strategies, emphasizing that memory integration benefits vary across models.

Poker Arena: Multi-Axis Profiling of Strategic Reasoning and Memory in LLMs

Introduction and Motivation

Evaluating strategic reasoning under uncertainty is critical for understanding the operational capabilities of frontier LLMs, especially as they encounter scenarios requiring planning, adaptation, and opponent modeling. No-limit Texas Hold'em poker is a canonical test domain given its requirements for bluffing, bet calibration, and adaptation to partial observability and shifting opponent profiles. Existing benchmarks for LLMs in games typically collapse heterogeneous reasoning skills into a unidimensional metric, thereby obscuring the capability structure underlying observed performance. Furthermore, prior work pays little attention to persistent memory and its potential role in agent adaptation in adversarial multi-agent contexts.

Poker Arena Framework

Poker Arena introduces a comprehensive benchmarking environment for LLM agents in no-limit Texas Hold'em tournaments, focusing on decomposing strategy and measuring temporal adaptation. The core architectural contribution is a three-layer memory system. Layer 1 handles within-hand ephemeral context; Layer 2, a session-level memory, is agent-writable and selectively updatable; and Layer 3 provides persistent cross-session memory, enabling explicit ablation of information retention and transfer across tournaments.

Seven leading LLMs (Claude Opus 4.6, Grok 4, GPT-5.4, DeepSeek V3.1, Qwen3-max, Gemini 3.1 Pro, Kimi K2 (thinking)) are benchmarked over 50 tournaments, each spanning 20 hands, under controlled bankroll and escalating blind structures.

Figure 1: Poker Arena architecture: a three-layer memory hierarchy (Layer 1: within-hand context, Layer 2: session notebook, Layer 3: cross-session lifetime store) interacting with seven LLMs on top of a deterministic Texas Hold’em engine, all scored via a nine-axis cognitive profile.

The architecture ensures thorough isolation of short-term, medium-term, and long-term strategic considerations, and renders precise ablation experiments feasible.

Cognitive Profiling: Moving Beyond Scalar Leaderboards

Poker Arena operationalizes model evaluation along a nine-axis cognitive profile:

1. Bet Sizing Calibration: $L_1$ histogram distance to GTO bet sizing.
2. Bluffing / Deception: Synthesis of forced low-equity aggression with LLM-judged intent/texture/success.
3. Opponent Reading: Behavioral differentiation and explicit textual evidence of theory-of-mind.
4. Composure: VPIP and AF stability post-bad beat.
5. Adaptability: Profitable style drift intra- and cross-session.
6. Prediction Accuracy: Correctness of hand-type/equity prediction and vocabulary use.
7. Strategic Mixing: Action entropy in positional and street buckets.
8. Factual Accuracy: Verification/hallucination in action descriptions and odds calculations.
9. Positional Awareness: VPIP gradient and deviation from GTO by position.

Each axis is normalized to $[0,1]$ and metrics are sourced deterministically, via regex, or using LLM judge panels as appropriate. Crucially, this multidimensional diagnostic exposes fragmented leadership patterns and enables robust hypothesis tests about cross-axis performance invariance.

Figure 2: (a) Playing styles (VPIP vs Aggression Factor) show diverse strategic regimes; (b) Nine-axis cognitive profiles for all seven LLMs, each axis scaled to $[0,1]$.

Experimental Results

Financial Performance and Strategic Diversity

Claude Opus 4.6 achieves the highest cumulative chip gain, outpacing all competitors with $+15,730$ chips over the tournament cascade, whereas the rest of the field clusters within a $\$2,000$$band of the starting buy-in. Kimi K2 (thinking) is consistently outperformed and registers the only unambiguously negative cumulative outcome. <img src="https://emergentmind-storage-cdn-c7atfsgud9cecchk.z01.azurefd.net/paper-images/2606-13815/fig1_chip_growth.png" alt="Figure 3" title="" class="markdown-image" loading="lazy"> <p class="figure-caption">Figure 3: Cumulative chip totals across 1,000 hands: Claude establishes a substantial margin over the runner-up Grok; the median cluster of models remain close to breakeven, and Kimi exhibits marked underperformance.</p></p> <p>Despite this, the leader on aggregate axis score is Grok, not Claude. Per-axis victories are dispersed: DeepSeek leads bet-sizing, Grok dominates bluffing and opponent modeling, <a href="https://www.emergentmind.com/topics/generative-pretrained-transformer-gpt" title="" rel="nofollow" data-turbo="false" class="assistant-link" x-data x-tooltip.raw="">GPT</a> prevails on composure and positional acuity, and Gemini excels in factual accuracy. No model claims more than two axes, underscoring a lack of dominance and supporting the argument that scalar leaderboards systematically obscure the nuanced distribution of competence.</p> <p>The Spearman correlation between mean-axis and chip-rank orders is only$$\rho_S = 0.571$($p=0.18$for$n=7$$), indicating only moderate and statistically non-significant alignment between generalized cognition and realized financial returns for the given sample size.</p> <h3 class='paper-heading' id='strategic-style-differentiation'>Strategic Style Differentiation</h3> <p>Agents display substantial heterogeneity in VPIP (voluntary pot entry percentage) and aggression factor (bet/raise to call ratio), with GPT manifesting the loosest, most active profile, and Grok exhibiting distinct hyper-aggression. Claude’s combination of above-median activity and aggression aligns with its overall chip lead—cross-axis balance, not outlier behavior, accounts for durable profit.</p> <h3 class='paper-heading' id='memory-and-adaptation-controlled-ablation'>Memory and Adaptation: Controlled Ablation</h3> <p><a href="https://www.emergentmind.com/topics/ablation-studies" title="" rel="nofollow" data-turbo="false" class="assistant-link" x-data x-tooltip.raw="">Ablation studies</a> show persistent memory exerts model-specific, non-uniform effects on agent performance. GPT’s chip gain increases with access to cross-session memory ($$+114.6$per session), Kimi is hindered ($-109.4$$per session), and Claude remains approximately unchanged ($$[0,1]$0, all $[0,1]$1 at $[0,1]$2, but directions are clear). Theoretical implications are significant: memory is not an unadulterated asset—model-specific adherence to priors versus online evidence integration dictates financial consequences. This observation harmonizes with classical findings in Bayesian opponent modeling for poker.

Figure 4: Mean chip delta per session under memory ablation; bars contrast no memory (A) and persistent memory (B), revealing model-dependent swings in profitability.

Implications and Future Directions

These findings have several critical theoretical and methodological implications:

• Decomposed Evaluation is Essential: Scalar metrics alone are systematically insufficient for ranking agent competence in complex, adversarial domains. Multi-axis decomposition surfaces non-obvious deficits and strengths, particularly important for safety and alignment research where hidden weaknesses may underlie apparently strong aggregate performance.
• Memory Integration Must Be Benchmarked Per Model: The sign and magnitude of persistent memory effects are model-dependent. Future AI evaluation must systematically examine (model, memory-interface) cross-products instead of fixing memory configuration globally.
• Consistent Cross-Axis Competence Outweighs Peak Specialization: Over a tournament horizon, agents with no dominant holes in their cognitive profiles perform more robustly, matching HEURISTIC predictions about leak minimization in adversarial meta-games.
• Evaluation Technologies: There is a growing need for interpretability-driven, adversarial multi-axis testbeds. Poker Arena’s architecture supports explicit judge bias mitigation, adversarial prompt anonymization, and role separation, contributing to methodological best practices in the evaluation of AI social reasoning.

The authors suggest that future work could incorporate larger panels, human baselines, and additional axes such as risk sensitivity or counterfactual reasoning. Such developments are likely to further illuminate the adaptation, robustness, and social reasoning faculties in advanced LLMs.

Conclusion

Poker Arena demonstrates the inadequacy of unidimensional leaderboards for evaluating strategic reasoning in LLMs, introducing a granular, multi-axis analysis tied to a controlled, memory-aware environment. The observed disjunction between financial and cognitive rankings, as well as the model-specific dependencies on memory, delineate a path forward for the assessment of AI strategic competence in imperfect-information, adversarial domains. This work lays a methodological foundation for the adoption of multi-axis adversarial benchmarking in strategic AI development and evaluation, providing incentives for robustness, consistency, and interpretability in agent design and analysis (2606.13815).