Poker Arena: LLM Strategic Evaluation

• Poker Arena is a no-limit Texas Hold'em tournament platform designed to evaluate LLM strategic reasoning by decomposing gameplay into interpretable dimensions.
• It employs a three-layer memory architecture and a nine-axis cognitive profile to capture nuanced behaviors beyond standard chip counts.
• The benchmark reveals that persistent memory can either aid or hinder performance, highlighting the importance of cross-dimensional consistency in decision-making.

Poker Arena is a no-limit Texas Hold’em tournament platform for evaluating strategic reasoning under uncertainty in LLMs. It couples a three-layer memory architecture—within-hand, session, and cross-session—with a nine-axis cognitive profile that decomposes strategic reasoning into interpretable dimensions such as bet-sizing calibration and positional awareness. In the reported evaluation, seven frontier models were assessed across 50 sessions of 1,000 hands, and the central result is that tournament chips and aggregate axis score order the field differently: Claude Opus 4.6 wins +$15,730 chips with 14 first-place finishes, yet ranks only fifth of seven on mean axis score, while persistent memory helps some models and hurts others (Singla et al., 11 Jun 2026).

1. Research objective and evaluative premise

Poker Arena was introduced to address a benchmarking limitation: prevailing game-play benchmarks collapse heterogeneous reasoning dimensions into a single scalar, leaving the capability structure of frontier LLMs unexamined. Its design therefore treats poker not merely as a win-rate task, but as an adversarial, payoff-driven environment in which strategic reasoning can be decomposed into multiple interpretable dimensions. The benchmark’s explicit target is no-limit Texas Hold’em, chosen as a domain in which uncertainty, sequential action, concealed information, positional asymmetry, and adaptive response all interact (Singla et al., 11 Jun 2026).

The evaluative premise is that scalar leaderboards conflate heterogeneous skills and misrank agents. In Poker Arena, chip accumulation remains an outcome variable, but it is paired with a nine-axis profile in which each axis returns $M_k \in [0,1]$and the overall mean is</p> <p>$\bar M = \tfrac{1}{9}\sum_{k=1}^9 M_k.$$</p> <p>This separation between economic outcome and decomposed capability is fundamental to the platform. It permits cases in which one model is the strongest tournament finisher while another exhibits a stronger mean cognitive profile, and it supports the further claim that cross-dimensional consistency outweighs peak performance on any single axis (<a href="/papers/2606.13815" title="" rel="nofollow" data-turbo="false" class="assistant-link" x-data x-tooltip.raw="">Singla et al., 11 Jun 2026</a>).</p> <h2 class='paper-heading' id='tournament-environment-and-game-mechanics'>2. Tournament environment and game mechanics</h2> <p>The environment uses seven agents per table, each with a$$1 000 starting stack. Each session runs 20 hands, and the reported study conducts 50 sessions, yielding 1 000 hands total. Blinds escalate every 5 hands through four tiers:$$5/10 \rightarrow 10/20 \rightarrow 25/50 \rightarrow 50/100$$. This induces nonstationarity in stack pressure and changes the relative value of passivity, aggression, and positional exploitation over the course of a session (Singla et al., 11 Jun 2026).

Game resolution is implemented with deterministic hand-ranking through a five-of-seven solver and Monte Carlo equity estimation using 1 000 run-outs. Side-pot handling is implemented via per-player contribution tracking to ensure exact payoff allocation under uneven all-ins. These mechanics matter because the benchmark is not limited to coarse outcome labels; it also computes axis-level metrics that depend on action histories, equity thresholds, positional statistics, and post hoc factual consistency. A plausible implication is that the environment is intended to minimize ambiguity in outcome accounting so that variance in scores can be attributed to agent behavior rather than to evaluator imprecision (Singla et al., 11 Jun 2026).

3. Three-layer memory architecture

The benchmark formalizes memory at three levels. Let agent $i$ in session $k$ at hand $h$ and street $s$ have hole cards $\mathcal{H}_i^{(h)}$, community cards $\mathcal{C}^{(h,s)}$, and action $a_i^{(h,s)}$. Layer 1 is within-hand context: $\bar M = \tfrac{1}{9}\sum_{k=1}^9 M_k.$0 The function $\bar M = \tfrac{1}{9}\sum_{k=1}^9 M_k.$1 anonymizes opponents to stable aliases, injects private reasoning, and is discarded after each decision. This layer therefore supports decision-local inference without creating persistent hidden state beyond the current action point (Singla et al., 11 Jun 2026).

Layer 2 is session-level memory. Each agent maintains a text buffer $\bar M = \tfrac{1}{9}\sum_{k=1}^9 M_k.$2 with maximum 16 K characters, written by the agent after each hand from an anonymized hand summary $\bar M = \tfrac{1}{9}\sum_{k=1}^9 M_k.$3. The update rule is

$\bar M = \tfrac{1}{9}\sum_{k=1}^9 M_k.$4

This design makes memory persistence conditional rather than automatic: the model must emit “UPDATE:yes” for the buffer to change. The session buffer is thus both a storage mechanism and an endogenous memory policy (Singla et al., 11 Jun 2026).

Layer 3 is cross-session memory. A lifetime store $\bar M = \tfrac{1}{9}\sum_{k=1}^9 M_k.$5 is consolidated from final $\bar M = \tfrac{1}{9}\sum_{k=1}^9 M_k.$6 via $\bar M = \tfrac{1}{9}\sum_{k=1}^9 M_k.$7 at session end. At the next session start, the session memory is seeded as $\bar M = \tfrac{1}{9}\sum_{k=1}^9 M_k.$8 under three policies: normal, which propagates prior session memory; past (Ablation A), which loads an offline file; and fresh (Ablation B), which uses empty memory. The cross-session layer is therefore the locus of the memory ablation experiments and of the paper’s claim that persistent memory is a conditional asset rather than a uniform benefit (Singla et al., 11 Jun 2026).

4. Nine-axis cognitive profile

The nine-axis profile decomposes strategic reasoning into interpretable dimensions. The axes combine deterministic statistics, judge-based hybrid assessments, and regex-based factual extraction. Their purpose is not to replace tournament outcome, but to expose which strategic faculties are responsible for performance and where scalar chip totals conceal asymmetries.

Axis	Modality	Basis
$\bar M = \tfrac{1}{9}\sum_{k=1}^9 M_k.$9 Bet-Sizing Calibration	Deterministic	7-bucket histogram of bet-ratios vs GTO reference; preflop/postflop weighting
$1 000 starting stack. Each session runs 20 hands, and the reported study conducts 50 sessions, yielding 1 000 hands total. Blinds escalate every 5 hands through four tiers:$0 Bluffing / Deception	Hybrid: 1-judge	Low-equity postflop aggression, judge-labeled intent, board texture, bluff frequency, low-equity calls
$1 000 starting stack. Each session runs 20 hands, and the reported study conducts 50 sessions, yielding 1 000 hands total. Blinds escalate every 5 hands through four tiers:$1 Opponent Reading	Hybrid: 3-judge	JSD across opponents’ action distributions and reasoning quality
$1 000 starting stack. Each session runs 20 hands, and the reported study conducts 50 sessions, yielding 1 000 hands total. Blinds escalate every 5 hands through four tiers:$2 Composure	Deterministic	VPIP and AF deviation pre-/post-“bad beat”
$1 000 starting stack. Each session runs 20 hands, and the reported study conducts 50 sessions, yielding 1 000 hands total. Blinds escalate every 5 hands through four tiers:$3 Adaptability	Deterministic	EV gap trend slope, window-drift JSD weighted by profit improvement, between-session JSD
$1 000 starting stack. Each session runs 20 hands, and the reported study conducts 50 sessions, yielding 1 000 hands total. Blinds escalate every 5 hands through four tiers:$4 Prediction Accuracy	Regex	HandID, equity MAE relative to Monte Carlo equity, range-vocab ratio
$1 000 starting stack. Each session runs 20 hands, and the reported study conducts 50 sessions, yielding 1 000 hands total. Blinds escalate every 5 hands through four tiers:$5 Strategic Mixing	Deterministic	Action diversity within $1 000 starting stack. Each session runs 20 hands, and the reported study conducts 50 sessions, yielding 1 000 hands total. Blinds escalate every 5 hands through four tiers:$6 buckets with at least 10 observations
$1 000 starting stack. Each session runs 20 hands, and the reported study conducts 50 sessions, yielding 1 000 hands total. Blinds escalate every 5 hands through four tiers:$7 Factual Accuracy	Regex	Hand-type correctness, pot-odds within 5%, hallucinated cards, draw identification
$1 000 starting stack. Each session runs 20 hands, and the reported study conducts 50 sessions, yielding 1 000 hands total. Blinds escalate every 5 hands through four tiers:$8 Positional Awareness	Deterministic	Spearman relation between position rank and VPIP, plus deviation from $1 000 starting stack. Each session runs 20 hands, and the reported study conducts 50 sessions, yielding 1 000 hands total. Blinds escalate every 5 hands through four tiers:$9

Several axes are defined by explicit formulas. For $i$0, each bet type $i$1 is scored by

$i$2

with preflop and postflop components combined by empirical weighting. For $i$3, adaptation and reasoning quality are combined as $i$4. For $i$5, composure is measured from normalized changes in VPIP and aggression factor around “bad beat” events. For $i$6, the score is

$i$7

where $i$8 and $i$9 (Singla et al., 11 Jun 2026).

A common misconception is that such axes merely restate chip outcomes in finer detail. The reported construction argues otherwise: the axes include factual accuracy, prediction accuracy, composure, and strategic mixing, which are not reducible to end-of-session bankroll changes. This suggests that the profile is intended as a structural probe of agent behavior rather than a relabeling of tournament success (Singla et al., 11 Jun 2026).

5. Experimental protocol

The evaluation seats seven frontier LLMs together across 50 sessions $k$0 20 hands, yielding 1 000 hands and 9 115 logged actions. Chips are aggregated as cumulative $k$1 per session and across sessions. Axis scores are computed per agent over all hands, and both per-axis $k$2 and mean $k$3 are reported. The tournament design therefore combines repeated interaction, escalating blinds, and long-horizon aggregation rather than isolated hands or synthetic static prompts (Singla et al., 11 Jun 2026).

A controlled memory ablation was also conducted. Three models—Claude, GPT, and Kimi—ran 10 paired sessions of 10 hands under two Layer 3 conditions, no memory versus seeded memory, with identical random seeds, for 600 hands total. Because random seeds are matched across conditions, the ablation isolates the effect of cross-session memory policy more directly than an unmatched comparison would. The platform’s methodological emphasis is thus dual: multi-axis profiling of strategic behavior and controlled variation of memory interfaces (Singla et al., 11 Jun 2026).

6. Leaderboards, rank divergence, and memory effects

The chip leaderboard over 50 sessions is: Claude Opus 4.6 at +$k$43 705, GPT-5.4 at −$k$5937, Qwen3-Max at −$k$62 095, and Kimi K2 at −$k$7\sigma_\Delta = \$k$8. By contrast, the mean-axis leaderboard is ordered differently: Grok at 0.6137, DeepSeek at $k$9, GPT at $h$0, Qwen at $h$1, Claude at 0.5754, Gemini at $h$2, and Kimi at $h$3 (Singla et al., 11 Jun 2026).

Model	Chip leaderboard	Mean-axis leaderboard
Claude Opus 4.6	+$15 730	0.5754
Grok 4	+$3 705	0.6137
GPT-5.4	−$h$4\sim 0.605$
DeepSeek V3.1	−$h$5\sim 0.610$$</td> <td style="text-align: right"></td> </tr> <tr> <td>Qwen3-Max</td> <td style="text-align: right">−$$h$6\sim 0.580$
Gemini 3.1 Pro	−$h$7\sim 0.550$$</td> <td style="text-align: right"></td> </tr> <tr> <td>Kimi K2</td> <td style="text-align: right">−$$h$8\sim 0.490$

Per-axis leadership is also distributed: DeepSeek leads $h$9 bet sizing at 0.79; Grok leads $s$0 bluffing and $s$1 opponent reading at 0.83 and 0.46; GPT leads $s$2 composure and $s$3 position at 0.89 and 0.83; Gemini leads $s$4 factual accuracy at 0.74; and no model wins more than two axes. Rank correlation between chip rank and mean $s$5 is Spearman $s$6, $s$7, with $s$8. The interpretation given is positive but loose alignment: scalar chip rank does not equal profile rank (Singla et al., 11 Jun 2026).

The memory ablation sharpens this point. GPT shows +114.6 chips per session gain with prior memory ($s$9, $\mathcal{H}_i^{(h)}$0), Kimi shows −109.4 chips per session with memory ($\mathcal{H}_i^{(h)}$1, $\mathcal{H}_i^{(h)}$2), and Claude shows −42.5 chips per session ($\mathcal{H}_i^{(h)}$3, $\mathcal{H}_i^{(h)}$4). The directions are consistent but $\mathcal{H}_i^{(h)}$5; effects are model-specific. The benchmark’s conclusion is therefore twofold: scalar leaderboards misrank agents, and persistent memory helps some models and hurts others. It further states that consistency across axes, reflected in low $\mathcal{H}_i^{(h)}$6, matters more than peak on any single axis for stable long-run chip accumulation (Singla et al., 11 Jun 2026).

7. Relation to decentralized and instruction-driven poker systems

Related work places Poker Arena within a broader technical landscape of poker platforms. “Instantaneous Decentralized Poker” formalizes a stateful contract model $\mathcal{H}_i^{(h)}$7 with deposits, balance, nonce, and status in $\mathcal{H}_i^{(h)}$8; it separates on-chain calls from off-chain updates via signed state proofs; and it describes setup, off-chain MPC, and settlement or penalty phases for an unbounded sequence of poker hands. The same work provides an Ethereum/Solidity reference implementation with functions such as openChannel, updateState, dispute, resolve, and withdraw, and describes amortized per-hand cost under dispute-free play and simulation-based security goals including fairness, no-counterparty risk, and off-chain liveness (Bentov et al., 2017).

A different line of work, “Instruction-Driven Game Engine: A Poker Case Study,” treats poker-engine learning as next-state prediction. Its architecture includes a Game-Script Parser, State Manager, Core-Function Library, LLM Next-State Predictor, and Code Interpreter & Merger. It distinguishes Full Next-State Prediction from Differential State Prediction, represents state as a Python dict with keys including deck, players, community, pot, bets, and game_flow, and emphasizes precision by off-loading heavy numerical logic to core functions and applying validators such as deck-size consistency, pot accounting, duplicate-card checks, and nonnegative chip totals. It also reports an “Easy → Medium → Hard” curriculum with warmup on Core Functions, Standard NSP/DSP, and Diverse via Segment Rephrasing (Wu et al., 2024).

These adjacent systems do not redefine the benchmark reported in Poker Arena, but they illuminate complementary interpretations of poker as a research substrate. This suggests a broader design space in which a “Poker Arena” can function as a multi-axis evaluation platform for frontier LLMs, as an off-chain secure multiparty protocol with stateful contracts, or as an instruction-driven engine that generates gameplay from natural-language rules. Across these settings, poker serves as a compact but technically demanding testbed for sequential decision-making, memory, verification, and adversarial interaction (Bentov et al., 2017, Wu et al., 2024).