---
title: 'Poker Arena: LLM Strategic Evaluation'
url: https://www.emergentmind.com/topics/poker-arena
type: topic
---

# Poker Arena: LLM Strategic Evaluation

Poker Arena is a no-limit Texas Hold’em tournament platform for evaluating strategic reasoning under uncertainty in large language models. It couples a three-layer memory architecture—within-hand, session, and cross-session—with a nine-axis cognitive profile that decomposes strategic reasoning into interpretable dimensions such as bet-sizing calibration and positional awareness. In the reported evaluation, seven frontier models were assessed across 50 sessions of 1,000 hands, and the central result is that tournament chips and aggregate axis score order the field differently: Claude Opus 4.6 wins +$15,730 chips with 14 first-place finishes, yet ranks only fifth of seven on mean axis score, while persistent memory helps some models and hurts others [2606.13815].

## 1. Research objective and evaluative premise

Poker Arena was introduced to address a benchmarking limitation: prevailing game-play benchmarks collapse heterogeneous reasoning dimensions into a single scalar, leaving the capability structure of frontier LLMs unexamined. Its design therefore treats poker not merely as a win-rate task, but as an adversarial, payoff-driven environment in which strategic reasoning can be decomposed into multiple interpretable dimensions. The benchmark’s explicit target is no-limit Texas Hold’em, chosen as a domain in which uncertainty, sequential action, concealed information, positional asymmetry, and adaptive response all interact [2606.13815].

The evaluative premise is that scalar leaderboards conflate heterogeneous skills and misrank agents. In Poker Arena, chip accumulation remains an outcome variable, but it is paired with a nine-axis profile in which each axis returns $M_k \in [0,1]$ and the overall mean is
\[
\bar M = \tfrac{1}{9}\sum_{k=1}^9 M_k.
\]
This separation between economic outcome and decomposed capability is fundamental to the platform. It permits cases in which one model is the strongest tournament finisher while another exhibits a stronger mean cognitive profile, and it supports the further claim that cross-dimensional consistency outweighs peak performance on any single axis [2606.13815].

## 2. Tournament environment and game mechanics

The environment uses seven agents per table, each with a $1 000 starting stack. Each session runs 20 hands, and the reported study conducts 50 sessions, yielding 1 000 hands total. Blinds escalate every 5 hands through four tiers: $5/10 \rightarrow 10/20 \rightarrow 25/50 \rightarrow 50/100$. This induces nonstationarity in stack pressure and changes the relative value of passivity, aggression, and positional exploitation over the course of a session [2606.13815].

Game resolution is implemented with deterministic hand-ranking through a five-of-seven solver and Monte Carlo equity estimation using 1 000 run-outs. Side-pot handling is implemented via per-player contribution tracking to ensure exact payoff allocation under uneven all-ins. These mechanics matter because the benchmark is not limited to coarse outcome labels; it also computes axis-level metrics that depend on action histories, equity thresholds, positional statistics, and post hoc factual consistency. A plausible implication is that the environment is intended to minimize ambiguity in outcome accounting so that variance in scores can be attributed to agent behavior rather than to evaluator imprecision [2606.13815].

## 3. Three-layer memory architecture

The benchmark formalizes memory at three levels. Let agent $i$ in session $k$ at hand $h$ and street $s$ have hole cards $\mathcal{H}_i^{(h)}$, community cards $\mathcal{C}^{(h,s)}$, and action $a_i^{(h,s)}$. Layer 1 is within-hand context:
\[
C_i^{(h,s)}=\phi(\mathcal{H}_i^{(h)}, \mathcal{C}^{(h,s)}, history^{(h,s)}, \mathcal{M}_i^{(k)}).
\]
The function $\phi$ anonymizes opponents to stable aliases, injects private reasoning, and is discarded after each decision. This layer therefore supports decision-local inference without creating persistent hidden state beyond the current action point [2606.13815].

Layer 2 is session-level memory. Each agent maintains a text buffer $\mathcal{M}_i^{(k)}$ with maximum 16 K characters, written by the agent after each hand from an anonymized hand summary $\sigma(h_t)$. The update rule is
\[
\mathcal{M}_i^{(k)} \leftarrow
\begin{cases}
\mathrm{LLM}_i\bigl(\mathcal{M}_i^{(k)},\,\sigma(h_t)\bigr) & \mathrm{if\ } \mathrm{UPDATE:yes},\\
\mathcal{M}_i^{(k)} & \mathrm{otherwise.}
\end{cases}
\]
This design makes memory persistence conditional rather than automatic: the model must emit “UPDATE:yes” for the buffer to change. The session buffer is thus both a storage mechanism and an endogenous memory policy [2606.13815].

Layer 3 is cross-session memory. A lifetime store $\mathcal{L}_i$ is consolidated from final $\mathcal{M}_i^{(k)}$ via $\rho$ at session end. At the next session start, the session memory is seeded as $\mathcal{M}_i^{(k+1)} \leftarrow \pi(\mathcal{M}_i^{(k)}, \mathcal{L}_i)$ under three policies: **normal**, which propagates prior session memory; **past (Ablation A)**, which loads an offline file; and **fresh (Ablation B)**, which uses empty memory. The cross-session layer is therefore the locus of the memory ablation experiments and of the paper’s claim that persistent memory is a conditional asset rather than a uniform benefit [2606.13815].

## 4. Nine-axis cognitive profile

The nine-axis profile decomposes strategic reasoning into interpretable dimensions. The axes combine deterministic statistics, judge-based hybrid assessments, and regex-based factual extraction. Their purpose is not to replace tournament outcome, but to expose which strategic faculties are responsible for performance and where scalar chip totals conceal asymmetries.

| Axis | Modality | Basis |
|---|---|---|
| $M_1$ Bet-Sizing Calibration | Deterministic | 7-bucket histogram of bet-ratios vs GTO reference; preflop/postflop weighting |
| $M_2$ Bluffing / Deception | Hybrid: 1-judge | Low-equity postflop aggression, judge-labeled intent, board texture, bluff frequency, low-equity calls |
| $M_3$ Opponent Reading | Hybrid: 3-judge | JSD across opponents’ action distributions and reasoning quality |
| $M_4$ Composure | Deterministic | VPIP and AF deviation pre-/post-“bad beat” |
| $M_5$ Adaptability | Deterministic | EV gap trend slope, window-drift JSD weighted by profit improvement, between-session JSD |
| $M_6$ Prediction Accuracy | Regex | HandID, equity MAE relative to Monte Carlo equity, range-vocab ratio |
| $M_7$ Strategic Mixing | Deterministic | Action diversity within $(\text{street}, \text{position})$ buckets with at least 10 observations |
| $M_8$ Factual Accuracy | Regex | Hand-type correctness, pot-odds within 5%, hallucinated cards, draw identification |
| $M_9$ Positional Awareness | Deterministic | Spearman relation between position rank and VPIP, plus deviation from $VPIP_{GTO}$ |

Several axes are defined by explicit formulas. For $M_1$, each bet type $t \in \{\text{cbet}, \text{value}, \text{3bet}, \text{4bet}, \text{overbet}\}$ is scored by
\[
S_t = 1 - \frac{\lVert \hat p_t - p_t^*\rVert_1}{2},
\]
with preflop and postflop components combined by empirical weighting. For $M_3$, adaptation and reasoning quality are combined as $M_3 = 0.5\,A_{\mathrm{adapt}} + 0.5\,Q_{\mathrm{reason}}$. For $M_4$, composure is measured from normalized changes in VPIP and aggression factor around “bad beat” events. For $M_9$, the score is
\[
M_9 = 0.50\,G + 0.50\,D,
\]
where $G = \max(0,\mathrm{Spearman}(\text{position rank}, VPIP))$ and $D = \max(0, 1-(\mathrm{mean}|VPIP_{\mathrm{obs}}-VPIP_{GTO}|)/0.30)$ [2606.13815].

A common misconception is that such axes merely restate chip outcomes in finer detail. The reported construction argues otherwise: the axes include factual accuracy, prediction accuracy, composure, and strategic mixing, which are not reducible to end-of-session bankroll changes. This suggests that the profile is intended as a structural probe of agent behavior rather than a relabeling of tournament success [2606.13815].

## 5. Experimental protocol

The evaluation seats seven frontier LLMs together across 50 sessions $\times$ 20 hands, yielding 1 000 hands and 9 115 logged actions. Chips are aggregated as cumulative $\Delta$ per session and across sessions. Axis scores are computed per agent over all hands, and both per-axis $M_k$ and mean $\bar M$ are reported. The tournament design therefore combines repeated interaction, escalating blinds, and long-horizon aggregation rather than isolated hands or synthetic static prompts [2606.13815].

A controlled memory ablation was also conducted. Three models—Claude, GPT, and Kimi—ran 10 paired sessions of 10 hands under two Layer 3 conditions, no memory versus seeded memory, with identical random seeds, for 600 hands total. Because random seeds are matched across conditions, the ablation isolates the effect of cross-session memory policy more directly than an unmatched comparison would. The platform’s methodological emphasis is thus dual: multi-axis profiling of strategic behavior and controlled variation of memory interfaces [2606.13815].

## 6. Leaderboards, rank divergence, and memory effects

The chip leaderboard over 50 sessions is: Claude Opus 4.6 at +$15 730, Grok 4 at +$3 705, GPT-5.4 at −$1 060, DeepSeek V3.1 at −$937, Qwen3-Max at −$2 785, Gemini 3.1 Pro at −$2 095, and Kimi K2 at −$11 558. Claude records 14 first-place finishes and $\sigma_\Delta = \$715$. By contrast, the mean-axis leaderboard is ordered differently: Grok at 0.6137, DeepSeek at $\sim 0.610$, GPT at $\sim 0.605$, Qwen at $\sim 0.580$, Claude at 0.5754, Gemini at $\sim 0.550$, and Kimi at $\sim 0.490$ [2606.13815].

| Model | Chip leaderboard | Mean-axis leaderboard |
|---|---:|---:|
| Claude Opus 4.6 | +$15 730 | 0.5754 |
| Grok 4 | +$3 705 | 0.6137 |
| GPT-5.4 | −$1 060 | $\sim 0.605$ |
| DeepSeek V3.1 | −$937 | $\sim 0.610$ |
| Qwen3-Max | −$2 785 | $\sim 0.580$ |
| Gemini 3.1 Pro | −$2 095 | $\sim 0.550$ |
| Kimi K2 | −$11 558 | $\sim 0.490$ |

Per-axis leadership is also distributed: DeepSeek leads $M_1$ bet sizing at 0.79; Grok leads $M_2$ bluffing and $M_3$ opponent reading at 0.83 and 0.46; GPT leads $M_4$ composure and $M_9$ position at 0.89 and 0.83; Gemini leads $M_8$ factual accuracy at 0.74; and no model wins more than two axes. Rank correlation between chip rank and mean $\bar M$ is Spearman $\rho_S = +0.571$, $p = 0.180$, with $n=7$. The interpretation given is positive but loose alignment: scalar chip rank does not equal profile rank [2606.13815].

The memory ablation sharpens this point. GPT shows +114.6 chips per session gain with prior memory ($t = +1.72$, $p = 0.120$), Kimi shows −109.4 chips per session with memory ($t = -1.84$, $p = 0.099$), and Claude shows −42.5 chips per session ($t = -1.92$, $p = 0.087$). The directions are consistent but $p > 0.05$; effects are model-specific. The benchmark’s conclusion is therefore twofold: scalar leaderboards misrank agents, and persistent memory helps some models and hurts others. It further states that consistency across axes, reflected in low $\sigma_{M_k}$, matters more than peak on any single axis for stable long-run chip accumulation [2606.13815].

## 7. Relation to decentralized and instruction-driven poker systems

Related work places Poker Arena within a broader technical landscape of poker platforms. “Instantaneous Decentralized Poker” formalizes a stateful contract model $\mathcal{F}_{\mathsf{stateful}}$ with deposits, balance, nonce, and status in $\{\mathsf{OPEN}, \mathsf{DISPUTE}, \mathsf{CLOSED}\}$; it separates on-chain calls from off-chain updates via signed state proofs; and it describes setup, off-chain MPC, and settlement or penalty phases for an unbounded sequence of poker hands. The same work provides an Ethereum/Solidity reference implementation with functions such as `openChannel`, `updateState`, `dispute`, `resolve`, and `withdraw`, and describes amortized per-hand cost under dispute-free play and simulation-based security goals including fairness, no-counterparty risk, and off-chain liveness [1701.06726].

A different line of work, “Instruction-Driven Game Engine: A Poker Case Study,” treats poker-engine learning as next-state prediction. Its architecture includes a Game-Script Parser, State Manager, Core-Function Library, LLM Next-State Predictor, and Code Interpreter & Merger. It distinguishes Full Next-State Prediction from Differential State Prediction, represents state as a Python dict with keys including `deck`, `players`, `community`, `pot`, `bets`, and `game_flow`, and emphasizes precision by off-loading heavy numerical logic to core functions and applying validators such as deck-size consistency, pot accounting, duplicate-card checks, and nonnegative chip totals. It also reports an “Easy → Medium → Hard” curriculum with warmup on Core Functions, Standard NSP/DSP, and Diverse via Segment Rephrasing [2410.13441].

These adjacent systems do not redefine the benchmark reported in Poker Arena, but they illuminate complementary interpretations of poker as a research substrate. This suggests a broader design space in which a “Poker Arena” can function as a multi-axis evaluation platform for frontier LLMs, as an off-chain secure multiparty protocol with stateful contracts, or as an instruction-driven engine that generates gameplay from natural-language rules. Across these settings, poker serves as a compact but technically demanding testbed for sequential decision-making, memory, verification, and adversarial interaction [1701.06726][2410.13441].

Source: https://www.emergentmind.com/topics/poker-arena