ToolPoker: A Solver-Integrated Poker AI Framework

• ToolPoker is a framework that integrates language model reasoning with external solvers and analysis tools to achieve game-theoretic optimal poker play.
• It employs a structured reasoning loop with explicit tool calls and a unified API to overcome LLM limitations and align strategic actions with solver outputs.
• Experimental results in two-player poker settings show that ToolPoker narrows the performance gap to equilibrium play through combined supervised and reinforcement learning.

Searching arXiv for the cited ToolPoker-related papers and adjacent poker/tool-use work to ground the article. ToolPoker denotes a poker AI framework in which language-model reasoning is coupled to external computational tools rather than relying on internal textual heuristics alone. In the broad systems sense, it is “a system that helps you compute, understand, and use game-theory-optimal (GTO) and exploitative strategies in poker” (Sonawane et al., 2024). In its most explicit current instantiation, ToolPoker is a tool-integrated reasoning framework that combines an LLM for explanation with external poker solvers, equity calculators, and range analyzers through a unified tool API and reinforcement learning, with the dual objective of near-GTO play and professional, game-theoretic explanations (Lin et al., 31 Jan 2026).

1. Definition and conceptual scope

ToolPoker emerged from a specific failure mode of general-purpose LLMs in poker. Systematic evaluation in Kuhn Poker, Leduc Hold’em, and Heads-up Limit Texas Hold’em found that raw LLMs consistently underperform traditional algorithmic baselines and display three recurring defects: reliance on heuristics, factual misunderstandings, and a “knowing-doing” gap in which the model’s action diverges from its own reasoning trace (Lin et al., 31 Jan 2026). In that sense, ToolPoker is not merely a solver wrapper; it is an attempt to make reasoning, tool use, and final action mutually consistent.

The framework is grounded in the long-standing role of poker as a benchmark for imperfect-information AI. Poker is modeled as an extensive-form game with hidden private information, strategic randomization, and opponent-dependent value. In the heads-up case, GTO corresponds to a Nash equilibrium strategy profile in a two-player zero-sum game, while exploitative play deliberately deviates from equilibrium to gain EV against specific opponent tendencies (Sonawane et al., 2024). ToolPoker inherits both strands: equilibrium computation supplies defensible baseline actions, while natural-language reasoning exposes the strategic rationale.

A common misconception is that scale alone makes LLMs competent poker reasoners. The empirical record does not support that view. Larger closed models improve some aspects of reasoning quality, but even strong models remain below CFR+/DeepCFR-style baselines in gameplay and continue to exhibit factual and action-consistency errors (Lin et al., 31 Jan 2026). ToolPoker therefore treats external computation as a structural necessity rather than an optional aid.

2. Game-theoretic substrate and solver foundations

ToolPoker is built on standard imperfect-information game theory. In the formalization used for poker reasoning, the game can be viewed as a POMDP

$(\mathcal{S}, \mathcal{A}, T, R, \mathcal{O}, O),$

with public state, private state, observation history, and a policy

$\pi_\theta(a_t \mid f(h_t)).$

The target is not generic reward maximization alone, but policy improvement toward equilibrium play under partial observability (Lin et al., 31 Jan 2026).

The solver side of ToolPoker draws on the CFR lineage. In two-player zero-sum extensive-form games, CFR’s average strategy converges to a Nash equilibrium, and CFR+ weakly solved Heads-Up Limit Texas Hold’em at exploitability approximately $0.986$ milli-big-blinds per game (Sonawane et al., 2024). This matters because ToolPoker’s external tools are not arbitrary heuristics: they are intended to return solver-derived actions, mix frequencies, equities, and range summaries consistent with equilibrium analysis.

The two-player focus is also principled. In poker with three or more players, CFR’s computational complexity rises exponentially with the number of players, and following Nash equilibrium no longer guarantees a non-losing outcome (Maugin et al., 26 Sep 2025). This sharply limits the meaning of “GTO” outside the heads-up zero-sum setting. ToolPoker, as currently formulated, therefore fits most naturally in domains where solver outputs retain clear game-theoretic semantics.

3. Architecture and interaction protocol

ToolPoker’s core design is a tool-integrated reasoning loop. The LLM reads a textual game state, emits a structured solver call, receives tool output, refines its reasoning using that output, and then produces a final action. The interaction protocol is enforced through explicit tags: > ..., <tool>...</tool>, <output>...</output>, and <answer>...</answer> (Lin et al., 31 Jan 2026).

The unified tool interface is central. Rather than exposing many loosely coupled tools, ToolPoker uses a single solver-style API that can internally invoke a CFR-style GTO solver, an equity calculator, and a range analyzer. An example call includes arguments such as player_card, public_card, my_pot, opponent_pot, my_raise_num, opponent_raise_num, and legal_actions (Lin et al., 31 Jan 2026). This design reduces tool-learning complexity and limits error propagation across multi-step tool chains.

Component	Function	Typical output
LLM reasoning layer	Parses state, formulates solver request, explains action	Tagged reasoning trace
Unified poker tool API	Computes solver-consistent action and statistics	GTO action, action distribution, equities, ranges
Action layer	Maps tool-grounded reasoning to play	Final move in <answer>

This architecture changes the role of the LLM. The LLM is no longer the primary source of strategic truth; it becomes an orchestrator and explainer whose reasoning is constrained by external computation. The framework’s explicit purpose is to replace vague heuristic narratives with reasoning grounded in solver-derived quantities such as equity, hand histograms, and mixed strategies (Lin et al., 31 Jan 2026).

4. Training pipeline and optimization

ToolPoker is preceded by a two-stage internal-policy baseline, BC-RIRL, which clarifies why tool grounding is needed. The behavior-cloning stage uses a dataset of approximately $5$k high-quality reasoning-augmented samples, where CFR+ provides expert actions and a strong LLM produces professional-style explanations conditioned on solver statistics (Lin et al., 31 Jan 2026). The supervised objective is

$$\mathcal{L}_{\text{BC}} = - \mathbb{E}_{(h_t, a_t)\sim D_b}\left[\log \pi_\theta(a_t \mid h_t)\right].$$

The reinforcement-learning stage introduces regret-inspired step rewards derived from CFR-style cumulative regret, then applies PPO optimization. This improves reasoning style and some gameplay metrics, but remains insufficient because factual alignment still lags behind solver-grounded performance (Lin et al., 31 Jan 2026). That shortfall motivates the shift to explicit tool use.

ToolPoker’s own training pipeline first augments the BC data with explicit tool tags and real solver outputs, then fine-tunes the model to produce structured tool-using rollouts. The RL phase uses a composite reward

$$R(a_t, \hat{a}_t, p_t) = R_{\text{answer}}(a_t, \hat{a}_t) + \alpha_f R_{\text{format}}(p_t) + \alpha_t R_{\text{tool}}(p_t),$$

where $R_{\text{answer}}$ rewards agreement with the solver action, $R_{\text{format}}$ rewards correct tagged structure, and $R_{\text{tool}}$ measures successful tool execution (Lin et al., 31 Jan 2026). This reward is optimized with PPO, and loss masking is used so the model is not directly trained to imitate solver-output tokens.

The technical significance of this pipeline is that it trains three coupled competencies simultaneously: when to call the tool, how to incorporate the tool’s output into a valid explanation, and how to align the final action with the returned solver recommendation. ToolPoker’s improvement is therefore not attributable to solver access alone, but to supervised and RL-based alignment of language, tool protocol, and strategic action.

5. Empirical performance and reasoning quality

In Leduc Hold’em, vanilla Qwen2.5-7B posts strongly negative net-chip results against NFSP, DQN, DMC, and CFR+, while the BC-RIRL variant improves but remains below CFR+ (Lin et al., 31 Jan 2026). ToolPoker, using the same Qwen2.5-7B backbone, records $+11.5$, $\pi_\theta(a_t \mid f(h_t)).$0, and $\pi_\theta(a_t \mid f(h_t)).$1 net chips against NFSP, DQN, and DMC respectively, while finishing only $\pi_\theta(a_t \mid f(h_t)).$2 against CFR+, for an average of approximately $\pi_\theta(a_t \mid f(h_t)).$3 across those baselines (Lin et al., 31 Jan 2026).

The effect is stronger in Heads-up Limit Texas Hold’em. Vanilla Qwen2.5-7B and its BC-RIRL variant remain substantially negative against DeepCFR and several RL baselines, whereas ToolPoker achieves $\pi_\theta(a_t \mid f(h_t)).$4, $\pi_\theta(a_t \mid f(h_t)).$5, and $\pi_\theta(a_t \mid f(h_t)).$6 against NFSP, DQN, and DMC, and only $\pi_\theta(a_t \mid f(h_t)).$7 against DeepCFR, for an average near $\pi_\theta(a_t \mid f(h_t)).$8 across the evaluated set (Lin et al., 31 Jan 2026). These results place ToolPoker close to equilibrium-level play in the tested two-player settings.

Reasoning quality is evaluated with three scores: heuristic reasoning (HR), factual alignment (FA), and action-reasoning consistency (AC). Professional traces score $\pi_\theta(a_t \mid f(h_t)).$9 on all three axes. Vanilla models are notably weaker, and BC-RIRL mainly improves style and consistency while leaving factual precision behind. ToolPoker reaches near-professional values, with Leduc scores reported at HR $0.986$0, FA $0.986$1, and AC $0.986$2 on the Qwen2.5-7B backbone (Lin et al., 31 Jan 2026). In practical terms, this means the framework materially reduces the three baseline defects it was designed to address.

The system’s residual gap to exact solver baselines is attributed to occasional mis-specified tool calls or action mismatches, not to the underlying game-theoretic competence of the external solver itself (Lin et al., 31 Jan 2026). That distinction is important: ToolPoker’s remaining errors arise at the tool-orchestration boundary.

6. Position within the broader poker-AI ecosystem

ToolPoker belongs to a broader family of poker systems, but it occupies a distinctive position. Traditional GTO poker systems rely on abstraction, discretized betting models, and equilibrium approximation, often using CFR or CFR+ as the computational backbone (Sonawane et al., 2024). ToolPoker retains that equilibrium foundation at the tool level while delegating explanation and interface management to an LLM.

A different line of work, exemplified by SpinGPT, applies LLM fine-tuning directly to a three-player Spin & Go format. SpinGPT uses supervised fine-tuning on $0.986$3k expert decisions and offline RL on $0.986$4k solver-generated hands, matching the solver’s actions in $0.986$5 of decisions under tolerant accuracy, but it does not attempt to compute exact Nash equilibria for the full three-player tournament game (Maugin et al., 26 Sep 2025). ToolPoker differs in that it does not try to internalize all strategic computation; it externalizes it through runtime solver access.

ToolPoker also fits naturally into an emerging infrastructure stack for poker research. The PHH format provides a standardized, human-readable and machine-friendly representation of hand histories across multiple variants, which suggests a natural interchange layer for ToolPoker-style systems that need reproducible trajectories, replay, and cross-tool compatibility (Kim, 2023). For evaluation, GTO Wizard Benchmark supplies a public API for Heads-Up No-Limit Texas Hold’em against a superhuman benchmark agent and integrates AIVAT, a provably unbiased variance reduction technique that achieves equivalent statistical significance with ten times fewer hands than naive Monte Carlo evaluation (Provost et al., 24 Mar 2026).

Opponent adaptation defines another complementary direction. StratFormer shows that a transformer-based meta-agent can model and exploit opponents in Leduc Hold’em, achieving an average exploitation gain of $0.986$6 BB per hand over GTO while maintaining near-equilibrium safety (Caen et al., 28 Apr 2026). ToolPoker and StratFormer address different points on the same spectrum: the former emphasizes solver-grounded reasoning and action alignment, while the latter emphasizes online opponent modeling and best-response deviation.

7. Limitations, extensions, and open problems

ToolPoker’s current formulation has explicit constraints. It depends on the availability of reliable external solvers and equity tools, incurs additional computational cost at both training and inference time, and has not yet been evaluated directly against human professionals in live play (Lin et al., 31 Jan 2026). Its strongest results are in two-player settings where solver semantics remain well defined.

Several extensions follow directly from the current design. The framework can be expanded to more complex and multi-agent strategic environments, richer tool ecosystems, and reward functions that better enforce faithful use of solver outputs in the textual explanation itself (Lin et al., 31 Jan 2026). A plausible implication is that future versions will need to decide not only how to use tools, but also how to manage disagreement between internal model priors and external solver advice.

If ToolPoker evolves from a unified solver API into a broader multi-tool ecosystem, it will also inherit the general tool-selection security problems documented for LLM agents. Tool metadata can be adversarially optimized to bias selection rates, and malicious tool documents can hijack retrieval-plus-selection pipelines even in no-box settings (Sneh et al., 2 Oct 2025, Shi et al., 28 Apr 2025). That issue is not a demonstrated weakness of the current unified ToolPoker interface, but it is a plausible systems-level concern for any future marketplace-style extension.

Taken together, ToolPoker represents a shift in poker AI from monolithic language-model policy learning toward tool-grounded strategic reasoning. Its central claim is not that LLMs have solved poker, but that solver-backed tool use can close much of the gap between fluent explanation and game-theoretic correctness (Lin et al., 31 Jan 2026).