TavernGym: A Full-Content Hearthstone AI Simulator

• TavernGym is a full-content, player-view-faithful Hearthstone Battlegrounds simulator that models recruit and combat phases, stochastic shops and battles, shared card pools, heroes, cards, quests, anomalies, trinkets, tribes, and persistent effects.
• Its Rust engine supports 245 legal actions with exact masks, seeded bit-for-bit replay, serialization, save and restore, batched execution, and Python bindings, enabling reinforcement-learning training, tactical lookahead, benchmark generation, and controlled experiments.
• TavernGym powered TavernAgent development and achieved 2.33 average placement with 41.5% first places in simulator evaluations, while live validation showed how replay and reality checks exposed 184 rule, encoding, controller, and opponent-model issues.

TavernGym is a Rust-based, full-content simulation environment for Hearthstone Battlegrounds developed within the AutoTavern system. It reproduces the live game from the player’s perspective, including recruit and combat phases, stochastic shops and battles, shared card-pool effects, heroes, cards, quests, anomalies, trinkets, tribes, persistent effects, and other patch-specific mechanics. TavernGym provides exact legal-action masks, seeded reproducibility, serialization, save/restore, batched execution, Python bindings, and interfaces for reinforcement learning, tactical lookahead, benchmark generation, and live-game validation. It was developed to train and evaluate TavernAgent, a PPO-based policy, and constitutes the central simulation artifact described in “Build, Don't Play: Self-Improving LLM Research Agents Develop a Hearthstone Battlegrounds AI That Rivals Top Players” (Ma et al., 24 Oct 2026).

1. Origins and role within AutoTavern

TavernGym was constructed as part of AutoTavern, a self-improving research system in which specialized LLM agents developed a game simulator, trained agents, modeled opponents, evaluated results, and deployed policies against the live game. Hearthstone Battlegrounds presents a difficult environment for game AI because it is an eight-player auto-battler with stochastic shops and combat, partial information about opponents, a shared card pool, delayed placement-based rewards, and nearly one thousand interacting game objects.

The live patch studied contained 978 game objects. A complete match generally involves approximately 15 turns and about 450 atomic actions, while the final outcome is a placement from first through eighth. The game combines long-horizon economy management, positioning, targeting, combat randomness, card effects, hero powers, quests, anomalies, trinkets, tribes, golden cards, and persistent effects.

Before TavernGym, available Battlegrounds environments were substantially narrower: some simplified the game, omitted heroes, modeled only combat, or supported advice rather than autonomous play. TavernGym was designed to provide a full-content, player-view-faithful, trainable reproduction of the live game rather than a combat-only or abstracted simulator.

Its principal functions were:

1. Training: supplying large numbers of simulated decisions for PPO training without requiring live matches.
2. Evaluation: enabling deterministic seeds, exact legality checks, replay, save/restore, and controlled comparisons.
3. Reality anchoring: allowing live-game records to be replayed in the simulator so that discrepancies could be identified and corrected.

The broader AutoTavern process operated through three nested loops. The training loop updated policies over seconds or minutes; the research loop evaluated simulator, opponent, and policy changes over hours; and the ladder loop used live games and replay-based validation over days. The system registered approximately 10,000 experiments, produced 23 TavernAgent versions, and recorded 184 fixes prompted by live games (Ma et al., 24 Oct 2026).

2. Software architecture and research-process integration

TavernGym consists of a Rust game engine of approximately 63,000 lines, approximately 27,000 lines of card scripts, thin Python bindings, batched calls that release the Python interpreter lock, an arena-based entity representation, a shared effect context, a single seeded random-number generator, JSON serialization, and source and rule hashes for build identity.

The shared effect context allows card effects to operate consistently across recruit and combat phases. Cards are represented as event hooks rather than as unrelated implementations for each phase. This architecture supports a unified treatment of triggers and persistent effects while reducing the risk of phase-specific behavioral divergence.

TavernGym was integrated into AutoTavern through a set of verification gates:

1. Build gate: code changes had to pass the full test suite.
2. Contract gate: a policy had to be associated with the simulator build and rule hash on which it was trained.
3. Comparison gate: model comparisons had to occur within one simulator build and across multiple seeds.
4. Delivery gate: candidate models had to be evaluated on seeds not used for selection.
5. Reality gate: delivered models had to play on the live ladder, after which the resulting games were replayed in TavernGym.

The research agents communicated through a typed, versioned research memory rather than unrestricted conversational state. This memory contained a numbered findings ledger, corrections linked to earlier findings, a rule-decision registry, open mechanics questions, handover notes, an experiment registry, and persistent lessons.

The live deployment controller read the live game state, translated it into TavernGym’s input encoding, obtained an action from TavernAgent, named the operation, and used an official plug-in shortcut key to execute it. A human pressed the corresponding shortcut, but did not make game decisions in the counted games. Live-game logs were then returned to the research loop.

The human intervened mainly to set goals and resolve ambiguous mechanics. The paper reports 418 human messages, representing 0.6% of all messages. The resulting development process produced 375 rule-verification records, 220 confirmed rule decisions, and 184 live-game-driven fixes comprising 35 game-rule and card-behavior fixes, 11 card-data and offer-rule fixes, 73 input-encoding fixes, 51 client-controller fixes, and 14 opponent-library or examination recalibrations (Ma et al., 24 Oct 2026).

3. State representation, observations, and action semantics

TavernGym represents the game state as an arena of entities with integer identifiers. This makes states cheap to copy, serializable, suitable for exact save/restore, useful for replays and lookahead, and convenient for randomized testing and branching analysis.

The observation is player-view faithful. The player sees their own complete state and the public leaderboard. Opponent warbands are represented by the last warband seen in combat, while hidden opponent information is not exposed. Observations include recruit-phase information, combat-relevant state, pending choices, cards, offers, counters, and persistent effects. The supplied paper does not provide a complete flattened tensor specification for every field, but it describes the semantic contents and the entity-token representation used by TavernAgent.

The RL interface exposes 245 discrete actions with an exact legality mask. Action categories include:

• Ending the turn
• Refreshing the tavern
• Freezing offers
• Upgrading the tavern
• Using the Hero Power
• Buying or selling minions
• Playing cards
• Magnetizing cards to positions
• Moving minions
• Choosing Discover or other offers
• Selecting targets and effect-specific choices

The policy produces logits over all 245 action slots, after which illegal actions are masked. The environment also performs an exact legal-action-mask guard, verifying that the selected operation is legal rather than relying solely on the policy’s output mask. This prevents invalid transitions and avoids training the policy to distinguish legal actions from impossible ones.

TavernGym models the game as a partially observable stochastic game with terminal utility based on placement. A match begins with a seeded state or a restored configuration. Each atomic action advances the game and produces a new player-view observation and legal-action mask. The episode terminates when only one player remains, yielding a placement from first through eighth.

The supplied paper does not specify a conventional Gym method signature, exact Python class names, or whether every intermediate step returns zero reward. It does establish the semantics of reset or start, action application, observations, legality masks, terminal placement utility, save/restore, and batched execution.

All simulator randomness comes from one seeded generator. Given a seed and an action sequence, a game can be reproduced bit-for-bit. This supports exact replay, invariant checking, fuzzing, regression testing, combat analysis, and controlled comparisons. A critical implementation issue arose when copied simulator states retained the random-number state: search procedures thereby gained access to future shops and combat outcomes. The apparent improvement was 23.6 percentage points in first-place rate, but after reseeding copied samples the effect became $-0.65 \pm 1.14$ percentage points.

4. Modeled game mechanics

TavernGym models the complete recruit-and-combat structure of the studied Battlegrounds patch. Each match has eight players. Each player has a hero, a Hero Power, approximately 30–40 health, and a warband of up to seven minions.

During the recruit phase, players can spend gold, buy and sell minions, refresh or freeze tavern offers, upgrade the tavern, play cards, use Hero Powers, arrange their warband, and make Discover, trinket, quest, anomaly, or other choices. During combat, a player automatically fights an opponent. Attack order and targets are randomized according to game rules and effects, while Deathrattles, Reborn, shields, cleaves, Venomous effects, auras, and other combat systems resolve. Players are eliminated as their health reaches zero.

The economy begins with three gold and rises to ten gold. TavernGym models gold spending, refreshes, freezing, upgrades from tavern tier 1 through tier 7, buying and selling, gold chains, Discover effects, triple formation, and shared card-pool implications.

Tavern offers are drawn from a card pool shared by all players. The simulator includes tier-dependent offers, offer generation, refresh and freeze behavior, card availability, pool effects, golden-card formation, Discover offers, and patch-specific card data. Card tables are generated from game data files, while later patch modifications are applied as overlays. The active build is identified by a rule hash.

Three copies of a minion merge into a golden version. Golden cards have enhanced characteristics or effects according to the card rules and provide a Discover choice from the next tavern tier. The benchmark specifically tests triple logic, and TavernGym includes golden-card behavior as part of its full-content implementation.

The 978 modeled objects include minions, heroes, Hero Powers, spells, tavern spells, trinkets, and other game objects. The simulator supports interactions involving Battlecries, Deathrattles, auras, magnetic attachments, tavern spells, Discover, Reborn, Divine Shield, Taunt, Cleave, Venomous effects, Hero Powers, golden versions, tribes, persistent effects, quests, anomalies, Dark Gifts, and other attachments or effects.

Combat is procedurally resolved rather than described through a single closed-form equation. TavernGym executes attack order, target selection, Taunt and shield interactions, Deathrattles, Reborn, Cleave, Venomous effects, start-of-combat effects, and other card-specific triggers. The supplied paper does not provide complete pseudocode for attacker selection, target sampling, damage, tie handling, matchmaking, simultaneous triggers, or every priority rule.

The player’s observation excludes hidden opponent information. Opponent models may internally maintain hidden counters because they represent simulated seats, but those counters are not necessarily visible to the learning player. This restriction is essential for transfer to the live game: an omniscient simulator would make policy training easier while creating an observation mismatch at deployment.

5. Performance, persistence, and reinforcement-learning integration

On one CPU core, TavernGym executes full eight-player scripted games in 3.6–5.3 milliseconds, corresponding to 190–280 complete games per second. Late-game combats require 13–19 microseconds, corresponding to 52,000–76,000 combats per second. These rates support PPO rollouts, large-scale opponent-library simulation, repeated combat estimation, benchmark-label generation, and one-step lookahead.

TavernGym supports exact save and restore of running games, JSON-compatible entity serialization, batched execution, source and rule hashes, and copied states for lookahead. These capabilities permit repeated combat simulations and controlled counterfactual comparisons. The simulator’s performance is reported on a single core, while the complete PPO training system used eight H200 GPUs and 128 CPU cores.

TavernAgent uses an entity-transformer architecture operating on up to 56 entity tokens plus one global token. Tokens represent the player’s minions, hand cards, tavern offers, pending choices, and opponents. Opponent tokens contain public leaderboard information, the last observed warband, and the result of the corresponding combat. Attachments are associated with the relevant minion token, while game-wide effects, trinkets, and counters are pooled into the global token.

The policy has hidden dimension 384, eight transformer layers, eight attention heads, approximately 18 million parameters, a pointer head for action logits, a value head, and an auxiliary placement head. Card representations combine identity embeddings, static features such as type, tier, statistics, tribe, keywords, and mechanics, and an embedding of rule text. The simulator executes card effects through scripts, whereas the policy receives learned representations of cards and rules.

TavernAgent is trained with PPO on approximately one million decisions per iteration. Each major training stage uses 300 iterations, approximately 310 million decisions, and approximately 570,000 games, with the previous stage’s model used for initialization. The final-stage configuration includes 512 environments or games per member and 256 steps, approximately $1.05 \times 10^6$ decisions per iteration, minibatches of 8,192, one training epoch, a learning rate declining from $6\times10^{-6}$ to $5\times10^{-6}$, PPO clip value 0.2, target KL 0.05, $\gamma=1.0$, $\lambda=0.995$, gradient-norm clipping at 1.0, entropy coefficient 0.002, value coefficient 0.5, auxiliary coefficient 0.1, and a KL-anchor coefficient of 0.1 for the first 100 iterations.

Training proceeded through simplified rules, full content, player-view observations, first-place optimization against human-statistics-based opponents, calibrated opponent libraries, graded opponent difficulty, and continued environment and opponent updates. Each delivered version was evaluated on unseen seeds before deployment.

On one simulator build, three unseen seeds, and 24,576 games per row, the progression from random legal actions and a scripted bot to TavernAgent versions showed improvement. TA-1 achieved a main average placement of 3.40 and 1.9% first places; TA-11 achieved 2.83 and 18.5%; and TA-23 achieved 2.33 and 41.5%. In an eight-version head-to-head league, TA-23 achieved average placement 3.95, first place in 26.2% of games, and top-four placement in 57.9% of games.

6. Opponent modeling, validation, and benchmarks

TavernGym’s training environment generally fills the seven opposing seats with opponent-library policies rather than unrestricted self-play. Each library entry contains a board at a particular turn, associated hand information, hidden counters, and a target strength curve. The libraries are derived from top-human statistics and recorded games, including tribe shares, upgrade curves, purchase rates, warband strength by turn, and progression patterns.

Opponent libraries were later calibrated to the opponents encountered by TavernAgent in live play. In an ablation, uncalibrated libraries produced 20.9% first place on the main examination and 10.3% on the hard examination, whereas reality-calibrated libraries produced 27.2% and 13.5%, respectively.

TavernGym was validated through unit tests, fuzzing with random legal actions, state-invariant checks after every action, exact action-mask checks, live replay of recruit actions, combat, and tavern offers, and rule and source hashes. Live games exposed failures that internal simulator metrics did not reveal. Among the reported issues were mismatched encodings between the simulator and controller, errors in card and offer rules, and opponent-model mismatches.

TavernBench is a 960-task benchmark across 40 scenarios and eight categories: rules, economy, combat, positioning, ordering, targets, selection, and strategy. Tasks include legal-action execution, one-on-one fights, gold accounting, optimal purchases, combat probabilities, warband arrangements, trigger ordering, target selection, Hero, trinket, and Discover choices, and early- and late-game plans.

Benchmark labels are generated by TavernGym or controlled play-outs. The procedure uses 4,096 combats for combat questions, 2,048 combats per positioning arrangement, 1,024 combats per decision option, 128 full play-outs per strategy plan, and label filtering when the winning option is not sufficiently separated. For combat questions, the most likely outcome must match and the Brier score must be at most 0.1.

On the 368 tasks shared by all systems, TavernAgent achieved 91% play accuracy, 90% real-time accuracy, 100% combat accuracy, 98% positioning accuracy, 100% ordering accuracy, 98% target accuracy, 64% selection accuracy, and 75% strategy accuracy, with a median decision time of 0.03 seconds. Frontier LLMs achieved lower play performance and generally required more time. When LLMs received TavernGym tools, their performance improved substantially; GPT-6 Astra, for example, rose from 62% text-only accuracy on the tool subset to 97% with tools.

Live deployment provided an additional validation layer. TavernAgent played 392 live ladder games over days 5–18. Its rating rose from 6,000 to 14,007 and later reached approximately 15,000. Excluding client-failure games, the overall average placement was 3.22, first-place frequency was 16.4%, and top-four frequency was 77.1%. For TA-19 through TA-23, average placement was 2.37, first-place frequency was 32.6%, and top-four frequency was 91.3%.

The last 20 public leaderboard records were summarized as an average placement of 1.60, 80% first places, and 90% top-four finishes for TavernAgent, compared with 2.34, 59%, and 81% for the server’s ten highest-rated players. These leaderboard statistics are not identical to the authors’ own game logs, and rating accumulation depends on the number of games played. The deployment also used an official plug-in and human shortcut presses, so it was not a completely autonomous computer-vision-and-input system.

7. Limitations, reproducibility, and significance

TavernGym’s reported full-content implementation is specific to patch 36.6.3. Later patches require new data overlays, rule changes, tests, and likely new live validation. The paper does not provide a complete combat-resolution algorithm, formal matchmaking equation, complete placement-to-utility transformation, full observation-tensor specification, exact Python API signatures, exact random-number-generator algorithm, or a complete list of unsupported edge cases.

The implementation’s 184 live-game-driven fixes demonstrate both the value of reality-based validation and the incompleteness of early simulator versions. Some mechanics required human decisions because source data were ambiguous. No independent third-party fidelity audit is reported, and “all 978 objects implemented” does not guarantee identical behavior for every undocumented timing interaction.

Opponent libraries are models of human behavior rather than direct human self-play. They are calibrated using human statistics, recorded games, and encountered live opponents, but their realism remains dependent on the selected statistics and modeling assumptions. Similarly, live deployment included 96 of 388 games with placement records flagged for client failures; these were attributed to the controller rather than the policy and analyzed separately.

The complete research campaign required substantial computation: eight H200 GPUs, 128 CPU cores, approximately 1,900 GPU-hours for the 23 delivered-agent stages, approximately 16,700 GPU-hours for other experiments, approximately 1,300 GPU-hours for evaluation and analysis, and approximately 100 million LLM output tokens. TavernGym itself is computationally efficient on a single core, but reproducing the full AutoTavern training and validation process is resource-intensive.

The authors state that TavernGym and TavernBench will be open-sourced at the AutoTavern repository. The supplied paper does not provide a fixed software release hash, container image, complete pretrained TavernAgent checkpoints, detailed hardware-independent reproduction recipe, or licensing terms for every Hearthstone-derived data file and asset.

TavernGym’s principal significance lies in its combination of full-content simulation, player-view partial observability, exact action legality, high-throughput stochastic execution, deterministic replay, serializable state, patch-specific rule tracking, human-calibrated opponent interfaces, and live-game correction. It functions simultaneously as an RL environment, a tactical lookahead engine, a benchmark-label generator, a reproducibility framework, and a reality-anchored executable specification of a complex live game.

The broader methodological conclusion is that a carefully gated simulator can convert a partially observable, stochastic, long-horizon game into a tractable environment for large-scale policy optimization and evaluation. TavernGym’s results also demonstrate that simulator fidelity cannot be assessed solely through internal performance: copied random-number states, opponent mismatches, encoding errors, and client failures were exposed only through targeted validation and live-game feedback.