MTG-Causal-RL: A Causal RL Benchmark

• MTG-Causal-RL is a benchmark for causal RL that integrates sequential decision making, partial observability, and a masked 478-action space using Magic: The Gathering.
• It employs a hand-specified Structural Causal Model to expose intervention effects and enable granular causal credit assignment, enhancing policy auditability.
• The benchmark combines multiple reward schemes and competitive archetypes to evaluate policy performance and diagnostic calibration in complex strategic settings.

MTG-Causal-RL is a Gymnasium benchmark for causal reinforcement learning built on Magic: The Gathering and designed for settings that combine sequential decision making, hidden information, large masked action spaces, and an explicit causal interface. It exposes a 3,077-dimensional partial observation, a 478-action masked discrete action space, five competitive Standard archetypes, three reward schemes, and a hand-specified Structural Causal Model (SCM) over strategic variables. During each episode, it publishes causal variables, SCM-predicted intervention effects, and per-factor credit traces, so that causal credit assignment, leave-one-out cross-archetype transfer, and policy auditability become benchmark targets rather than auxiliary diagnostics (Cunha et al., 7 May 2026).

1. Conceptual scope and research setting

MTG-Causal-RL was introduced to address a benchmark gap rather than a purely algorithmic one. The motivating claim is that earlier causal-RL settings often isolated only part of the problem—either relatively clean intervention structure in control domains or strategic sequential reasoning in games—whereas Magic: The Gathering naturally combines stochastic draws, hidden information, masked legality constraints, and strategically meaningful abstractions such as mana, board pressure, card advantage, and tempo (Cunha et al., 7 May 2026).

This positioning places MTG-Causal-RL within a broader causal-RL lineage. Earlier work on reinforcement learning of causal variables argued that standard RL can maximize return without identifying the right abstract variables to reason about, and proposed learning a mediator $Z$ that is both manipulable by a policy and predictive of the outcome via a mediation-analysis objective (Herlau et al., 2020). A separate theoretical line argued that online RL is already causal in the sense that, under action sufficiency and self-generated interaction, conditional probabilities estimated from online data can coincide with interventional probabilities; under that view, causal modeling becomes especially important for offline RL, counterfactual queries, and settings with observation mismatch (Schulte et al., 2024). MTG-Causal-RL does not contradict those arguments; a plausible implication is that it shifts emphasis from “are actions causal?” to “can causal structure be surfaced, audited, and exploited under partial observability and action masking?”

2. Environment architecture, game structure, and reward design

The benchmark is built around Standard-format MTG decks. Each player starts at 20 life with a 60-card deck and a 7-card opening hand, and an episode ends either at lethal damage or at a user-chosen turn cap (Cunha et al., 7 May 2026). The observation is intentionally partial: the 3,077-dimensional vector contains visible information about the hand, battlefield, graveyard, life totals, phase, and mana state. The action space is a 478-action masked discrete space divided into 16 action categories; at a given state, usually only 2 to 15 actions are legal. The action layout includes PASS, KEEP, MULLIGAN, CONFIRM, CANCEL, AUTO_PAY, PLAY_LAND, CAST_SORCERY, CAST_INSTANT, ACTIVATE, ATTACK_TOGGLE, blocking and targeting categories, and MANA_SOURCE selection. The environment therefore treats each turn as a sequence of 10 to 20 decisions, not a single move-selection event (Cunha et al., 7 May 2026).

The benchmark ships with five fixed Standard 2025 archetypes, chosen to induce distinct strategic regimes.

Archetype	Strategic emphasis
Mono-Red Aggro	Tempo and immediate pressure
Azorius Control	Defensive sequencing and resource management
Dimir Midrange	Pressure plus interaction
Domain Ramp	Mana acceleration and delayed payoff
Boros Convoke	Board development and pressure

MTG-Causal-RL includes three reward schemes. The first is sparse terminal reward. The second is a shaped potential-based reward

$$R^{\mathrm{shaped}}_t = \gamma\,\Phi(s_{t+1}) - \Phi(s_t) + R^{\mathrm{terminal}}_t$$

where $\Phi(s)$ is a weighted combination of observable strategic variables such as mana, card advantage, board pressure, tempo, and life buffer. The third is a dense reward variant that adds per-step signals such as damage, card draw, and creature-related feedback. The shaping potential is computed only from agent-observable quantities, which the benchmark presents as preserving policy invariance relative to the observation process (Cunha et al., 7 May 2026).

3. Hand-specified SCM and the benchmark’s intervention interface

A distinguishing feature of MTG-Causal-RL is that it exposes a hand-designed SCM rather than requiring agents to infer causal structure from scratch. The SCM is organized into four layers: resource layer, board state layer, strategic position layer, and outcome layer. Its variables include $\mathrm{Mana}_t$, $\mathrm{LandDrop}_t$, $\mathrm{ManaCreatures}$, $\mathrm{Mana}_{t+1}$, $\mathrm{CardCount}$, $\mathrm{HasRemoval}$, $\mathrm{BoardPress}$, $$R^{\mathrm{shaped}}_t = \gamma\,\Phi(s_{t+1}) - \Phi(s_t) + R^{\mathrm{terminal}}_t$$0, $$R^{\mathrm{shaped}}_t = \gamma\,\Phi(s_{t+1}) - \Phi(s_t) + R^{\mathrm{terminal}}_t$$1, $$R^{\mathrm{shaped}}_t = \gamma\,\Phi(s_{t+1}) - \Phi(s_t) + R^{\mathrm{terminal}}_t$$2, $$R^{\mathrm{shaped}}_t = \gamma\,\Phi(s_{t+1}) - \Phi(s_t) + R^{\mathrm{terminal}}_t$$3, $$R^{\mathrm{shaped}}_t = \gamma\,\Phi(s_{t+1}) - \Phi(s_t) + R^{\mathrm{terminal}}_t$$4, and $$R^{\mathrm{shaped}}_t = \gamma\,\Phi(s_{t+1}) - \Phi(s_t) + R^{\mathrm{terminal}}_t$$5 (Cunha et al., 7 May 2026).

The benchmark highlights six strategic parents of $$R^{\mathrm{shaped}}_t = \gamma\,\Phi(s_{t+1}) - \Phi(s_t) + R^{\mathrm{terminal}}_t$$6: card advantage, board pressure, tempo, life buffer, threat density, and removal availability. The outcome equation is

$$R^{\mathrm{shaped}}_t = \gamma\,\Phi(s_{t+1}) - \Phi(s_t) + R^{\mathrm{terminal}}_t$$7

Illustrative structural equations include

$$R^{\mathrm{shaped}}_t = \gamma\,\Phi(s_{t+1}) - \Phi(s_t) + R^{\mathrm{terminal}}_t$$8

$$R^{\mathrm{shaped}}_t = \gamma\,\Phi(s_{t+1}) - \Phi(s_t) + R^{\mathrm{terminal}}_t$$9

and

$\Phi(s)$0

The SCM is explicitly hand-specified, not learned from scratch; the stated reason is to prioritize interpretability and clear edge orientation (Cunha et al., 7 May 2026).

Within this interface, actions are treated as interventions. A canonical example is playing a land, represented as

$\Phi(s)$1

The benchmark then computes downstream changes in strategic factors and exposes per-factor intervention effects

$\Phi(s)$2

where $\Phi(s)$3 is the $\Phi(s)$4-th strategic factor. During episodes, the environment publishes $\Phi(s)$5, factor rewards

$\Phi(s)$6

the SCM-predicted intervention effects $\Phi(s)$7, and factor-level traces for credit assignment (Cunha et al., 7 May 2026). This means that MTG-Causal-RL is not only a partially observed game environment; it is also an explicit intervention-analysis interface over a known strategic abstraction.

4. Reference baselines and CGFA-PPO

The benchmark is accompanied by a panel of reference methods rather than a single canonical solver. These include Random, Heuristic, Masked PPO, a causal-world-model PPO variant, and an architecture-matched scalar control (Cunha et al., 7 May 2026).

Method	Role in evaluation
Random	Uniformly samples legal actions
Heuristic	Rule-based script, one per archetype
Masked PPO	Standard PPO with invalid-action masking
Causal-world-model PPO variant	Auxiliary causal prediction baseline
Scalar control / CGFA scalar-only	Capacity-matched non-causal control

The benchmark’s reference causal agent is Causal Graph-Factored Advantage PPO (CGFA-PPO). It retains a masked PPO actor and scalar critic $\Phi(s)$8, but adds per-factor critic heads $\Phi(s)$9 trained on factor-specific returns

$\mathrm{Mana}_t$0

It also introduces learnable mixture weights

$\mathrm{Mana}_t$1

initialized from the SCM’s logistic-regression weights, and a state-conditional residual gate $\mathrm{Mana}_t$2 that blends scalar and factor-based learning: $\mathrm{Mana}_t$3

A further component is the intervention-calibration loss

$\mathrm{Mana}_t$4

which encourages alignment between learned factor advantages and SCM-predicted intervention effects. The full objective is

$\mathrm{Mana}_t$5

During training, the wrapper logs per-factor Pearson correlation between $\mathrm{Mana}_t$6 and $\mathrm{Mana}_t$7, sign agreement, per-factor credit shares, and gate statistics $\mathrm{Mana}_t$8, making causal usage inspectable rather than implicit (Cunha et al., 7 May 2026).

5. Evaluation protocol and empirical findings

MTG-Causal-RL adopts an unusually explicit statistical protocol. The full evaluation uses at least $\mathrm{Mana}_t$9 environment steps per opponent, 7 paired seeds per agent/deck cell, and 300 deterministic evaluation episodes per agent-seed-opponent cell. Comparisons use paired bootstrap confidence intervals, Wilson confidence intervals for win rates, Wilcoxon signed-rank tests, and Holm-Bonferroni correction within pre-registered comparison families. The benchmark also notes that some headline learned-agent results in the main tables use only $\mathrm{LandDrop}_t$0 paired seeds and are therefore exploratory (Cunha et al., 7 May 2026).

The main aggregate finding is deliberately restrained: Masked PPO and CGFA-PPO both beat random play across all five archetypes, but neither dominates uniformly. In the reported in-distribution comparisons, CGFA-PPO exceeds PPO on Azorius Control (25.6% vs 20.6%) and Mono-Red Aggro (70.6% vs 67.8%), while PPO is stronger on Boros Convoke, Dimir Midrange, and Domain Ramp. The benchmark also reports that simple heuristics remain strong on some aggressive decks, whereas slower archetypes such as Azorius Control and Domain Ramp remain difficult under the current training budget (Cunha et al., 7 May 2026).

The benchmark’s causal diagnostics are at least as important as scalar win rate. Calibration trajectories show that several factor correlations become positive over training, that factor credit concentrates heavily on the life-buffer factor, and that the residual gate remains active rather than collapsing entirely to scalar PPO. Leave-one-out transfer experiments report similar negative measured gaps for both learned agents:

• PPO: $\mathrm{LandDrop}_t$1 percentage points
• CGFA-PPO: $\mathrm{LandDrop}_t$2 percentage points

The interpretation given is not that cross-archetype transfer is solved, but that the held-out opponent mix happened to be easier in those runs, which is precisely why paired-seed transfer evaluation is necessary (Cunha et al., 7 May 2026). A common misconception is therefore that MTG-Causal-RL is primarily a win-rate leaderboard. Its actual diagnostic center of gravity is the joint inspection of performance, factor calibration, and transfer gaps.

6. Relation to causal-RL methodology and principal limitations

MTG-Causal-RL is most naturally understood as a benchmark that operationalizes several themes already present in causal RL. One such theme is the search for coarse-grained causal variables that are actionable and reward-relevant, rather than merely predictive; mediation-based RL formalized this through variables that are easier to induce under one policy than another and that materially affect return (Herlau et al., 2020). Another is the online coupling of intervention, causal structure learning, and policy guidance, where better interventions improve the learned causal graph and the graph in turn constrains policy search (Cai et al., 2024). A third is the representation of environments by context-dependent causal subgraphs, as in Meta-Causal Graph world models, where latent meta states determine which causal relations are active (Zhao et al., 29 Jun 2025). MTG-Causal-RL does not itself implement those algorithms, but it provides a domain in which comparable questions about intervention effects, structural transfer, and policy auditability can be asked in a strategically rich game setting.

Several limitations follow directly from the benchmark design. First, the causal structure is not discovered from raw data; the SCM is hand-specified, so the benchmark tests reasoning over a known abstraction rather than causal discovery under full ambiguity (Cunha et al., 7 May 2026). Second, the causal label does not imply uniform empirical superiority: CGFA-PPO is competitive, but the reported benefit is deck-dependent, and scalar PPO is better on several archetypes (Cunha et al., 7 May 2026). Third, transfer remains unresolved: the reported leave-one-out gaps are negative for both PPO and CGFA-PPO, and the benchmark treats that as a diagnostic result rather than a solved generalization story (Cunha et al., 7 May 2026).

Those constraints are integral to what MTG-Causal-RL is meant to measure. It is not a claim that Magic strategies can be reduced to six factors, nor a claim that causal agents should automatically dominate associative baselines. Rather, it is a benchmark in which masked-action control, partial observability, SCM-grounded intervention queries, and auditability of strategic credit are exposed together. In that sense, it provides a shared testbed for causal RL, world-model methods, and LLM-agent research on questions that simpler environments do not pose in a single setting (Cunha et al., 7 May 2026).