Equilibrium Residuals Expose Three Regimes of Matrix-Game Strategic Reasoning in Language Models

Abstract: LLMs can score well on named game-theory benchmarks while failing on the same strategic computation once semantic cues are removed. We show this gap with procedurally generated zero-sum matrix games: a model that recognizes familiar games drops to 34%, 18%, and 2% success on anonymous $2{\times}2$, $3{\times}3$, and $5{\times}5$ payoff matrices. The benchmark separates semantic recall, learned approximate Nash computation, and an output-interface bottleneck that limits scale. Training only on $2{\times}2$ and $3{\times}3$ games, supervised fine-tuning raises unseen $5{\times}5$--$7{\times}7$ success from 2% to 61%, while exploitability-reward training averages 37% with high seed variance. We prove that the exploitability residual is $2$-Lipschitz in payoff perturbations, unlike discontinuous vertex-returning LP equilibrium selectors, explaining why residual training can transfer under payoff shifts even when formatting instability limits mean performance. A dominated-action padding experiment provides causal evidence: trained models solve $3{\times}3$ games embedded in much larger matrices, while random-padded controls fail and dense $12{\times}12$ games remain near failure. Procedural evaluation is therefore necessary for measuring strategic reasoning, and residual rewards expose a real but format-limited route to approximate equilibrium computation.

• The paper reveals three distinct strategic reasoning regimes in LLMs by analyzing equilibrium residuals in zero-sum matrix games.
• The paper compares supervised fine-tuning and RL-based VERGE, showing that both improve performance on small games but fail on larger matrices due to interface constraints.
• The paper establishes theoretical insights such as the Lipschitz continuity of exploitability, highlighting serialization bottlenecks that limit scalable strategic reasoning in LLMs.

Equilibrium Residuals Reveal Regimes of Strategic Reasoning in LLMs

Overview

The paper "Equilibrium Residuals Expose Three Regimes of Matrix-Game Strategic Reasoning in LLMs" (2605.10410) presents a rigorous analysis of LLMs' (LLMs) capacity for strategic reasoning in zero-sum matrix games. Challenging prior benchmark-driven claims of game-theoretic competence, the study demonstrates a pronounced gap between semantic recall and algorithmic computation in LLMs. Through a procedural game framework, the authors dissect how current models operate in three distinct regimes, expose their limitations in generalization and scalability, and provide theoretical insights into the stability of residual-based optimization versus solver imitation.

Methodology and Experimental Setup

The authors investigate LLM reasoning in matrix games by constructing a strictly procedural evaluation: rather than relying on 'named' textbook games commonly found in LLM training corpora, they generate random payoff matrices at each training and test instance, precluding memorization-based solutions. Two principal training approaches are compared:

• Supervised Fine-Tuning (SFT): Models are fine-tuned on oracle (LP solver-labeled) pairs of payoff matrices and their Nash equilibria.
• VERGE (Verifiable Equilibrium-Regret GRPO): Models receive only a scalar 'exploitability' reward denoting proximity to Nash equilibrium but no explicit label, employing GRPO-based RL with exploitability as the optimization objective.

Both approaches are evaluated on the Qwen3.5-9B model, using LoRA-based adaptation, under strict JSON-formatted output constraints. The primary metric is $[email protected]$—the proportion of games where the best-of-4 model outputs yield normalized exploitability below $0.10$.

Key Findings: Three Regimes of Strategic Reasoning

The study identifies three sharply delineated regimes of strategic reasoning emerging from the data (Figure 1):

Figure 1: Three regimes of matrix-game strategic reasoning in LLMs. (a) Base model excels on named games but fails with random matrices (memorization gap). (b) SFT and VERGE generalize to larger, out-of-distribution games. (c) $3{\times}3$ games embedded in $12{\times}12$ matrices succeed, while dense or randomly-padded $12{\times}12$ games fail, revealing a serialization bottleneck.

Regime I: Semantic Lookup

Base LLMs perform at near-perfect levels on textbook game names but their accuracy collapses to 34%, 18%, and 2% on randomly generated $2{\times}2$, $3{\times}3$, and $5{\times}5$ games, respectively. Few-shot prompting using solved examples further degrades performance, indicating that LLMs rely on template-matching rather than actual algorithmic strategy computation in such settings.

Regime II: Learned Approximate Computation

Models fine-tuned via SFT or VERGE on $2{\times}2$ and $3{\times}3$ games can generalize to much larger instances. SFT achieves $0.10$0 on $0.10$1 and $0.10$2 on $0.10$3, compared to the base model's $0.10$4 and $0.10$5 on the same. VERGE attains $0.10$6 and $0.10$7 (with higher seed variance), highlighting the feasibility—but instability—of training with only exploitability rewards. These solutions exhibit key game-theoretic invariances (permutation and payoff shift/scale equivariance errors $0.10$8), supporting claims of non-trivial learned computation.

Regime III: Output-Interface Failure (Depth/Serialization Bottleneck)

Substantial performance degradation is observed for $0.10$9 (matrix dimension), where both SFT and VERGE exhibit near-total failure. Controlled experiments, including dominated-action padding—embedding a $3{\times}3$0 solvable game within a larger $3{\times}3$1 matrix by appending iteratively dominated actions—demonstrate that output length alone does not explain this cliff: success persists for padded $3{\times}3$2 games but not for genuine $3{\times}3$3 dense or random-padded games (Figure 2).

Figure 2: Dominated-padding experiment: $3{\times}3$4 as a function of padded matrix size $3{\times}3$5 for $3{\times}3$6 to $3{\times}3$7 embeddings. Dominated embeddings remain far above both controls through $3{\times}3$8.

Theoretical Contributions

The authors prove that the exploitability residual (used in VERGE) is $3{\times}3$9-Lipschitz in the payoff matrix (Theorem 1): for any fixed mixed strategies, small perturbations in the payoff matrix result in commensurately small changes in exploitability. In contrast, LP-based equilibrium selectors can be discontinuous at support boundaries—infinitesimal payoff changes can yield $12{\times}12$0 jumps in selected equilibria. This suggests a formal underpinning for observed stability differences in transfer: residual-based RL can facilitate smoother policy adaptations under distribution shift, while SFT is tightly coupled to the training distribution.

Additionally, the paper rigorously demonstrates that symmetric zero-sum self-play using role-merged GRPO yields exactly zero advantage-weighted gradient signal, causing failure in policy improvement. This motivates the cooperative exploitability setup in VERGE.

Empirical Analysis and Comparative Benchmarks

Empirical evidence substantiates the theoretical findings:

• On OOD $12{\times}12$1 games, SFT and VERGE outperform strong pure-strategy and uniform-mixing baselines.
• VERGE, despite lacking equilibrium labels, achieves strong select-seed transfer on Gaussian-distributed payoffs (up to $12{\times}12$2 on $12{\times}12$3 Gaussian), though with significant checkpoint variance.
• Uniform strategies become optimal for very large, unstructured random matrices, limiting the advantage of learned models to $12{\times}12$4--$12{\times}12$5 for most practical distributions.
• Output format accuracy (valid JSON, proper simplex) is a crucial bottleneck at scale; SFT preserves higher output validity over VERGE in out-of-distribution large games.

Implications and Prospective Directions

Practical Implications

The results provide a diagnostic blueprint for evaluating LLMs' capacity for true algorithmic reasoning, independent of semantic leakage from the training corpus. The separation between algorithmic computation and serialization/output constraints indicates that, for deployment in structured decision-support or agent-planning contexts, improving LLM output interfaces and generalized serialization mechanisms is as crucial as enhancing policy computation itself.

Theoretical Implications

The Lipschitz continuity of exploitability offers a solid foundation for understanding which optimization signals are robust under OOD conditions in RL for discrete structured tasks. However, selector instability remains an inherent hurdle in achieving stable generalization via imitation-based objectives.

Speculations on Future AI Developments

• Techniques that combine procedural evaluation, residual-based optimization, and modular output interfaces will likely be necessary for scalable, robust strategic reasoning in future LLMs.
• Full fine-tuning, structured or constrained decoding, and more stable RL-from-verifier strategies may overcome the depth/serialization bottleneck evidenced herein.
• Generalization to extensive-form or general-sum games, and rigorous behavioral auditing through proceduralization, could become standard evaluation components for advanced LLM agents.

Conclusion

This work advances the understanding of the contours of strategic reasoning in LLMs by delineating three operational regimes across memorization, computation, and output scaffolding. The findings clarify that current LLMs' strategic performance is chiefly an artifact of semantic lookup unless explicitly trained with procedural data, and that residual-driven RL offers principled but format-limited improvements. Eliminating interface bottlenecks and expanding mechanistic understanding of RL for algorithmic tasks remain priority areas for further research (2605.10410).