Papers
Topics
Authors
Recent
Search
2000 character limit reached

Equilibrium Refinements Improve Subgame Solving in Imperfect-Information Games

Published 23 Jan 2026 in cs.GT | (2601.17131v1)

Abstract: Subgame solving is a technique for scaling algorithms to large games by locally refining a precomputed blueprint strategy during gameplay. While straightforward in perfect-information games where search starts from the current state, subgame solving in imperfect-information games must account for hidden states and uncertainty about the opponent's past strategy. Gadget games were developed to ensure that the improved subgame strategy is robust against any possible opponent's strategy in a zero-sum game. Gadget games typically contain infinitely many Nash equilibria. We demonstrate that while these equilibria are equivalent in the gadget game, they yield vastly different performance in the full game, even when facing a rational opponent. We propose gadget game sequential equilibria as the preferred solution concept. We introduce modifications to the sequence-form linear program and counterfactual regret minimization that converge to these refined solutions with only mild additional computational cost. Additionally, we provide several new insights into the surprising superiority of the resolving gadget game over the max-margin gadget game. Our experiments compare different Nash equilibria of gadget games in several standard benchmark games, showing that our refined equilibria consistently outperform unrefined Nash equilibria, and can reduce the exploitability of the overall strategy by more than 50%

Citations (1)

Summary

  • The paper introduces the gadget game sequential equilibrium (GGSE) as a more robust solution concept in imperfect-information games, showing that refining traditional Nash equilibria in gadget games can vastly reduce exploitability.
  • GGSE ensures immediate improvements in handling uncertainty in subgames, outperforming traditional methods by consistently reducing exploitability by over 50% in key games like Goofspiel and Leduc hold'em.
  • The study underscores the importance of using the blueprint prior in GGSE, which results in solving algorithms that are not only robust but also computational feasible and practical across different solvers and game settings.

Subgame solving is the dominant technique for scaling game-solving algorithms to large imperfect-information games: a coarse blueprint strategy is precomputed for the entire game, and then, at decision time, a local strategy within the currently reached subgame is computed and substituted into the blueprint. Because the true state of the world is hidden from each player, this substitution must be robust to uncertainty about how the opponent reached the current public state. Gadget games—augmented subgames that let the opponent adversarially select its past reach probabilities—were introduced precisely to provide such safety guarantees. This paper, by Kubíček, Lisý, and Sandholm (2601.17131), identifies a fundamental weakness in how gadget games have been solved to date: they typically admit infinitely many Nash equilibria that are indistinguishable by value inside the gadget yet produce vastly different exploitability when spliced back into the full game. The authors propose a new solution concept tailored to gadget games—the gadget game sequential equilibrium (GGSE)—show that it can be computed with minor modifications to standard solvers, and use it to overturn a long-standing belief about the relative merits of resolving versus max-margin gadget constructions.

The problem with Nash equilibria in gadget games

In common-knowledge subgame solving, the subgame rooted at public state ss is initialized with chance probabilities proportional to the players' blueprint reach probabilities π(h)\pi(h) over histories h∈H(s)h \in H(s). Unsafe solving simply solves this subgame assuming both players followed the blueprint; it often performs well but carries no guarantee. Safe alternatives—resolving, max-margin, and reach max-margin gadget games [cfrd2014; maxmargin2016; reachmaxmargin2017]—add auxiliary opponent information sets so that any Nash equilibrium of the gadget yields a combined strategy whose exploitability is no worse than the blueprint's.

The paper's central observation is that this safety guarantee is far weaker than what one would want. A theorem (inspired by DeepStack) characterizes the exploitability of the resolved strategy through the counterfactual best-response values in the opponent's root information sets S−i(s)S_{-i}(s): if the refined strategy does not decrease these values relative to the blueprint, exploitability cannot increase. Every gadget-game Nash equilibrium satisfies this constraint, but among the equilibria—which are payoff-equivalent inside the gadget—some improve the root counterfactual values substantially while others do not. The authors prove three negative results via explicit constructions:

  • Non-uniqueness matters: there exist games where different Nash equilibria of the same gadget game yield different full-game exploitabilities.
  • Non-existence: no Nash equilibrium of either the resolving or the max-margin gadget may belong to an optimal continuation strategy—the best strategy obtainable by re-solving the subgame while keeping past play fixed. Their example uses sequential rock-paper-scissors with a degenerate blueprint.
  • Max-margin can be worse than resolving: some Nash equilibria of the resolving gadget dominate every Nash equilibrium of the corresponding max-margin gadget.

The intuition behind the last result is that the max-margin gadget lets the opponent choose at its root which information set to enter based on achievable improvement, introducing a blueprint-dependent bias into every equilibrium; the resolving gadget imposes only per-information-set lower bounds and thus preserves a larger set of acceptable solutions. This offers a new explanation for empirical observations by DeepStack and Obscuro that were previously attributed to imperfect value functions in nested solving—and it directly contradicts the widely held view that max-margin strictly dominates resolving.

Gadget game sequential equilibria

To select among gadget equilibria, the paper introduces GGSE, a deliberately weakened analogue of Kreps–Wilson sequential equilibrium [kreps1982sequential]. Standard refinements require trembles at every information set with uncorrelated mistakes, which is numerically fragile. In a gadget game, however, the divergence between the gadget and the original game lies solely in the auxiliary information sets, and each trajectory passes through at most one auxiliary decision point. GGSE therefore requires only that every action in the auxiliary information sets be played with probability exceeding ϵ⋅μ(σ−i,a−i)\epsilon \cdot \mu(\sigma_{-i}, a_{-i}) as ϵ→0+\epsilon \to 0^+, where μ\mu is a prior over opponent reaches.

Two properties make GGSE attractive. First, it is computationally benign: since trembles are confined to auxiliary sets and can be scaled by a prior, the resulting perturbed problems remain well-conditioned. Second, the authors prove that searching only over GGSEs is lossless: for any gadget Nash equilibrium, there exists a GGSE whose per-root-counterfactual-best-response values are pointwise no lower. Consequently, restricting attention to GGSE never excludes a lower-exploitability solution.

The choice of prior matters. The ideal target is the optimal continuation strategy, but computing it naively requires fixing player ii's strategy outside the subgame and re-solving the whole game; the "full gadget" alternative needs a value function whose size grows linearly with game length. The paper proposes using the opponent's part of the blueprint as the prior—proportional to β2σ(I2)\beta^{\sigma}_{2}(I_2), clipped below at 10−310^{-3}—which retains safety while injecting the blueprint's information about plausible opponent play. This generalizes DeepStack's use of the blueprint as a warm start, which was applied only in the first CFR iteration and therefore did not converge to a refined equilibrium.

Algorithms

Both standard solver families admit lightweight modifications to converge to GGSE.

Sequence-form LP: adding a perturbation vector π(h)\pi(h)0—nonzero only on sequences terminating in auxiliary actions, weighted by the prior—to the objective of Koller–Megiddo–von Stengel LP turns it into an approximate GGSE solver. The construction resembles Farina and Sandholm's one-sided quasi-perfect equilibrium [farina2021osqpe] but differs in two ways: trembles apply only to auxiliary information sets, and minimal action probabilities within a set vary according to the prior rather than being uniform. If the optimum persists at π(h)\pi(h)1, the solution is exact; otherwise π(h)\pi(h)2 is decreased and the program re-solved.

CFR: adopting Farina et al.'s behaviorally-constrained regret minimization [farina2017regret], the authors replace Regret Matching in auxiliary sets by a basis-matrix transform π(h)\pi(h)3, where columns of the symmetric matrix π(h)\pi(h)4 contain prior-scaled action weights. Per-iteration cost is essentially unchanged; convergence may require more iterations depending on π(h)\pi(h)5 and π(h)\pi(h)6, but the experiments indicate no performance downside even at reduced iteration counts.

A caveat noted explicitly: the resulting GGSE need not be consistent with the prior unless the equilibrium itself plays into no part of the subgame. Max-margin solutions are always inconsistent with the prior in this sense; resolving solutions are consistent exactly when all counterfactual values fall below their optima.

Experimental results

Experiments cover Goofspiel 5, Leduc hold'em, and Liar's dice 1,4, using 25 blueprints drawn from CFR checkpoints and 20 Dirichlet-random blueprints per game, with subgames solved by CFR+ and (in the appendix) by SQF. Three priors—none, uniform, and blueprint—are compared against unsafe solving and the optimal continuation strategy. Key findings:

Finding Evidence
Blueprint prior reduces exploitability Consistent gains across all games/solvers; >50% exploitability reduction vs. vanilla SQF/CFR
Resolving + prior beats max-margin Confirmed empirically in Leduc hold'em, refuting universal max-margin superiority
Resolving + blueprint prior ≈ unsafe, but safe Difference analysis shows it always performs at least as well as unsafe resolving, without its risk

Notably, unsafe solving degrades as the blueprint improves, whereas safe resolving with a blueprint prior matches or exceeds it throughout—combining the strengths of both approaches with a formal safety guarantee. Statistical significance is supported by t-tests across technique pairs.

For nested depth-limited solving with multi-valued-state value functions in Goofspiel 7 and Liar's dice 2,4, head-to-head evaluation against each technique's worst matchup confirms that resolving with a blueprint prior is consistently strong across opponents, while max-margin performs poorly in Goofspiel regardless of prior and the ranking reverses in Liar's dice. Convergence curves confirm the main practical cost of the refinement is additional iterations, not degraded accuracy.

Limitations and open questions

Several boundaries on the results deserve emphasis. The theoretical constructions show that neither resolving nor max-margin gadgets can always recover the optimal continuation strategy; developing a gadget variant that does so without the "full gadget"'s linear size growth remains open. The GGSE framework has not been evaluated with the reach max-margin gadget or under knowledge-limited subgame solving, where auxiliary structure differs. Empirical validation covers benchmark games of moderate size; application to Texas hold'em-scale games or dark chess, and to real-world settings, is left untested. Finally, the nested-solving experiments rely on win-rate against worst matchups as only a limited proxy for exploitability, since no technique in those matchups aims to exploit.

Conclusion

This paper establishes that the choice among payoff-equivalent gadget-game Nash equilibria materially affects the quality of subgame solving, and supplies both the right solution concept—GGSE—and practical algorithms (modified SQF and CFR) to compute it at negligible overhead. Its disproof of the presumed dominance of the max-margin gadget reconciles previously anomalous empirical results, and its blueprint-prior formulation delivers unsafe-solving-level performance with formal safety. Together, these results make gadget game sequential equilibrium the preferred solution concept for subgame solving in two-player zero-sum imperfect-information games.

Paper to Video (Beta)

No one has generated a video about this paper yet.

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.