Subgame Perfect Equilibrium (SPE)

• Subgame Perfect Equilibrium (SPE) is a refinement of Nash equilibrium that requires strategies to be optimal in every subgame of a sequential game.
• It eliminates non-credible threats and underpins solution concepts in dynamic, repeated, and extensive-form games across economics, control, and multiagent systems.
• Algorithmic methods compute SPE payoffs via iterative hypercube partitioning, linear/mixed-integer programming, and extraction of finite automata strategies.

A subgame perfect equilibrium (SPE) is a refinement of Nash equilibrium in sequential games, requiring that strategies constitute a Nash equilibrium in every subgame of the original game. SPE eliminates non-credible threats and ensures sequential rationality: after every history—whether on or off the equilibrium path—no player can profit by deviating, given the strategies of others. SPE underpins the solution theory of repeated, extensive-form, and dynamic games, with foundational practical and algorithmic significance across economics, formal methods, and control.

1. Formal Definition and Conceptual Foundations

Let $\Gamma$ denote an extensive-form or dynamic game, with histories denoted $h$ and subgames $\Gamma|_h$. A strategy profile $\sigma$ is a subgame perfect equilibrium if for every history $h$, the restriction $\sigma|_h$ (the continuation strategies from $h$ onward) forms a Nash equilibrium in the subgame $\Gamma|_h$:

$$\forall h, \ \forall i,\ \forall \sigma'_i, \quad u_i(\sigma|_h) \geq u_i((\sigma'_i, \sigma_{-i})|_h)$$

where $u_i$ denotes player $i$'s expected payoff. This local equilibrium condition must hold in every reachable or non-reachable (off-path) subgame.

SPE is particularly significant in infinite-horizon and repeated games, extensive-form (with perfect or imperfect information), and games with complex objectives such as mean-payoff or $\omega$-regular objectives. It is strictly stronger than Nash equilibrium, as every SPE is a Nash equilibrium, but not conversely.

2. Algorithmic Approaches for SPE Computation in Discounted Repeated Games

A central algorithmic paradigm for approximating the set of SPE payoffs in infinite-horizon discounted repeated games involves iterative partitioning of the space of candidate payoffs via hypercube covering and verification by mathematical programming (Burkov et al., 2010):

• Initialization: Start with a large hypercube containing all possible payoff profiles, e.g., with side length $l = \bar{r} - \underline{r}$, where $\underline{r} = \min_{a, i} r_i(a)$ and $\bar{r} = \max_{a, i} r_i(a)$.
• Iterative refinement: At each iteration, for each hypercube $c$, verify via a mathematical program (a linear program for pure strategies, or a mixed-integer program for mixed strategies) whether $c$ may contain an SPE payoff. The essential verification conditions are:

$w' = (1-\gamma) r(\alpha) + \gamma w$

$$(1-\gamma) r_i(\alpha) + \gamma w_i \geq (1-\gamma) r_i(BR_i(\alpha), \alpha_{-i}) + \gamma \underline{w}_i, \ \forall i$$

If no such $(\alpha, w)$ can be found, the hypercube is removed; otherwise, it may be subdivided for further refinement.

• Convergence: Continue until all retained hypercubes have side length at most $\epsilon$, yielding an $\epsilon$-approximation of the SPE payoff set.

Three algorithmic variants address different support sets and assumptions regarding randomization:

• Pure strategy approximation: Enumerate over pure actions, use linear feasibility checks.
• Mixed strategy without coordination: Employ mixed-integer programming to jointly select the support of mixed strategies; some payoffs may be missed due to lack of coordination.
• Mixed strategy with public correlation: Assume a public randomization device; convexify the continuation payoffs, ensuring the full SPE set is captured.

3. Extraction and Implementation of Strategies via Finite Automata

Upon termination, the union of surviving hypercubes provides an $\epsilon$-approximation of the SPE payoff set. Strategies can be extracted and represented as finite automata:

• State encoding: Each hypercube corresponds to a finite automaton state; continuation payoffs within a hypercube are approximately identical.
• Action and transition: In each state (hypercube), the prescribed (possibly mixed) action is played, and the next state is selected according to the realized outcome (and public signal, if correlation is used).
• Guarantees: Each induced automaton strategy ensures approximate adherence to equilibrium conditions: no player can gain more than $\epsilon$ by deviating unilaterally.
• Scalability: The memory requirements for automaton-based strategies scale with the granularity of the hypercube partition ($O(\epsilon^{-|N|})$ for $|N|$ players).

This automaton-based encoding is especially suitable for artificial agent systems with finite memory and computation constraints, providing verifiable and real-time executable strategies.

4. Mathematical Foundations and Program Formulations

The algorithmic approach leverages the recursive structure of discounted repeated games:

$$u_i^\gamma(\sigma) = (1-\gamma) r_i(a^0(\sigma)) + \gamma u_i^\gamma(\sigma|_{a^0(\sigma)})$$

This motivates approximation schemes utilizing continuation payoffs and feasibility conditions (above) that enforce incentive compatibility recursively.

Key mathematical program formulations differ for pure and mixed strategies:

• Pure strategy (LP): Variables are continuation values $w$; linear constraints ensure incentive compatibility.
• Mixed strategy (MIP): Variables include action selection probabilities and binary support indicators $y_i^a$; constraints must link support, payoffs, and ensure payoffs for off-support actions are appropriately penalized.
• Public correlation (convexification): Augment with variables to express convex combinations and constrain continuation vectors to the convex hull of feasible payoffs.

This iterative feasibility-checking process can be framed as solving a succession of linear or mixed-integer programs, whose computational complexity depends on action cardinality and desired approximation precision.

5. Practical Applications and Trade-offs

The presented approximation technique and automata extraction enable:

• Design of robust equilibrium strategies in repeated games where agents have limited computation or recall.
• Behavior synthesis for multiagent systems requiring verifiable and finite-memory (Moore machine) implementations.
• Tunable accuracy and resource constraints: The $\epsilon$ parameter directly controls both the fidelity of the equilibrium and the size of the synthesis (number of automaton states).
• Algorithmic flexibility: The three formulations permit adaptation based on scenario—pure actions for simple settings, public correlation for richer payoff sets, and scalability trade-offs between coverage and computation.

The method generalizes to settings with arbitrary stage-game payoff structure and is particularly effective for controller synthesis and verification applications where subgame perfection is vital.

6. Limitations and Implementation Considerations

• Computational demands: As the approximation parameter $\epsilon$ decreases or the number of players/actions increases, the number of hypercubes and corresponding automaton states grows exponentially.
• Completeness: Only the public correlation (external coordination) formulation captures the entire set of mixed strategy SPE payoffs; the pure and uncoordinated mixed variants under-approximate the set in general.
• Precision vs. tractability: Smaller $\epsilon$ improves approximation but increases computational cost. Practical implementations may trade off accuracy for scalability, especially in large many-agent systems.
• Deployment: For real-world multiagent systems, extracted automata can be integrated into control software as finite-state machines. Verification of equilibrium properties reduces to checking properties of the automaton’s transition graph.

These considerations inform the choice among algorithmic formulations in practical deployments, depending on available coordination mechanisms, the admissible complexity for automata, and required equilibrium precision.

---

This technical framework for approximating subgame perfect equilibria in discounted repeated games, built on hypercube partitioning, programmatic verification, and finite automaton extraction, defines both the theoretical and practical landscape for deploying SPE-compliant strategies in artificial and engineered systems (Burkov et al., 2010).