Characterisation of reactive Nash equilibria in repeated additive games

Abstract: In this paper, we study reactive strategies in repeated additive games between two players with finitely many actions. Reactive strategies condition only on the opponent's previous action, making them one of the simplest ways players can respond to past interactions. Additive games include important models of cooperation, such as the donation game and games with a punishment option. We show that, for this class of games and strategies, the conditions for symmetric Nash equilibria reduce to a system of linear equalities and inequalities in the strategy parameters, allowing us to characterise all such equilibria. We establish a one-to-one correspondence between non-empty subsets S of the action set and equilibrium classes, which we call S-supporting equilibria. These are equilibria that use exactly the actions in S when playing against themselves. As a special case, we recover the well-known equalizer strategies as the equilibria supported on the entire action set. To assess which equilibrium classes are most evolutionarily relevant, we complement our analytical characterisation with simulations of social learning dynamics. We find that their prevalence is determined by two factors: how likely they are to be generated and how robust they are against invasion.

• The paper introduces explicit algebraic conditions to characterize symmetric Nash equilibria for memory-1 reactive strategies in additive games.
• It presents a convex decomposition method for self-payoff computation that dramatically reduces complexity compared to full Markov transition analyses.
• Simulations show that equilibria with small action supports exhibit greater evolutionary robustness and lower invasibility in dynamic settings.

Characterization of Reactive Nash Equilibria in Repeated Additive Games

Introduction and Motivation

The paper "Characterisation of reactive Nash equilibria in repeated additive games" (2606.27653) delivers a rigorous characterisation of symmetric Nash equilibria within the scope of reactive strategies in repeated two-player additive games with a finite set of actions. Reactive strategies are memory-1 strategies that condition solely on the opponent's previous action, and additive games have stage payoffs decomposable into summands each depending only on one player's action. This framework generalizes classic iterated contexts such as the donation game and scenarios incorporating punishment mechanics.

Prior research has shown that the verification of Nash equilibrium for memory-1 strategies is tractable due to the existence of best responses among pure strategies. However, explicit characterization in multi-action settings has remained technically challenging and typically confined to two-action cases or limited subclasses. This paper extends such characterization to arbitrary finite action sets by synthesizing analytical reductions with actionable algebraic conditions.

Equilibrium Classification: $S$-Supporting Equilibria

The principal result is the explicit algebraic characterization of symmetric Nash equilibria among reactive strategies in additive games. For any reactive strategy $p$ with action set $A$, there exists a one-to-one mapping between non-empty subsets $S \subseteq A$ and classes of equilibria termed $S$-supporting equilibria. Such equilibria only employ actions in $S$ when played against themselves, and the Nash conditions manifest as a system of linear equalities and inequalities on strategy parameters:

• The stationary distribution is supported entirely on $S \times S$.
• For any $a_1, a_2 \in S$, the payoffs $\pi^i(Alla_1, p)$ and $\pi^i(Alla_2, p)$ are equal.
• Any action $p$0 yields strictly lower or equal payoffs: $p$1 for all $p$2.

This structural partitioning implies the equilibrium set is indexed by the power set of $p$3, with each class corresponding to a distinct subset. Notably, when $p$4, the class coincides with equalizer strategies, a prominent subclass in evolutionary game theory.

Payoff Computation and Self-Payoff Representation

A critical technical innovation is the derived convex decomposition of a strategy's self-payoff. The self-payoff $p$5 can be computed as a convex combination of payoffs against pure unconditional strategies, weighted by entries in the stationary distribution. This eliminates explicit calculation of stationary distributions in most cases, vastly reducing computational complexity and enabling tractable enumeration of equilibrium classes. The conditions for equilibrium can be checked without computing $p$6 Markov transition matrices.

Dimensions and Constraints of Equilibrium Classes

The paper quantifies the degrees of freedom in each equilibrium class as a function of the support size $p$7 and total actions $p$8:

Figure 1: Upper bound on the degrees of freedom for $p$9-supporting equilibrium classes in a 10-action game, symmetric around the midpoint, highlighting maximal dimension for equalizers and singleton-support equilibria.

For fixed $A$0, the maximal dimension for a class supported on $A$1 is $A$2. This reveals symmetry around $A$3, establishing that both full-support and singleton-support classes are the most unconstrained.

Evolutionary Robustness and Empirical Simulation

To analyze evolutionary relevance, social learning dynamics are simulated under pairwise comparison processes using a three-action donation game with cooperation ($A$4), moderation ($A$5), and defection ($A$6). Simulations indicate that equilibrium classes supported on small subsets, particularly moderation-supporting or cooperation-supporting equilibria, are more prevalent and robust in dynamic populations. This robustness stems from both their evolutionary stability against invasion and their likelihood of generation during mutation events.

Figure 2: Evolutionary dynamics of reactive strategies, showing that $A$7-supporting equilibria dominate for low $A$8, while $A$9-supporting equilibria prevail for high $S \subseteq A$0, and quantifying class invasibility and sampling probability.

Empirical results show that invasibility and dimensionality jointly determine evolutionary abundance; equilibrium classes of intermediate support rarely arise due to their reduced degrees of freedom, while equalizer classes, despite high dimension, are more susceptible to invasion.

Theoretical and Practical Implications

The characterization provided significantly advances tractable analysis of equilibria in repeated additive games. The algebraic formulation allows explicit enumeration and identification of Nash equilibria for any finite action set, bridging a prior gap in cases exceeding two actions. The partitioning into $S \subseteq A$1-supporting classes informs both theoretical understanding—by revealing equilibrium structure as indexed by action subsets—and practical computation in strategic modeling and evolutionary simulations.

These results are extensible to diverse domains where repeated interactions and strategic memory are pivotal, e.g., cooperative AI systems, mechanism design in multi-agent environments, and evolutionary social dilemmas. The convex representation of self-payoff may inspire analogous frameworks in more general repeated games, potentially enabling new algorithms for equilibrium computation in non-additive or complex memory settings.

Speculation on Future Developments

Further research may generalize this payoff representation beyond reactive strategies, potentially covering memory-$S \subseteq A$2 or state-dependent approaches in non-additive games. This approach could facilitate systematic construction and analysis of equilibrium classes in complex multi-agent or sequential AI settings, and lead to rigorous evolutionary stability analysis for learning-based strategy protocols. The explicit partitioning and algebraic criteria set a foundation for automated equilibrium detection and synthesis in computational game-theoretic modeling.

Conclusion

This paper establishes a rigorous, tractable characterization of reactive Nash equilibria in repeated additive games for arbitrary finite action sets. The findings partition the equilibrium space into disjoint $S \subseteq A$3-supporting classes identified by linear conditions, provide explicit algebraic computation of self-payoff, and clarify evolutionary abundance through simulation and dimensional analysis. The results enrich both theoretical understanding and practical tools for analyzing equilibria in repeated strategic interaction frameworks.