Program Equilibria: Code-Based Strategic Interaction

• Program equilibria are a generalization of Nash equilibrium where strategies are computer programs that inspect opponents’ source code to decide actions.
• Key methods include proof-based and simulation-based bots, ensuring cooperation through recursive reasoning and probabilistic decision rules.
• SDP-based techniques and MPEC frameworks provide tractable solutions for optimizing equilibria in both two-player dilemmas and multi-agent systems.

Program equilibria generalize Nash equilibrium to settings in which strategies are themselves computer programs capable of inspecting and simulating the opponents’ source code prior to selecting actions. This paradigm models interactions among mutually transparent agents, including AI systems and contractual institutions. Major frameworks include the study of “program games” for classical dilemmas such as the Prisoner’s Dilemma, multi-agent generalizations, simulation-based approaches for robustness, and mathematical programs with equilibrium constraints (MPECs) for optimization problems. Conceptual advances focus on guaranteeing robust cooperative outcomes, characterizing equilibria under informational and computational constraints, and providing SDP-based solution methods for polynomial program games.

1. Formal Definitions and Program Game Structures

In a two-player program game for the Prisoner’s Dilemma, each player submits a program $p_i \in \mathit{PROG}_i$, which, given read access to the opponent’s source code $p_{-i}$, outputs either $C$ (cooperate) or $D$ (defect) (Oesterheld, 2022). These programs are assumed (almost surely) to halt; otherwise, non-termination is interpreted as $D$. The payoff is given by $u_i(p_1, p_2) = u_i(a_1, a_2)$, where $a_i = p_i(p_{-i})$ and $u_i$ is the standard payoff function with $T > R > P > S$.

Generalizing to $n$ players, let $p_{-i}$0 be an $p_{-i}$1-player normal-form game with finite actions. A program $p_{-i}$2 for player $p_{-i}$3 is a function

$p_{-i}$4

where $p_{-i}$5 denotes a random bitstream (sources of randomness may be independent or shared among players for coordination) (Cooper et al., 2024). Program equilibria are profiles $p_{-i}$6 such that no player benefits from unilaterally switching to a different halting program.

2. Types of Program Equilibria: Fair Bots and Simulation-Based Strategies

The literature distinguishes proof-based bots (which inspect opponent code for provable cooperation), simulation-based bots (which actively run opponent programs on simulated inputs), and hybrids. Two principal constructions are:

• $p_{-i}$7-Grounded Fair Bot ($p_{-i}$8GFB): On input opponent $p_{-i}$9, with probability $C$0 return $C$1, otherwise invoke and copy $C$2GFB$C$3 (Oesterheld, 2022). This introduces randomization and ensures almost sure halting.
• Proof-Based Fair Bot (PFB): On input opponent $C$4, return $C$5 if and only if $C$6(PFB) $C$7 is provable within Peano arithmetic ($C$8), else return $C$9 (Oesterheld, 2022).

Simulation-based strategies (e.g., $D$0-Grounded$D$1Bot) generalize $D$2GFB: players simulate opponents on truncated random streams, halting with probability $D$3, and select actions via policies $D$4 based on accumulated simulated histories (Cooper et al., 2024). These are robust to code obfuscation and enable extension to the multi-player case.

3. Existence and Characterization of Cooperative Program Equilibria

For the two-player Prisoner’s Dilemma, several robust cooperative Nash equilibria have been formally established:

• $D$5 is a Nash equilibrium yielding $D$6, as the recursion ensures mutual cooperation almost surely.
• $D$7 constitutes an equilibrium yielding $D$8, established via Löb’s theorem: $D$9 implies $D$0 (Oesterheld, 2022).
• $D$1 also achieves cooperation, as $D$2 can provably infer the cooperative tendency embedded in $D$3.

These constructions are compatible, supporting families of robust cooperative equilibria that tolerate syntactic and algorithmic variation.

In $D$4-player settings, simulation-based program equilibria are characterized as follows:

• With shared randomness (correlated program game), a folk theorem holds: any feasible and strictly individually rational payoff vector can be supported by a program equilibrium of correlated $D$5-Grounded$D$6Bots. Coordination on triggers and punishment is enabled by shared random cutoffs (Cooper et al., 2024).
• With private randomness (uncorrelated program game), stricter constraints apply. The attainable payoffs are those strictly dominating the best mixtures under undetectable deviations, as codified by a penalty parameter $D$7 that controls the detection trade-off. If utilities are additively separable, the set of feasible equilibria is widened (Cooper et al., 2024).

4. Compatibility, Generalizations, and Limits

Syntactically distinct strategies (randomized grounding vs. proof-search) are shown to be compatible and jointly support cooperative outcomes. Proof-based bots can cooperate with any $D$8GFB program via Löb-style arguments, and multi-parameter mixtures of $D$9GFBs are stable. PrudentBot variants using extended consistency (e.g., $u_i(p_1, p_2) = u_i(a_1, a_2)$0), and probabilistic proof requirements, also integrate into this robust equilibrium family (Oesterheld, 2022).

Key limitations arise in multi-agent settings without shared randomness. The impossibility of full folk theorem enforcement: in the pirates’ dilemma (three players, strictly Pareto-optimal payoffs), cooperation cannot be sustained by simulationist programs because unobservable deviations elude collective punishment (Cooper et al., 2024). Coordination failure in private random times is fundamental.

5. Mathematical Programs with Equilibrium Constraints (MPECs)

A related but distinct instantiation involves optimization under equilibrium or complementarity constraints (MPECs), especially when data is polynomial. Formally, for $u_i(p_1, p_2) = u_i(a_1, a_2)$1, one minimizes $u_i(p_1, p_2) = u_i(a_1, a_2)$2 subject to semialgebraic constraints and complementarity ($u_i(p_1, p_2) = u_i(a_1, a_2)$3). Equilibrium can be expressed via a lower-level value function $u_i(p_1, p_2) = u_i(a_1, a_2)$4 (Jiao et al., 2019).

Global minimizers are found using moment–sum-of-squares (SOS) hierarchies, solved by semidefinite programming (SDP). Each relaxation $u_i(p_1, p_2) = u_i(a_1, a_2)$5 corresponds to a truncated moment matrix and localizing matrices indexed by degree. The sequence of SDP solutions converges monotonically to the global minimum under Archimedean (compactness) conditions. Experiments demonstrate computational feasibility for small examples; matrix size is the main bottleneck (Jiao et al., 2019).

Program Type	Halting Guarantee	Syntactic Robustness	Multi-Player Generality
Proof-Based Fair Bot (PFB)	Yes	Low–Medium	2-player
$u_i(p_1, p_2) = u_i(a_1, a_2)$6-Grounded Fair Bot	Yes	High	2-player (original)
$u_i(p_1, p_2) = u_i(a_1, a_2)$7-Grounded$u_i(p_1, p_2) = u_i(a_1, a_2)$8Bot	Yes	High	$u_i(p_1, p_2) = u_i(a_1, a_2)$9-player (Cooper et al., 2024)

6. Illustrative Examples and Applications

In the canonical one-shot Prisoner’s Dilemma ($a_i = p_i(p_{-i})$0), correlated $a_i = p_i(p_{-i})$1-Grounded$a_i = p_i(p_{-i})$2Bots with grim-trigger policies support mutual cooperation for small $a_i = p_i(p_{-i})$3; a single deviation triggers punishment with probability $a_i = p_i(p_{-i})$4, implying approximately optimal payoffs $a_i = p_i(p_{-i})$5 (Cooper et al., 2024).

The trust game demonstrates mixed-strategy equilibrium with uncorrelated $a_i = p_i(p_{-i})$6-Grounded$a_i = p_i(p_{-i})$7Bots: mixing greedy and charitable responses enforces a stable equilibrium indistinguishable from true mixed strategies, with detection and punishment dependent on random stopping (Cooper et al., 2024).

Polynomial MPECs are solved by SDP relaxations, as evidenced by small-scale benchmarks in (Jiao et al., 2019). Realistic use is currently limited to problems with $a_i = p_i(p_{-i})$8 and relaxation order $a_i = p_i(p_{-i})$9.

7. Implications for AI, Multi-Agent Systems, and Economic Mechanisms

The program equilibrium paradigm expands the set of attainable cooperative solutions in strategic settings once agents have transparent access to each other’s source code. Randomization, proof-based reasoning, and simulation protocols are mutually compatible, suggesting robust frameworks for multi-agent trust and commitment without requiring identical implementations. The necessity of shared randomness for full coordination in $u_i$0-player games is fundamental. SDP-based techniques for polynomial games with equilibrium and complementarity constraints provide tractable global optimization methods for small to medium scale problems. These developments underpin advances in programmatic contract design, AI alignment, and mechanism design in transparent agent environments (Oesterheld, 2022, Cooper et al., 2024, Jiao et al., 2019).