The Game Changer Problem: Controlling Equilibria with Discrete Rewards

Abstract: We introduce the game changer problem, where an external designer modifies a game's reward matrix to make a target pure action profile the unique equilibrium, subject to the constraint that all entries of the reward matrix come from a finite set. We give simple feasibility characterizations for two-player zero-sum games and general-sum games, and the discrete reward structure yields exact optimality and enables efficient dynamic programming algorithms, providing a sharper alternative to prior continuous reward redesign formulations based on linear programming.

• The paper introduces a novel discrete reward design method that enforces unique pure-strategy equilibrium in both zero-sum (requiring |Ω| ≥ 3) and general-sum games (|Ω| ≥ 2).
• It details efficient algorithms, including dynamic programming and greedy rounding, that overcome the limitations of continuous payoff designs and optimize modification costs.
• The study highlights practical applications in digital reward systems, robust policy design in multi-agent reinforcement learning, and potential risks from adversarial reward poisoning.

Discrete Reward Design for Unique Equilibrium Control in Strategic Games

Problem Statement and Motivation

The paper formulates the "game changer problem," a reward redesign task where an external agent adjusts a game's reward matrix using only values from a finite set, with the explicit goal of enforcing a unique target pure-strategy equilibrium. Unlike classical mechanism design, which often centers on continuous payoff spaces and may rely on indifference conditions or transfers, this formulation operates under hard constraints of reward discreteness and focuses on controlling strategic behavior through the reward matrix alone. The motivation is both practical—mirroring real-world reward systems such as discrete grades, fixed payments, or limited rating levels—and algorithmic, given that discrete sets admit exact strict inequalities and efficient algorithmic solutions.

Formalization and Feasibility Results

The core optimization problem involves three criteria: (1) maximizing behavioral predictability (ideally, a unique pure Nash or dominant-strategy equilibrium), (2) maximizing the extrinsic value to the game designer assigned to the induced joint action, and (3) minimizing the modification cost (such as the L1 norm from the original game matrix), all under the constraint that every new reward matrix entry comes from the set $\Omega$.

The main feasibility results are:

• Zero-sum games: Installing a unique strict pure Nash equilibrium requires $|\Omega| \geq 3$.
• General-sum games: Installing a strict dominant strategy equilibrium requires $|\Omega| \geq 2$.

Both criteria are shown to be tight; that is, for zero-sum games with only two available reward levels, the necessary ordering constraints for a strict Nash equilibrium cannot be satisfied due to a lack of sufficient separation.

Algorithmic Contributions

The paper departs from traditional linear or integer programming approaches, which are often inexact when dealing with strict inequalities and finite sets, and demonstrates that the discrete reward structure enables efficient and exact dynamic programming solutions.

Zero-sum Algorithm

For two-player zero-sum games, the paper provides an $O((m+|A|)\log m)$ algorithm, where $m=|\Omega|$ and $|A|$ is the action space size, to optimally reconstruct a reward matrix installing the desired action profile as the unique equilibrium. This procedure systematically explores possible reward assignments (i.e., anchor points for the desired equilibrium payoff), then efficiently fills in the off-diagonal rewards using greedy rounding and adjustment procedures based on the ordering structure of $\Omega$.

A salient property of this procedure is that strict Nash equilibria coincide with uniqueness of all standard equilibrium concepts in finite zero-sum games, including correlated and coarse correlated equilibria, thus strengthening the practical control that the designer can exert.

General-sum Games

In general-sum games, the algorithm scales to $n$ players and seeks to install a strict dominant strategy equilibrium. The solution leverages strong separability: for each player and fixed actions of others, the appropriate rewards are selected from $\Omega$ via efficient rounding and candidate list optimization using the $ComputeInducedCost$ subroutine. The process is exponentially complex in the number of players, as necessitated by the exponential size of the action space, but each subproblem is solved with low-order polynomial time.

Both algorithms avoid the pitfalls of strict inequality relaxations necessitated by continuous LP/IP-based frameworks, yielding lexicographically minimal deviations from the original game.

Experimental Evidence

The paper provides empirical results substantiating the correctness and practical efficiency of the proposed algorithms. In synthetic experiments, the algorithms consistently install the intended unique Nash or dominant-strategy equilibrium and achieve modification costs commensurate with the exact solutions returned by integer programming approaches (up to float precision). Scaling experiments validate the claimed complexity: runtime is linear in action space size and logarithmic in $|\Omega| \geq 3$0 when the latter dominates.

Theoretical, Practical, and Security Implications

The theoretical advancement is the exact characterization and efficient solvability of equilibrium control under discrete rewards—bridging a gap between idealized, often unrealistic continuous reward design and real-world, integral-valued reward setups.

Practically, the results have implications for both benign and adversarial reward designers. A system administrator or policy designer can optimally and provably install a unique and predictable joint behavior using a feasible and minimal intervention within realistic reward constraints. Symmetrically, the paper highlights security risks: an adversarial party can use these tools to subtly yet deterministically coerce agent behaviors (so-called "reward poisoning"). This quantifies the threat landscape and provides guidance for robust reward monitoring: increasing the cost of undetected reward modification can directly limit the adversary's influence.

The algorithms naturally extend to Markov games and applications in offline or adversarial multi-agent reinforcement learning, opening avenues for robust policy teaching, attack, and defense in practical multi-agent RL systems. The explicit focus on discrete rewards makes these results particularly germane to digital platforms and economic settings with quantized incentives.

Future Directions

Potential directions include characterizing the optimal trade-offs between designer value, unpredictability, and distance to original rewards under more complex solution concepts (e.g., mixed-strategy equilibria, Bayesian equilibria), extending to sequential decision settings, and integrating detection constraints informed by statistical anomalies in observed reward modifications. The structure of the provided algorithms may form a foundation for scalable reward engineering in large-scale reinforcement learning and economic mechanisms, particularly where trustee transparency or auditability is required.

Conclusion

The paper precisely characterizes and efficiently solves the problem of installing unique predictable equilibria in finite games through reward matrix redesign constrained to discrete sets. These contributions sharpen the mechanism design and reward engineering toolbox for both optimally controlling agent behavior under practical constraints and elucidating the attack surfaces for reward-based manipulation in strategic and reinforcement learning environments (2606.29012).