Procedurally Generated Zero-Sum Matrix Games

• Procedurally generated zero-sum matrix games are instances defined by automatic or adaptive payoff-generation mechanisms that challenge traditional equilibrium computation.
• They include models with fixed unknown matrices learned via bandit feedback, query-based reconstruction, and time-varying payoffs that require joint saddle-point analysis.
• The research leverages advanced methods such as UCB, K-learning, and variational quantum approaches while addressing sparse support recovery and the limitations of standard regret minimization.

Searching arXiv for the cited works and closely related zero-sum matrix game papers. Procedurally generated zero-sum matrix games are zero-sum matrix-game instances whose payoff structure is generated automatically, revealed adaptively, or induced endogenously by the learning process rather than being fully specified ex ante. Across the literature, this umbrella includes at least three distinct but related models: repeated play with a fixed but unknown payoff matrix learned from bandit feedback (O'Donoghue et al., 2020); query-based access to an incompletely observed matrix whose equilibrium structure must be reconstructed from sampled entries (Maiti et al., 2023); and time-varying or endogenously evolving payoff matrices, where the relevant equilibrium object is attached either to a larger static reduction or to the long-run average game (Skoulakis et al., 2020, Cardoso et al., 2019). In all of these formulations, the central technical question is how procedural generation or partial revelation alters equilibrium computation, regret analysis, feedback requirements, and the complexity of identifying minimax strategies.

1. Formal models and equilibrium objects

A two-player zero-sum matrix game is specified by a payoff matrix. In the bandit-feedback formulation, the unknown matrix is

$A \in \mathbb{R}^{m \times k},$

where the row player chooses $i \in \{1,\dots,m\}$, the column player chooses $j \in \{1,\dots,k\}$, and the row player pays the column player $A_{ij}$ (O'Donoghue et al., 2020). The value of the game is

$$V_A^\star = \min_{y \in \Delta_m}\max_{x \in \Delta_k} y^\top A x = \max_{x \in \Delta_k}\min_{y \in \Delta_m} y^\top A x,$$

with Nash equilibrium strategies $(x^\star,y^\star)$ solving the saddle-point problem (O'Donoghue et al., 2020).

In the query-complexity formulation, the input matrix is $A \in \mathbb{R}^{n \times n}$, the row player chooses $x \in \Delta_n$, and the column player chooses $y \in \Delta_n$. A pair $(x^\star,y^\star)$ is a Nash equilibrium iff

$i \in \{1,\dots,m\}$0

equivalently,

$i \in \{1,\dots,m\}$1

(Maiti et al., 2023). The paper emphasizes that in zero-sum games, $i \in \{1,\dots,m\}$2 and $i \in \{1,\dots,m\}$3 are convex polytopes (Maiti et al., 2023).

When the payoff matrix evolves over time, the model changes. In Online Matrix Games, the matrix may change on each round: $i \in \{1,\dots,m\}$4 players choose mixed strategies $i \in \{1,\dots,m\}$5 and $i \in \{1,\dots,m\}$6 before seeing $i \in \{1,\dots,m\}$7, and receive payoff $i \in \{1,\dots,m\}$8 (Cardoso et al., 2019). The benchmark is then the saddle point of the cumulative or average game rather than of any single round (Cardoso et al., 2019).

A further extension arises when the game itself is generated endogenously by strategic interaction. In that setting, the changing payoff matrix can be encoded as part of a larger static polymatrix game, with environment variables treated as additional players (Skoulakis et al., 2020). This suggests that “procedural generation” has two mathematically distinct interpretations: exogenous instance generation or revelation, and endogenous generation by the current state of the dynamics.

2. Unknown fixed matrices: bandit feedback and minimax learning

One canonical model of procedurally generated play assumes that a matrix is generated once, fixed throughout the interaction, but initially unknown. In the repeated-learning setting, the matrix is fixed but unknown, and learning occurs from bandit feedback (O'Donoghue et al., 2020). At round $i \in \{1,\dots,m\}$9, the row player chooses $j \in \{1,\dots,k\}$0, the column player chooses $j \in \{1,\dots,k\}$1, and the observed payoff is

$j \in \{1,\dots,k\}$2

where $j \in \{1,\dots,k\}$3 is zero-mean i.i.d. noise from a known distribution (O'Donoghue et al., 2020). Both players observe the opponent’s action and the noisy payoff, and the history before round $j \in \{1,\dots,k\}$4 is

$j \in \{1,\dots,k\}$5

(O'Donoghue et al., 2020).

The regret notion is equilibrium-relative rather than action-relative. For the learner, regret relative to the Nash value is

$j \in \{1,\dots,k\}$6

with Bayesian and worst-case variants defined as

$j \in \{1,\dots,k\}$7

and

$j \in \{1,\dots,k\}$8

(O'Donoghue et al., 2020). This covers both a fixed unknown generated instance and a prior over procedurally generated instances.

The paper studies matrix-aware variants of UCB and K-learning (O'Donoghue et al., 2020). Under the assumptions that $j \in \{1,\dots,k\}$9 is 1-sub-Gaussian and $A_{ij}$0, UCB forms the optimistic matrix

$A_{ij}$1

and plays the minimax policy for $A_{ij}$2: $A_{ij}$3 (O'Donoghue et al., 2020). The regret guarantee is

$A_{ij}$4

for $A_{ij}$5 and $A_{ij}$6, and it holds against any opponent strategy, even an informed best-responder (O'Donoghue et al., 2020).

K-learning is presented as a Bayesian optimistic algorithm based on cumulant generating functions. For each column $A_{ij}$7 of $A_{ij}$8,

$A_{ij}$9

and the algorithm solves

$$V_A^\star = \min_{y \in \Delta_m}\max_{x \in \Delta_k} y^\top A x = \max_{x \in \Delta_k}\min_{y \in \Delta_m} y^\top A x,$$0

then chooses

$$V_A^\star = \min_{y \in \Delta_m}\max_{x \in \Delta_k} y^\top A x = \max_{x \in \Delta_k}\min_{y \in \Delta_m} y^\top A x,$$1

(O'Donoghue et al., 2020). Its Bayes regret satisfies

$$V_A^\star = \min_{y \in \Delta_m}\max_{x \in \Delta_k} y^\top A x = \max_{x \in \Delta_k}\min_{y \in \Delta_m} y^\top A x,$$2

again robust to the opponent’s strategy, including adversarial best-response behavior (O'Donoghue et al., 2020).

The central methodological point is that these methods exploit matrix structure. Standard adversarial bandit algorithms such as Exp3 obtain $$V_A^\star = \min_{y \in \Delta_m}\max_{x \in \Delta_k} y^\top A x = \max_{x \in \Delta_k}\min_{y \in \Delta_m} y^\top A x,$$3-type regret relative to the best action in hindsight, but they do not naturally target the Nash equilibrium and may fail to “solve” the game (O'Donoghue et al., 2020). This distinction is fundamental in procedurally generated settings: the objective is not merely to outperform a realized opponent sequence, but to identify a robust minimax strategy for the underlying generated game.

3. Query-efficient equilibrium recovery and sparse support structure

A different line of work studies procedurally hidden games through query access rather than repeated play. The question is not how to learn from noisy rewards over time, but how many matrix entries must be queried in order to reconstruct the equilibrium set of a partially observed game (Maiti et al., 2023).

The key structural parameters are the unions of equilibrium supports: $$V_A^\star = \min_{y \in \Delta_m}\max_{x \in \Delta_k} y^\top A x = \max_{x \in \Delta_k}\min_{y \in \Delta_m} y^\top A x,$$4 (Maiti et al., 2023). These are explicitly defined as the paper’s notion of solution size (Maiti et al., 2023). Bohnenblust’s support lemma gives

$$V_A^\star = \min_{y \in \Delta_m}\max_{x \in \Delta_k} y^\top A x = \max_{x \in \Delta_k}\min_{y \in \Delta_m} y^\top A x,$$5

where

$$V_A^\star = \min_{y \in \Delta_m}\max_{x \in \Delta_k} y^\top A x = \max_{x \in \Delta_k}\min_{y \in \Delta_m} y^\top A x,$$6

(Maiti et al., 2023). This identifies the rows and columns that participate in some equilibrium support.

The main contribution is a randomized query-efficient algorithm that returns the entire set of Nash equilibria $$V_A^\star = \min_{y \in \Delta_m}\max_{x \in \Delta_k} y^\top A x = \max_{x \in \Delta_k}\min_{y \in \Delta_m} y^\top A x,$$7, not just a single equilibrium (Maiti et al., 2023). Its high-level strategy is to isolate the relevant $$V_A^\star = \min_{y \in \Delta_m}\max_{x \in \Delta_k} y^\top A x = \max_{x \in \Delta_k}\min_{y \in \Delta_m} y^\top A x,$$8 optimal submatrix, lift the problem to a larger combinatorial matrix, reduce the task to finding a strict pure-strategy Nash equilibrium in that lifted matrix, and simulate the needed oracle calls with relatively few queries to the original matrix (Maiti et al., 2023). A crucial subroutine identifies a strict PSNE using only

$$V_A^\star = \min_{y \in \Delta_m}\max_{x \in \Delta_k} y^\top A x = \max_{x \in \Delta_k}\min_{y \in \Delta_m} y^\top A x,$$9

oracle calls, where $(x^\star,y^\star)$0 is the number of rows plus columns in the searched matrix (Maiti et al., 2023).

The resulting guarantee is: $(x^\star,y^\star)$1 entries of $(x^\star,y^\star)$2 and returns $(x^\star,y^\star)$3 (Maiti et al., 2023). If $(x^\star,y^\star)$4 are unknown, trying all support-size pairs contributes another factor $(x^\star,y^\star)$5, yielding the final $(x^\star,y^\star)$6 bound (Maiti et al., 2023).

The lower bound is

$(x^\star,y^\star)$7

expected queries in the worst case, even when

$(x^\star,y^\star)$8

(Maiti et al., 2023). More precisely, there exists an $(x^\star,y^\star)$9 matrix with $A \in \mathbb{R}^{n \times n}$0 and $A \in \mathbb{R}^{n \times n}$1 such that any correct randomized algorithm must query at least

$A \in \mathbb{R}^{n \times n}$2

entries in expectation (Maiti et al., 2023). This shows that sparse equilibrium structure can reduce the effective complexity from $A \in \mathbb{R}^{n \times n}$3 to roughly $A \in \mathbb{R}^{n \times n}$4, but does not remove the need to inspect a substantial fraction of the relevant matrix.

For procedurally generated games, this framework captures cases where a large matrix is generated by a hidden procedure but equilibrium play depends only on a small support union. A plausible implication is that procedural structure is most algorithmically valuable when it induces support sparsity, because that sparsity can be translated directly into lower query complexity.

4. Evolving and endogenous payoff generation

Procedural generation need not mean that a hidden matrix is sampled once. In Online Matrix Games, the matrix itself may change every round, possibly adversarially and even depending on the history of play (Cardoso et al., 2019). The appropriate benchmark is therefore the Nash equilibrium of the long-term averaged payoff matrix,

$A \in \mathbb{R}^{n \times n}$5

or equivalently of the cumulative game $A \in \mathbb{R}^{n \times n}$6 (Cardoso et al., 2019).

The paper defines Nash Equilibrium regret as

$A \in \mathbb{R}^{n \times n}$7

and aims for sublinear growth in $A \in \mathbb{R}^{n \times n}$8 (Cardoso et al., 2019). A key negative result states that no algorithm can simultaneously guarantee vanishing NE regret together with both players’ standard individual no-regret guarantees; minimizing each player’s regret independently is therefore insufficient (Cardoso et al., 2019). This separates joint equilibrium tracking from classical no-regret learning.

The full-information algorithm is Saddle-Point Regularized Follow-the-Leader (SP-RFTL). For convex-concave losses $A \in \mathbb{R}^{n \times n}$9, it forms

$x \in \Delta_n$0

and updates by solving the saddle point of the cumulative regularized game: $x \in \Delta_n$1 (Cardoso et al., 2019). For matrix games on simplices, negative entropy regularizers are used, together with a shrunk simplex

$x \in \Delta_n$2

(Cardoso et al., 2019). The main full-information theorem yields a bound summarized as

$x \in \Delta_n$3

(Cardoso et al., 2019).

In the bandit setting, only the realized entry is observed. The paper introduces the one-point matrix estimator: if $x \in \Delta_n$4 and $x \in \Delta_n$5, define $x \in \Delta_n$6 by

$x \in \Delta_n$7

so that

$x \in \Delta_n$8

(Cardoso et al., 2019). The resulting bandit algorithm achieves

$x \in \Delta_n$9

against an adaptive adversary that does not observe current-round actions when choosing $y \in \Delta_n$0 (Cardoso et al., 2019).

A more endogenous notion of evolving payoff generation appears in coevolutionary models. There, the game changes over time as a function of the agents’ current behavior, and many such systems can be rewritten as a static polymatrix game with extra nodes representing environments (Skoulakis et al., 2020). In the canonical model, the environment state $y \in \Delta_n$1 modifies the payoff matrix seen by the population strategy $y \in \Delta_n$2, with

$y \in \Delta_n$3

and dynamics

$y \in \Delta_n$4

(Skoulakis et al., 2020). The system is shown to be exactly replicator dynamics in a static two-player polymatrix game with

$y \in \Delta_n$5

(Skoulakis et al., 2020). This reduction is the core modeling insight: the evolving payoff matrix is encoded as strategic evolution in a larger static game.

5. Learning dynamics, recurrence, and finite-sample convergence

When the matrix is hidden or generated procedurally, the information structure strongly shapes the available learning guarantees. In the minimally informative repeated-play setting, one recent approach studies best-response type dynamics under two information models: full information and minimal information, also called the radically uncoupled case (Faizal et al., 2024).

The game is a finite two-player zero-sum matrix game with payoff matrices $y \in \Delta_n$6, bounded by

$y \in \Delta_n$7

(Faizal et al., 2024). Suboptimality is measured by the Nash gap

$y \in \Delta_n$8

and $y \in \Delta_n$9 implies that $(x^\star,y^\star)$0 is an $(x^\star,y^\star)$1-Nash equilibrium (Faizal et al., 2024).

In full information, each player observes both payoff matrices and the opponent’s mixed strategy, and the dynamics use the entropy-smoothed best response

$(x^\star,y^\star)$2

with softmax form

$(x^\star,y^\star)$3

(Faizal et al., 2024). The damped smoothed best-response update is

$(x^\star,y^\star)$4

(Faizal et al., 2024).

In minimal information, each player observes only their own realized payoff and own action, not the opponent’s actions and not the payoff matrices (Faizal et al., 2024). The algorithm combines the strategy update

$(x^\star,y^\star)$5

with an importance-weighted TD update

$(x^\star,y^\star)$6

whose expected drift satisfies

$(x^\star,y^\star)$7

(Faizal et al., 2024). The scheme is explicitly two-timescale stochastic approximation.

The finite-sample guarantees are polynomial-time. In the full-information setting, with appropriate smoothing,

$(x^\star,y^\star)$8

for constant stepsize, and

$(x^\star,y^\star)$9

for inverse-linear decay (Faizal et al., 2024). In the minimal-information setting, the theorem gives a polynomial bound of order roughly $i \in \{1,\dots,m\}$00 up to constants and logs; specifically, for constant stepsize,

$i \in \{1,\dots,m\}$01

(Faizal et al., 2024). A notable point is that no additional exploration is required beyond the entropy smoothing itself (Faizal et al., 2024).

The coevolutionary setting produces a different form of long-run regularity. In rescaled zero-sum polymatrix games with an interior Nash equilibrium, replicator dynamics are Poincaré recurrent: for almost all interior initial conditions, trajectories return arbitrarily close to their initial point infinitely often (Skoulakis et al., 2020). The proof uses log-ratio coordinates, volume preservation via

$i \in \{1,\dots,m\}$02

and a conserved quantity

$i \in \{1,\dots,m\}$03

(Skoulakis et al., 2020). The invariant can be rewritten in weighted KL-divergence form,

$i \in \{1,\dots,m\}$04

up to additive constants (Skoulakis et al., 2020). If the rescaled zero-sum polymatrix game has a unique interior Nash equilibrium, then the time averages converge: $i \in \{1,\dots,m\}$05 (Skoulakis et al., 2020). This distinguishes pointwise non-convergence from statistical equilibrium in evolving procedurally generated systems.

6. Structural exploitation, failure modes, and computational frontiers

A recurrent theme is that the success of learning or computation depends on exploiting the specific structure induced by the generation mechanism. In hidden fixed-matrix games, UCB and K-learning exploit the facts that the matrix is fixed, the rows and columns are jointly structured, and the learner’s goal is to compute a minimax policy over a matrix estimate (O'Donoghue et al., 2020). In query models, the exploitable structure is support sparsity across all equilibria (Maiti et al., 2023). In evolving-payoff games, the relevant structure is the aggregate saddle problem of the whole sequence rather than separate per-round losses (Cardoso et al., 2019). In coevolutionary systems, the exploitable structure is a rescaled zero-sum polymatrix representation (Skoulakis et al., 2020).

The literature also highlights failure modes. A key result is that Thompson sampling can fail catastrophically in matrix games with bandit feedback (O'Donoghue et al., 2020). The paper gives the $i \in \{1,\dots,m\}$06 game

$i \in \{1,\dots,m\}$07

and shows that if the true value is $i \in \{1,\dots,m\}$08 and the opponent knows this and plays the Nash equilibrium, the Thompson sampler never resolves the uncertainty about $i \in \{1,\dots,m\}$09, keeps sampling a distribution that sometimes plays the bad action, and incurs constant expected loss (O'Donoghue et al., 2020). The conclusion is that Thompson sampling can have linear regret against an informed opponent because stochastic optimism can be pessimistic on some rounds, allowing exploitation (O'Donoghue et al., 2020).

A second misconception concerns the role of ordinary adversarial bandit or no-regret algorithms. Exp3 may have low regret relative to the best action in hindsight, but it does not naturally target Nash equilibrium and may perform poorly compared with matrix-aware methods (O'Donoghue et al., 2020). Likewise, independently minimizing each player’s individual regret in evolving-payoff games does not control NE regret (Cardoso et al., 2019). These are distinct failures, but both reflect the same principle: equilibrium learning is not reducible to single-agent regret minimization when the benchmark is minimax or saddle-point based.

A newer computational frontier appears in variational quantum methods for two-player zero-sum matrix games (Do et al., 8 Apr 2026). In that work, mixed strategies are parameterized as Born distributions of parameterized quantum circuits, transforming the saddle-point problem into a smooth but generally nonconvex–nonconcave problem in circuit-parameter space (Do et al., 8 Apr 2026). To handle arbitrary game sizes, the method uses a dominated embedding with

$i \in \{1,\dots,m\}$10

and embedded matrix

$i \in \{1,\dots,m\}$11

so that dummy actions are strictly dominated (Do et al., 8 Apr 2026). The projected variational quantum extragradient method uses finite-shot parameter-shift gradient estimates and establishes variance bounds scaling as $i \in \{1,\dots,m\}$12 in the number of measurement shots $i \in \{1,\dots,m\}$13, together with convergence to approximate first-order stationarity under standard assumptions (Do et al., 8 Apr 2026). The paper emphasizes, however, that stationarity does not imply equilibrium optimality, so performance is evaluated by the game-space Nash gap

$i \in \{1,\dots,m\}$14

with

$i \in \{1,\dots,m\}$15

(Do et al., 8 Apr 2026). Numerical results are strong on structured instances up to $i \in \{1,\dots,m\}$16, but weaker on random or unstructured games (Do et al., 8 Apr 2026). This suggests that the distinction between structured and unstructured procedural generation remains central even in alternative computational paradigms.

7. Interpretation and significance

Procedurally generated zero-sum matrix games are best understood not as a single model but as a family of information and generation regimes. If a game instance is generated once and then hidden, the core problem is equilibrium learning under partial feedback, with matrix-aware optimism yielding $i \in \{1,\dots,m\}$17 regret against arbitrary opponents (O'Donoghue et al., 2020). If the matrix is only accessible through queries, the main issue is how equilibrium-support structure compresses the information needed to reconstruct $i \in \{1,\dots,m\}$18, leading to an $i \in \{1,\dots,m\}$19 upper bound and an $i \in \{1,\dots,m\}$20 lower bound (Maiti et al., 2023). If the payoffs evolve over time, either exogenously or adversarially, the correct benchmark is the equilibrium of the average game, and sublinear NE regret requires joint saddle-point methods rather than separate no-regret updates (Cardoso et al., 2019). If the payoff matrix is generated endogenously by the current state, then a reduction to a rescaled zero-sum polymatrix game reveals conservation laws, recurrence, and time-average convergence to Nash equilibrium values (Skoulakis et al., 2020).

Taken together, these results show that procedural generation affects zero-sum matrix games along three principal axes: what is known about the matrix, how the matrix is revealed or evolves, and which equilibrium object remains meaningful under that revelation process. This suggests that the decisive technical question is not whether a game is procedurally generated in a generic sense, but which procedural mechanism is operating: hidden-instance generation, sparse support hiding, online payoff generation, or endogenous coevolution. Different mechanisms induce different complexity measures, different failure modes, and different notions of successful equilibrium computation.