---
title: Procedurally Generated Zero-Sum Matrix Games
url: https://www.emergentmind.com/topics/procedurally-generated-zero-sum-matrix-games
type: topic
---

# Procedurally Generated Zero-Sum Matrix Games

Searching arXiv for the cited works and closely related zero-sum matrix game papers.
Procedurally generated zero-sum matrix games are zero-sum matrix-game instances whose payoff structure is generated automatically, revealed adaptively, or induced endogenously by the learning process rather than being fully specified ex ante. Across the literature, this umbrella includes at least three distinct but related models: repeated play with a fixed but unknown payoff matrix learned from bandit feedback [2006.05145]; query-based access to an incompletely observed matrix whose equilibrium structure must be reconstructed from sampled entries [2310.16236]; and time-varying or endogenously evolving payoff matrices, where the relevant equilibrium object is attached either to a larger static reduction or to the long-run average game [2012.08382], [1907.07723]. In all of these formulations, the central technical question is how procedural generation or partial revelation alters equilibrium computation, regret analysis, feedback requirements, and the complexity of identifying minimax strategies.

## 1. Formal models and equilibrium objects

A two-player zero-sum matrix game is specified by a payoff matrix. In the bandit-feedback formulation, the unknown matrix is
\[
A \in \mathbb{R}^{m \times k},
\]
where the row player chooses \(i \in \{1,\dots,m\}\), the column player chooses \(j \in \{1,\dots,k\}\), and the row player pays the column player \(A_{ij}\) [2006.05145]. The value of the game is
\[
V_A^\star = \min_{y \in \Delta_m}\max_{x \in \Delta_k} y^\top A x = \max_{x \in \Delta_k}\min_{y \in \Delta_m} y^\top A x,
\]
with Nash equilibrium strategies \((x^\star,y^\star)\) solving the saddle-point problem [2006.05145].

In the query-complexity formulation, the input matrix is \(A \in \mathbb{R}^{n \times n}\), the row player chooses \(x \in \Delta_n\), and the column player chooses \(y \in \Delta_n\). A pair \((x^\star,y^\star)\) is a Nash equilibrium iff
\[
\langle x, A y^\star\rangle \le \langle x^\star, A y^\star\rangle \le \langle x^\star, A y\rangle \quad \forall x,y \in \Delta_n,
\]
equivalently,
\[
x^\star \in \mathcal X_\star := \arg\max_{x\in\Delta_n}\min_{y\in\Delta_n}\langle x,Ay\rangle,\qquad
y^\star \in \mathcal Y_\star := \arg\min_{y\in\Delta_n}\max_{x\in\Delta_n}\langle x,Ay\rangle
\]
[2310.16236]. The paper emphasizes that in zero-sum games, \(\mathcal X_\star\) and \(\mathcal Y_\star\) are convex polytopes [2310.16236].

When the payoff matrix evolves over time, the model changes. In Online Matrix Games, the matrix may change on each round:
\[
A_t \in [-1,1]^{d_1 \times d_2}, \qquad t=1,\dots,T,
\]
players choose mixed strategies \(x_t \in \Delta_{d_1}\) and \(y_t \in \Delta_{d_2}\) before seeing \(A_t\), and receive payoff \(x_t^\top A_t y_t\) [1907.07723]. The benchmark is then the saddle point of the cumulative or average game rather than of any single round [1907.07723].

A further extension arises when the game itself is generated endogenously by strategic interaction. In that setting, the changing payoff matrix can be encoded as part of a larger static polymatrix game, with environment variables treated as additional players [2012.08382]. This suggests that “procedural generation” has two mathematically distinct interpretations: exogenous instance generation or revelation, and endogenous generation by the current state of the dynamics.

## 2. Unknown fixed matrices: bandit feedback and minimax learning

One canonical model of procedurally generated play assumes that a matrix is generated once, fixed throughout the interaction, but initially unknown. In the repeated-learning setting, the matrix is fixed but unknown, and learning occurs from bandit feedback [2006.05145]. At round \(t\), the row player chooses \(i_t\), the column player chooses \(j_t\), and the observed payoff is
\[
r_t = A_{i_t j_t} + \eta_t,
\]
where \(\eta_t\) is zero-mean i.i.d. noise from a known distribution [2006.05145]. Both players observe the opponent’s action and the noisy payoff, and the history before round \(t\) is
\[
F_t = (i_1,j_1,r_1,\ldots,i_{t-1},j_{t-1},r_{t-1})
\]
[2006.05145].

The regret notion is equilibrium-relative rather than action-relative. For the learner, regret relative to the Nash value is
\[
R(A,\mathrm{alg},T) = \mathbb{E}_{\eta,\mathrm{alg}}\!\left[\sum_{t=1}^T V_A^\star - r_t\right],
\]
with Bayesian and worst-case variants defined as
\[
\mathrm{BayesRegret}(\phi,\mathrm{alg},T) = \mathbb{E}_{A\sim \phi} R(A,\mathrm{alg},T),
\]
and
\[
\mathrm{WorstCaseRegret}(A,\mathrm{alg},T) = \max_{A\in \mathcal{A}} R(A,\mathrm{alg},T)
\]
[2006.05145]. This covers both a fixed unknown generated instance and a prior over procedurally generated instances.

The paper studies matrix-aware variants of UCB and K-learning [2006.05145]. Under the assumptions that \(\eta_t\) is 1-sub-Gaussian and \(A \in [0,1]^{m\times k}\), UCB forms the optimistic matrix
\[
\tilde A_{ij}^t = \bar A_{ij}^t + \sqrt{2\log(2T^2mk)/(1\vee n_{ij}^t)},
\]
and plays the minimax policy for \(\tilde A^t\):
\[
x_t \in \arg\min_{x\in\Delta_k}\max_{y\in\Delta_m} y^\top \tilde A^t x
\]
[2006.05145]. The regret guarantee is
\[
\mathrm{WorstCaseRegret}(A,\mathrm{UCB},T) \le 1 + 2\sqrt{mkT\log(2mkT^2)} = \tilde O(\sqrt{mkT}),
\]
for \(T \ge mk \ge 2\) and \(\delta = 1/(2T^2mk)\), and it holds against any opponent strategy, even an informed best-responder [2006.05145].

K-learning is presented as a Bayesian optimistic algorithm based on cumulant generating functions. For each column \(a_j\) of \(A\),
\[
K_{a_j}^t(y) = \log \mathbb{E}^t \exp(a_j^\top y),
\]
and the algorithm solves
\[
(y_t^\star,\tau_t^\star) \in \arg\min_{y\in\Delta_m,\tau\ge 0} \tau \sum_{j=1}^k K_{a_j}^t(y/\tau),
\]
then chooses
\[
x_t^\star \propto \exp K_{a_j}^t(y_t^\star/\tau_t^\star)
\]
[2006.05145]. Its Bayes regret satisfies
\[
\mathrm{BayesRegret}(\phi,\mathrm{Klearn},T) \le 2\sqrt{mkT\log k\, (1+\log T)} = \tilde O(\sqrt{mkT}),
\]
again robust to the opponent’s strategy, including adversarial best-response behavior [2006.05145].

The central methodological point is that these methods exploit matrix structure. Standard adversarial bandit algorithms such as Exp3 obtain \(O(\sqrt{kT\log k})\)-type regret relative to the best action in hindsight, but they do not naturally target the Nash equilibrium and may fail to “solve” the game [2006.05145]. This distinction is fundamental in procedurally generated settings: the objective is not merely to outperform a realized opponent sequence, but to identify a robust minimax strategy for the underlying generated game.

## 3. Query-efficient equilibrium recovery and sparse support structure

A different line of work studies procedurally hidden games through query access rather than repeated play. The question is not how to learn from noisy rewards over time, but how many matrix entries must be queried in order to reconstruct the equilibrium set of a partially observed game [2310.16236].

The key structural parameters are the unions of equilibrium supports:
\[
k_1 := \left|\bigcup_{x\in\mathcal X_\star}\operatorname{supp}(x)\right|, \qquad
k_2 := \left|\bigcup_{y\in\mathcal Y_\star}\operatorname{supp}(y)\right|, \qquad
k := \max\{k_1,k_2\}
\]
[2310.16236]. These are explicitly defined as the paper’s notion of solution size [2310.16236]. Bohnenblust’s support lemma gives
\[
\bigcup_{x\in\mathcal X_\star}\operatorname{supp}(x)=\bigcap_{y\in\mathcal Y_\star} I_y, \qquad
\bigcup_{y\in\mathcal Y_\star}\operatorname{supp}(y)=\bigcap_{x\in\mathcal X_\star} I_x,
\]
where
\[
I_x := \{j\in[n] : V_A^\star = \langle x, A e_j\rangle\},\qquad
I_y := \{i\in[n] : V_A^\star = \langle e_i, A y\rangle\}
\]
[2310.16236]. This identifies the rows and columns that participate in some equilibrium support.

The main contribution is a randomized query-efficient algorithm that returns the entire set of Nash equilibria \(\mathcal X_\star \times \mathcal Y_\star\), not just a single equilibrium [2310.16236]. Its high-level strategy is to isolate the relevant \(k_1 \times k_2\) optimal submatrix, lift the problem to a larger combinatorial matrix, reduce the task to finding a strict pure-strategy Nash equilibrium in that lifted matrix, and simulate the needed oracle calls with relatively few queries to the original matrix [2310.16236]. A crucial subroutine identifies a strict PSNE using only
\[
O(\log^2(r))
\]
oracle calls, where \(r\) is the number of rows plus columns in the searched matrix [2310.16236].

The resulting guarantee is:
\[
\text{With probability at least } 1-\delta,\quad \text{the algorithm queries at most } O\!\left(nk^5\cdot \log n\cdot \log\!\left(\frac{n}{\delta}\right)\right)
\]
entries of \(A\) and returns \(\mathcal X_\star \times \mathcal Y_\star\) [2310.16236]. If \(k_1,k_2\) are unknown, trying all support-size pairs contributes another factor \(O(k^2)\), yielding the final \(O(nk^5\cdot \mathrm{polylog}(n/\delta))\) bound [2310.16236].

The lower bound is
\[
\Omega(nk)
\]
expected queries in the worst case, even when
\[
\min\{k_1,k_2\}=1
\]
[2310.16236]. More precisely, there exists an \(n\times n\) matrix with \(\max\{k_1,k_2\}=k\) and \(\min\{k_1,k_2\}=1\) such that any correct randomized algorithm must query at least
\[
\frac{(n-1)(k+1)}{2}
\]
entries in expectation [2310.16236]. This shows that sparse equilibrium structure can reduce the effective complexity from \(n^2\) to roughly \(n\cdot \mathrm{poly}(k)\), but does not remove the need to inspect a substantial fraction of the relevant matrix.

For procedurally generated games, this framework captures cases where a large matrix is generated by a hidden procedure but equilibrium play depends only on a small support union. A plausible implication is that procedural structure is most algorithmically valuable when it induces support sparsity, because that sparsity can be translated directly into lower query complexity.

## 4. Evolving and endogenous payoff generation

Procedural generation need not mean that a hidden matrix is sampled once. In Online Matrix Games, the matrix itself may change every round, possibly adversarially and even depending on the history of play [1907.07723]. The appropriate benchmark is therefore the Nash equilibrium of the long-term averaged payoff matrix,
\[
\frac{1}{T}\sum_{t=1}^T A_t,
\]
or equivalently of the cumulative game \(\sum_{t=1}^T A_t\) [1907.07723].

The paper defines Nash Equilibrium regret as
\[
\left| \sum_{t=1}^T x_t^{\top}A_t y_t - x \sum_{t=1}^T x^{\top}A_t y \right|
\]
and aims for sublinear growth in \(T\) [1907.07723]. A key negative result states that no algorithm can simultaneously guarantee vanishing NE regret together with both players’ standard individual no-regret guarantees; minimizing each player’s regret independently is therefore insufficient [1907.07723]. This separates joint equilibrium tracking from classical no-regret learning.

The full-information algorithm is Saddle-Point Regularized Follow-the-Leader (SP-RFTL). For convex-concave losses \(\bar L_t(x,y)\), it forms
\[
L_t(x,y) \gets \bar{L}_t(x,y) + \frac{1}{\eta}R_X(x)-\frac{1}{\eta}R_Y(y),
\]
and updates by solving the saddle point of the cumulative regularized game:
\[
x_{t+1}\leftarrow \arg \min_{x\in X} \max_{y \in Y} \sum_{\tau=1}^t L_t(x,y), \qquad
y_{t+1}\leftarrow \arg \max_{y \in Y} \min_{x\in X} \sum_{\tau=1}^t L_t(x,y)
\]
[1907.07723]. For matrix games on simplices, negative entropy regularizers are used, together with a shrunk simplex
\[
\Delta_\theta = \{z \in \mathbb{R}^d : \|z\|_1=1,\ z_i\ge \theta \ \forall i\}
\]
[1907.07723]. The main full-information theorem yields a bound summarized as
\[
O\!\left(\ln(T)\sqrt{T} + \sqrt{T}\max\{\ln d_1,\ln d_2\}\right) + o(1)\max\{d_1,d_2\}
\]
[1907.07723].

In the bandit setting, only the realized entry is observed. The paper introduces the one-point matrix estimator: if \(i'\sim x\) and \(j'\sim y\), define \(\hat A\) by
\[
\hat{A}_{i,j}=0 \quad \text{for } (i,j)\neq(i',j'),
\qquad
\hat{A}_{i',j'} = \frac{A_{i',j'}}{x(i')y(j')},
\]
so that
\[
\mathbb{E}_{i' \sim x,\, j' \sim y}[\hat A] = A
\]
[1907.07723]. The resulting bandit algorithm achieves
\[
\mathbb{E}\left[\left| \sum_{t=1}^T e_{x,t}^{\top}A_t e_{y,t} - x \sum_{t=1}^T x^{\top}A_t y \right|\right] \le O\big((d_1+d_2)\ln(T)T^{5/6}\big)
\]
against an adaptive adversary that does not observe current-round actions when choosing \(A_t\) [1907.07723].

A more endogenous notion of evolving payoff generation appears in coevolutionary models. There, the game changes over time as a function of the agents’ current behavior, and many such systems can be rewritten as a static polymatrix game with extra nodes representing environments [2012.08382]. In the canonical model, the environment state \(w\) modifies the payoff matrix seen by the population strategy \(y\), with
\[
P(w)=P+\mu W
\]
and dynamics
\[
\dot{w}_i = w_i \sum_{j=1}^n w_j (y_j-y_i), \qquad
\dot{y}_i = y_i\big((P(w)y)_i - y^\top P(w)y\big)
\]
[2012.08382]. The system is shown to be exactly replicator dynamics in a static two-player polymatrix game with
\[
A^{wy}=-I,\qquad A^{yy}=P,\qquad A^{yw}=\mu I
\]
[2012.08382]. This reduction is the core modeling insight: the evolving payoff matrix is encoded as strategic evolution in a larger static game.

## 5. Learning dynamics, recurrence, and finite-sample convergence

When the matrix is hidden or generated procedurally, the information structure strongly shapes the available learning guarantees. In the minimally informative repeated-play setting, one recent approach studies best-response type dynamics under two information models: full information and minimal information, also called the radically uncoupled case [2407.20128].

The game is a finite two-player zero-sum matrix game with payoff matrices \(R^i\), bounded by
\[
|R^i(a^i,a^{-i})|\le 1
\]
[2407.20128]. Suboptimality is measured by the Nash gap
\[
NG(\pi):=\sum_{i=1}^2 \max_{\bar\pi^i\in \Delta^i} \big( \bar\pi^i-\pi^i\big)^\top R^i \pi^{-i},
\]
and \(NG(\pi)\le \epsilon\) implies that \(\pi\) is an \(\epsilon\)-Nash equilibrium [2407.20128].

In full information, each player observes both payoff matrices and the opponent’s mixed strategy, and the dynamics use the entropy-smoothed best response
\[
\sigma_\tau(q^i)\in \arg\max_{\pi^i\in \Delta^i} \left\{ (\pi^i)^\top q^i+\tau H(\pi^i)\right\},
\]
with softmax form
\[
\sigma_\tau(q^i)(a^i) = \frac{e^{q^i(a^i)/\tau}}{\sum_{a\in A^i} e^{q^i(a)/\tau}}
\]
[2407.20128]. The damped smoothed best-response update is
\[
\pi^i_{k+1} = \pi^i_k+\beta_k\big(\sigma_\tau(R^i\pi_k^{-i})-\pi^i_k\big)
\]
[2407.20128].

In minimal information, each player observes only their own realized payoff and own action, not the opponent’s actions and not the payoff matrices [2407.20128]. The algorithm combines the strategy update
\[
\pi^i_{k+1} = \pi^i_k+\beta_k\big(\sigma_\tau(q_k^i)-\pi_k^i\big)
\]
with an importance-weighted TD update
\[
q^i_{k+1} = q^i_k - \alpha_k\, \frac{\mathbf e^i(A_k^i)}{\pi_k^i(A_k^i)} \Big(R^i(A_k^i,A_k^{-i})-q_k^i(A_k^i)\Big),
\]
whose expected drift satisfies
\[
\mathbb E\!\left[q^i_{k+1}-q^i_k\mid \pi_k,q_k\right] = R^i\pi_k^{-i}-q_k^i
\]
[2407.20128]. The scheme is explicitly two-timescale stochastic approximation.

The finite-sample guarantees are polynomial-time. In the full-information setting, with appropriate smoothing,
\[
K(\epsilon)\ \lesssim\ \frac{A_{\max}^3}{\epsilon^2}\log\!\left(\frac{V_1}{\epsilon}\right)
\]
for constant stepsize, and
\[
K(\epsilon)\ \lesssim\ \frac{A_{\max}^3}{\epsilon^2}
\]
for inverse-linear decay [2407.20128]. In the minimal-information setting, the theorem gives a polynomial bound of order roughly \((1/\epsilon)^{8+}\) up to constants and logs; specifically, for constant stepsize,
\[
K(\epsilon) \le \frac{A_{\max}^{10}}{g_2(\cdot)\,\epsilon^{8+}} \log\!\left(\frac{1}{\epsilon}\right)
\]
[2407.20128]. A notable point is that no additional exploration is required beyond the entropy smoothing itself [2407.20128].

The coevolutionary setting produces a different form of long-run regularity. In rescaled zero-sum polymatrix games with an interior Nash equilibrium, replicator dynamics are Poincaré recurrent: for almost all interior initial conditions, trajectories return arbitrarily close to their initial point infinitely often [2012.08382]. The proof uses log-ratio coordinates, volume preservation via
\[
\mathrm{tr}(DF(z))=0,
\]
and a conserved quantity
\[
\Phi(t)=\sum_{i\in V}\sum_{\alpha\in A_i}\eta_i x_{i\alpha}^\ast \ln x_{i\alpha},
\qquad
\frac{d\Phi(t)}{dt}=0
\]
[2012.08382]. The invariant can be rewritten in weighted KL-divergence form,
\[
\Psi(t)=\sum_{i\in V}\eta_i \mathrm{KL}(x_i^\ast\|x_i),
\]
up to additive constants [2012.08382]. If the rescaled zero-sum polymatrix game has a unique interior Nash equilibrium, then the time averages converge:
\[
\lim_{T\to\infty}\frac1T\int_0^T x(\tau)\,d\tau = x^\ast,
\qquad
\lim_{T\to\infty}\frac1T\int_0^T u_i(x(\tau))\,d\tau = u_i(x^\ast)
\]
[2012.08382]. This distinguishes pointwise non-convergence from statistical equilibrium in evolving procedurally generated systems.

## 6. Structural exploitation, failure modes, and computational frontiers

A recurrent theme is that the success of learning or computation depends on exploiting the specific structure induced by the generation mechanism. In hidden fixed-matrix games, UCB and K-learning exploit the facts that the matrix is fixed, the rows and columns are jointly structured, and the learner’s goal is to compute a minimax policy over a matrix estimate [2006.05145]. In query models, the exploitable structure is support sparsity across all equilibria [2310.16236]. In evolving-payoff games, the relevant structure is the aggregate saddle problem of the whole sequence rather than separate per-round losses [1907.07723]. In coevolutionary systems, the exploitable structure is a rescaled zero-sum polymatrix representation [2012.08382].

The literature also highlights failure modes. A key result is that Thompson sampling can fail catastrophically in matrix games with bandit feedback [2006.05145]. The paper gives the \(2\times 2\) game
\[
\begin{bmatrix} r & 0 \\ 0 & -1 \end{bmatrix}, \qquad
r = \begin{cases} 1 & \text{w.p. } 1/2,\\ -1 & \text{w.p. } 1/2, \end{cases}
\]
and shows that if the true value is \(r=1\) and the opponent knows this and plays the Nash equilibrium, the Thompson sampler never resolves the uncertainty about \(r\), keeps sampling a distribution that sometimes plays the bad action, and incurs constant expected loss [2006.05145]. The conclusion is that Thompson sampling can have linear regret against an informed opponent because stochastic optimism can be pessimistic on some rounds, allowing exploitation [2006.05145].

A second misconception concerns the role of ordinary adversarial bandit or no-regret algorithms. Exp3 may have low regret relative to the best action in hindsight, but it does not naturally target Nash equilibrium and may perform poorly compared with matrix-aware methods [2006.05145]. Likewise, independently minimizing each player’s individual regret in evolving-payoff games does not control NE regret [1907.07723]. These are distinct failures, but both reflect the same principle: equilibrium learning is not reducible to single-agent regret minimization when the benchmark is minimax or saddle-point based.

A newer computational frontier appears in variational quantum methods for two-player zero-sum matrix games [2604.16466]. In that work, mixed strategies are parameterized as Born distributions of parameterized quantum circuits, transforming the saddle-point problem into a smooth but generally nonconvex–nonconcave problem in circuit-parameter space [2604.16466]. To handle arbitrary game sizes, the method uses a dominated embedding with
\[
M=2^{\lceil \log_2 m\rceil},\qquad N=2^{\lceil \log_2 n\rceil},
\]
and embedded matrix
\[
\widetilde A_{ij}= \begin{cases}
A_{ij}, & i\le m,\ j\le n,\\
-C, & i>m,\ j\le n,\\
+C, & i\le m,\ j>n,\\
0, & i>m,\ j>n,
\end{cases}
\qquad C>\|A\|_\infty
\]
so that dummy actions are strictly dominated [2604.16466]. The projected variational quantum extragradient method uses finite-shot parameter-shift gradient estimates and establishes variance bounds scaling as \(O(1/S)\) in the number of measurement shots \(S\), together with convergence to approximate first-order stationarity under standard assumptions [2604.16466]. The paper emphasizes, however, that stationarity does not imply equilibrium optimality, so performance is evaluated by the game-space Nash gap
\[
\mathcal{G}(x,y)=\alpha(y)-\beta(x)
\]
with
\[
\alpha(y)=\max_{x'\in\Delta_m}\langle x',Ay\rangle, \qquad
\beta(x)=\min_{y'\in\Delta_n}\langle x,Ay'\rangle
\]
[2604.16466]. Numerical results are strong on structured instances up to \(32\times 32\), but weaker on random or unstructured games [2604.16466]. This suggests that the distinction between structured and unstructured procedural generation remains central even in alternative computational paradigms.

## 7. Interpretation and significance

Procedurally generated zero-sum matrix games are best understood not as a single model but as a family of information and generation regimes. If a game instance is generated once and then hidden, the core problem is equilibrium learning under partial feedback, with matrix-aware optimism yielding \(\tilde O(\sqrt{mkT})\) regret against arbitrary opponents [2006.05145]. If the matrix is only accessible through queries, the main issue is how equilibrium-support structure compresses the information needed to reconstruct \(\mathcal X_\star \times \mathcal Y_\star\), leading to an \(O(nk^5\cdot \mathrm{polylog}(n/\delta))\) upper bound and an \(\Omega(nk)\) lower bound [2310.16236]. If the payoffs evolve over time, either exogenously or adversarially, the correct benchmark is the equilibrium of the average game, and sublinear NE regret requires joint saddle-point methods rather than separate no-regret updates [1907.07723]. If the payoff matrix is generated endogenously by the current state, then a reduction to a rescaled zero-sum polymatrix game reveals conservation laws, recurrence, and time-average convergence to Nash equilibrium values [2012.08382].

Taken together, these results show that procedural generation affects zero-sum matrix games along three principal axes: what is known about the matrix, how the matrix is revealed or evolves, and which equilibrium object remains meaningful under that revelation process. This suggests that the decisive technical question is not whether a game is procedurally generated in a generic sense, but which procedural mechanism is operating: hidden-instance generation, sparse support hiding, online payoff generation, or endogenous coevolution. Different mechanisms induce different complexity measures, different failure modes, and different notions of successful equilibrium computation.

Source: https://www.emergentmind.com/topics/procedurally-generated-zero-sum-matrix-games