Fictitious-Play Approximation Procedure

• Fictitious-play approximation is an iterative best-response method that leverages empirical averages of opponent strategies to approach Nash equilibria across diverse game types.
• It is applied in finite normal-form, stochastic differential, and mean field games, with variants employing Monte Carlo sampling, neural networks, and PDE solvers to enhance convergence.
• Despite guaranteed convergence in two-player zero-sum games, its performance may suffer from slow convergence rates and exponential iteration barriers in general-sum or potential games.

Searching arXiv for recent and foundational papers on fictitious play approximation procedures, including classical finite games, stochastic differential games, and numerical variants. Fictitious-play approximation procedure denotes a family of iterative best-response methods that use beliefs formed from past play, previous-stage strategies, or averaged population states to approximate Nash equilibria. Across finite normal-form games, zero-sum games, stochastic differential games, mean field games, and imperfect-information games, the common template is to freeze or average opponents’ behavior, compute a best response, and update an empirical or stagewise aggregate that serves as the next belief. The procedure is classical in two-player zero-sum games, where fictitious play was introduced by Brown and shown by Robinson to converge, but later work established that worst-case approximation quality and convergence speed can be poor under adversarial tie-breaking or in broader game classes (Daskalakis et al., 2014). Subsequent work generalized the procedure to multiplayer normal-form games with multiple initializations (Ganzfried, 2022), stochastic differential games using deep neural networks (Hu, 2019, Han et al., 2019), fully coupled FBSDE systems with geometric convergence guarantees (Andersson et al., 9 Jul 2026), convex-concave saddle-point problems over convex sets via randomized sampling (Elbassioni et al., 2013), large-scale games via Monte Carlo estimation (Swenson et al., 2015), and mean field games via finite-difference fictitious play (Inoue et al., 2022, Shen et al., 2023).

1. Classical finite-game formulation

In finite two-player zero-sum games, a game is specified by a matrix $A=(a_{ij})$, with the row player choosing $i$, the column player choosing $j$, and row payoff $a_{ij}$. Mixed strategies are $x\in\Delta^m$ and $y\in\Delta^n$, and the expected payoff is $x^{\mathsf T}Ay$ (Daskalakis et al., 2014). Von Neumann’s min-max theorem yields the game value

$$\max_x\min_y x^{\mathsf T}Ay = \min_y\max_x x^{\mathsf T}Ay = z,$$

and an $\varepsilon$-approximate equilibrium can be measured by the duality gap

$$f_A(x,y) := \max_i e_i^{\mathsf T}Ay - \min_j x^{\mathsf T}Ae_j.$$

This gap is nonnegative and equals zero if and only if $i$0 is an equilibrium (Daskalakis et al., 2014).

Discrete-time fictitious play updates empirical distributions of realized pure actions: $i$1 and at step $i$2 each player chooses a pure best response to the opponent’s empirical distribution: $i$3 The approximation error is then $i$4 (Daskalakis et al., 2014).

In normal-form multiplayer games, the paper on initialization strategies uses the classic Brown–Robinson update

$i$5

with exploitability

$i$6

This $i$7-equilibrium notion is the metric used to evaluate fictitious play as an approximation procedure in multiplayer normal-form games (Ganzfried, 2022). A related empirical study of multiplayer games adopts the same viewpoint, using

$i$8

as the final approximation measure for FP and CFR (Ganzfried, 2020). A plausible implication is that, across these works, fictitious play is treated less as a pathwise prediction model and more as a solver whose output is the current empirical or averaged profile.

2. Convergence guarantees and worst-case approximation limits

Robinson proved that in finite two-player zero-sum games fictitious play satisfies

$i$9

and Robinson’s proof gives the rate

$j$0

Karlin conjectured the stronger universal rate $j$1, but this was disproved in a very strong sense (Daskalakis et al., 2014). For the identity game $j$2, the paper constructs valid fictitious-play executions for which

$j$3

equivalently

$j$4

showing that under arbitrary tie-breaking fictitious play may converge much more slowly than $j$5 (Daskalakis et al., 2014). The lower bound is worst-case over tie-breaking, and the same paper explicitly states that the weak conjecture for lexicographic or random tie-breaking remains open (Daskalakis et al., 2014).

The identity-game construction is algorithmically significant because the number of iterations needed to reach error $j$6 scales roughly like

$j$7

so the worst-case dependence on dimension is exponential (Daskalakis et al., 2014). This stands in contrast to the dimension-free $j$8 rates of many no-regret algorithms in zero-sum games, up to log factors, as explicitly noted in the same source (Daskalakis et al., 2014).

A different worst-case perspective appears in finite two-player general-sum games. The paper "On the Approximation Performance of Fictitious Play in Finite Games" shows that fictitious play fails to find an additive approximation guarantee significantly better than $j$9, exhibiting games where both players may perpetually have mixed strategies whose payoffs fall short of the best response by

$a_{ij}$0

for arbitrarily small $a_{ij}$1, together with an essentially matching upper bound of $a_{ij}$2 (Goldberg et al., 2011). This establishes a distinct approximation barrier in finite bimatrix games: asymptotic convergence is not the same as strong finite-time equilibrium approximation.

Potential games exhibit another negative phenomenon. In identical-payoff games, fictitious play converges asymptotically to a Nash equilibrium, but a recent lower-bound construction proves that fictitious play can take exponential time in the number of strategies to reach even an approximate Nash equilibrium, and this holds for arbitrary tie-breaking rules (Panageas et al., 2023). The paper’s main theorem gives a lower bound

$a_{ij}$3

for the time until an $a_{ij}$4-approximate equilibrium is reached in a two-player identical-payoff game with a unique pure Nash equilibrium (Panageas et al., 2023). The same construction also shows that every approximate equilibrium in the game must be close to the pure equilibrium in $a_{ij}$5-distance (Panageas et al., 2023). This suggests that, even in structured coordination settings where asymptotic convergence is known, fictitious play may be unsuitable as a polynomial-time black-box approximation routine.

3. Initialization, multi-start heuristics, and empirical multiplayer behavior

In multiplayer normal-form games, fictitious play is used explicitly as a heuristic approximation procedure with multiple initializations (Ganzfried, 2022). The paper defines a multi-initialization framework: run fictitious play from $a_{ij}$6 initial mixed-strategy profiles, compute exploitability

$a_{ij}$7

and return the run with minimal $a_{ij}$8 (Ganzfried, 2022). The main practical claim is that initialization matters substantially. The baseline initializes each player uniformly,

$a_{ij}$9

while the proposed maximin initialization selects diverse starting profiles, either by sampling or by solving a nonconvex QCQP over the full simplex (Ganzfried, 2022).

The unsampled maximin formulation chooses each new initialization to maximize its minimum squared Euclidean distance from previously selected centers: $x\in\Delta^m$0 The paper solves this directly with Gurobi’s nonconvex quadratic solver and reports that, with $x\in\Delta^m$1 initializations, the best-performing maximin approach reduces approximation error by nearly 75% relative to classic uniform initialization in random 3-player games (Ganzfried, 2022). In the reported 3-player, 5-action experiment over 100,000 games, classic yields $x\in\Delta^m$2 while maximin-u yields $x\in\Delta^m$3 (Ganzfried, 2022).

An independent empirical study compares fictitious play to CFR in multiplayer and non-zero-sum games (Ganzfried, 2020). In two-player zero-sum games, CFR generally performs slightly better for larger action sets, but in two-player general-sum and in many multiplayer settings fictitious play consistently produces lower $x\in\Delta^m$4 than CFR on average (Ganzfried, 2020). For example, in random 5-player, 10-action games with 1,000 iterations, the reported average $x\in\Delta^m$5 is $x\in\Delta^m$6 for CFR and $x\in\Delta^m$7 for FP (Ganzfried, 2020). On 5-player GAMUT benchmarks, FP significantly outperforms CFR on Bertrand oligopoly, Bidirectional LEG, Congestion, Covariant, Random graphical, and Uniform LEG, and ties on Collaboration, Polymatrix, and Random LEG (Ganzfried, 2020).

The same paper also studies multiple random initializations on classical counterexamples (Ganzfried, 2020). In Shapley’s classic counterexample, with $x\in\Delta^m$8 random initializations and $x\in\Delta^m$9 iterations per run, 33,403 initializations yield $y\in\Delta^n$0 (Ganzfried, 2020). In the Doctrines Game, under the same $y\in\Delta^n$1 and $y\in\Delta^n$2, 182 initializations yield $y\in\Delta^n$3 (Ganzfried, 2020). This suggests that non-convergence from a standard initialization does not preclude near-equilibrium convergence from a set of positive-measure initial conditions.

4. Computational variants for large-scale finite and convex games

Large-scale games motivate approximation procedures that preserve the fictitious-play logic while reducing per-iteration cost. In finite normal-form games, Sampled FP replaces exact expected-utility computation with Monte Carlo estimates, but requires $y\in\Delta^n$4 samples per round with $y\in\Delta^n$5 (Swenson et al., 2015). The paper "A Computationally Efficient Implementation of Fictitious Play for Large-Scale Games" introduces Computationally Efficient Sampled FP (CESFP), which uses only one sample per round and updates utility estimates via stochastic approximation: $y\in\Delta^n$6 with $y\in\Delta^n$7, $y\in\Delta^n$8, and $y\in\Delta^n$9 (Swenson et al., 2015). Under these conditions, CESFP converges almost surely to the set of Nash equilibria in potential games, two-player zero-sum games, and generic $x^{\mathsf T}Ay$0 games, in the same sense as classical FP and Sampled FP (Swenson et al., 2015). The paper’s simulation study on a 1,000-driver traffic routing game reports that CESFP has substantially lower wall-clock cost than Sampled FP while showing similar decay in total travel time over iterations (Swenson et al., 2015).

A different generalization concerns saddle-point problems over continuous convex sets. "On Randomized Fictitious Play for Approximating Saddle Points Over Convex Sets" extends Grigoriadis–Khachiyan randomized fictitious play from matrix games to convex-concave problems $x^{\mathsf T}Ay$1 where $x^{\mathsf T}Ay$2 and $x^{\mathsf T}Ay$3 are bounded convex sets given by membership oracles (Elbassioni et al., 2013). At iteration $x^{\mathsf T}Ay$4, the method samples

$x^{\mathsf T}Ay$5

and updates

$x^{\mathsf T}Ay$6

For functions of constant width $x^{\mathsf T}Ay$7, the paper shows that an $x^{\mathsf T}Ay$8-approximate saddle point can be computed in

$x^{\mathsf T}Ay$9

iterations, with total oracle complexity

$$\max_x\min_y x^{\mathsf T}Ay = \min_y\max_x x^{\mathsf T}Ay = z,$$0

using log-concave sampling (Elbassioni et al., 2013). This is not classical fictitious play over finite action sets, but the update is still an averaging of randomized best-response surrogates and is presented as a continuous randomized fictitious-play approximation procedure.

In imperfect-information games, Neural Fictitious Self-Play (NFSP) implements the same decomposition into average strategy and best response, and Monte Carlo Neural Fictitious Self-Play (MC-NFSP) replaces DQN best response with MCTS guided by a policy–value network (Zhang et al., 2019). The average strategy network is trained on best-response behavior, while self-play uses the anticipatory mixture

$$\max_x\min_y x^{\mathsf T}Ay = \min_y\max_x x^{\mathsf T}Ay = z,$$1

where $$\max_x\min_y x^{\mathsf T}Ay = \min_y\max_x x^{\mathsf T}Ay = z,$$2 is the average policy and $$\max_x\min_y x^{\mathsf T}Ay = \min_y\max_x x^{\mathsf T}Ay = z,$$3 is the best-response policy (Zhang et al., 2019). The paper reports that MC-NFSP can converge to approximate Nash equilibrium in games with large-scale search depth while NFSP cannot, and that asynchronous parallel actor-learners further accelerate and stabilize training (Zhang et al., 2019). A plausible implication is that neural fictitious-play systems inherit the same conceptual split as classical FP—belief averaging plus best response—but replace explicit normal-form calculations by function approximation and search.

5. Continuous-time stochastic differential games and FBSDE formulations

In continuous-time stochastic differential games, fictitious play is reformulated as a stagewise procedure over control policies rather than action frequencies. "Deep Fictitious Play for Stochastic Differential Games" considers asymmetric $$\max_x\min_y x^{\mathsf T}Ay = \min_y\max_x x^{\mathsf T}Ay = z,$$4-player non-zero-sum stochastic differential games with open-loop controls $$\max_x\min_y x^{\mathsf T}Ay = \min_y\max_x x^{\mathsf T}Ay = z,$$5, state dynamics

$$\max_x\min_y x^{\mathsf T}Ay = \min_y\max_x x^{\mathsf T}Ay = z,$$6

and costs

$$\max_x\min_y x^{\mathsf T}Ay = \min_y\max_x x^{\mathsf T}Ay = z,$$7

At stage $$\max_x\min_y x^{\mathsf T}Ay = \min_y\max_x x^{\mathsf T}Ay = z,$$8, each player solves

$$\max_x\min_y x^{\mathsf T}Ay = \min_y\max_x x^{\mathsf T}Ay = z,$$9

with opponents’ previous-stage controls frozen (Hu, 2019). Time is discretized, each best-response control is approximated by a deep neural network, and all $\varepsilon$0 stochastic control problems are solved in parallel (Hu, 2019). The paper proves convergence in a linear-quadratic game under a contraction condition, establishing that $\varepsilon$1 converges in $\varepsilon$2 to an open-loop Nash equilibrium independent of the initial belief (Hu, 2019). In experiments with $\varepsilon$3, after about 10 stages the maximum relative cost error is at most about 3%, and trajectory $\varepsilon$4 errors are of order $\varepsilon$5 (Hu, 2019).

A related development targets Markovian feedback equilibria. "Deep Fictitious Play for Finding Markovian Nash Equilibrium in Multi-Agent Games" recasts an $\varepsilon$6-player stochastic differential game into $\varepsilon$7 decoupled decision problems, one per player, where each player at stage $\varepsilon$8 solves an HJB equation with the opponents’ previous-stage policies frozen (Han et al., 2019). The resulting semilinear PDE is handled via a deep BSDE method, using a neural network approximation $\varepsilon$9 for the value function and automatic differentiation for $$f_A(x,y) := \max_i e_i^{\mathsf T}Ay - \min_j x^{\mathsf T}Ae_j.$$0 (Han et al., 2019). The algorithm is parallel over players and is demonstrated on inter-bank borrowing and lending games, risk-sensitive linear–exponential–quadratic games, and a nonlinear 50-player example with common noise (Han et al., 2019). For the 10-player inter-bank case, the reported relative squared errors are about 4.6% for $$f_A(x,y) := \max_i e_i^{\mathsf T}Ay - \min_j x^{\mathsf T}Ae_j.$$1 and about 0.2% for $$f_A(x,y) := \max_i e_i^{\mathsf T}Ay - \min_j x^{\mathsf T}Ae_j.$$2 (Han et al., 2019).

The paper "Convergence of fictitious play for fully coupled FBSDEs in finite-player stochastic differential games" provides a theoretical counterpart for closed-loop Markov policies represented via fully coupled FBSDE systems (Andersson et al., 9 Jul 2026). The procedure starts from zero policy $$f_A(x,y) := \max_i e_i^{\mathsf T}Ay - \min_j x^{\mathsf T}Ae_j.$$3, and at iteration $$f_A(x,y) := \max_i e_i^{\mathsf T}Ay - \min_j x^{\mathsf T}Ae_j.$$4 each player solves a best-response FBSDE with opponents’ policy $$f_A(x,y) := \max_i e_i^{\mathsf T}Ay - \min_j x^{\mathsf T}Ae_j.$$5 frozen, obtaining a Markov map $$f_A(x,y) := \max_i e_i^{\mathsf T}Ay - \min_j x^{\mathsf T}Ae_j.$$6 for the BSDE control and updating

$$f_A(x,y) := \max_i e_i^{\mathsf T}Ay - \min_j x^{\mathsf T}Ae_j.$$7

Under either a gradient-bound assumption or a global Lipschitz plus smallness condition, the paper proves geometric convergence: $$f_A(x,y) := \max_i e_i^{\mathsf T}Ay - \min_j x^{\mathsf T}Ae_j.$$8 where $$f_A(x,y) := \max_i e_i^{\mathsf T}Ay - \min_j x^{\mathsf T}Ae_j.$$9 is the squared $i$00 error between the fictitious-play FBSDEs and the Nash FBSDE (Andersson et al., 9 Jul 2026). Under the additional structural condition that $i$01 is independent of $i$02, the rate improves to super-exponential: $i$03 A numerical experiment for a linear-quadratic interbank game reports exponential decay of state, backward, and control errors for both $i$04 and $i$05 (Andersson et al., 9 Jul 2026).

These continuous-time formulations depart from the empirical-frequency interpretation of Brown’s original FP. Instead of averaging observed actions, they use previous-stage opponent controls as beliefs. This suggests a broader editor’s term, “stagewise fictitious play,” for settings where the approximation procedure is iterative best response against last-stage policies rather than empirical averages.

6. Mean field, smoothing, and robustness issues

Mean field games provide yet another reinterpretation of fictitious play. In linearly solvable MFGs, an iterative finite-difference scheme updates the average density

$i$06

then solves a backward HJB and forward Fokker–Planck system against $i$07 (Inoue et al., 2022). Under a Cole–Hopf transform,

$i$08

the nonlinear MFG becomes a linear advection–diffusion–reaction system (Inoue et al., 2022). The fictitious-play iterates in the transformed variables satisfy linear PDEs, and the finite-difference method converges to the true MFG solution as the mesh is refined and the number of FP iterations grows (Inoue et al., 2022).

In mean field games with optimal stopping, the paper "Fictitious Play via Finite Differences for Mean Field Games with Optimal Stopping" proposes a generalized fictitious play with learning rates $i$09 satisfying

$i$10

At iteration $i$11, one solves the obstacle problem and associated pure-strategy forward equation for a proposed density $i$12, then updates the belief by

$i$13

Under a potential-game assumption and regularity hypotheses, any regular cluster point is a mixed-strategy equilibrium of the relaxed OSMFG system (Shen et al., 2023). The paper’s experiments show that when only mixed equilibria exist, $i$14 leads to oscillation, whereas $i$15 and $i$16 yield convergence of exploitability and $i$17-error to zero (Shen et al., 2023).

A different robustness issue concerns smoothing. Classical fictitious play is not consistent as a no-regret algorithm, but vanishing smooth fictitious play (VSFP) replaces exact best response by a smooth best response $i$18 with a time-varying smoothing parameter $i$19 (Benaïm et al., 2011). The main theorem states that if

$i$20

then VSFP is consistent: $i$21 However, the same paper gives a matching negative example showing that VSFP with $i$22 is not consistent in matching pennies (Benaïm et al., 2011). This establishes a rate-sensitive sense in which smoothed fictitious play approximates ordinary fictitious play while avoiding its worst no-regret pathology.

Finally, fictitious play can be strategically manipulated. "Intelligent Players in a Fictitious Play Framework" studies finite normal-form games where all but one player follow alternating fictitious play, while an intelligent player knows the full payoff matrix (Vundurthy et al., 2021). In two-player games, the intelligent player’s optimal exploitation problem reduces to solving $i$23 linear programs: $i$24 subject to the opponent best-responding with action $i$25 (Vundurthy et al., 2021). In the paper’s 2×3 example, the Nash payoff under mutual FP is 6, the Stackelberg payoff is 7, and the exploiting mixed strategy achieves 14.17 by inducing the opponent to play a fixed pure best response (Vundurthy et al., 2021). This does not alter the formal definition of fictitious-play approximation, but it shows that uncoupled FP procedures may be fragile when other agents can predict and exploit the learning rule.

The literature therefore presents a sharply differentiated picture. Fictitious-play approximation procedures are conceptually simple and broadly extensible: they can be implemented through empirical frequencies, random sampling, neural best-response solvers, FBSDE iterations, or finite-difference PDE solvers. Yet their approximation quality depends strongly on game class, tie-breaking, initialization, and information structure. In two-player zero-sum games, convergence is guaranteed but worst-case rates can be as slow as $i$26 (Daskalakis et al., 2014). In finite general-sum games, worst-case additive approximation is essentially limited to $i$27 (Goldberg et al., 2011). In potential games, the time to an approximate equilibrium can be exponential (Panageas et al., 2023). At the same time, carefully initialized or domain-adapted variants can be empirically effective in multiplayer normal-form games (Ganzfried, 2022, Ganzfried, 2020), scalable in stochastic differential games (Hu, 2019, Han et al., 2019, Andersson et al., 9 Jul 2026), and rigorously convergent in several PDE-based mean field settings (Inoue et al., 2022, Shen et al., 2023).