--- title: Fictitious-Play Approximation Procedure url: https://www.emergentmind.com/topics/fictitious-play-approximation-procedure type: topic --- # Fictitious-Play Approximation Procedure Searching arXiv for recent and foundational papers on fictitious play approximation procedures, including classical finite games, stochastic differential games, and numerical variants. Fictitious-play approximation procedure denotes a family of iterative best-response methods that use beliefs formed from past play, previous-stage strategies, or averaged population states to approximate Nash equilibria. Across finite normal-form games, zero-sum games, stochastic differential games, mean field games, and imperfect-information games, the common template is to freeze or average opponents’ behavior, compute a best response, and update an empirical or stagewise aggregate that serves as the next belief. The procedure is classical in two-player zero-sum games, where fictitious play was introduced by Brown and shown by Robinson to converge, but later work established that worst-case approximation quality and convergence speed can be poor under adversarial tie-breaking or in broader game classes [1412.4840]. Subsequent work generalized the procedure to multiplayer normal-form games with multiple initializations [2203.10774], stochastic differential games using deep neural networks [1903.09376, 1912.01809], fully coupled FBSDE systems with geometric convergence guarantees [2607.08861], convex-concave saddle-point problems over convex sets via randomized sampling [1301.5290], large-scale games via Monte Carlo estimation [1506.04320], and mean field games via finite-difference fictitious play [2204.07278, 2310.05741]. ## 1. Classical finite-game formulation In finite two-player zero-sum games, a game is specified by a matrix \(A=(a_{ij})\), with the row player choosing \(i\), the column player choosing \(j\), and row payoff \(a_{ij}\). Mixed strategies are \(x\in\Delta^m\) and \(y\in\Delta^n\), and the expected payoff is \(x^{\mathsf T}Ay\) [1412.4840]. Von Neumann’s min-max theorem yields the game value \[ \max_x\min_y x^{\mathsf T}Ay = \min_y\max_x x^{\mathsf T}Ay = z, \] and an \(\varepsilon\)-approximate equilibrium can be measured by the duality gap \[ f_A(x,y) := \max_i e_i^{\mathsf T}Ay - \min_j x^{\mathsf T}Ae_j. \] This gap is nonnegative and equals zero if and only if \((x,y)\) is an equilibrium [1412.4840]. Discrete-time fictitious play updates empirical distributions of realized pure actions: \[ x(t) = \frac1t\sum_{\tau\le t} e_{i_\tau}, \qquad y(t) = \frac1t\sum_{\tau\le t} e_{j_\tau}, \] and at step \(t+1\) each player chooses a pure best response to the opponent’s empirical distribution: \[ i_{t+1} \in \arg\max_i e_i^{\mathsf T}Ay(t), \qquad j_{t+1} \in \arg\min_j x(t)^{\mathsf T}Ae_j. \] The approximation error is then \(f_A(x(t),y(t))\) [1412.4840]. In normal-form multiplayer games, the paper on initialization strategies uses the classic Brown–Robinson update \[ \sigma'^t_i = \argmax_{\sigma_i \in \Sigma_i} u_i(\sigma_i,\sigma^{t-1}_{-i}), \qquad \sigma^t_i = \left( 1 - \frac{1}{t+1} \right)\sigma^{t-1}_i + \frac{1}{t+1}\sigma'^t_i, \] with exploitability \[ \epsilon(\sigma^*) = \max_i \max_{\sigma_i \in \Sigma_i} \left[ u_i(\sigma_i,\sigma^*_{-i}) - u_i(\sigma^*_i,\sigma^*_{-i}) \right]. \] This \(\epsilon\)-equilibrium notion is the metric used to evaluate fictitious play as an approximation procedure in multiplayer normal-form games [2203.10774]. A related empirical study of multiplayer games adopts the same viewpoint, using \[ \epsilon(\sigma) = \max_i \max_{\sigma_i \in \Sigma_i} \bigl[ u_i(\sigma_i,\sigma_{-i}) - u_i(\sigma_i,\sigma_{-i}) \bigr] \] as the final approximation measure for FP and CFR [2001.11165]. A plausible implication is that, across these works, fictitious play is treated less as a pathwise prediction model and more as a solver whose output is the current empirical or averaged profile. ## 2. Convergence guarantees and worst-case approximation limits Robinson proved that in finite two-player zero-sum games fictitious play satisfies \[ f_A(x(t),y(t)) \to 0 \quad\text{as } t\to\infty, \] and Robinson’s proof gives the rate \[ f_A(x(t),y(t)) = O\Bigl(t^{-\frac{1}{m+n-2}}\Bigr). \] Karlin conjectured the stronger universal rate \(O(t^{-1/2})\), but this was disproved in a very strong sense [1412.4840]. For the identity game \(A=I_n\), the paper constructs valid fictitious-play executions for which \[ f_{I_n}(x(t),y(t)) = \Theta(t^{-1/n}), \] equivalently \[ \max V(t) - \min U(t) = \Theta\bigl(t^{(n-1)/n}\bigr), \] showing that under arbitrary tie-breaking fictitious play may converge much more slowly than \(1/\sqrt{t}\) [1412.4840]. The lower bound is worst-case over tie-breaking, and the same paper explicitly states that the weak conjecture for lexicographic or random tie-breaking remains open [1412.4840]. The identity-game construction is algorithmically significant because the number of iterations needed to reach error \(\varepsilon\) scales roughly like \[ t \approx \varepsilon^{-n}, \] so the worst-case dependence on dimension is exponential [1412.4840]. This stands in contrast to the dimension-free \(O(1/\varepsilon^2)\) rates of many no-regret algorithms in zero-sum games, up to log factors, as explicitly noted in the same source [1412.4840]. A different worst-case perspective appears in finite two-player general-sum games. The paper "On the Approximation Performance of Fictitious Play in Finite Games" shows that fictitious play fails to find an additive approximation guarantee significantly better than \(1/2\), exhibiting games where both players may perpetually have mixed strategies whose payoffs fall short of the best response by \[ 1/2 - O(1/n^{1-\delta}) \] for arbitrarily small \(\delta\), together with an essentially matching upper bound of \(1/2 - O(1/n)\) [1103.1040]. This establishes a distinct approximation barrier in finite bimatrix games: asymptotic convergence is not the same as strong finite-time equilibrium approximation. Potential games exhibit another negative phenomenon. In identical-payoff games, fictitious play converges asymptotically to a Nash equilibrium, but a recent lower-bound construction proves that fictitious play can take exponential time in the number of strategies to reach even an approximate Nash equilibrium, and this holds for arbitrary tie-breaking rules [2310.02387]. The paper’s main theorem gives a lower bound \[ T \ge \Omega\left( 4^n \left(\left(\frac{n}{2}-2\right)!\right)^4 + \frac{1}{n\sqrt{\epsilon}} \right) \] for the time until an \(\epsilon\)-approximate equilibrium is reached in a two-player identical-payoff game with a unique pure Nash equilibrium [2310.02387]. The same construction also shows that every approximate equilibrium in the game must be close to the pure equilibrium in \(\ell_1\)-distance [2310.02387]. This suggests that, even in structured coordination settings where asymptotic convergence is known, fictitious play may be unsuitable as a polynomial-time black-box approximation routine. ## 3. Initialization, multi-start heuristics, and empirical multiplayer behavior In multiplayer normal-form games, fictitious play is used explicitly as a heuristic approximation procedure with multiple initializations [2203.10774]. The paper defines a multi-initialization framework: run fictitious play from \(K\) initial mixed-strategy profiles, compute exploitability \[ \epsilon_k = \max_i \max_{\sigma_i \in \Sigma_i} \left[ u_i(\sigma_i,\sigma^T_{k,-i}) - u_i(\sigma^T_{k,i}, \sigma^T_{k,-i}) \right], \] and return the run with minimal \(\epsilon_k\) [2203.10774]. The main practical claim is that initialization matters substantially. The baseline initializes each player uniformly, \[ \sigma^0_i(j) = \frac{1}{|S_i|}, \] while the proposed maximin initialization selects diverse starting profiles, either by sampling or by solving a nonconvex QCQP over the full simplex [2203.10774]. The unsampled maximin formulation chooses each new initialization to maximize its minimum squared Euclidean distance from previously selected centers: \[ \begin{array}{rrl} &\max_{\mathbf{x}, y} & y \\ &\text{s.t.} & y \leq \sum_j \sum_k \Big( x_{jk}^2 - 2\tau^i_{jk} x_{jk} + (\tau^i_{jk})^2 \Big) \quad \forall i,\\ && 0 \leq x_{ij} \leq 1 \quad \forall i,j,\\ && \sum_{j} x_{ij} = 1 \quad \forall i. \end{array} \] The paper solves this directly with Gurobi’s nonconvex quadratic solver and reports that, with \(K=5\) initializations, the best-performing maximin approach reduces approximation error by nearly 75% relative to classic uniform initialization in random 3-player games [2203.10774]. In the reported 3-player, 5-action experiment over 100,000 games, classic yields \(0.02234 \pm 2.18\times10^{-4}\) while maximin-u yields \(0.00603 \pm 9.05\times10^{-5}\) [2203.10774]. An independent empirical study compares fictitious play to CFR in multiplayer and non-zero-sum games [2001.11165]. In two-player zero-sum games, CFR generally performs slightly better for larger action sets, but in two-player general-sum and in many multiplayer settings fictitious play consistently produces lower \(\epsilon\) than CFR on average [2001.11165]. For example, in random 5-player, 10-action games with 1,000 iterations, the reported average \(\epsilon\) is \(0.06350\) for CFR and \(0.04341\) for FP [2001.11165]. On 5-player GAMUT benchmarks, FP significantly outperforms CFR on Bertrand oligopoly, Bidirectional LEG, Congestion, Covariant, Random graphical, and Uniform LEG, and ties on Collaboration, Polymatrix, and Random LEG [2001.11165]. The same paper also studies multiple random initializations on classical counterexamples [2001.11165]. In Shapley’s classic counterexample, with \(K=100{,}000\) random initializations and \(T=100{,}000\) iterations per run, 33,403 initializations yield \(\epsilon_k < 10^{-4}\) [2001.11165]. In the Doctrines Game, under the same \(K\) and \(T\), 182 initializations yield \(\epsilon_k < 10^{-4}\) [2001.11165]. This suggests that non-convergence from a standard initialization does not preclude near-equilibrium convergence from a set of positive-measure initial conditions. ## 4. Computational variants for large-scale finite and convex games Large-scale games motivate approximation procedures that preserve the fictitious-play logic while reducing per-iteration cost. In finite normal-form games, Sampled FP replaces exact expected-utility computation with Monte Carlo estimates, but requires \(k_t = \lceil C t^\gamma \rceil\) samples per round with \(\gamma > 1/2\) [1506.04320]. The paper "A Computationally Efficient Implementation of Fictitious Play for Large-Scale Games" introduces Computationally Efficient Sampled FP (CESFP), which uses only one sample per round and updates utility estimates via stochastic approximation: \[ \widehat U_i(\alpha_i,t) = (1-\rho(t))\,\widehat U_i(\alpha_i,t-1) + \rho(t)\, U_i(\alpha_i,a_{-i}^*(t)), \] with \(0 < \rho(t) \le 1\), \(\sum_t \rho(t)^2 < \infty\), and \(\lim_{t\to\infty} \frac{1}{t\rho(t)}=0\) [1506.04320]. Under these conditions, CESFP converges almost surely to the set of Nash equilibria in potential games, two-player zero-sum games, and generic \(2\times m\) games, in the same sense as classical FP and Sampled FP [1506.04320]. The paper’s simulation study on a 1,000-driver traffic routing game reports that CESFP has substantially lower wall-clock cost than Sampled FP while showing similar decay in total travel time over iterations [1506.04320]. A different generalization concerns saddle-point problems over continuous convex sets. "On Randomized Fictitious Play for Approximating Saddle Points Over Convex Sets" extends Grigoriadis–Khachiyan randomized fictitious play from matrix games to convex-concave problems \(\min_{x\in X}\max_{y\in Y} F(x,y)\) where \(X\) and \(Y\) are bounded convex sets given by membership oracles [1301.5290]. At iteration \(t\), the method samples \[ \xi \sim \widetilde p_t(\xi) \propto \exp\Big(-\frac{\varepsilon}{2}\, t\, F(\xi,y(t))\Big), \qquad \eta \sim \widetilde q_t(\eta) \propto \exp\Big(\frac{\varepsilon}{2}\, t\, F(x(t),\eta)\Big), \] and updates \[ x(t+1) = \frac{t}{t+1}x(t) + \frac{1}{t+1}\xi,\qquad y(t+1) = \frac{t}{t+1}y(t) + \frac{1}{t+1}\eta. \] For functions of constant width \(\rho = O(1)\), the paper shows that an \(\varepsilon\)-approximate saddle point can be computed in \[ O\!\left(\frac{\rho^2 (n+m)}{\varepsilon^2}\ln\frac{R}{\varepsilon}\right) \] iterations, with total oracle complexity \[ O^*\!\left(\frac{\rho^2 (n+m)^6}{\varepsilon^{2}\ln R}\right) \] using log-concave sampling [1301.5290]. This is not classical fictitious play over finite action sets, but the update is still an averaging of randomized best-response surrogates and is presented as a continuous randomized fictitious-play approximation procedure. In imperfect-information games, Neural Fictitious Self-Play (NFSP) implements the same decomposition into average strategy and best response, and Monte Carlo Neural Fictitious Self-Play (MC-NFSP) replaces DQN best response with MCTS guided by a policy–value network [1903.09569]. The average strategy network is trained on best-response behavior, while self-play uses the anticipatory mixture \[ \sigma = (1-\eta)\Pi + \eta B, \] where \(\Pi\) is the average policy and \(B\) is the best-response policy [1903.09569]. The paper reports that MC-NFSP can converge to approximate Nash equilibrium in games with large-scale search depth while NFSP cannot, and that asynchronous parallel actor-learners further accelerate and stabilize training [1903.09569]. A plausible implication is that neural fictitious-play systems inherit the same conceptual split as classical FP—belief averaging plus best response—but replace explicit normal-form calculations by function approximation and search. ## 5. Continuous-time stochastic differential games and FBSDE formulations In continuous-time stochastic differential games, fictitious play is reformulated as a stagewise procedure over control policies rather than action frequencies. "Deep Fictitious Play for Stochastic Differential Games" considers asymmetric \(N\)-player non-zero-sum stochastic differential games with open-loop controls \(\alpha^i\in \mathbb{A}\), state dynamics \[ d X_t^i = b^i(t,\boldsymbol{X}_t,\boldsymbol{\alpha}_t)\,dt + \sigma^i(t,\boldsymbol{X}_t,\boldsymbol{\alpha}_t)\,dW_t^i + \sigma^0(t,\boldsymbol{X}_t,\boldsymbol{\alpha}_t)\,dW_t^0, \] and costs \[ J^i(\boldsymbol{\alpha}) := \mathbb{E}\Big[ \int_0^T f^i(t, \boldsymbol{X}_t,\boldsymbol{\alpha}_t)\,dt + g^i(\boldsymbol{X}_T)\Big]. \] At stage \(n+1\), each player solves \[ \alpha^{i,n+1} := \arg\min_{\beta^i\in A} J^i(\beta^i; \boldsymbol{\alpha}^{-i,n}), \] with opponents’ previous-stage controls frozen [1903.09376]. Time is discretized, each best-response control is approximated by a deep neural network, and all \(N\) stochastic control problems are solved in parallel [1903.09376]. The paper proves convergence in a linear-quadratic game under a contraction condition, establishing that \(\{\boldsymbol{\alpha}^n\}\) converges in \(L^2\) to an open-loop Nash equilibrium independent of the initial belief [1903.09376]. In experiments with \(N=5,10,24\), after about 10 stages the maximum relative cost error is at most about 3%, and trajectory \(L^1\) errors are of order \(10^{-2}\) [1903.09376]. A related development targets Markovian feedback equilibria. "Deep Fictitious Play for Finding Markovian Nash Equilibrium in Multi-Agent Games" recasts an \(N\)-player stochastic differential game into \(N\) decoupled decision problems, one per player, where each player at stage \(m+1\) solves an HJB equation with the opponents’ previous-stage policies frozen [1912.01809]. The resulting semilinear PDE is handled via a deep BSDE method, using a neural network approximation \(\text{Net}(t,\bm x;\theta) \approx V^i(t,\bm x)\) for the value function and automatic differentiation for \(\nabla_x V^i\) [1912.01809]. The algorithm is parallel over players and is demonstrated on inter-bank borrowing and lending games, risk-sensitive linear–exponential–quadratic games, and a nonlinear 50-player example with common noise [1912.01809]. For the 10-player inter-bank case, the reported relative squared errors are about 4.6% for \(V^i\) and about 0.2% for \(\nabla_x V^i\) [1912.01809]. The recent paper "Convergence of fictitious play for fully coupled FBSDEs in finite-player stochastic differential games" provides a theoretical counterpart for closed-loop Markov policies represented via fully coupled FBSDE systems [2607.08861]. The procedure starts from zero policy \(a_{i,0}\equiv 0\), and at iteration \(m\) each player solves a best-response FBSDE with opponents’ policy \(a_{-i,m-1}\) frozen, obtaining a Markov map \(\phi_{i,m}\) for the BSDE control and updating \[ a_{i,m}(t,x) := \kappa_i\Bigl(t,x,\psi(t,x)\phi_{i,m}(t,x),\, a_{-i,m-1}(t,x)\Bigr). \] Under either a gradient-bound assumption or a global Lipschitz plus smallness condition, the paper proves geometric convergence: \[ \Upsilon_m \le C K^m, \qquad 0