Fictitious Play: Adaptive Game Dynamics

• Fictitious Play is a learning dynamic in finite games where players update empirical beliefs from past actions and best respond, linking adaptive behavior to Nash equilibrium analysis.
• Extensions like continuous-time FP, diminishing step sizes, and aggregate FP enhance convergence properties and scalability in complex or large-scale game settings.
• The distinction between weak and strong learning, along with varying convergence rates, underscores FP’s role in informing equilibrium stability and decentralized learning strategies.

to=arxiv_search 娱乐开号json {"query":"fictitious play convergence potential games zero-sum empirical centroid aggregate fictitious play", "max_results": 10} to=search_arxiv ացինjson {"query":"fictitious play arXiv", "max_results": 5} Fictitious play (FP) is a repeated-play learning dynamic for finite games in which each player treats opponents’ past actions as draws from stationary mixed strategies, forms empirical beliefs from those observations, and then plays a myopic best response to the resulting forecast. In its classical form, if $a_i(t)$ denotes player $i$’s realized action at stage $t$, the empirical distribution is

$q_i(t):=\frac{1}{t}\sum_{s=1}^t a_i(s),$

and the next action satisfies

$a_i(t+1)\in BR_i(q_{-i}(t)).$

FP is central because it links adaptive behavior to Nash equilibrium analysis: in important game classes the empirical distributions converge to equilibrium, while in broader settings the same update rule serves as a template for compressed, decentralized, stochastic, and continuous-control learning procedures (Swenson et al., 2015).

1. Canonical formulation

The standard FP process is defined on a finite normal-form game $\Gamma=(N,(Y_i,u_i)_{i\in N})$ with mixed-strategy simplices $\Delta_i$ and mixed extensions $U_i$. Each player uses observed action histories to build empirical beliefs and then best-responds to the opponents’ empirical profile. This is the canonical discrete-time recursion used throughout the modern literature, including classical FP, generalized weakened FP, and several large-scale variants (Swenson et al., 2015).

A common generalization replaces uniform averaging by diminishing step sizes. In that formulation,

$q_i(t+1)=q_i(t)+\gamma_t\big(a_i(t+1)-q_i(t)\big),$

with $\gamma_t\ge 0$, $i$0, and $i$1. This preserves the “belief update plus best response” structure while permitting non-uniform weighting of observations, including recency-sensitive updates (Swenson et al., 2015).

Continuous-time FP is typically written as the differential inclusion

$i$2

or, in the autonomous coordinate form used for potential games, as an absolutely continuous process whose velocity points from the current mixed strategy toward a current best response. In $i$3 games, continuous-time FP and best-response dynamics have the same orbit shapes inside each best-response region and differ only by time reparametrization (Swenson et al., 2017, Wu et al., 2024).

2. Equilibrium learning and its precise meaning

The standard convergence notion for classical FP is weak learning: the empirical distribution approaches the Nash equilibrium set,

$i$4

This is known to hold in two-player zero-sum games, potential games, and generic $i$5 games, and it extends to several structurally related settings such as identical-interest games and certain decomposable bimatrix games (Swenson et al., 2015, Chen et al., 2022).

Weak learning is distinct from period-by-period stabilization. A central observation in the FP literature is that empirical convergence does not imply that the realized actions, or even the period-by-period mixed strategies generated by the algorithm, converge to equilibrium. The discontinuity of the best-response correspondence can induce cycling around mixed equilibria, so agents may “learn” an equilibrium in the empirical sense while never asymptotically playing it round by round (Swenson et al., 2015).

This distinction motivates strong learning, where the actual mixed strategies used at each stage converge to an equilibrium set. A general construction achieves this by letting each player take deliberate approximate best responses only with probabilities $i$6 such that $i$7, $i$8, and the deliberate-update counts remain asymptotically synchronized across players. Under robustness assumptions, this upgrades weakly convergent FP-type procedures to strongly convergent ones (Swenson et al., 2015).

The literature also identifies new game classes with the fictitious-play property. A geometrical approach for $i$9 games without an internal indifferent point shows that every continuous-time FP trajectory converges in that class. The analysis proceeds by projecting the strategy space onto a planar system, locating mixed equilibria as vertices of a central cell, and handling nonsmoothness through redefined saddle and sink equilibria (Wu et al., 2024).

3. Rates of convergence and their limitations

Asymptotic convergence of FP is compatible with very different finite-time behaviors. In regular potential games, continuous-time FP converges to the set of Nash equilibria at an exponential rate from almost every initial condition: $t$0 The mechanism is local: generic trajectories converge to pure regular equilibria, and near such a point the best response becomes constant, reducing FP to the linear ODE $t$1 (Swenson et al., 2017).

For two-player zero-sum games, the long-standing focus has been the rate conjectured by Karlin. In diagonal payoff matrices with strictly positive diagonal entries and lexicographic tie-breaking, FP attains

$t$2

and this $t$3 rate is tight in the identity-matrix case, where a matching $t$4 lower bound is proved under the same tie-breaking assumption (Abernethy et al., 2019).

That positive result is not universal. A tie-breaking-agnostic lower bound constructs a $t$5 zero-sum matrix game in which FP converges only at rate $t$6, with no ties after the first step. This disproves the weaker form of Karlin’s conjecture as well: slow convergence is not merely an artifact of adversarial tie-breaking (Wang, 14 Jul 2025).

Potential games exhibit an even sharper worst-case contrast between asymptotic convergence and computational usefulness. A recursively constructed two-player identical-payoff coordination game with a unique pure Nash equilibrium yields the lower bound

$t$7

before the empirical strategy profile becomes an $t$8-approximate Nash equilibrium. The lower bound is independent of the tie-breaking rule (Panageas et al., 2023).

A related misconception is that convergence to the equilibrium set should imply convergence to a single equilibrium point. In zero-sum games this can fail. Under geometric conditions on the equilibrium set—specifically positive measure, interiority, and a technical boundary-instability condition—FP cannot converge pointwise to any single equilibrium of Player 1, regardless of the tie-breaking rule. The result isolates a difference between setwise convergence and pointwise stabilization (Moon, 8 Apr 2026).

4. Robustness, perturbations, and decentralized information

A broad FP-type theory treats belief-based best-response learning through an observation state $t$9, a forecast map $q_i(t):=\frac{1}{t}\sum_{s=1}^t a_i(s),$0, and the recursion

$q_i(t):=\frac{1}{t}\sum_{s=1}^t a_i(s),$1

In this framework, players may use $q_i(t):=\frac{1}{t}\sum_{s=1}^t a_i(s),$2-best responses with $q_i(t):=\frac{1}{t}\sum_{s=1}^t a_i(s),$3. The main robustness result states that weakened FP-type processes converge to the chain recurrent set of the associated differential inclusion, so asymptotically vanishing best-response errors do not alter the limiting continuous-time geometry (Swenson et al., 2016).

This robustness perspective yields several concrete consequences. Classical FP in potential games, two-player zero-sum games, and generic $q_i(t):=\frac{1}{t}\sum_{s=1}^t a_i(s),$4 games remains convergent under asymptotically vanishing best-response perturbations. More generally, it justifies distributed and asynchronous deployments in which agents do not observe the exact empirical state but instead rely on local estimates whose error vanishes (Swenson et al., 2016).

Empirical Centroid Fictitious Play (ECFP) extends the same principle to large-scale games with symmetry. Players are partitioned into equivalence classes and best-respond to class centroids rather than to every individual empirical distribution. Under identical-interest assumptions and permutation-invariant partitions, robust ECFP satisfies

$q_i(t):=\frac{1}{t}\sum_{s=1}^t a_i(s),$5

even with general diminishing step sizes and vanishing best-response errors (Swenson et al., 2015).

Decentralized FP in near-potential games pushes this further: each agent maintains local estimates of others’ empirical frequencies, communicates over a time-varying network, and best-responds to those local beliefs. The local estimates satisfy

$q_i(t):=\frac{1}{t}\sum_{s=1}^t a_i(s),$6

and the empirical frequencies converge around a single Nash equilibrium, provided the game has finitely many Nash equilibria and is sufficiently close to a potential game in maximum pairwise difference (Aydin et al., 2022).

5. Scalability: sampling, centroids, and aggregation

The central scalability problem in classical FP is the cost of evaluating expected utility against the full opponents’ empirical profile. In large games this can be exponential in the number of players. Several variants compress the information state while preserving equilibrium learning guarantees on structured classes (Swenson et al., 2013, Swenson et al., 2015, Kara et al., 26 Aug 2025).

ECFP is one such compression. When all players share the same action space, the empirical centroid

$q_i(t):=\frac{1}{t}\sum_{s=1}^t a_i(s),$7

replaces the full profile $q_i(t):=\frac{1}{t}\sum_{s=1}^t a_i(s),$8. Players then use

$q_i(t):=\frac{1}{t}\sum_{s=1}^t a_i(s),$9

In permutation-invariant potential games, the average empirical profile $a_i(t+1)\in BR_i(q_{-i}(t)).$0 converges to the set of consensus equilibria $a_i(t+1)\in BR_i(q_{-i}(t)).$1, and the distributed variant achieves centroid-tracking error

$a_i(t+1)\in BR_i(q_{-i}(t)).$2

under connected-graph, doubly stochastic communication (Swenson et al., 2013).

Computationally Efficient Sampled Fictitious Play (CESFP) addresses the utility-evaluation bottleneck through stochastic approximation. Sampled FP requires a growing number of Monte Carlo samples, with a sufficient condition

$a_i(t+1)\in BR_i(q_{-i}(t)).$3

CESFP replaces this with a single sample per round and the recursion

$a_i(t+1)\in BR_i(q_{-i}(t)).$4

In potential games, zero-sum games, and generic $a_i(t+1)\in BR_i(q_{-i}(t)).$5 games, it preserves almost sure convergence to the Nash equilibrium set while substantially reducing per-round sampling cost (Swenson et al., 2015).

Aggregate Fictitious Play (agg-FP) compresses FP still further by exploiting anonymity. In an anonymous game, payoffs depend only on how many opponents choose each action, not on their identities. The aggregation map is

$a_i(t+1)\in BR_i(q_{-i}(t)).$6

This reduces the reward representation from $a_i(t+1)\in BR_i(q_{-i}(t)).$7 entries to

$a_i(t+1)\in BR_i(q_{-i}(t)).$8

With 3 actions and 5 agents, the representation shrinks from $a_i(t+1)\in BR_i(q_{-i}(t)).$9 reward entries to $\Gamma=(N,(Y_i,u_i)_{i\in N})$0. In anonymous polymatrix games,

$\Gamma=(N,(Y_i,u_i)_{i\in N})$1

so agg-FP and classical FP choose the same best response at each step and converge under exactly the same conditions, while using a smaller belief state (Kara et al., 26 Aug 2025).

6. Dynamic, model-free, and strategic extensions

FP has been extended beyond static finite normal-form games in several directions. In Markov games, a two-timescale FP-plus-$\Gamma=(N,(Y_i,u_i)_{i\in N})$2-learning update combines belief recursion over stationary strategies with value estimation. For $\Gamma=(N,(Y_i,u_i)_{i\in N})$3-player identical-interest Markov games with a single controller, and also for two-player zero-sum Markov games, the resulting process converges to a Markov stationary Nash equilibrium under standard two-timescale conditions on $\Gamma=(N,(Y_i,u_i)_{i\in N})$4 and $\Gamma=(N,(Y_i,u_i)_{i\in N})$5 (Sayin et al., 2022).

Model-free anonymous polymatrix learning admits a related two-timescale agg-FP construction. A fast $\Gamma=(N,(Y_i,u_i)_{i\in N})$6-update estimates aggregate payoffs on the compressed state space $\Gamma=(N,(Y_i,u_i)_{i\in N})$7, while the slower belief update tracks aggregate action-count frequencies. If the $\Gamma=(N,(Y_i,u_i)_{i\in N})$8-greedy best-response dynamics for the expected-payoff game make $\Gamma=(N,(Y_i,u_i)_{i\in N})$9 attractive, then two-timescale agg-FP converges to $\Delta_i$0 in anonymous polymatrix games with random payoffs (Kara et al., 26 Aug 2025).

Continuous-action and high-dimensional partially observable settings motivate still different approximations to the best-response operator. DiffFP retains the fictitious-play mixture update

$\Delta_i$1

but represents approximate best responses by diffusion policies in continuous-space zero-sum games. The paper reports convergence toward $\Delta_i$2-Nash equilibria and, in its benchmark suite, up to $\Delta_i$3 faster convergence and $\Delta_i$4 higher success rates on average against RL-based baselines (Karthikeyan et al., 17 Nov 2025).

Other modifications target either nonstationarity or anticipatory reasoning. Adaptive Forgetting Factor Fictitious Play relaxes the stationary-opponent assumption by discounting old observations through a learned forgetting factor $\Delta_i$5; the paper recommends $\Delta_i$6 and $\Delta_i$7 based on its parameter study (Smyrnakis et al., 2011). Anticipatory Fictitious Play replaces “best respond to the current average” with “best respond to an anticipated average after the opponent best-responds to you.” It preserves FP’s worst-case $\Delta_i$8 guarantee in two-player zero-sum matrix games and achieves $\Delta_i$9 on transitive games and on cyclic games $U_i$0 for $U_i$1 (Cloud et al., 2022).

FP is also not strategically neutral. In a setting with one intelligent player who knows the full payoff matrix while the others follow alternating FP using only local utility information and observed actions, the informed player can deviate from FP and obtain a payoff strictly better than the Nash-equilibrium payoff by manipulating the opponents’ empirical beliefs. This identifies a fragility of FP under informational asymmetry and strategic heterogeneity (Vundurthy et al., 2021).

Taken together, these developments suggest a stable conceptual core. FP remains a best-response-to-empirical-beliefs paradigm, but the “belief” can be full, centroidal, aggregate, network-estimated, model-free, or history-discounted; the “best response” can be exact, approximate, smoothed, diffusion-parameterized, or team-coordinated; and the limiting object can be a Nash equilibrium, a symmetric or mean-centric equilibrium, a Markov stationary equilibrium, or an approximate equilibrium set. What varies across the literature is not the myopic logic itself, but the structural assumptions under which that logic is computationally tractable, behaviorally robust, and dynamically convergent (Swenson et al., 2015, Swenson et al., 2016).