Aggregate Fictitious Play Overview

• Aggregate fictitious play (agg-FP) is a learning framework that uses lower-dimensional aggregate statistics of opponents' actions instead of tracking full empirical profiles.
• It reduces computational complexity in large-scale games by summarizing actions into aggregate count vectors and centroids, enabling efficient best-response updates.
• Variations such as empirical centroid fictitious play and mean-field formulations demonstrate agg-FP’s effectiveness in distributed settings and convergence to various equilibrium notions.

Aggregate fictitious play (agg-FP) denotes fictitious-play-style learning in which agents respond to a lower-dimensional summary of play rather than to the full empirical profile of individual opponents. In anonymous polymatrix games, agg-FP is introduced as a variant of FP where each agent tracks the frequency of the number of other agents playing each action, rather than these agents’ individual actions (Kara et al., 26 Aug 2025). Closely related formulations include empirical centroid fictitious play (ECFP), in which players respond to the centroid of all players’ actions rather than track and respond to the individual actions of every player (Swenson et al., 2013), and mean-field formulations in which the aggregate state is an averaged population distribution updated by convex averaging (Cardaliaguet et al., 2015, Yu et al., 2024). The shared structure is best response to an aggregate empirical state followed by an averaging recursion, but the exact aggregate, update rule, and equilibrium notion vary with the game class.

1. Classical fictitious play and the aggregate reformulation

Classical fictitious play (FP) is the canonical best-response learning rule in which each player forms beliefs about opponents’ mixed strategies from empirical frequencies and then best-responds to those beliefs. In one standard formulation, the empirical distribution is

$q_i(t) := \frac{1}{t}\sum_{s=1}^t a_i(s),$

and the action rule is

$a_i(t+1)\in BR_i(q_{-i}(t)).$

A count-based formulation writes the belief update through action counts,

$$\kappa_{t}^{j}(s^{j}) = \kappa_{t-1}^{j}(s^{j})+\mathfrak{I}_{s^j_t=s^j},$$

with induced mixed-strategy belief given by normalized counts. This is the standard empirical-frequency mechanism: learn beliefs from empirical frequencies, then best respond (Smyrnakis, 2013, Swenson et al., 2015).

Agg-FP modifies this architecture by replacing the full opponent profile with an aggregate statistic. The main motivation is scalability. In standard FP, if each agent has $n$ actions, the full joint profile space has size

$n^N,$

which becomes quickly intractable for reward estimation or exploration when the agents do not know the payoff function and need to learn it from experience. Anonymous structure mitigates this by making payoffs depend on which actions others choose, but not on who chooses them, so that the relevant representation becomes an aggregate count vector rather than the full joint action profile (Kara et al., 26 Aug 2025).

The same large-scale motivation underlies ECFP. In that formulation, classical FP is theoretically attractive, but in large games it is often impractical because each player must store and process the empirical distribution of every other player and must compute best responses against a high-dimensional joint belief. ECFP replaces that requirement with response to the empirical centroid, reducing both memory and computational burden, and enabling distributed implementation through local communication (Swenson et al., 2013).

2. Aggregate state variables and representations

The literature uses several aggregate state variables rather than a single universal construction. In each case, the aggregate compresses past play into a statistic that is behaviorally sufficient for the corresponding update rule.

Aggregate state	Definition	Setting
Empirical centroid	$\bar q(t)=\frac{q_1(t)+q_2(t)+\cdots+q_n(t)}{n}$	Basic ECFP
Class centroid	$\bar p^k := \frac{1}{|C_k|}\sum_{i\in C_k} p_i$	ECFP with equivalence classes
Aggregate count vector	$x^{-i}:=\sigma(a^{-i})\in\mathbb X$, with $$\mathbb X:=\{\xi\in\mathbb N^n:\sum_{m=1}^n \xi_m=N-1\}$$	Anonymous polymatrix agg-FP
Aggregate population belief	$\bar m^n=\frac1n\sum_{k=1}^n m^k$	Mean-field and optimal-stopping formulations
Aggregate interference message	$a_i(t+1)\in BR_i(q_{-i}(t)).$0	Channel selection aggregation game

In ECFP, the players are partitioned into permutation-invariant equivalence classes $a_i(t+1)\in BR_i(q_{-i}(t)).$1, where players in the same class possess similar properties and are interchangeable in the sense of the symmetry condition. The centroid profile replaces individual opponent tracking by class-level averages, so that players respond to aggregate statistics related to each equivalence class. The centroids are essentially aggregate state variables; they make the learning problem lower dimensional and more scalable (Swenson et al., 2015).

In anonymous polymatrix games, the aggregate statistic is the count vector of how many of the other $a_i(t+1)\in BR_i(q_{-i}(t)).$2 agents chose each action. The state space becomes

$a_i(t+1)\in BR_i(q_{-i}(t)).$3

The reward table size drops from $a_i(t+1)\in BR_i(q_{-i}(t)).$4 entries to

$a_i(t+1)\in BR_i(q_{-i}(t)).$5

For $a_i(t+1)\in BR_i(q_{-i}(t)).$6 actions and $a_i(t+1)\in BR_i(q_{-i}(t)).$7 agents, the number of reward entries drops from $a_i(t+1)\in BR_i(q_{-i}(t)).$8 to $a_i(t+1)\in BR_i(q_{-i}(t)).$9 (Kara et al., 26 Aug 2025).

In mean-field formulations, the aggregate is not a finite-dimensional action histogram but a population object: either a time-indexed density $$\kappa_{t}^{j}(s^{j}) = \kappa_{t-1}^{j}(s^{j})+\mathfrak{I}_{s^j_t=s^j},$$0, a pair $$\kappa_{t}^{j}(s^{j}) = \kappa_{t-1}^{j}(s^{j})+\mathfrak{I}_{s^j_t=s^j},$$1, or a probability measure on path space. This suggests that agg-FP is best understood structurally—as best response to an aggregate empirical state—rather than as a single finite-game statistic (Cardaliaguet et al., 2015, Shen et al., 2023, Yu et al., 2024).

3. Update rules and algorithmic structure

The discrete-time ECFP update allows a general step-size process: $$\kappa_{t}^{j}(s^{j}) = \kappa_{t-1}^{j}(s^{j})+\mathfrak{I}_{s^j_t=s^j},$$2 with

$$\kappa_{t}^{j}(s^{j}) = \kappa_{t-1}^{j}(s^{j})+\mathfrak{I}_{s^j_t=s^j},$$3

and the action rule

$$\kappa_{t}^{j}(s^{j}) = \kappa_{t-1}^{j}(s^{j})+\mathfrak{I}_{s^j_t=s^j},$$4

Thus ECFP is a best-response process to aggregated beliefs rather than to a full belief vector. In the ideal centralized version, each player best-responds to repeated copies of the centroid; in the imperfect-information and distributed versions, each player uses a local estimate of the centroid (Swenson et al., 2015, Swenson et al., 2013).

In anonymous polymatrix agg-FP, each agent $$\kappa_{t}^{j}(s^{j}) = \kappa_{t-1}^{j}(s^{j})+\mathfrak{I}_{s^j_t=s^j},$$5 tracks an empirical distribution over aggregate count vectors,

$$\kappa_{t}^{j}(s^{j}) = \kappa_{t-1}^{j}(s^{j})+\mathfrak{I}_{s^j_t=s^j},$$6

and updates it via

$$\kappa_{t}^{j}(s^{j}) = \kappa_{t-1}^{j}(s^{j})+\mathfrak{I}_{s^j_t=s^j},$$7

with Robbins–Monro stepsizes

$$\kappa_{t}^{j}(s^{j}) = \kappa_{t-1}^{j}(s^{j})+\mathfrak{I}_{s^j_t=s^j},$$8

The next action is chosen by maximizing expected aggregate payoff against $$\kappa_{t}^{j}(s^{j}) = \kappa_{t-1}^{j}(s^{j})+\mathfrak{I}_{s^j_t=s^j},$$9, while empirical action frequencies are tracked separately through

$n$0

The central change from classical FP is therefore not the best-response principle, but the state on which beliefs are formed (Kara et al., 26 Aug 2025).

The same aggregate-averaging pattern appears in continuum games. In mean-field games, the aggregate belief is updated by

$n$1

or, more generally,

$n$2

For mean field games with optimal stopping, the generalized fictitious play recursion is

$n$3

with

$n$4

and when $n$5, it reduces to the classic fictitious play (Cardaliaguet et al., 2015, Yu et al., 2024, Shen et al., 2023).

4. Convergence statements and equilibrium notions

In anonymous polymatrix games, the key bridge between agg-FP and FP is the equality

$n$6

Because expected rewards coincide, the best responses coincide, and the paper states that in anonymous polymatrix games agg-FP converges to a Nash equilibrium if and only if FP does. Since classical FP converges in polymatrix zero-sum games and potential games, agg-FP also converges in those classes, provided they are anonymous and polymatrix. For the model-free two-timescale extension, empirical action frequencies converge to an $n$7-Nash equilibrium set when the corresponding $n$8-greedy best-response dynamics make the target equilibrium attractive (Kara et al., 26 Aug 2025).

ECFP changes the equilibrium target. In the basic formulation, convergence is stated as

$n$9

where $n^N,$0 is the set of consensus equilibria. In the generalized partition-based formulation, convergence becomes

$n^N,$1

where $n^N,$2 is the set of symmetric Nash equilibria relative to the partition. The robustness analysis sharpens this by proving that, under identical-interests games and permutation-invariant partitions,

$n^N,$3

with $n^N,$4 the set of mean-centric equilibria. The papers are explicit that this is not the same as classical convergence of the full empirical profile to the full Nash set (Swenson et al., 2013, Swenson et al., 2015).

In mean-field games, the convergence theory depends on game structure. For potential mean field games, any cluster point of the fictitious-play sequence solves the mean-field game system, and under monotonicity the whole sequence converges. For general non-potential mean-field games, the 2024 analysis introduces the gain

$n^N,$5

and proves the recursion

$n^N,$6

With diminishing weights $n^N,$7, the rate is $n^N,$8; with constant weight $n^N,$9, the decay is linear: $\bar q(t)=\frac{q_1(t)+q_2(t)+\cdots+q_n(t)}{n}$0 For optimal-stopping mean-field games, any regular cluster point of the generalized fictitious-play iterates solves the mixed strategy system under the stated potential and regularity assumptions (Cardaliaguet et al., 2015, Yu et al., 2024, Shen et al., 2023).

5. Scalability, distributed implementation, and acceleration

The principal computational argument for agg-FP is dimensional reduction. In anonymous polymatrix games, aggregating the agents’ actions reduces the relevant representation from the full joint profile space $\bar q(t)=\frac{q_1(t)+q_2(t)+\cdots+q_n(t)}{n}$1 to $\bar q(t)=\frac{q_1(t)+q_2(t)+\cdots+q_n(t)}{n}$2. The reported simulations on a $\bar q(t)=\frac{q_1(t)+q_2(t)+\cdots+q_n(t)}{n}$3-agent rock-paper-scissors polymatrix game show that agg-FP learns faster than standard two-timescale FP in terms of Q-table error, also converges faster in terms of distance to Nash equilibrium, and produces smoother trajectories, likely because the aggregate representation reduces the effective dimensionality of the learning problem (Kara et al., 26 Aug 2025).

ECFP emphasizes three practical gains: memory reduction, computational simplification, and distributed implementability. In the distributed version, players communicate over a fixed, connected graph and maintain local estimates $\bar q(t)=\frac{q_1(t)+q_2(t)+\cdots+q_n(t)}{n}$4 of the centroid. The update

$\bar q(t)=\frac{q_1(t)+q_2(t)+\cdots+q_n(t)}{n}$5

uses a matrix $\bar q(t)=\frac{q_1(t)+q_2(t)+\cdots+q_n(t)}{n}$6 that is doubly stochastic, aperiodic, irreducible, and sparse according to the graph. A key intermediate result is

$\bar q(t)=\frac{q_1(t)+q_2(t)+\cdots+q_n(t)}{n}$7

which verifies the error assumption required for convergence under imperfect centroid estimates (Swenson et al., 2013).

A distinct aggregation-based implementation appears in channel selection games. Because the game is an aggregation game, the receiver can broadcast only the aggregate vector

$\bar q(t)=\frac{q_1(t)+q_2(t)+\cdots+q_n(t)}{n}$8

and each transmitter updates

$\bar q(t)=\frac{q_1(t)+q_2(t)+\cdots+q_n(t)}{n}$9

This reproduces the same best responses as standard fictitious play while requiring only local channel gains and the common broadcast of aggregate interference/noise (Perlaza et al., 2010).

In mean-field computation, aggregation also enables acceleration. The 2024 mean-field analysis proposes a backtracking line search for choosing $\bar p^k := \frac{1}{|C_k|}\sum_{i\in C_k} p_i$0 and a hierarchical grid strategy for coarse-to-fine continuation. Under the paper’s scaling assumptions, the hierarchical scheme is estimated to reduce complexity from

$\bar p^k := \frac{1}{|C_k|}\sum_{i\in C_k} p_i$1

because the number of iterations per level is $\bar p^k := \frac{1}{|C_k|}\sum_{i\in C_k} p_i$2 rather than $\bar p^k := \frac{1}{|C_k|}\sum_{i\in C_k} p_i$3 (Yu et al., 2024).

6. Conceptual boundaries, related variants, and observed limitations

Agg-FP is closely related to, but conceptually distinct from, several other fictitious-play variants. The EKF-based variant of fictitious play summarizes past play in a latent propensity vector $\bar p^k := \frac{1}{|C_k|}\sum_{i\in C_k} p_i$4 and updates beliefs through an Extended Kalman Filter rather than through empirical counts. The paper explicitly notes that this is not pure aggregate frequency learning: it is a filtered latent-state approximation of action propensities, closer to a state-space / Bayesian-filter extension of FP than to classical agg-FP (Smyrnakis, 2013).

Other neighboring variants change how the average is used rather than which aggregate is tracked. Anticipatory Fictitious Play computes a one-step anticipated average by folding an opponent best response into the empirical mean before best-responding; the paper does not use the term agg-FP, but it is explicitly framed as a modification of how the empirical average is used. In extensive-form games, GXFP updates behavior strategies by averaging local best decisions at information sets,

$\bar p^k := \frac{1}{|C_k|}\sum_{i\in C_k} p_i$5

and is described as realization equivalent to a generalized form of fictitious play. These formulations reinforce the broader interpretation of agg-FP as a family of average-response dynamics rather than a single count-based recipe (Cloud et al., 2022, Schulze, 2023).

The main limitations are structural. ECFP converges to consensus equilibria, symmetric Nash equilibria relative to a partition, or mean-centric equilibria, not to all Nash equilibria of the full game. In channel selection, fictitious play can converge empirically to a mixed-strategy equilibrium while the realized action profiles cycle between two same-channel profiles; the paper states that the resulting expected utility can be worse than the worst NE in pure or mixed strategies. In mean-field games with optimal stopping, the numerical experiments show that proposed distributions $\bar p^k := \frac{1}{|C_k|}\sum_{i\in C_k} p_i$6 can oscillate while the averaged distribution $\bar p^k := \frac{1}{|C_k|}\sum_{i\in C_k} p_i$7 converges, which underscores that the averaging step is not merely auxiliary but constitutive of the learned mixed equilibrium (Swenson et al., 2013, Perlaza et al., 2010, Shen et al., 2023).

Empirical findings across the literature are therefore consistent but not uniform. In anonymous polymatrix games, aggregation accelerates learning while preserving convergence guarantees under the same conditions as classical FP. In distributed and wireless settings, aggregation reduces communication and information requirements. In PDE-based mean-field settings, aggregation stabilizes iterative best-response schemes and admits acceleration mechanisms. At the same time, the equilibrium object learned by an aggregate process is model-dependent, and aggregation can change both the convergence target and the observable action-level behavior (Kara et al., 26 Aug 2025, Swenson et al., 2015, Yu et al., 2024).