---
title: Almost Greedy Fictitious Play
url: https://www.emergentmind.com/topics/almost-greedy-fictitious-play
type: topic
---

# Almost Greedy Fictitious Play

Searching arXiv for the specified papers and closely related context.
Almost greedy fictitious play denotes a family of fictitious-play variants in which the stagewise update is not fully exact greedy play, yet the deviation from exact best-response behavior is controlled so that equilibrium convergence can still be established. In the literature represented by "No-regret Dynamics and Fictitious Play" [1207.0660] and "Do Not Discretize, Optimize: Almost Greedy Fictitious Play" [2606.12149], the term refers to two related but distinct constructions: a vanishing-error $\varepsilon_t$-best-reply process induced by potential-based no-regret dynamics, and a zero-sum algorithm that performs a constrained greedy line search along the segment joining the current mixed strategy to the current pure best-response profile. Both formulations retain the core fictitious-play motif—responding to empirical or current opponent behavior by moving toward best replies—while modifying the exact notion of greediness.

## 1. Terminological scope and historical placement

In the 2012 formulation, almost greedy fictitious play is introduced as a natural variant of classical fictitious play in which, at each stage, players choose not exact best replies but $\varepsilon$-best replies to the empirical distribution of their opponents’ past play, with the perturbation $\varepsilon$ vanishing over time [1207.0660]. In the 2026 formulation, Almost-Greedy Fictitious Play (AGFP) is a specific zero-sum algorithm in which each iteration first computes pure best responses and then chooses a stepsize greedily by minimizing the duality gap along a constrained portion of the line from the current mixed strategy to that pure best-response profile [2606.12149].

| Variant | Core update | Context |
|---|---|---|
| Vanishing-error almost greedy fictitious play | Choose $\varepsilon_t$-best replies with $\varepsilon_t \to 0$ | Realized by potential-based no-regret dynamics |
| AGFP with line search | Best-response step with $\eta_t=\arg\min_{\eta\in[\delta,1]} \psi(z(\eta))$ | Zero-sum games and duality-gap minimization |

The first variant is primarily a dynamical-systems and learning-theoretic equivalence result: potential-based no-regret dynamics can be viewed as an $\varepsilon_t$-perturbed fictitious-play process. The second is an optimization-oriented algorithmic redesign: rather than discretizing continuous fictitious play, it directly optimizes the one-step update and obtains an instance-dependent $\mathcal{O}(1/T)$ rate with respect to the duality gap [2606.12149].

A common misconception is that "almost greedy fictitious play" names a single standard algorithm. The literature here indicates otherwise. The 2012 and 2026 usages share the idea of approximate greediness but differ in state variable, update rule, and proof technique. This suggests a broader organizing principle: fictitious-play dynamics can be relaxed either through controlled best-response error or through constrained optimization of the interpolation step.

## 2. Vanishing-error best replies and regret-based foundations

The 2012 development begins with repeated play in a finite game. Each player $i$ has finite action set $A_i$, realized average payoff
$$
\frac{1}{T}\sum_{s=1}^T u_i(a_i(s),a_{-i}(s)),
$$
and, for any fixed action $k\in A_i$, the counterfactual average payoff
$$
\frac{1}{T}\sum_{s=1}^T u_i(k,a_{-i}(s)).
$$
The external regret of player $i$ for action $k$ at time $t$ is
$$
R^i_k(t)=\frac{1}{t}\sum_{s=1}^t\bigl[u_i(k,a_{-i}(s)) - u_i(a_i(s),a_{-i}(s))\bigr],
$$
and the maximal regret is
$$
R^i_t=\max_{k\in A_i} R^i_k(t).
$$
A no-regret strategy guarantees almost surely
$$
\limsup_{t\to\infty} R^i_t\le 0
$$
[1207.0660].

Potential-based no-regret dynamics of Hart–Mas-Colell style select in period $t+1$ a mixed action $q_i(t+1)$ whose probability on action $k$ is proportional to the partial derivative of a convex potential $P_i$ at the regret vector $R^i(t)=(R^i_k(t))_{k\in A_i}$:
$$
q_{i,k}(t+1)=\frac{\partial_k P_i\bigl(R^i(t)\bigr)}{\sum_{\ell\in A_i}\partial_\ell P_i\bigl(R^i(t)\bigr)},
$$
whenever $R^i_t>0$; if $R^i_t\le 0$ the player repeats some fixed pure action [1207.0660]. Under the stated conditions on $P_i$—twice differentiable, convex, zero on the nonpositive orthant, and with positive gradient only on strictly positive coordinates—two properties hold: $R^i_t\to 0$ almost surely, and every realized action $k$ played satisfies $R^i_k(t)>0$ whenever $R^i_t>0$ [1207.0660].

The key lemma is the $\varepsilon$-best-reply selection statement. If $R^i_t\le \varepsilon$, then every action $k$ with $R^i_k(t)>0$ is an $\varepsilon$-best reply to the opponents’ empirical distribution $z_{-i}(t)$. Consequently, with
$$
\varepsilon_t=\max_i R^i_t\to 0,
$$
the mixed action satisfies
$$
q_i(t)\in BR_{\varepsilon_t}\bigl(z_{-i}(t)\bigr),
$$
where
$$
BR_\varepsilon(x_{-i})=\{x_i\in \Delta(A_i):\, u_i(x_i,x_{-i})\ge \max_{k\in A_i}u_i(k,x_{-i})-\varepsilon\}.
$$
This is exactly the 2012 notion of almost greedy fictitious play: at each stage, players choose $\varepsilon_t$-best replies with vanishing $\varepsilon_t$ [1207.0660].

## 3. Equivalence with potential-based no-regret dynamics

The principal conceptual contribution of the 2012 paper is that a wide class of potential-based no-regret dynamics can be realized as almost greedy fictitious play, and that this realization permits alternative and sometimes much shorter proofs of convergence results [1207.0660]. The equivalence is not merely heuristic. It identifies the stagewise mixed actions generated by regret-based procedures with an explicit $\varepsilon_t$-best-response correspondence, where the perturbation level is the maximal regret.

This equivalence has two immediate consequences. First, it clarifies the operational meaning of potential-based no-regret dynamics: rather than viewing them only as regret-minimizing procedures, one may interpret them as fictitious-play processes with asymptotically vanishing greediness error. Second, it transfers convergence information from continuous fictitious play to no-regret dynamics whenever the corresponding perturbed differential inclusion has the same attractor structure [1207.0660].

The special case of the $\ell_2$-norm potential, identified in the paper as "regret matching," yields a concrete rate statement. If
$$
R^i_t=O\bigl(1/\sqrt{t}\bigr),
$$
then
$$
\varepsilon_t=O\bigl(1/\sqrt{t}\bigr).
$$
Thus the perturbation level in almost greedy fictitious play decays at the same rate as the maximal regret induced by the selected potential [1207.0660].

A plausible implication is that almost greediness provides a unifying interface between online learning and game dynamics: regret bounds determine the admissible best-response error, while fictitious-play attractor theory determines the limiting empirical behavior. The 2012 exposition makes this interface explicit.

## 4. Convergence through perturbed continuous fictitious play

The convergence theorem of Viossatt and Zapechelnyuk considers any two-player game in which continuous fictitious play (CFP) has a global attractor $A$; examples given in the data include zero-sum games and potential games, where $A$ is the set of Nash equilibria [1207.0660]. For any potential-based no-regret dynamics, viewed as almost greedy fictitious play with $\varepsilon_t\to 0$, two conclusions follow: the perturbation level $\varepsilon_t$ vanishes at the same rate as the maximal regret under the chosen potential, and the empirical distribution
$$
z(t)=\frac{1}{t}\sum_{s\le t}\delta_{a(s)}
$$
converges almost surely to $A$ [1207.0660].

The proof strategy relies on perturbed differential inclusions. The discrete-time beliefs $x(t)=(z_1(t),z_2(t))$ satisfy
$$
x(t+1)-x(t)=q(t)-x(t),
$$
with $q_i(t)\in BR_{\varepsilon_t}(x_{-i}(t))$ and $\varepsilon_t\to 0$, so the process is an $\varepsilon_t$-payoff-perturbed fictitious-play trajectory [1207.0660]. A graph-perturbation lemma is then invoked to replace payoff perturbations by the standard sup-norm perturbation used in Benaïm–Hofbauer–Sorin (2005). After linear interpolation over $[t,t+1]$, one obtains a continuous-time path $x(\cdot)$ that is a perturbed solution of the CFP inclusion
$$
\dot x(t)\in BR\bigl(x(t)\bigr)-x(t),
$$
with perturbation parameter tending to zero [1207.0660].

By Theorem 3.6 of Benaïm–Hofbauer–Sorin, the limit set of the interpolated path is internally chain transitive (ICT) for CFP; because $A$ is the unique global attractor, every ICT set must lie in $A$ [1207.0660]. The resulting implication chain is summarized in the paper as
$$
\text{no-regret} \to \varepsilon\text{-best reply} \to \text{perturbed CFP} \to \text{ICT} \to \text{attractor } A.
$$

In zero-sum games, and more generally in any game whose attractor under CFP is the Nash set, this yields convergence of the empirical distribution to Nash equilibrium. The same conclusion is stated for potential games and strictly dominance-solvable games [1207.0660]. The significance is methodological as much as substantive: the equivalence with almost greedy fictitious play replaces more specialized regret-matching martingale arguments with a shorter dynamical-systems proof.

## 5. The 2026 AGFP algorithm for zero-sum games

The 2026 AGFP algorithm is formulated for a two-player zero-sum game with payoff matrix $A\in\mathbb{R}^{m\times n}$, row and column pure-strategy sets $[m]$ and $[n]$, and mixed strategies $x\in \Delta_m$, $y\in \Delta_n$ [2606.12149]. A pair $(x^*,y^*)$ is a Nash equilibrium iff, for every $i\in[m]$ and $j\in[n]$,
$$
(x^*)^\top A e_j \le (x^*)^\top A y^* \le e_i^\top A y^*.
$$
The distance to equilibrium is measured by the duality gap
$$
\psi(x,y)=\max_i e_i^\top A y - \min_j x^\top A e_j,
$$
which satisfies $\psi(x,y)\ge 0$ by von Neumann’s minimax theorem and vanishes exactly at Nash equilibrium; if $\psi(x,y)\le \varepsilon$, then $(x,y)$ is an $\varepsilon$-NE [2606.12149].

At iteration $t$, given $(x_{t-1},y_{t-1})$, the row player computes
$$
i_t\in \arg\max_{i\in[m]} e_i^\top A y_{t-1},
$$
and the column player computes
$$
j_t\in \arg\min_{j\in[n]} x_{t-1}^\top A e_j.
$$
For any $\eta\in[0,1]$, the candidate profile is
$$
z(\eta)=(x(\eta),y(\eta)),
$$
with
$$
x(\eta)=(1-\eta)x_{t-1}+\eta e_{i_t}, \qquad
y(\eta)=(1-\eta)y_{t-1}+\eta e_{j_t}.
$$
The AGFP step then chooses
$$
\eta_t=\arg\min_{\eta\in[\delta,1]} \psi(z(\eta)),
$$
where $\delta>0$ is a small lower bound [2606.12149].

Because
$$
\psi(z(\eta))=\max_i e_i^\top A y(\eta)-\min_j x(\eta)^\top A e_j
$$
is convex and piecewise-linear in $\eta$, the minimizer can be found exactly in $O(m+n)$ or via a binary search on the slopes, approximately $50$ steps to machine precision [2606.12149]. The update is then
$$
x_t=(1-\eta_t)x_{t-1}+\eta_t e_{i_t}, \qquad
y_t=(1-\eta_t)y_{t-1}+\eta_t e_{j_t}.
$$

The design principle differs sharply from standard discrete fictitious play with step-sizes $1/(t+1)$. Instead of following a predetermined averaging schedule, AGFP greedily optimizes the current step over an interval that includes almost all the line between the cumulative mixed strategy and the current best response. The paper explicitly presents this as an alternative to discretization of continuous fictitious play [2606.12149].

## 6. Rates, empirical behavior, and limitations

The main theoretical result of the 2026 paper is an instance-dependent $\mathcal{O}(1/T)$ convergence rate for the duality gap [2606.12149]. For any non-equilibrium $(x,y)$, the quantities
$$
\ell_x= \max_i e_i^\top A y - \max_{i\notin BR(y)} e_i^\top A y,
$$
and
$$
\ell_y= \min_{j\notin BR(x)} x^\top A e_j - \min_j x^\top A e_j
$$
measure separation between best and non-best responses. The condition-like number is
$$
\kappa_A= \sup_{(x,y)\notin NE} \frac{\min\{\ell_x,\ell_y\}}{\psi(x,y)}.
$$
If $\delta>0$ is chosen sufficiently small, then after $T$ iterations AGFP satisfies
$$
\psi(x_T,y_T)=O\bigl(1/(\kappa_A T)\bigr),
$$
and, in particular, $(x_T,y_T)$ is an $O(\sqrt{\delta/\kappa_A})$-NE whenever $T\to\infty$ [2606.12149].

The proof sketch has four components. First, if the chosen pure best responses remain best responses at the next iterate, then
$$
\psi(x_{t+1},y_{t+1})=(1-\eta_t)\psi(x_t,y_t).
$$
Second, unique best responses remain optimal under small $\eta$, and multiple best responses that are "linearly indistinguishable" remain so under small $\eta$. Third, because the search is restricted to $\eta\ge \delta$, there are tie-breaking steps in which the unconstrained minimizer would satisfy $\eta<\delta$; these moves make the best-response set well-behaved for the next round and can only increase $\psi$ by $O(\delta)$. Fourth, combining the preceding ingredients yields, over any two consecutive iterations starting from a large $\psi$,
$$
\psi(z_{t+2}) \le \psi(z_t) - (\kappa_A/4)\cdot \psi(z_t)^2,
$$
from which a standard discrete-ODE argument gives $\psi(z_T)=O(1/T)$ [2606.12149].

The paper contrasts this with two benchmarks. Classical discrete fictitious play with step-sizes $1/(t+1)$ is stated to converge in zero-sum games but only at $O(1/\sqrt{T})$ in many settings, and it can be exponentially slow under adversarial tie-breaking. Continuous-time fictitious play enjoys an $O(1/T)$ rate, but discretization typically slows it down. AGFP is presented as matching the $O(1/T)$ rate of the continuous version without discretizing a differential equation [2606.12149].

Implementation details and experiments reinforce the algorithmic claim. The line search is implemented by binary-searching on the slope of $\psi(z(\eta))$, requiring $O(1)$ matrix-vector products per slope evaluation and about $50$ slope checks to reach machine precision. In practice, $\delta\approx 10^{-8}$–$10^{-11}$, though the paper notes that one can also adapt $\delta_t$ per iteration to satisfy the theoretical bounds. Experiments are reported on Rock-Paper-Scissors $(3\times 3)$ and random Gaussian games of size $50\times 50$ and $500\times 500$ with entries i.i.d. $N(0,1)$ normalized to $[0,1]$; the metric is $\psi(x_t,y_t)$ versus $t$ and versus wall-clock time [2606.12149].

The empirical observations are specific. In Rock-Paper-Scissors, the AGFP duality gap falls like $O(1/t)$ until it plateaus near $O(\sqrt{\delta})$. In random $n\times n$ games, the empirical rate is $O(n/T)$, namely $\psi\approx (\text{const}\cdot n)/t$, matching the reported behavior of continuous fictitious play in high dimensions. For the $50\times 50$ wall-clock comparison, to reach duality-gap $\approx 10^{-4}$, classical fictitious play takes approximately $195\,\mathrm{s}$ while AGFP takes approximately $4\,\mathrm{s}$, a speed-up of approximately $50\times$ [2606.12149].

The paper also records limitations and open problems. The method requires choosing $\delta$ or a schedule $\delta_t$ small enough to guarantee net decrease; too large $\delta$ yields a plateau at $O(\sqrt{\delta})$. The convergence constant $1/\kappa_A$ is game-dependent and may be large in degenerate cases. Each iteration incurs the extra line-search overhead, though the paper characterizes this as only tens of vector multiplies. Open questions include bounding $\kappa_A$ in natural ensembles such as random games, adaptive $\delta_t$ schedules that eliminate the plateau altogether, extension to smooth or strongly-convex payoff settings, AGFP in non-zero-sum games—especially potential games where pure NE exist—and fully characterizing optimal step sequences in higher-dimensional Rock-Paper-Scissors variants [2606.12149].

Source: https://www.emergentmind.com/topics/almost-greedy-fictitious-play