Almost Greedy Fictitious Play

• The paper demonstrates that almost greedy fictitious play unifies no-regret learning with approximate best-response dynamics, ensuring convergence to Nash equilibria.
• It shows that employing ε-best replies with vanishing error or a constrained greedy line search recovers continuous fictitious play behavior and achieves an O(1/T) convergence rate.
• The methodology highlights the trade-off between line search overhead and improved convergence performance, opening avenues for adaptive strategies in various game settings.

Searching arXiv for the specified papers and closely related context. Almost greedy fictitious play denotes a family of fictitious-play variants in which the stagewise update is not fully exact greedy play, yet the deviation from exact best-response behavior is controlled so that equilibrium convergence can still be established. In the literature represented by "No-regret Dynamics and Fictitious Play" (Viossat et al., 2012) and "Do Not Discretize, Optimize: Almost Greedy Fictitious Play" (Markakis et al., 10 Jun 2026), the term refers to two related but distinct constructions: a vanishing-error $\varepsilon_t$-best-reply process induced by potential-based no-regret dynamics, and a zero-sum algorithm that performs a constrained greedy line search along the segment joining the current mixed strategy to the current pure best-response profile. Both formulations retain the core fictitious-play motif—responding to empirical or current opponent behavior by moving toward best replies—while modifying the exact notion of greediness.

1. Terminological scope and historical placement

In the 2012 formulation, almost greedy fictitious play is introduced as a natural variant of classical fictitious play in which, at each stage, players choose not exact best replies but $\varepsilon$-best replies to the empirical distribution of their opponents’ past play, with the perturbation $\varepsilon$ vanishing over time (Viossat et al., 2012). In the 2026 formulation, Almost-Greedy Fictitious Play (AGFP) is a specific zero-sum algorithm in which each iteration first computes pure best responses and then chooses a stepsize greedily by minimizing the duality gap along a constrained portion of the line from the current mixed strategy to that pure best-response profile (Markakis et al., 10 Jun 2026).

Variant	Core update	Context
Vanishing-error almost greedy fictitious play	Choose $\varepsilon_t$-best replies with $\varepsilon_t \to 0$	Realized by potential-based no-regret dynamics
AGFP with line search	Best-response step with $\eta_t=\arg\min_{\eta\in[\delta,1]} \psi(z(\eta))$	Zero-sum games and duality-gap minimization

The first variant is primarily a dynamical-systems and learning-theoretic equivalence result: potential-based no-regret dynamics can be viewed as an $\varepsilon_t$-perturbed fictitious-play process. The second is an optimization-oriented algorithmic redesign: rather than discretizing continuous fictitious play, it directly optimizes the one-step update and obtains an instance-dependent $\mathcal{O}(1/T)$ rate with respect to the duality gap (Markakis et al., 10 Jun 2026).

A common misconception is that "almost greedy fictitious play" names a single standard algorithm. The literature here indicates otherwise. The 2012 and 2026 usages share the idea of approximate greediness but differ in state variable, update rule, and proof technique. This suggests a broader organizing principle: fictitious-play dynamics can be relaxed either through controlled best-response error or through constrained optimization of the interpolation step.

2. Vanishing-error best replies and regret-based foundations

The 2012 development begins with repeated play in a finite game. Each player $i$ has finite action set $A_i$, realized average payoff

$\varepsilon$0

and, for any fixed action $\varepsilon$1, the counterfactual average payoff

$\varepsilon$2

The external regret of player $\varepsilon$3 for action $\varepsilon$4 at time $\varepsilon$5 is

$\varepsilon$6

and the maximal regret is

$\varepsilon$7

A no-regret strategy guarantees almost surely

$\varepsilon$8

(Viossat et al., 2012).

Potential-based no-regret dynamics of Hart–Mas-Colell style select in period $\varepsilon$9 a mixed action $\varepsilon$0 whose probability on action $\varepsilon$1 is proportional to the partial derivative of a convex potential $\varepsilon$2 at the regret vector $\varepsilon$3:

$\varepsilon$4

whenever $\varepsilon$5; if $\varepsilon$6 the player repeats some fixed pure action (Viossat et al., 2012). Under the stated conditions on $\varepsilon$7—twice differentiable, convex, zero on the nonpositive orthant, and with positive gradient only on strictly positive coordinates—two properties hold: $\varepsilon$8 almost surely, and every realized action $\varepsilon$9 played satisfies $\varepsilon_t$0 whenever $\varepsilon_t$1 (Viossat et al., 2012).

The key lemma is the $\varepsilon_t$2-best-reply selection statement. If $\varepsilon_t$3, then every action $\varepsilon_t$4 with $\varepsilon_t$5 is an $\varepsilon_t$6-best reply to the opponents’ empirical distribution $\varepsilon_t$7. Consequently, with

$\varepsilon_t$8

the mixed action satisfies

$\varepsilon_t$9

where

$\varepsilon_t \to 0$0

This is exactly the 2012 notion of almost greedy fictitious play: at each stage, players choose $\varepsilon_t \to 0$1-best replies with vanishing $\varepsilon_t \to 0$2 (Viossat et al., 2012).

3. Equivalence with potential-based no-regret dynamics

The principal conceptual contribution of the 2012 paper is that a wide class of potential-based no-regret dynamics can be realized as almost greedy fictitious play, and that this realization permits alternative and sometimes much shorter proofs of convergence results (Viossat et al., 2012). The equivalence is not merely heuristic. It identifies the stagewise mixed actions generated by regret-based procedures with an explicit $\varepsilon_t \to 0$3-best-response correspondence, where the perturbation level is the maximal regret.

This equivalence has two immediate consequences. First, it clarifies the operational meaning of potential-based no-regret dynamics: rather than viewing them only as regret-minimizing procedures, one may interpret them as fictitious-play processes with asymptotically vanishing greediness error. Second, it transfers convergence information from continuous fictitious play to no-regret dynamics whenever the corresponding perturbed differential inclusion has the same attractor structure (Viossat et al., 2012).

The special case of the $\varepsilon_t \to 0$4-norm potential, identified in the paper as "regret matching," yields a concrete rate statement. If

$\varepsilon_t \to 0$5

then

$\varepsilon_t \to 0$6

Thus the perturbation level in almost greedy fictitious play decays at the same rate as the maximal regret induced by the selected potential (Viossat et al., 2012).

A plausible implication is that almost greediness provides a unifying interface between online learning and game dynamics: regret bounds determine the admissible best-response error, while fictitious-play attractor theory determines the limiting empirical behavior. The 2012 exposition makes this interface explicit.

4. Convergence through perturbed continuous fictitious play

The convergence theorem of Viossatt and Zapechelnyuk considers any two-player game in which continuous fictitious play (CFP) has a global attractor $\varepsilon_t \to 0$7; examples given in the data include zero-sum games and potential games, where $\varepsilon_t \to 0$8 is the set of Nash equilibria (Viossat et al., 2012). For any potential-based no-regret dynamics, viewed as almost greedy fictitious play with $\varepsilon_t \to 0$9, two conclusions follow: the perturbation level $\eta_t=\arg\min_{\eta\in[\delta,1]} \psi(z(\eta))$0 vanishes at the same rate as the maximal regret under the chosen potential, and the empirical distribution

$\eta_t=\arg\min_{\eta\in[\delta,1]} \psi(z(\eta))$1

converges almost surely to $\eta_t=\arg\min_{\eta\in[\delta,1]} \psi(z(\eta))$2 (Viossat et al., 2012).

The proof strategy relies on perturbed differential inclusions. The discrete-time beliefs $\eta_t=\arg\min_{\eta\in[\delta,1]} \psi(z(\eta))$3 satisfy

$\eta_t=\arg\min_{\eta\in[\delta,1]} \psi(z(\eta))$4

with $\eta_t=\arg\min_{\eta\in[\delta,1]} \psi(z(\eta))$5 and $\eta_t=\arg\min_{\eta\in[\delta,1]} \psi(z(\eta))$6, so the process is an $\eta_t=\arg\min_{\eta\in[\delta,1]} \psi(z(\eta))$7-payoff-perturbed fictitious-play trajectory (Viossat et al., 2012). A graph-perturbation lemma is then invoked to replace payoff perturbations by the standard sup-norm perturbation used in Benaïm–Hofbauer–Sorin (2005). After linear interpolation over $\eta_t=\arg\min_{\eta\in[\delta,1]} \psi(z(\eta))$8, one obtains a continuous-time path $\eta_t=\arg\min_{\eta\in[\delta,1]} \psi(z(\eta))$9 that is a perturbed solution of the CFP inclusion

$\varepsilon_t$0

with perturbation parameter tending to zero (Viossat et al., 2012).

By Theorem 3.6 of Benaïm–Hofbauer–Sorin, the limit set of the interpolated path is internally chain transitive (ICT) for CFP; because $\varepsilon_t$1 is the unique global attractor, every ICT set must lie in $\varepsilon_t$2 (Viossat et al., 2012). The resulting implication chain is summarized in the paper as

$\varepsilon_t$3

In zero-sum games, and more generally in any game whose attractor under CFP is the Nash set, this yields convergence of the empirical distribution to Nash equilibrium. The same conclusion is stated for potential games and strictly dominance-solvable games (Viossat et al., 2012). The significance is methodological as much as substantive: the equivalence with almost greedy fictitious play replaces more specialized regret-matching martingale arguments with a shorter dynamical-systems proof.

5. The 2026 AGFP algorithm for zero-sum games

The 2026 AGFP algorithm is formulated for a two-player zero-sum game with payoff matrix $\varepsilon_t$4, row and column pure-strategy sets $\varepsilon_t$5 and $\varepsilon_t$6, and mixed strategies $\varepsilon_t$7, $\varepsilon_t$8 (Markakis et al., 10 Jun 2026). A pair $\varepsilon_t$9 is a Nash equilibrium iff, for every $\mathcal{O}(1/T)$0 and $\mathcal{O}(1/T)$1,

$\mathcal{O}(1/T)$2

The distance to equilibrium is measured by the duality gap

$\mathcal{O}(1/T)$3

which satisfies $\mathcal{O}(1/T)$4 by von Neumann’s minimax theorem and vanishes exactly at Nash equilibrium; if $\mathcal{O}(1/T)$5, then $\mathcal{O}(1/T)$6 is an $\mathcal{O}(1/T)$7-NE (Markakis et al., 10 Jun 2026).

At iteration $\mathcal{O}(1/T)$8, given $\mathcal{O}(1/T)$9, the row player computes

$i$0

and the column player computes

$i$1

For any $i$2, the candidate profile is

$i$3

with

$i$4

The AGFP step then chooses

$i$5

where $i$6 is a small lower bound (Markakis et al., 10 Jun 2026).

Because

$i$7

is convex and piecewise-linear in $i$8, the minimizer can be found exactly in $i$9 or via a binary search on the slopes, approximately $A_i$0 steps to machine precision (Markakis et al., 10 Jun 2026). The update is then

$A_i$1

The design principle differs sharply from standard discrete fictitious play with step-sizes $A_i$2. Instead of following a predetermined averaging schedule, AGFP greedily optimizes the current step over an interval that includes almost all the line between the cumulative mixed strategy and the current best response. The paper explicitly presents this as an alternative to discretization of continuous fictitious play (Markakis et al., 10 Jun 2026).

6. Rates, empirical behavior, and limitations

The main theoretical result of the 2026 paper is an instance-dependent $A_i$3 convergence rate for the duality gap (Markakis et al., 10 Jun 2026). For any non-equilibrium $A_i$4, the quantities

$A_i$5

and

$A_i$6

measure separation between best and non-best responses. The condition-like number is

$A_i$7

If $A_i$8 is chosen sufficiently small, then after $A_i$9 iterations AGFP satisfies

$\varepsilon$00

and, in particular, $\varepsilon$01 is an $\varepsilon$02-NE whenever $\varepsilon$03 (Markakis et al., 10 Jun 2026).

The proof sketch has four components. First, if the chosen pure best responses remain best responses at the next iterate, then

$\varepsilon$04

Second, unique best responses remain optimal under small $\varepsilon$05, and multiple best responses that are "linearly indistinguishable" remain so under small $\varepsilon$06. Third, because the search is restricted to $\varepsilon$07, there are tie-breaking steps in which the unconstrained minimizer would satisfy $\varepsilon$08; these moves make the best-response set well-behaved for the next round and can only increase $\varepsilon$09 by $\varepsilon$10. Fourth, combining the preceding ingredients yields, over any two consecutive iterations starting from a large $\varepsilon$11,

$\varepsilon$12

from which a standard discrete-ODE argument gives $\varepsilon$13 (Markakis et al., 10 Jun 2026).

The paper contrasts this with two benchmarks. Classical discrete fictitious play with step-sizes $\varepsilon$14 is stated to converge in zero-sum games but only at $\varepsilon$15 in many settings, and it can be exponentially slow under adversarial tie-breaking. Continuous-time fictitious play enjoys an $\varepsilon$16 rate, but discretization typically slows it down. AGFP is presented as matching the $\varepsilon$17 rate of the continuous version without discretizing a differential equation (Markakis et al., 10 Jun 2026).

Implementation details and experiments reinforce the algorithmic claim. The line search is implemented by binary-searching on the slope of $\varepsilon$18, requiring $\varepsilon$19 matrix-vector products per slope evaluation and about $\varepsilon$20 slope checks to reach machine precision. In practice, $\varepsilon$21–$\varepsilon$22, though the paper notes that one can also adapt $\varepsilon$23 per iteration to satisfy the theoretical bounds. Experiments are reported on Rock-Paper-Scissors $\varepsilon$24 and random Gaussian games of size $\varepsilon$25 and $\varepsilon$26 with entries i.i.d. $\varepsilon$27 normalized to $\varepsilon$28; the metric is $\varepsilon$29 versus $\varepsilon$30 and versus wall-clock time (Markakis et al., 10 Jun 2026).

The empirical observations are specific. In Rock-Paper-Scissors, the AGFP duality gap falls like $\varepsilon$31 until it plateaus near $\varepsilon$32. In random $\varepsilon$33 games, the empirical rate is $\varepsilon$34, namely $\varepsilon$35, matching the reported behavior of continuous fictitious play in high dimensions. For the $\varepsilon$36 wall-clock comparison, to reach duality-gap $\varepsilon$37, classical fictitious play takes approximately $\varepsilon$38 while AGFP takes approximately $\varepsilon$39, a speed-up of approximately $\varepsilon$40 (Markakis et al., 10 Jun 2026).

The paper also records limitations and open problems. The method requires choosing $\varepsilon$41 or a schedule $\varepsilon$42 small enough to guarantee net decrease; too large $\varepsilon$43 yields a plateau at $\varepsilon$44. The convergence constant $\varepsilon$45 is game-dependent and may be large in degenerate cases. Each iteration incurs the extra line-search overhead, though the paper characterizes this as only tens of vector multiplies. Open questions include bounding $\varepsilon$46 in natural ensembles such as random games, adaptive $\varepsilon$47 schedules that eliminate the plateau altogether, extension to smooth or strongly-convex payoff settings, AGFP in non-zero-sum games—especially potential games where pure NE exist—and fully characterizing optimal step sequences in higher-dimensional Rock-Paper-Scissors variants (Markakis et al., 10 Jun 2026).