Smooth Regret-Matching Methods

• Smooth regret-matching is a family of methods that replace jumpy threshold-and-renormalize updates with smoother, stable dynamics in game-theoretic settings.
• It enhances stability in two-player zero-sum games by employing geometric interpolation, projected extragradient, and predictive corrections to maintain monotonic regret norms.
• These techniques enable faster equilibrium convergence with sublinear regret bounds and preserve practical properties like parameter-freeness and scale invariance in extensive-form games.

Smooth regret-matching denotes a line of modifications of regret matching, regret matching$^+$, and predictive regret matching$^+$ that replace jumpy threshold-and-renormalize dynamics with smoother or more stable updates on the simplex or on a lifted positive cone. In two-player zero-sum games, these methods are studied through online learning and counterfactual regret minimization (CFR), where sublinear regret implies equilibrium convergence through the duality gap of average play. Across the literature, “smoothness” has been instantiated geometrically, through projected extragradient or optimistic corrections, through removing the singular region near the origin, and, most recently, through enforcing monotonic growth of the regret norm without adding a smoothing hyperparameter (Zhang et al., 6 Oct 2025, Cai et al., 2023, Farina et al., 2023, Lan, 2019).

1. Problem setting and role in zero-sum game solving

The standard setting is a two-player zero-sum game with strategy spaces $X$ and $Y$. On round $t$, the players choose strategies $x^{(t)} \in X$, $y^{(t)} \in Y$, observe linear feedback

$$u_x^{(t)} = Ay^{(t)}, \qquad u_y^{(t)} = -A^\top x^{(t)},$$

and seek sublinear regret: $$Reg^{(T)}_x=\max_{x^* \in X}\sum_{t=1}^T \langle x^*-x^{(t)},u_x^{(t)}\rangle, \qquad Reg^{(T)}_y=\max_{y^* \in Y}\sum_{t=1}^T \langle y^*-y^{(t)},u_y^{(t)}\rangle.$$ For zero-sum games, the average profile $(\bar x^{(T)},\bar y^{(T)})$ satisfies

$^+$0

so regret minimization directly yields equilibrium convergence (Zhang et al., 6 Oct 2025).

This is the operational reason regret matching matters: it is the backbone of CFR, which is the dominant practical framework for solving large extensive-form games such as poker. Its appeal is not only theoretical but also algorithmic: regret matching and its variants are parameter-free and scale-invariant, which makes them robust in large, heterogeneous game trees (Zhang et al., 6 Oct 2025). The central tension in the literature is that the practically dominant family—especially predictive regret matching$^+$1 with non-uniform averaging—has often outperformed classical first-order methods empirically, even though optimal $^+$2 convergence had long been associated with mirror-prox and excessive-gap techniques rather than with regret-matching-style updates (Zhang et al., 6 Oct 2025).

2. From jumpy updates to predictive and geometrical variants

An early explicit smoothing proposal is “Geometrical Regret Matching” (Lan, 2019). Its motivation is that standard regret matching, in the Hart–Mas-Colell sense, can be “jumpy” because future play probabilities are set proportional to positive regret measures. The geometrical update keeps the past mixed strategy relevant and continuously moves the strategy vector toward the current regret vector: $^+$3 In this construction, less profitable pure strategies are continuously suppressed rather than abruptly discarded, the path remains in the simplex, and the iteration is framed as a fixed-point scheme for approximating Nash equilibrium (Lan, 2019).

A different development came from Blackwell approachability. “Faster Game Solving via Predictive Blackwell Approachability: Connecting Regret Matching and Mirror Descent” showed that RM and RM$^+$4 arise exactly from FTRL and OMD, respectively, when those methods are used to choose the separating halfspace in a Blackwell approachability game (Farina et al., 2020). This perspective yields predictive regret matching (PRM) and predictive regret matching$^+$5 (PRM$^+$6), and it places the regret-matching family inside the same analytical ecosystem as predictive FTRL and predictive OMD. In the simplex case, the forcing action is

$^+$7

applied to the positive part of the cumulative regret direction, so normalization cancels the Euclidean stepsize and preserves scale invariance (Farina et al., 2020).

Empirically, predictive RM$^+$8 coupled with CFR was reported to converge vastly faster than the fastest prior algorithms across 18 common zero-sum extensive-form benchmark games, across all games but two of the poker games, sometimes by two or more orders of magnitude (Farina et al., 2020). This practical success made PRM$^+$9 the benchmark to explain, rather than merely a baseline to replace.

3. Instability as the central obstacle

The modern smooth regret-matching literature is driven by a specific diagnosis: the raw regret-matching family can be unstable. In the 2025 scale-invariant analysis, the basic flaw of PRM$X$0 is that the $X$1-norm of the regret vector can decrease. Interpreting the regret norm as the inverse learning rate, this means the effective learning rate can increase unpredictably, which destabilizes iterates and makes prediction harder for the opponent (Zhang et al., 6 Oct 2025). This provides a direct mechanism linking thresholded regret dynamics to failures of smooth optimistic behavior.

The instability picture had already been made explicit in “Regret Matching+: (In)Stability and Fast Convergence in Games” (Farina et al., 2023). There, RM$X$2 and predictive RM$X$3 were shown to alternate between

$X$4

on a finite loss sequence in $X$5. The geometric reason is that normalization is highly sensitive near the origin: if the aggregate regret vector is small, a tiny perturbation can produce a large swing in the normalized strategy (Farina et al., 2023).

The last-iterate literature sharpened the same point. “Last-Iterate Convergence Properties of Regret-Matching Algorithms in Games” reports numerical evidence that RM$X$6, predictive RM$X$7, and alternating RM$X$8 all lack last-iterate convergence guarantees even on a simple $X$9 matrix game, while the main analytical obstacle is that their regret operators lack Lipschitzness and (pseudo)monotonicity (Cai et al., 2023). The 2025 scale-invariant paper adds a sharper convergence-theoretic statement: although PRM$Y$0 often performs well empirically, it can still converge as slowly as $Y$1, even in simple normal-form zero-sum games and even in self-play (Zhang et al., 6 Oct 2025).

A common misconception is therefore that predictive or thresholded regret matching is automatically “smooth” because it is optimistic or because it works well in CFR benchmarks. The papers do not support that conclusion. They instead show that empirical strength and dynamical stability are distinct properties, and that the latter requires additional structure (Cai et al., 2023, Zhang et al., 6 Oct 2025).

4. Main smoothing mechanisms

Several non-equivalent constructions have been used to smooth regret matching.

Variant	Smoothing mechanism	Representative source
Geometrical regret matching	Continuous interpolation toward regret vector	(Lan, 2019)
Smooth predictive RM$Y$2	Projection onto a chopped-off positive orthant	(Farina et al., 2023)
ExRM$Y$3, SPRM$Y$4	Projected extragradient / optimistic correction on lifted operator	(Cai et al., 2023)
IREG-PRM$Y$5	Monotone $Y$6-norm of regret vector	(Zhang et al., 6 Oct 2025)

In the chopped-off-orthant construction, the singular region near the origin is removed by restricting the lifted state to

$Y$7

Projected predictive updates on this domain define Smooth Predictive RM$Y$8, and the point of the restriction is that the normalization map $Y$9 becomes uniformly Lipschitz on the feasible set (Farina et al., 2023). This stabilizes dynamics without explicit resets.

The extragradient line of work lifts regret matching to a monotone-optimization-style operator on

$t$0

with operator

$t$1

ExRM$t$2 then applies projected extragradient,

$t$3

while SPRM$t$4 uses an OGDA-style pair of projected steps (Cai et al., 2023). The smoothing here comes from replacing raw threshold-and-renormalize dynamics by stable extragradient or optimistic correction on a Lipschitz lifted map.

The most recent construction, IREG-PRM$t$5, is explicitly parameter-free and scale-invariant. It maintains a nonnegative shifted-regret vector $t$6, and on each round chooses $t$7 so that

$t$8

Then

$t$9

followed by

$x^{(t)} \in X$0

Its extra-gradient instantiation uses lookahead predictions

$x^{(t)} \in X$1

The defining invariant is

$x^{(t)} \in X$2

so the regret norm never shrinks and the implicit learning rate $x^{(t)} \in X$3 never increases (Zhang et al., 6 Oct 2025). The key scalar $x^{(t)} \in X$4 can be computed exactly in linear time by solving $x^{(t)} \in X$5, using either sorting or linear-time selection (Zhang et al., 6 Oct 2025).

5. Convergence guarantees and the RVU perspective

A unifying analytical motif is the RVU template,

$x^{(t)} \in X$6

in which a positive variation term is offset by a negative stability term (Zhang et al., 6 Oct 2025). Predictive FTRL and predictive OMD already exhibit this structure, which is one reason the Blackwell approachability interpretation was so consequential (Farina et al., 2020).

For IREG-PRM$x^{(t)} \in X$7, the RVU-style inequality is intrinsic to the monotone-regret geometry rather than enforced by an extra smoothness hyperparameter. In the exact form given in the paper,

$x^{(t)} \in X$8

This is notable precisely because the negative term is generated by the construction itself (Zhang et al., 6 Oct 2025).

The resulting equilibrium guarantees are strong. For IREG-PRM, the average profile

$x^{(t)} \in X$9

is an $y^{(t)} \in Y$0-Nash equilibrium: $y^{(t)} \in Y$1 and there exists some iterate $y^{(t)} \in Y$2 whose Nash gap is $y^{(t)} \in Y$3 (Zhang et al., 6 Oct 2025). This closes the theory-practice gap identified in the paper: the method retains the aggressive, scale-free behavior associated with regret matching while recovering optimal average-iterate convergence.

The last-iterate line obtains a different but complementary guarantee. For ExRM$y^{(t)} \in Y$4 and SPRM$y^{(t)} \in Y$5, the papers prove asymptotic last-iterate convergence, $y^{(t)} \in Y$6 best-iterate convergence, and, with restarting, linear-rate last-iterate convergence under metric subregularity (Cai et al., 2023). In particular, ExRM$y^{(t)} \in Y$7 satisfies

$y^{(t)} \in Y$8

which implies boundedness, summability of step differences, and convergence of the lifted iterate to a point whose normalization is Nash (Cai et al., 2023).

The stabilization paper proves yet another guarantee profile. Restarting yields Stable Predictive RM$y^{(t)} \in Y$9 with $$u_x^{(t)} = Ay^{(t)}, \qquad u_y^{(t)} = -A^\top x^{(t)},$$0 individual regret, while the chopped-off-domain Smooth Predictive RM$$u_x^{(t)} = Ay^{(t)}, \qquad u_y^{(t)} = -A^\top x^{(t)},$$1 yields $$u_x^{(t)} = Ay^{(t)}, \qquad u_y^{(t)} = -A^\top x^{(t)},$$2 social regret in multiplayer normal-form games. Conceptual and extragradient RM$$u_x^{(t)} = Ay^{(t)}, \qquad u_y^{(t)} = -A^\top x^{(t)},$$3 variants on the chopped-off domain also satisfy constant-regret guarantees akin to clairvoyant OMD and mirror-prox results (Farina et al., 2023). This suggests a broad pattern: once the singular geometry of the raw RM$$u_x^{(t)} = Ay^{(t)}, \qquad u_y^{(t)} = -A^\top x^{(t)},$$4 state is controlled, optimistic game-solving guarantees become available.

6. Relation to adaptive first-order methods and empirical behavior

The strongest recent conceptual claim is that regret matching can be understood as a scale-invariant adaptive first-order method. The 2025 scale-invariant paper draws an explicit analogy between IREG-PRM$$u_x^{(t)} = Ay^{(t)}, \qquad u_y^{(t)} = -A^\top x^{(t)},$$5 and optimistic gradient descent with adaptive learning rate, called AdOGD, whose step size is chosen from cumulative misprediction error,

$$u_x^{(t)} = Ay^{(t)}, \qquad u_y^{(t)} = -A^\top x^{(t)},$$6

In both analyses, the misprediction error appears directly: accurate predictions tighten the bound, and imperfect predictions are paid for only through observed error (Zhang et al., 6 Oct 2025). This suggests a unifying interpretation in which regret matching’s empirical strength is not mysterious or uniquely tied to CFR; rather, it reflects a scale-invariant adaptive step-size principle.

This interpretation is consistent with the earlier Blackwell approachability reduction, where RM and RM$$u_x^{(t)} = Ay^{(t)}, \qquad u_y^{(t)} = -A^\top x^{(t)},$$7 arise exactly as FTRL and OMD instantiations for halfspace selection, and predictive RM$$u_x^{(t)} = Ay^{(t)}, \qquad u_y^{(t)} = -A^\top x^{(t)},$$8 inherits prediction-sensitive regret bounds of the form

$$u_x^{(t)} = Ay^{(t)}, \qquad u_y^{(t)} = -A^\top x^{(t)},$$9

in the simplex game (Farina et al., 2020). The smooth variants can be read as repairing the parts of the regret-matching dynamics that prevented this optimistic interpretation from yielding the same stability guarantees available for more standard Euclidean methods.

Empirically, IREG-PRM, PRM, AdOGD, and DCFR were benchmarked on liar’s dice, Kuhn poker, Leduc poker, a small Goofspiel variant, a Battleship variant, and the counterexample game from the theory section. The main findings are that IREG-PRM performs on par with, and sometimes better than, PRM across the tested games, and that AdOGD performs similarly to IREG-PRM (Zhang et al., 6 Oct 2025). The experiments also suggest that simultaneous variants close much of the gap with alternating updates, and that last-iterate convergence is often strikingly fast, sometimes appearing linear even though the formal guarantees are $$Reg^{(T)}_x=\max_{x^* \in X}\sum_{t=1}^T \langle x^*-x^{(t)},u_x^{(t)}\rangle, \qquad Reg^{(T)}_y=\max_{y^* \in Y}\sum_{t=1}^T \langle y^*-y^{(t)},u_y^{(t)}\rangle.$$0 for a best iterate and $$Reg^{(T)}_x=\max_{x^* \in X}\sum_{t=1}^T \langle x^*-x^{(t)},u_x^{(t)}\rangle, \qquad Reg^{(T)}_y=\max_{y^* \in Y}\sum_{t=1}^T \langle y^*-y^{(t)},u_y^{(t)}\rangle.$$1 for the average (Zhang et al., 6 Oct 2025).

The last-iterate paper reports a closely related empirical pattern: on a $$Reg^{(T)}_x=\max_{x^* \in X}\sum_{t=1}^T \langle x^*-x^{(t)},u_x^{(t)}\rangle, \qquad Reg^{(T)}_y=\max_{y^* \in Y}\sum_{t=1}^T \langle y^*-y^{(t)},u_y^{(t)}\rangle.$$2 game with unique Nash equilibrium, RM$$Reg^{(T)}_x=\max_{x^* \in X}\sum_{t=1}^T \langle x^*-x^{(t)},u_x^{(t)}\rangle, \qquad Reg^{(T)}_y=\max_{y^* \in Y}\sum_{t=1}^T \langle y^*-y^{(t)},u_y^{(t)}\rangle.$$3, alternating RM$$Reg^{(T)}_x=\max_{x^* \in X}\sum_{t=1}^T \langle x^*-x^{(t)},u_x^{(t)}\rangle, \qquad Reg^{(T)}_y=\max_{y^* \in Y}\sum_{t=1}^T \langle y^*-y^{(t)},u_y^{(t)}\rangle.$$4, and PRM$$Reg^{(T)}_x=\max_{x^* \in X}\sum_{t=1}^T \langle x^*-x^{(t)},u_x^{(t)}\rangle, \qquad Reg^{(T)}_y=\max_{y^* \in Y}\sum_{t=1}^T \langle y^*-y^{(t)},u_y^{(t)}\rangle.$$5 remain at duality gap around $$Reg^{(T)}_x=\max_{x^* \in X}\sum_{t=1}^T \langle x^*-x^{(t)},u_x^{(t)}\rangle, \qquad Reg^{(T)}_y=\max_{y^* \in Y}\sum_{t=1}^T \langle y^*-y^{(t)},u_y^{(t)}\rangle.$$6 even after $$Reg^{(T)}_x=\max_{x^* \in X}\sum_{t=1}^T \langle x^*-x^{(t)},u_x^{(t)}\rangle, \qquad Reg^{(T)}_y=\max_{y^* \in Y}\sum_{t=1}^T \langle y^*-y^{(t)},u_y^{(t)}\rangle.$$7 iterations, whereas ExRM$$Reg^{(T)}_x=\max_{x^* \in X}\sum_{t=1}^T \langle x^*-x^{(t)},u_x^{(t)}\rangle, \qquad Reg^{(T)}_y=\max_{y^* \in Y}\sum_{t=1}^T \langle y^*-y^{(t)},u_y^{(t)}\rangle.$$8, SPRM$$Reg^{(T)}_x=\max_{x^* \in X}\sum_{t=1}^T \langle x^*-x^{(t)},u_x^{(t)}\rangle, \qquad Reg^{(T)}_y=\max_{y^* \in Y}\sum_{t=1}^T \langle y^*-y^{(t)},u_y^{(t)}\rangle.$$9, and especially alternating PRM$(\bar x^{(T)},\bar y^{(T)})$0 reach machine precision quickly on the same instance (Cai et al., 2023). The stabilization paper likewise finds that Smooth and Stable PRM$(\bar x^{(T)},\bar y^{(T)})$1 remove pathological cycling on hard matrix games while remaining competitive with predictive RM$(\bar x^{(T)},\bar y^{(T)})$2 on random matrix games and several extensive-form settings (Farina et al., 2023).

7. Limits, controversies, and terminological scope

The literature is careful not to overstate the reach of smoothing. Geometrical regret matching proves unilateral monotonicity—payoff does not decrease and total regret does not increase under a unilateral update—but these properties do not automatically survive simultaneous updates by all players. Regret sums can fluctuate, convergence can be slow, some games exhibit oscillation or periodic cycles, and equilibrium points may behave as attractors in some games and repellors in others (Lan, 2019). The method is therefore presented as a heuristic-plus-analysis approach rather than a universal convergence theorem.

Likewise, predictive RM$(\bar x^{(T)},\bar y^{(T)})$3 should not be conflated with smooth regret matching in the stronger sense used by later papers. The predictive variants explain important empirical gains, but by themselves they do not resolve last-iterate instability or guarantee optimal $(\bar x^{(T)},\bar y^{(T)})$4 average-iterate convergence in matrix games (Cai et al., 2023, Zhang et al., 6 Oct 2025). The 2025 scale-invariant paper explicitly distinguishes its contribution from prior smooth regret-matching variants that attained $(\bar x^{(T)},\bar y^{(T)})$5 convergence only by introducing an extra smoothing hyperparameter and were empirically less competitive (Zhang et al., 6 Oct 2025).

There is also a terminological boundary with “smooth regret” in contextual bandits. That literature defines smooth regret against the best action distribution with density bounded by $(\bar x^{(T)},\bar y^{(T)})$6 relative to a base measure and uses inverse gap weighting as a regret-matching-style exploration rule in large or continuous action spaces (Zhu et al., 2022). A plausible implication is that “smooth regret-matching” now names two related but distinct strands: one concerned with stabilizing simplex dynamics in games, the other with relaxing the benchmark in large-action bandits. The core game-theoretic usage, however, remains the stabilization of regret-matching dynamics so that scale invariance and parameter-freeness can coexist with the smoothness properties needed for fast and robust convergence (Zhang et al., 6 Oct 2025, Farina et al., 2023).

Taken together, the literature establishes a precise shift in viewpoint. Regret matching is no longer treated merely as a heuristic local rule inside CFR; smooth regret-matching methods recast it as a family of adaptive first-order procedures whose geometry can be engineered. The most successful constructions preserve the practical virtues of regret matching—scale invariance, parameter-freeness, and aggressiveness—while replacing jumpy or unstable behavior with monotone-regret, projected-extragradient, or chopped-domain dynamics that support RVU-type analysis and substantially stronger convergence guarantees (Zhang et al., 6 Oct 2025, Cai et al., 2023).