Iterated Regret Minimization in Game Theory

• Iterated Regret Minimization (IRM) is a solution concept that iteratively prunes strategies based on maximum regret minimization to capture more realistic decision making.
• IRM employs iterative deletion through algorithms like dynamic programming and backward induction, ensuring convergence even in complex or graph-based games.
• IRM outcomes often align with experimental results in games such as Traveler’s Dilemma and the Centipede Game, providing a robust alternative to Nash equilibrium.

Iterated Regret Minimization (IRM) is a solution concept in non-cooperative game theory grounded in the iterative deletion of strategies that do not locally minimize a well-defined notion of regret. Originating in the context of finite normal-form games to better match empirical choice behavior—especially in settings where Nash equilibrium predictions fail—IRM has been extended to graph-structured, infinite-duration, and quantitative games, as well as algorithmic frameworks for large-scale imperfect information games. It provides a principled refinement of rationalizability, yielding often more empirically-plausible solutions than Nash equilibria, particularly by robustly capturing cooperative and risk-averse decision tendencies.

1. Formal Definitions and Solution Procedures

Regret in Normal-Form Games

Given a finite normal-form game $$G=([n], A=A_1\times\cdots\times A_n, u=(u_1,\dots,u_n))$$, the regret $R_i(a_i|a_{-i})$ of playing $a_i\in A_i$ against $a_{-i}\in A_{-i}$ is defined as the difference between the best payoff achievable for player $i$ (relative to the available strategy set) and the actual payoff: $$u_i^{S_i}(a_{-i}) = \max_{a_i' \in S_i} u_i(a_i', a_{-i}),$$

$$R_i(a_i | a_{-i}) = u_i^{S_i}(a_{-i}) - u_i(a_i, a_{-i}).$$

The maximum regret of $a_i$ over all other players' strategies in $S_{-i}$ is $$R_i(a_i|S_{-i}) = \max_{a_{-i} \in S_{-i}} R_i(a_i|a_{-i})$$. The minimal regret among actions in $S_i$ is $m_i(S) = \min_{a_i \in S_i} R_i(a_i|S_{-i})$. The set of regret-minimizing actions is $$RM_i(S) = \{a_i \in S_i : R_i(a_i|S_{-i}) = m_i(S)\}$$ (0810.3023).

The Iterated Regret Minimization Operator

IRM is defined by iteratively applying a deletion operator: from the current strategy set, omit any action that does not achieve the minimal possible maximum regret given others' surviving strategies. Starting from all actions, the process proceeds in stages:

• Initialization: $S_i^0 = A_i$.
• Iteration $k$: $S_i^k = RM_i(S^{k-1})$ for each $i$.
• Termination: At the fixed-point $S^\infty = \cap_k S^k$, yielding the set of IRM-surviving profiles (0810.3023).

Extension to Game Graphs

For games with explicit graph structure $G=(V_1 \cup V_2, E)$ and edge weights $w_i(e) \in \mathbb{Z}$ for each player, payoffs are path-dependent: $u_i(\pi) = \sum_{k=0}^{\infty} w_i(e_k),$ with reachability objectives yielding infinite cost unless a target $T_i$ is reached. Strategies map histories to choices at each node (Filiot et al., 2010).

The single-shot regret $r_i(s_1, s_2)$ is $U_i(s_1, s_2) - \mathrm{BR}_i(s_{-i})$, with best-response $$\mathrm{BR}_i(s_{-i}) = \min_{s_i'} U_i(s_i', s_{-i})$$. The worst-case regret $R_i(s_i) = \max_{s_{-i}} r_i(s_i, s_{-i})$, and the minimal achievable regret $R_i(G) = \min_{s_i} R_i(s_i)$ (Filiot et al., 2010).

IRM on graphs iterates this regret calculation and successively prunes the strategy space at each node, analogously to the matrix case, but with deletion manifesting as edge pruning in the graph (Filiot et al., 2010).

2. Theoretical Properties and Fixed-Point Behavior

IRM possesses a number of formal properties:

• Existence and Convergence: For finite games, the set sequence stabilizes in at most $\sum_i |A_i|$ stages, always yielding a nonempty fixed point $S^\infty = RM(S^\infty)$.
• Order Independence: Simultaneous deletion of non-minimal regret actions yields a unique fixed point, independent of deletion order.
• Comparison to Standard Concepts: If a player has a dominant action, it is the unique IRM survivor. IRM outcomes are generically strict subsets of rationalizable strategies, but are not necessarily subsets of Nash equilibria nor conversely (0810.3023).

In graph-based (infinite-duration) settings, the strategy spaces become uncountably infinite, and computational properties depend on graph and weight structure. IRM correctly specializes to the matrix case, but the computational difficulty increases substantially (Filiot et al., 2010).

3. Algorithmic Procedures for IRM

The algorithmic instantiation of IRM depends on game representation:

Game Class	Algorithm Complexity	Remarks
Target-Weighted Arenas (TWA)	$O(\vert C_1\vert \log M_1(|S|+|E|))$	Best-value and best-alternative annotation; memoryless
General Weighted Graphs	$O(M^2 \log(|S|M) |S||C_1|(|S|+|E|))$	Requires pseudo-polynomial unfolding to TWA
Finite Trees	$O(|S|^2)$	Bottom-up DP, convergence in at most $|S|$ steps
Strictly-Positive Arenas	Pseudo-exponential	Unfold to bounded cost, reduce to tree case

Key techniques include backward induction, best-alternative graph augmentation, dynamic programming, and bounded unfolding. For trees, all IRM strategies are memoryless and amenable to efficient DP; in positive-weight graphs, solution size may be pseudo-exponential due to cost-bounding (Filiot et al., 2010).

4. Illustrative Examples and Empirical Phenomena

IRM outputs diverge substantially from Nash equilibrium in exemplary settings:

• Traveler’s Dilemma: Nash equilibrium predicts minimal bids, but IRM deletion sequence selects bids near the upper range (decreasing in penalty parameter), matching experimental data (0810.3023).
• Centipede Game: Nash equilibrium is immediate exit; IRM prescribes substantial cooperation (e.g., “stop at round $k-p+1$” for linear, or full cooperation for exponential payoffs) (0810.3023).
• Bertrand Competition and Nash Bargaining: IRM typically yields fair and high-profit equilibria, also aligning better with observed behavior (0810.3023).
• Small Graph Example: In a graph with nodes A, B, and C, with edges and weights as specified in (Filiot et al., 2010), IRM identifies "go to C" unambiguously as the unique rationalizable play for Player 1 after one deletion iteration.

IRM thus provides robust predictions in games where Nash equilibrium fails to align with empirical patterns, particularly in one-shot and finitely repeated contexts.

5. IRM in Extensive-Form and Imperfect-Information Games

IRM provides a conceptual umbrella for modern regret-minimization algorithms applied to large extensive-form settings, notably via counterfactual regret minimization (CFR). Variants such as DCFR, LCFR, and NormalHedge instantiate the IRM principle algorithmically by iteratively minimizing (discounted or weighted) regret at each information set and reweighting strategies over time (Brown et al., 2018).

Discounting positive and negative regret differently (parameterized by $(\alpha,\beta,\gamma)$) accelerates convergence—e.g., CFR$^+$ as DCFR$_{\infty,-\infty,2}$, or DCFR$_{3/2,0,2}$ for optimal trade-off between discounting and pruning. Practical advances include:

• Substantial reductions in exploitability convergence time for large poker subgames (2–3$\times$ over CFR$^+$).
• Compatibility with regret-based pruning and sampling methods, depending on negative-regret discounting.
• Parameter-free minimizers (NormalHedge) and optimistic updates accelerate early-stage convergence in certain regimes, but may increase per-node computational cost (Brown et al., 2018).

These methodologies realize IRM as an operational tool for approximating equilibrium and rational play in high-complexity games.

6. Comparative Perspective and Behavioral Motivation

IRM, as introduced by Halpern and Pass, was motivated by empirical failures of Nash equilibrium and rationalizability to predict actual choices in well-studied games. Key comparative insights:

• IRM captures robust, "hedging" attitudes by minimizing regret under maximal uncertainty about opponents.
• Iterative deletion mirrors cognitive hierarchy and bounded rationality models, interpreting successive rounds as increasingly sophisticated beliefs (0810.3023).
• In a variety of laboratory and field contexts—including Traveler’s Dilemma, Centipede Game, Bertrand Competition, and bargaining—IRM delivers solution profiles closely matching observed data, in contrast to Nash equilibrium's typically non-cooperative predictions.

A plausible implication is that IRM-based methodologies form a foundational basis for solution concepts emphasizing empirical adequacy and realistic decision-making under uncertainty.

---

References

• "Iterated Regret Minimization: A More Realistic Solution Concept" (0810.3023)
• "Iterated Regret Minimization in Game Graphs" (Filiot et al., 2010)
• "Solving Imperfect-Information Games via Discounted Regret Minimization" (Brown et al., 2018)