---
title: 'DCFR: Discounted Counterfactual Regret Minimization'
url: https://www.emergentmind.com/topics/discounted-cfr-dcfr
type: topic
---

# DCFR: Discounted Counterfactual Regret Minimization

Discounted Counterfactual Regret Minimization (DCFR) is a family of algorithms designed to accelerate convergence in extensive-form imperfect-information games by introducing temporal discounting into both regret and strategy averaging computations. DCFR directly generalizes Counterfactual Regret Minimization (CFR) and its variant CFR+, which are foundational for equilibrium computation in large extensive-form games such as poker. By modulating the influence of past iterations through discounting factors, DCFR enables the rapid decay of large historical mistakes and the attenuation of early strategy contributions, greatly improving empirical performance over its predecessors across a spectrum of benchmarks, particularly in high-variance gaming domains [1809.04040, 2404.09097, 2404.13891, 2511.08174].

## 1. Mathematical Formulation and Algorithmic Structure

DCFR parameterizes three discount exponents: $\alpha$ for positive regrets, $\beta$ for negative regrets, and $\gamma$ for strategy averaging. Let $R^t_i(I, a)$ denote the (possibly signed) cumulative regret for player $i$ at information set $I$ and action $a$ at iteration $t$. Let $r^t_i(I, a)$ be the instantaneous counterfactual regret, and $C^t_i(I, a)$ the cumulative strategy weight. The DCFR update equations are

\[
R^t_i(I, a) =
\begin{cases}
  \frac{(t-1)^\alpha}{(t-1)^\alpha+1} R^{t-1}_i(I,a) + r^t_i(I,a), & \text{if } R^{t-1}_i(I,a) > 0 \\
  \frac{(t-1)^\beta}{(t-1)^\beta+1} R^{t-1}_i(I,a) + r^t_i(I,a), & \text{otherwise}
\end{cases}
\]
\[
C^t_i(I, a) = C^{t-1}_i(I, a) \left( \frac{t-1}{t} \right)^\gamma + \pi_i^{\sigma^t}(I) \sigma_i^t(I, a)
\]

Here, $\pi_i^{\sigma^t}(I)$ denotes the reach probability of $I$ for player $i$ at iteration $t$. Regret-matching is performed on the nonnegative part $R^{t,+}_i(I,a) = \max(R^t_i(I,a), 0)$, and the strategy at $t+1$ is updated proportionally. Averaged strategies for output are weighted according to $w_t = t^\gamma$ [1809.04040, 2404.09097].

A summary of key DCFR hyperparameters is provided below:

| Symbol      | Control Target            | Typical Value   |
|-------------|--------------------------|-----------------|
| $\alpha$    | Pos. regret discount     | $1$ -- $2$      |
| $\beta$     | Neg. regret discount     | $0$ -- $1$      |
| $\gamma$    | Averaging discount       | $1$ -- $30$     |

The canonical parameter set for poker-like settings is $(\alpha, \beta, \gamma) = (1.5, 0, 2)$ [1809.04040].

## 2. Discounting Scheme and Regret-Weight Interaction

Unlike CFR, which uniformly accumulates all historical regrets, DCFR modulates the persistence of old updates. Positive regrets decay by a factor of $t^\alpha/(t^\alpha+1)$. Negative regrets can be decayed much more slowly or even left undiscounted (e.g., $\beta = 0$). This selective decay enables extremely rapid suppression of the influence of dominated or high-mistake actions, as old outlier regrets no longer dominate subsequent iterates [2404.13891, 1809.04040].

Strategy averaging is performed with weights decayed by $(t/(t+1))^\gamma$ per iteration, so that the effective average after $T$ iterations is

\[
\bar{\sigma}_i^T(I, a) = \frac{\sum_{t=1}^T t^\gamma \pi_i^{\sigma^t}(I) \sigma_i^t(I, a)}{\sum_{t=1}^T t^\gamma \pi_i^{\sigma^t}(I)}
\]

This polynomial weighting strongly emphasizes later iterations when $\gamma$ is large, in contrast to CFR's uniform averaging or CFR+'s linear weighting.

## 3. Theoretical Guarantees and Convergence Analysis

DCFR inherits the $O(1/\sqrt{T})$ convergence rate for $\epsilon$-Nash equilibrium in two-player zero-sum extensive-form games. Formally, if $\Delta$ is the maximal payoff range, $|\mathcal{I}|$ is the number of information sets, and $|A|$ is the maximal number of actions per infoset, then after $T$ iterations [1809.04040, 2404.09097]:

\[
\epsilon \le \frac{6 \Delta |\mathcal{I}| \left( \sqrt{|A|} + \frac{1}{\sqrt{T}} \right)}{\sqrt{T}}
\]

Empirically, DCFR achieves substantially lower exploitability faster than CFR or CFR+ for the same number of iterations when applied to large-scale, high-variance benchmarks such as Hold'em subgames. The hyperparameterization $(1.5, 0, 2)$ delivers 2--3$\times$ faster convergence than CFR+ in these settings [1809.04040, 2404.09097, 2404.13891].

## 4. Algorithmic Implementation and Pseudocode

A representative pseudocode outline for DCFR [1809.04040, 2404.13891]:

```python
# Inputs: iterations T, hyperparameters alpha, beta, gamma
for t in range(1, T+1):
    for each infoset I, action a:
        # Compute instantaneous regret
        r = v_sigma_t(I, a) - v_sigma_t(I)
        # Discounted regret update
        if R[I][a] > 0:
            discount = t**alpha / (t**alpha + 1)
        else:
            discount = t**beta / (t**beta + 1)
        R[I][a] = discount * R[I][a] + r
    # Regret-matching to form sigma^{t+1}
    # Discounted average strategy update
    decay = (t / (t+1))**gamma
    for each infoset I, action a:
        S[I][a] = decay * S[I][a] + pi_sigma_t(I) * sigma_t(I, a)
    D[I] = decay * D[I] + pi_sigma_t(I)
# Output: averaged strategy bar_sigma as S[I][a] / D[I]
```

Key implementation notes include in-place discounting to avoid numerical underflow, and the use of alternating updates for large-game efficiency [1809.04040].

## 5. Comparison to CFR, CFR+, and PCFR+; Advanced Extensions

DCFR generalizes CFR (no discounting: $(1, 1, 1)$) and CFR+ (linear averaging, regret clipping). CFR+ clips regrets at zero and applies $w_t = t$ weighting. DCFR introduces flexible polynomial decay and can be combined with regret clipping for variants referred to as DCFR+ [2404.13891, 2511.08174].

Further, PDCFR+ integrates DCFR-style discounting with the predictive/optimistic step of PCFR+, performing a predictive update on top of the discounted regrets before strategy computation. This hybrid yields state-of-the-art convergence, particularly in the presence of highly uneven losses typical in large poker domains [2404.13891]. In neural CFR, discounted and clipped cumulative advantages are bootstrapped in value networks to match the DCFR+ mechanism [2511.08174].

## 6. Empirical Performance, Hyperparameters, and Practical Guidelines

In poker-structured games and high-mistake normal-form games, DCFR and DCFR+ offer marked empirical speedup over both CFR+ and PCFR+, especially in early- to mid-stage learning (10$^3$–10$^4$ iterations). Dominated actions, if initially selected, have their adverse contribution rapidly suppressed. In small non-poker games, DCFR and PCFR+ are competitive, but the predictive-discounted hybrid (PDCFR+) ultimately yields the best performance [2404.09097, 2404.13891].

Recommended parameters for general use are $(\alpha, \beta, \gamma) = (1.5, 0, 2)$. When pruning is desired, $\beta$ should be increased to ensure large negative regrets for suboptimal actions (e.g., $(1.5, 0.5, 2)$). DCFR is also compatible with sampling/MCCFR methods, as periodic discounting at node-touched intervals preserves its acceleration properties [1809.04040].

Recent developments—such as hyperparameter schedules (HS-DCFR)—further accelerate game-solving by dynamically adapting discounting rates; e.g., starting with large $\gamma$ and linearly decreasing. This approach establishes a new state-of-the-art in practical convergence speed, outperforming both fixed-parameter DCFR and RL-based dynamic DCFR by orders of magnitude on standard benchmarks [2404.09097].

## 7. Extensions and Neural Implementations

Recent research has demonstrated that DCFR-style discounting can be efficiently integrated with neural policy/value approximation. At each update, the neural models are trained to simulate DCFR’s bootstrapped, clipped, and discounted advantage updates, and the cumulative strategy network uses polynomially downward-weighted averaging, as in the tabular version. In large-scale benchmarks, neural DCFR implementations exhibit improved convergence and adversarial robustness relative to vanilla neural CFR methods [2511.08174].

Integration with predictive updates, as in deep PDCFR+, combines both optimism and discounting, further improving stability and speed in settings with high variance and unbalanced payoff landscapes.

---

For comprehensive formal definitions, convergence theorems, and full algorithmic details see the original works: "Solving Imperfect-Information Games via Discounted Regret Minimization" [1809.04040], "Faster Game Solving via Hyperparameter Schedules" [2404.09097], "Minimizing Weighted Counterfactual Regret with Optimistic Online Mirror Descent" [2404.13891], and "Deep (Predictive) Discounted Counterfactual Regret Minimization" [2511.08174].

Source: https://www.emergentmind.com/topics/discounted-cfr-dcfr