---
title: Averaged Gradient Episodic Memory (A-GEM)
url: https://www.emergentmind.com/topics/averaged-gem-a-gem
type: topic
---

# Averaged Gradient Episodic Memory (A-GEM)

Averaged Gradient Episodic Memory (A-GEM) is a continual learning algorithm designed to balance computational efficiency, memory economy, and resistance to catastrophic forgetting in single-pass lifelong learning scenarios. A-GEM is an advancement over Gradient Episodic Memory (GEM), offering similar or superior accuracy with dramatically lower computational and memory costs by introducing a novel projection constraint on the average past gradient [1812.00420].

## 1. Lifelong Learning Setup and Evaluation Protocols

In lifelong learning (LLL), the objective is to learn a predictor $f_\theta : \mathcal{X} \times \mathcal{T} \to \mathcal{Y}$—for example, a neural network parameterized by $\theta \in \mathbb{R}^P$—over a sequence of $T$ tasks. Each task $k$ is associated with a dataset $D_k = \{(x_i^k, t_i^k, y_i^k)\}_{i=1}^{n_k}$, where $x_i^k$ is the input, $t_i^k$ is a task descriptor, and $y_i^k$ is the label. The learner observes each instance exactly once, with all tasks presented in sequence.

To mitigate catastrophic forgetting, methods maintain a small episodic memory $M_k \subset D_k$, typically much smaller than the task dataset ($|M_k| \ll n_k$). The union of all past task memories before task $t$ is denoted $M = \bigcup_{k<t} M_k$.

A-GEM evaluations are conducted via a two-stream protocol:

- $D^{CV}$: A held-out stream for hyper-parameter optimization, allowing arbitrary replay.
- $D^{EV}$: An evaluation stream processed in a single pass with fixed hyper-parameters.

This separation prevents information leakage from evaluation tasks during hyper-parameter search and enforces a strictly single-pass regime for reporting metrics.

## 2. Evaluation Metrics

A-GEM is evaluated using several metrics that quantify accuracy and the dynamics of knowledge retention and acquisition:

**(a) Final Average Accuracy $A_T$:**
\[
A_k = \frac{1}{k} \sum_{j=1}^k a_{k, B_k, j}
\]
Here, $a_{k, B_k, j}$ is the test accuracy on task $j$ after training on all minibatches of task $k$; $A_T$ is the terminal metric.

**(b) Forgetting $F_T$:**
\[
F_k = \frac{1}{k-1} \sum_{j=1}^{k-1} \left[ \max_{l \leq k-1} a_{l, B_l, j} - a_{k, B_k, j} \right]
\]
This quantifies the deterioration in performance on previous tasks due to new learning.

**(c) Learning Curve Area (LCA$_\beta$):**
\[
Z_b = \frac{1}{T} \sum_{k=1}^T a_{k,b,k} \qquad LCA_\beta = \frac{1}{\beta+1}\sum_{b=0}^\beta Z_b
\]
LCA evaluates both few-shot and progressive learning by averaging accuracy up to $\beta$ training steps.

## 3. From GEM to A-GEM: Mathematical Formulation

GEM constrains gradient updates to avoid loss increases on any previous task's memory, projecting the current gradient $g$ onto the intersection of half-spaces:
\[
\text{Find } \tilde g = \arg\min_{z} \frac{1}{2}\|g-z\|^2 \quad \text{s.t.} \ \langle z, g_k \rangle \geq 0 \ \forall k < t
\]
with $g_k = \nabla_\theta \ell(f_\theta, M_k)$. This requires solving a quadratic program with $t-1$ constraints and storing all $g_k$.

A-GEM simplifies the constraint to a single condition on the average past gradient $g_\mathrm{ref}$, computed from a mini-batch sampled from $M$:
\[
\tilde g = \arg\min_z \frac{1}{2}\|g - z\|^2 \quad \text{s.t.}\ \langle z, g_\mathrm{ref} \rangle \geq 0
\]
If $\langle g, g_\mathrm{ref} \rangle \geq 0$, $g$ is untouched; else, the projection has a closed form:
\[
\tilde g = g - \frac{\langle g, g_\mathrm{ref} \rangle}{\langle g_\mathrm{ref}, g_\mathrm{ref} \rangle} g_\mathrm{ref}
\]
This single-constraint projection reduces computational complexity and storage, enabling scalability to longer task sequences and larger networks.

## 4. Algorithmic Implementation and Complexity

A-GEM maintains a global episodic memory $M$ with reservoir sampling to ensure a uniform selection from all encountered data. At each training step, a mini-batch from $M$ provides $g_\mathrm{ref}$. The update is as follows:

```text
for (x, y) in D_t:
    if M ≠ ∅:
        sample (x_ref, y_ref) ~ M
        g_ref ← ∇_θ ℓ(f_θ(x_ref, t), y_ref)
    else:
        g_ref ← 0
    g ← ∇_θ ℓ(f_θ(x, t), y)
    if ⟨g, g_ref⟩ < 0:
        g ← g − (⟨g, g_ref⟩ / ⟨g_ref, g_ref⟩) · g_ref
    θ ← θ − η · g
Update M with reservoir sampling from D_t
```
Inputs: training stream $D$, test sets $D^{test}$, learning rate $\eta$. Outputs: final $\theta$, accuracy matrix $A$.

**Complexity Table**

| Method         | Time (per step)                   | Memory                      |
|---             |---                                |---                           |
| Vanilla        | $O(P)$                            | $O(P + BH)$                  |
| EWC            | $O(P)$ + diag-updates             | $O(4P + BH)$                 |
| GEM            | $O(P t + \text{QP})$              | $O(P t + (B+M) H)$           |
| A-GEM          | $O(P + M H + 1)$ (≈$O(P)$)        | $O(2P + (B+M) H)$            |

Here $P$=#parameters, $B$=mini-batch size, $H$=activation size, $M$=episodic memory size. In practice, A-GEM is approximately $100\times$ faster and $10\times$ more memory efficient than GEM on MNIST/CIFAR.

## 5. Empirical Results and Benchmark Performance

Experiments evaluate A-GEM on Permuted MNIST, Split CIFAR-100, Split CUB, and Split AWA, using MLP and ResNet architectures. A-GEM's final accuracy ($A_T$) matches or slightly trails GEM (e.g., 89.1% vs 89.5% on MNIST) while outperforming all regularization-based baselines (EWC, PI, MAS, RWalk) in the single-pass regime (e.g., EWC: 68%, A-GEM: 89% on MNIST). Forgetting $F_T$ remains lowest among methods with bounded memory.

Incorporating compositional task descriptors with a joint-embedding model ("–je" variant) further improves $A_T$, 0-shot performance (LCA$_0$), and learning speed for A-GEM and other methods.

Normalized summary (Permuted MNIST, Split CIFAR):

| Method     | $A_T$ (%) ↑ | LCA$_{10}$ ↑ | Time ↓ | Mem ↓  |
|------------|----------|-----------|--------|--------|
| Vanilla    |   47.9   |   0.26    | 0.06   | 0.06   |
| EWC        |   68.3   |   0.27    | 0.14   | 0.14   |
| GEM        |   89.5   |   0.23    | 1.00   | 1.00   |
| A-GEM      |   89.1   |   0.29    | 0.14   | 0.11   |

A-GEM is thus Pareto-optimal in the joint space of accuracy, forgetting, LCA, time, and memory.

## 6. Ablations, Sensitivity Analyses, and Algorithmic Variants

A-GEM projections are only required on a small fraction of steps, in contrast to GEM's frequent constraints as tasks accumulate. The "Stochastic GEM" (s-GEM) variant, which randomly samples past constraints, is still more costly and slightly less effective than A-GEM.

EWC's efficacy is highly sensitive to the number of epochs and model capacity; in single-pass and small-network settings, it only marginally outperforms vanilla SGD. Only with over-parameterized models and multiple passes does EWC approach A-GEM's performance.

Hyper-parameter search spaces and selected settings are detailed in the appendix of [1812.00420].

## 7. Key Insights, Limitations, and Future Directions

A-GEM achieves the core objectives of lifelong learning—retaining prior knowledge and enabling forward transfer—while being computationally and memory efficient. The Learning Curve Area (LCA) metric, introduced alongside A-GEM, provides a finer quantification of few-shot learning dynamics.

Observed limitations include the residual gap between single-pass continual learning (even with A-GEM) and the multi-task upper bound (IID setting). Differences in LCA among advanced continual methods converge when catastrophic forgetting is controlled; the field thus requires strategies for enhancing positive backward transfer.

Natural extensions include applying A-GEM to unsupervised, reinforcement, or streaming non-i.i.d. learning settings. The open-source codebase is provided at https://github.com/facebookresearch/agem [1812.00420].

Source: https://www.emergentmind.com/topics/averaged-gem-a-gem