Averaged Gradient Episodic Memory (A-GEM)

• A-GEM is a continual learning algorithm that mitigates catastrophic forgetting by using a single projection constraint based on the average past gradient.
• It achieves competitive accuracy, e.g., around 89.1% on MNIST, while drastically reducing computational and memory costs compared to GEM.
• The method employs reservoir sampling for episodic memory and a strict two-stream evaluation protocol to ensure robustness in single-pass lifelong learning.

Averaged Gradient Episodic Memory (A-GEM) is a continual learning algorithm designed to balance computational efficiency, memory economy, and resistance to catastrophic forgetting in single-pass lifelong learning scenarios. A-GEM is an advancement over Gradient Episodic Memory (GEM), offering similar or superior accuracy with dramatically lower computational and memory costs by introducing a novel projection constraint on the average past gradient (Chaudhry et al., 2018).

1. Lifelong Learning Setup and Evaluation Protocols

In lifelong learning (LLL), the objective is to learn a predictor $$f_\theta : \mathcal{X} \times \mathcal{T} \to \mathcal{Y}$$—for example, a neural network parameterized by $\theta \in \mathbb{R}^P$—over a sequence of $T$ tasks. Each task $k$ is associated with a dataset $D_k = \{(x_i^k, t_i^k, y_i^k)\}_{i=1}^{n_k}$, where $x_i^k$ is the input, $t_i^k$ is a task descriptor, and $y_i^k$ is the label. The learner observes each instance exactly once, with all tasks presented in sequence.

To mitigate catastrophic forgetting, methods maintain a small episodic memory $M_k \subset D_k$, typically much smaller than the task dataset ($|M_k| \ll n_k$). The union of all past task memories before task $\theta \in \mathbb{R}^P$0 is denoted $\theta \in \mathbb{R}^P$1.

A-GEM evaluations are conducted via a two-stream protocol:

• $\theta \in \mathbb{R}^P$2: A held-out stream for hyper-parameter optimization, allowing arbitrary replay.
• $\theta \in \mathbb{R}^P$3: An evaluation stream processed in a single pass with fixed hyper-parameters.

This separation prevents information leakage from evaluation tasks during hyper-parameter search and enforces a strictly single-pass regime for reporting metrics.

2. Evaluation Metrics

A-GEM is evaluated using several metrics that quantify accuracy and the dynamics of knowledge retention and acquisition:

(a) Final Average Accuracy $\theta \in \mathbb{R}^P$4:

$\theta \in \mathbb{R}^P$5

Here, $\theta \in \mathbb{R}^P$6 is the test accuracy on task $\theta \in \mathbb{R}^P$7 after training on all minibatches of task $\theta \in \mathbb{R}^P$8; $\theta \in \mathbb{R}^P$9 is the terminal metric.

(b) Forgetting $T$0:

$T$1

This quantifies the deterioration in performance on previous tasks due to new learning.

(c) Learning Curve Area (LCA$T$2):

$T$3

LCA evaluates both few-shot and progressive learning by averaging accuracy up to $T$4 training steps.

3. From GEM to A-GEM: Mathematical Formulation

GEM constrains gradient updates to avoid loss increases on any previous task's memory, projecting the current gradient $T$5 onto the intersection of half-spaces: $T$6 with $T$7. This requires solving a quadratic program with $T$8 constraints and storing all $T$9.

A-GEM simplifies the constraint to a single condition on the average past gradient $k$0, computed from a mini-batch sampled from $k$1: $k$2 If $k$3, $k$4 is untouched; else, the projection has a closed form: $k$5 This single-constraint projection reduces computational complexity and storage, enabling scalability to longer task sequences and larger networks.

4. Algorithmic Implementation and Complexity

A-GEM maintains a global episodic memory $k$6 with reservoir sampling to ensure a uniform selection from all encountered data. At each training step, a mini-batch from $k$7 provides $k$8. The update is as follows:

$t_i^k$5 Inputs: training stream $k$9, test sets $D_k = \{(x_i^k, t_i^k, y_i^k)\}_{i=1}^{n_k}$0, learning rate $D_k = \{(x_i^k, t_i^k, y_i^k)\}_{i=1}^{n_k}$1. Outputs: final $D_k = \{(x_i^k, t_i^k, y_i^k)\}_{i=1}^{n_k}$2, accuracy matrix $D_k = \{(x_i^k, t_i^k, y_i^k)\}_{i=1}^{n_k}$3.

Complexity Table

Method	Time (per step)	Memory
Vanilla	$D_k = \{(x_i^k, t_i^k, y_i^k)\}_{i=1}^{n_k}$4	$D_k = \{(x_i^k, t_i^k, y_i^k)\}_{i=1}^{n_k}$5
EWC	$D_k = \{(x_i^k, t_i^k, y_i^k)\}_{i=1}^{n_k}$6 + diag-updates	$D_k = \{(x_i^k, t_i^k, y_i^k)\}_{i=1}^{n_k}$7
GEM	$D_k = \{(x_i^k, t_i^k, y_i^k)\}_{i=1}^{n_k}$8	$D_k = \{(x_i^k, t_i^k, y_i^k)\}_{i=1}^{n_k}$9
A-GEM	$x_i^k$0 (≈$x_i^k$1)	$x_i^k$2

Here $x_i^k$3=#parameters, $x_i^k$4=mini-batch size, $x_i^k$5=activation size, $x_i^k$6=episodic memory size. In practice, A-GEM is approximately $x_i^k$7 faster and $x_i^k$8 more memory efficient than GEM on MNIST/CIFAR.

5. Empirical Results and Benchmark Performance

Experiments evaluate A-GEM on Permuted MNIST, Split CIFAR-100, Split CUB, and Split AWA, using MLP and ResNet architectures. A-GEM's final accuracy ($x_i^k$9) matches or slightly trails GEM (e.g., 89.1% vs 89.5% on MNIST) while outperforming all regularization-based baselines (EWC, PI, MAS, RWalk) in the single-pass regime (e.g., EWC: 68%, A-GEM: 89% on MNIST). Forgetting $t_i^k$0 remains lowest among methods with bounded memory.

Incorporating compositional task descriptors with a joint-embedding model ("–je" variant) further improves $t_i^k$1, 0-shot performance (LCA$t_i^k$2), and learning speed for A-GEM and other methods.

Normalized summary (Permuted MNIST, Split CIFAR):

Method	$t_i^k$3 (%) ↑	LCA$t_i^k$4 ↑	Time ↓	Mem ↓
Vanilla	47.9	0.26	0.06	0.06
EWC	68.3	0.27	0.14	0.14
GEM	89.5	0.23	1.00	1.00
A-GEM	89.1	0.29	0.14	0.11

A-GEM is thus Pareto-optimal in the joint space of accuracy, forgetting, LCA, time, and memory.

6. Ablations, Sensitivity Analyses, and Algorithmic Variants

A-GEM projections are only required on a small fraction of steps, in contrast to GEM's frequent constraints as tasks accumulate. The "Stochastic GEM" (s-GEM) variant, which randomly samples past constraints, is still more costly and slightly less effective than A-GEM.

EWC's efficacy is highly sensitive to the number of epochs and model capacity; in single-pass and small-network settings, it only marginally outperforms vanilla SGD. Only with over-parameterized models and multiple passes does EWC approach A-GEM's performance.

Hyper-parameter search spaces and selected settings are detailed in the appendix of (Chaudhry et al., 2018).

7. Key Insights, Limitations, and Future Directions

A-GEM achieves the core objectives of lifelong learning—retaining prior knowledge and enabling forward transfer—while being computationally and memory efficient. The Learning Curve Area (LCA) metric, introduced alongside A-GEM, provides a finer quantification of few-shot learning dynamics.

Observed limitations include the residual gap between single-pass continual learning (even with A-GEM) and the multi-task upper bound (IID setting). Differences in LCA among advanced continual methods converge when catastrophic forgetting is controlled; the field thus requires strategies for enhancing positive backward transfer.

Natural extensions include applying A-GEM to unsupervised, reinforcement, or streaming non-i.i.d. learning settings. The open-source codebase is provided at https://github.com/facebookresearch/agem (Chaudhry et al., 2018).