---
title: Complementary Anchor Replay (CAR)
url: https://www.emergentmind.com/topics/complementary-anchor-replay-car
type: topic
---

# Complementary Anchor Replay (CAR)

Complementary Anchor Replay (CAR), also referred to as Hindsight Anchor Learning (HAL), is a continual learning algorithm designed to mitigate catastrophic forgetting in neural networks exposed to non-stationary data streams. CAR augments standard experience replay strategies by incorporating a learned set of anchor points, optimized to identify and preserve predictions most susceptible to forgetting upon subsequent learning phases. The method leverages bilevel optimization, episodic memory, and a bespoke anchoring loss to ensure retention of decision boundaries crucial to prior tasks while accommodating new information, offering consistent performance improvements across standard lifelong learning benchmarks [2002.08165].

## 1. Objective and Core Formulation

The primary objective of CAR is to enable deep models to retain knowledge acquired from previous tasks while learning new ones, particularly in supervised continual learning scenarios. Conventional experience replay minimizes the standard supervised loss over the union of a current minibatch $D_\text{curr}$ and a sampled subset $B_M$ of episodic memory $M$:
$$
\ell(\theta; D_\text{curr} \cup B_M)
$$
where $\theta \in \mathbb{R}^P$ denotes the model parameters.

CAR augments this with an anchoring penalty, added to each SGD update, that constrains the change in model predictions on a set of learned anchor points $\{ e_i \}$. The two-step procedure for every gradient descent step is:
1. **Temporary update:** $\tilde{\theta} \leftarrow \theta - \alpha \, \nabla_\theta \ell(\theta; D_\text{curr} \cup B_M)$
2. **CAR update:**
$$
\theta \leftarrow \theta - \alpha \nabla_\theta \left[
\ell(\theta; D_\text{curr} \cup B_M) + \lambda \sum_i \left\| f_\theta(e_i,t_i) - f_{\tilde{\theta}}(e_i,t_i) \right\|^2
\right]
$$
where $f_\theta(\cdot, t_i)$ is the task-conditioned output, $\lambda > 0$ modulates the strength of anchoring, and the summation is over anchor points for all previous tasks. This combined loss penalizes deviation of the network’s predictions on anchors after a small future ("look-ahead") update, effectively preserving information most at risk of being lost.

## 2. Bilevel Optimization in CAR

CAR is formulated as a bilevel optimization problem in both the parameter update and anchor selection phases.

- **Parameter update as bilevel problem:**  
  For each step, CAR approximately solves:
  - **Outer:** $\min_{\theta} L_\text{rep}(\theta) + \lambda A(\theta; \{e_i\})$  
    where $L_\text{rep}(\theta) = \ell(\theta; D_\text{curr} \cup B_M)$ and $A(\theta; \{e_i\}) = \sum_i \left\| f_\theta(e_i,t_i) - f_{\tilde{\theta}}(e_i,t_i) \right\|^2$
  - **Inner:** $\tilde{\theta}(\theta) = \arg\min_u L_\text{rep}(u)$

- **Anchor selection as bilevel problem:**  
  For each past task $t$ and class $y_t$, an anchor $e_t$ is learned by maximizing a proxy for forgetting:
  - **Inner:** $\theta_M = \arg\min_u \ell(u; M)$ (fine-tuning on memory)
  - **Outer:** 
  $$
  e_t = \arg\max_{e \in \mathbb{R}^D} \left[\ell(f_{\theta_M}(e,t), y_t) - \ell(f_{\theta_t}(e,t), y_t) - \gamma \| \phi(e) - \phi_t \|^2 \right]
  $$
  Here, $\phi(e)$ is the feature embedding, $\phi_t$ a running mean feature for task $t$, and $\gamma > 0$ regularizes the anchor’s proximity to the data manifold.

## 3. Loss Functions

The CAR methodology involves three principal loss functions:
- **Classification loss:** For any set $A$,
  $$
  \ell(\theta; A) = \frac{1}{|A|} \sum_{(x, y, t) \in A} \ell_\text{ce}(f_\theta(x, t), y)
  $$
  where $\ell_\text{ce}$ is the cross-entropy loss.
- **Anchoring loss:** On learned anchors,
  $$
  A(\theta; \{e_i\}) = \sum_i \left\| f_\theta(e_i, t_i) - f_{\tilde{\theta}}(e_i, t_i) \right\|^2
  $$
- **Forgetting loss (anchor selection):**
  $$
  F(e; \theta_t, \theta_M) = \ell_\text{ce}(f_{\theta_M}(e, t), y_t) - \ell_\text{ce}(f_{\theta_t}(e, t), y_t) - \gamma \| \phi(e) - \phi_t \|^2
  $$

## 4. Anchor Selection and Update Procedure

Anchor points are learned at the completion of each task using gradient ascent to maximize the proxy forgetting loss:
1. **Initialize** $e_{t, c}$ for each class $c$ in task $t$ (random initialization or mean-embedding).
2. **Update** via $k$ gradient-ascent steps:
   $$
   e_{t, c} \leftarrow e_{t, c} + \eta \nabla_e \left[
     \ell_\text{ce}(f_{\theta_M}(e, t), c) - \ell_\text{ce}(f_\theta(e, t), c) - \gamma \| \phi(e) - \phi_t \|^2
   \right]
   $$
3. **Freeze** $e_{t, c}$ for use as an anchor in subsequent iterations.

The feature extractor $\phi$ is defined as the penultimate layer of the network; $\phi_t$ is tracked online with exponential moving average: $\phi_t \leftarrow \beta \, \phi_t + (1-\beta) \, (1/|B|) \sum_{x \in B} \phi(x)$, with decay $\beta$.

## 5. Algorithmic Workflow

The CAR/HAL workflow is structured as follows:

- Maintain an episodic memory $M$ with a per-class ring buffer for storage of past examples.
- After each new task is learned:
  1. Update $M$ with examples from the task.
  2. Compute $\phi_t$ (feature mean) online.
  3. After completing the task, fine-tune parameters on $M$ to obtain $\theta_M$.
  4. For each class, optimize the anchor $e_{t, c}$ via $k$ steps of gradient ascent on the forgetting objective.
- During training on each minibatch:
  1. Sample replay batch $B_M$ from $M$.
  2. Compute look-ahead parameters $\tilde{\theta}$.
  3. Update $\theta$ using the combined loss: standard replay plus anchoring on all current anchors.

### Pseudocode

```
Input: stream of tasks D={D_1,…,D_T}, learning rate α, anchor-loss weight λ,
       anchor-learning rate η, mean-embed weight γ, decay β,
       memory-size per class m, anchor steps k
Initialize θ randomly, memory M empty, empty anchor-list E

for t in 1…T:
  φ_t ← 0                           # running mean-embedding for task t
  for each minibatch B from D_t:
    B_M ← RandomSample(M, |B|)      # replay batch from memory
    θ˜ ← θ − α∇_θℓ(θ; B∪B_M)       # temporary replay update
    θ ← θ − α [∇_θℓ(θ; B∪B_M) + λ ∑_{(e,task)∈E} ∇_θ ∥f_θ(e,task)−f_{θ˜}(e,task)∥²]
    φ_t ← β φ_t + (1−β) (1/|B|)∑_{x∈B}φ(x)
    UpdateRingBuffer(M, B, m)        # ring buffer update
  end for

  θ_M ← θ
  for each minibatch B_M in one epoch over M:
    θ_M ← θ_M − α∇_θℓ(θ_M; B_M)
  end for

  for each class c in task t:
    initialize e_{t,c} randomly
    for step=1…k:
      e_{t,c} ← e_{t,c} + η∇_{e}[ℓ_ce(f_{θ_M}(e,t),c) − ℓ_ce(f_θ(e,t),c) − γ ∥φ(e)−φ_t∥²]
    end for
    add (e_{t,c},t) to E
  end for

return θ
```

## 6. Empirical Results and Comparative Analysis

CAR/HAL was evaluated on multiple supervised continual learning benchmarks: Permuted MNIST (23 tasks × 10 classes), Rotated MNIST (23 × 10), Split CIFAR-100 (20 × 5), and Split miniImageNet (20 × 5), across varying episodic memory budgets—1, 3, or 5 examples per class per task. Baselines included Finetune, EWC, VCL (with/without random replay), ICARL, AGEM, MER, ER-Ring, MIR, and oracle strategies.

Performance metrics comprised final average accuracy $(1/T)\sum_j a_{T,j}$ and final forgetting $(1/(T-1))\sum_{j=1}^{T-1} \max_{l \leq T-1}(a_{l,j} - a_{T,j})$. With 1 example per class memory:
- On Permuted MNIST: HAL achieved $73.6\%\pm0.31$ vs. MIR $71.1\%$, ER-Ring $70.2\%$, MER $69.9\%$
- On Rotated MNIST: HAL $68.4\%\pm0.72$, ER-Ring $65.9\%$
- On Split CIFAR: HAL $60.4\%\pm0.54$, MIR $57.1\%$, ER-Ring $56.2\%$
- On Split miniImageNet: HAL $51.6\%\pm2.02$, MIR $49.3\%$, ER-Ring $49.0\%$

Relative forgetting was consistently reduced by 15–25%. Increasing memory to 5 examples per class further increased HAL’s accuracy—approximately $78\%$ on MNIST tasks, $64\%$ on Split CIFAR—always outperforming the compared replay or meta-learning methods.

## 7. Hyperparameters and Implementation Considerations

Key hyperparameters and implementation aspects include:
- **Learning rate $\alpha$**: $\{0.003, 0.01, 0.03, 0.1, 0.3, 1.0\}$; selected as $0.1$ for MNIST, $0.03$ for CIFAR/ImageNet.
- **Anchor-loss weight $\lambda$**: same grid, set as $0.1$ for MNIST, $1.0$ for CIFAR, $0.3$ for miniImageNet.
- **Mean-embedding regularizer $\gamma$**: typically $0.1$.
- **Decay $\beta$ for running mean-embedding**: $0.5$.
- **Anchor-learning steps $k$**: $100$ per anchor, anchor step size $\eta=\alpha$.
- **Feature extractor $\phi(x)$**: penultimate network activation.
- **Episodic memory**: per-class ring buffer of size $m$; sampling is uniform.
- **Computational cost**: CAR approximately doubles computation time over vanilla replay and remains substantially faster than meta-experience replay (MER).
- **Anchor initialization**: random Gaussian initialization in input space is effective; mean-embedding initialization is a possible alternative.

A plausible implication is that the computational efficiency, combined with modest memory usage and scalable performance, makes CAR suitable for practical large-scale continual learning.

---

CAR/HAL thus functions as a principled extension to experience replay, leveraging trajectory-sensitive anchor points—discovered via explicit maximization of forgetting—to maintain essential decision boundaries. The method demonstrates consistent gains in both accuracy and forgetting metrics relative to multiple strong baselines, with transparent and reproducible optimization and implementation strategies [2002.08165].

Source: https://www.emergentmind.com/topics/complementary-anchor-replay-car