FlashbackCL: Flashcards for Continual Learning

• FlashbackCL is a continual learning approach that uses compact pseudo-samples—flashcards—to retain past knowledge without storing full historical data.
• It employs recursive autoencoder-based flashcard generation and bidirectional regularization to balance stability with plasticity across various learning scenarios.
• Empirical results demonstrate lower reconstruction errors and improved memory efficiency, making FlashbackCL effective for both standard and federated learning tasks.

Searching arXiv for the cited FlashbackCL-related papers and terminology. Flashback Continual Learning (FlashbackCL) denotes a family of continual learning procedures that use compact retained signals from earlier training stages to mitigate catastrophic forgetting without requiring full replay of historical data. In the literature, the term has not remained semantically fixed. In "Knowledge Capture and Replay for Continual Learning" (Gopalakrishnan et al., 2020), FlashbackCL is the continual-learning embodiment of the paper’s "flashcards" mechanism: pseudo-samples obtained by recursively passing structured random inputs through a trained autoencoder. In "Flashbacks to Harmonize Stability and Plasticity in Continual Learning" (Mahmoodi et al., 31 May 2025), the method is named Flashback Learning (FL), but is explicitly described as FlashbackCL in an overview aligned with that terminology. In "FlashbackCL: Mitigating Temporal Forgetting in Federated Learning" (Ojewale et al., 2 Jun 2026), FlashbackCL is a temporally aware extension of Flashback for federated learning. Related "flashback" terminology also appears in incremental instruction tuning for LLMs, where limited prompts from old tasks are used to constrain adaptation (Zhao et al., 21 May 2025).

1. Terminological scope and lineage

The expression "FlashbackCL" is best understood as a label for several related continual-learning designs rather than a single canonical algorithm. The most specific early usage in the provided literature is the flashcards-based protocol of (Gopalakrishnan et al., 2020), where the paper explicitly states that Flashback Continual Learning refers to the continual-learning protocol built around flashcards and that there is no separate algorithm called "FlashbackCL" beyond that embodiment. Later work reuses the flashback motif for different mechanisms: a two-phase bidirectional regularization framework in standard continual learning (Mahmoodi et al., 31 May 2025), and a temporal-drift-aware extension in federated learning (Ojewale et al., 2 Jun 2026). A related but distinct line appears in continual instruction tuning, where "flashbacks" are a limited number of prompts from old tasks used during adaptation (Zhao et al., 21 May 2025).

Usage	Core retained knowledge	Setting
Flashcards / FlashbackCL (Gopalakrishnan et al., 2020)	Recursively generated pseudo-samples	Reconstruction, denoising, Task-IL, NIL
Flashback Learning / FlashbackCL (Mahmoodi et al., 31 May 2025)	Stable and plastic knowledge bases	Class-Incremental and Task-Incremental CL
FlashbackCL (Ojewale et al., 2 Jun 2026)	Decayed counts, replay buffers, curated public distillation data	Federated learning with temporal drift
Joint Flashback Adaptation (Zhao et al., 21 May 2025)	Limited old-task prompts and output constraints	Incremental instruction tuning for LLMs

A common misconception is that FlashbackCL names one method with one objective. The available literature instead supports a narrower conclusion: "flashback" has become a recurring design motif for retaining old competence through compact, strategically chosen traces of prior knowledge.

2. Flashcards as knowledge capture and replay

In the flashcards formulation, a network trained on task-specific data is used to generate pseudo-inputs that capture the learned representation (Gopalakrishnan et al., 2020). The setting is centered on an autoencoder (AE) trained on task $t$ with mean absolute error:

$$\min_{\theta_t}\ \frac{1}{N_t}\sum_{n=1}^{N_t}\,\big\|X_n - f_t(X_n;\theta_t)\big\|_1.$$

After training, the reconstruction error is bounded on the task data:

$$\epsilon_1 \le \|X_n - f_t(X_n;\theta_t)\|_1 \le \epsilon_2,\quad \forall X_n\in D_t.$$

Flashcards are then constructed from random structured inputs $M_i \sim P(M)$ by recursive application of the trained AE:

$$F^{(0)}_i = M_i,\qquad F^{(r)}_i = f_t\big(F^{(r-1)}_i;\theta_t\big),\quad r=1,\dots,R,$$

so that the final flashcard is

$$\mathbf{F}_{Ri} \equiv F^{(R)}_i = f_t^{(R)}(M_i;\theta_t).$$

The recursion is accompanied by per-iteration change bounds

$$\gamma_1 \le \big\|F^{(r)}_i - F^{(r-1)}_i\big\|_1 \le \gamma_2,$$

and the desired flashcard set $F_t$ is characterized by two properties: P1, reconstruction consistency with $\gamma_2 < \epsilon_1$, and P2, alignment between the latent-space distribution of flashcards and the latent-space distribution of the original data.

P2 is quantified by the Frechet Latent Space Distance (FLSD):

$$\mathrm{FLSD}(F_t, D_t) = \|\mu_f - \mu_x\|_2^2 + \operatorname{Tr}\!\Big(\Sigma_f + \Sigma_x - 2\,(\Sigma_f\,\Sigma_x)^{1/2}\Big).$$

Empirically, FLSD decreases as the number of recursive passes $$\min_{\theta_t}\ \frac{1}{N_t}\sum_{n=1}^{N_t}\,\big\|X_n - f_t(X_n;\theta_t)\big\|_1.$$0 increases, up to a point, and then rises again because of drift. The practical implication is that recursion is not simply iterative denoising; it is a controlled transformation of random patterns toward the learned manifold. The paper reports that maze-like patterns are effective initial inputs, whereas pixel-wise Gaussian noise is ineffective, and that $$\min_{\theta_t}\ \frac{1}{N_t}\sum_{n=1}^{N_t}\,\big\|X_n - f_t(X_n;\theta_t)\big\|_1.$$1 in the range $$\min_{\theta_t}\ \frac{1}{N_t}\sum_{n=1}^{N_t}\,\big\|X_n - f_t(X_n;\theta_t)\big\|_1.$$2-$$\min_{\theta_t}\ \frac{1}{N_t}\sum_{n=1}^{N_t}\,\big\|X_n - f_t(X_n;\theta_t)\big\|_1.$$3 is effective, with $$\min_{\theta_t}\ \frac{1}{N_t}\sum_{n=1}^{N_t}\,\big\|X_n - f_t(X_n;\theta_t)\big\|_1.$$4 used in the paper.

This mechanism is task agnostic in the precise sense used in (Gopalakrishnan et al., 2020): flashcards are created without task labels or task identifiers, and the same principle is applied across reconstruction, denoising, and classification. The storage claim is equally central: flashcards need to be constructed only before learning the subsequent task, so the method does not require per-task archival of the original data and its memory overhead is independent of the number of tasks.

3. Replay protocol and objective structure

For unsupervised continual learning with reconstruction or denoising, FlashbackCL trains the next model on a mixture of current-task data and flashcards from the preceding task (Gopalakrishnan et al., 2020). With parameters initialized by the previous solution, the objective is

$$\min_{\theta_t}\ \frac{1}{N_t}\sum_{n=1}^{N_t}\,\big\|X_n - f_t(X_n;\theta_t)\big\|_1.$$5

with $$\min_{\theta_t}\ \frac{1}{N_t}\sum_{n=1}^{N_t}\,\big\|X_n - f_t(X_n;\theta_t)\big\|_1.$$6 in the reported experiments and minibatches typically containing a $$\min_{\theta_t}\ \frac{1}{N_t}\sum_{n=1}^{N_t}\,\big\|X_n - f_t(X_n;\theta_t)\big\|_1.$$7 mix of new samples and flashcards.

For Task-IL classification, the encoder is shared with the AE. After classifier training on a task, the encoder weights are copied to the AE, which is then trained to construct flashcards. During the next task, classification uses cross-entropy on current labeled data together with knowledge distillation on flashcards:

$$\min_{\theta_t}\ \frac{1}{N_t}\sum_{n=1}^{N_t}\,\big\|X_n - f_t(X_n;\theta_t)\big\|_1.$$8

$$\min_{\theta_t}\ \frac{1}{N_t}\sum_{n=1}^{N_t}\,\big\|X_n - f_t(X_n;\theta_t)\big\|_1.$$9

with optional latent regularization

$$\epsilon_1 \le \|X_n - f_t(X_n;\theta_t)\|_1 \le \epsilon_2,\quad \forall X_n\in D_t.$$0

The total Task-IL loss is

$$\epsilon_1 \le \|X_n - f_t(X_n;\theta_t)\|_1 \le \epsilon_2,\quad \forall X_n\in D_t.$$1

with typical $$\epsilon_1 \le \|X_n - f_t(X_n;\theta_t)\|_1 \le \epsilon_2,\quad \forall X_n\in D_t.$$2 and $$\epsilon_1 \le \|X_n - f_t(X_n;\theta_t)\|_1 \le \epsilon_2,\quad \forall X_n\in D_t.$$3 commonly $$\epsilon_1 \le \|X_n - f_t(X_n;\theta_t)\|_1 \le \epsilon_2,\quad \forall X_n\in D_t.$$4-$$\epsilon_1 \le \|X_n - f_t(X_n;\theta_t)\|_1 \le \epsilon_2,\quad \forall X_n\in D_t.$$5.

For new-instance learning (NIL), the same classes recur across sessions. The protocol again constructs unsupervised AE flashcards per session, obtains soft class scores from the prior classifier, and trains the new-session model with standard cross-entropy on current data plus distillation on flashcards. This unifies supervised and unsupervised replay under the same pseudo-sample mechanism.

4. Empirical behavior, memory profile, and implementation

The reported empirical profile of flashcards is unusually specific about the trade-off between retention and storage (Gopalakrishnan et al., 2020). On continual reconstruction and denoising over Sequence5 (MNIST, Fashion-MNIST, CIFAR10, SVHN, Omniglot), with an AE using 4 down/up-sampling layers, 64 convolutional filters per layer, and bottleneck size 256, Joint Training achieved reconstruction MAE $$\epsilon_1 \le \|X_n - f_t(X_n;\theta_t)\|_1 \le \epsilon_2,\quad \forall X_n\in D_t.$$6 and denoising MAE $$\epsilon_1 \le \|X_n - f_t(X_n;\theta_t)\|_1 \le \epsilon_2,\quad \forall X_n\in D_t.$$7, whereas Sequential Fine-Tuning gave reconstruction MAE $$\epsilon_1 \le \|X_n - f_t(X_n;\theta_t)\|_1 \le \epsilon_2,\quad \forall X_n\in D_t.$$8 with BWT $$\epsilon_1 \le \|X_n - f_t(X_n;\theta_t)\|_1 \le \epsilon_2,\quad \forall X_n\in D_t.$$9 and denoising MAE $M_i \sim P(M)$0 with BWT $M_i \sim P(M)$1. Episodic Replay with a 5K coreset reached reconstruction MAE $M_i \sim P(M)$2 and denoising MAE $M_i \sim P(M)$3, both with BWT $M_i \sim P(M)$4, at an external memory cost of $M_i \sim P(M)$5 MB. Generative Replay reached reconstruction MAE $M_i \sim P(M)$6 and denoising MAE $M_i \sim P(M)$7, with extra network capacity of approximately $M_i \sim P(M)$8 MB. Flashcards 5K reached reconstruction MAE $M_i \sim P(M)$9 with BWT $$F^{(0)}_i = M_i,\qquad F^{(r)}_i = f_t\big(F^{(r-1)}_i;\theta_t\big),\quad r=1,\dots,R,$$0 and denoising MAE $$F^{(0)}_i = M_i,\qquad F^{(r)}_i = f_t\big(F^{(r-1)}_i;\theta_t\big),\quad r=1,\dots,R,$$1 with BWT $$F^{(0)}_i = M_i,\qquad F^{(r)}_i = f_t\big(F^{(r-1)}_i;\theta_t\big),\quad r=1,\dots,R,$$2, while using no external memory.

In Task-IL classification on Sequence3 (CIFAR10-MNIST-FashionMNIST), flashcards performed similarly to baselines when task-ID was provided and outperformed regularization and generative replay methods when task-ID was unavailable. In NIL classification on CIFAR10 sessions with brightness and saturation jitter, Flashcards 1K achieved $$F^{(0)}_i = M_i,\qquad F^{(r)}_i = f_t\big(F^{(r-1)}_i;\theta_t\big),\quad r=1,\dots,R,$$3 through session 5, while A-GEM 1K reached up to $$F^{(0)}_i = M_i,\qquad F^{(r)}_i = f_t\big(F^{(r-1)}_i;\theta_t\big),\quad r=1,\dots,R,$$4 at session 5.

These results are paired with a distinctive memory claim. Episodic replay storage scales approximately as

$$F^{(0)}_i = M_i,\qquad F^{(r)}_i = f_t\big(F^{(r-1)}_i;\theta_t\big),\quad r=1,\dots,R,$$5

whereas flashcards require no per-task sample storage and only standard model parameters must be retained between tasks. Flashcard generation incurs computational cost $$F^{(0)}_i = M_i,\qquad F^{(r)}_i = f_t\big(F^{(r-1)}_i;\theta_t\big),\quad r=1,\dots,R,$$6, but no auxiliary generator is trained.

The implementation defaults are equally concrete: 100 epochs per task for the AE, Adam with learning rate $$F^{(0)}_i = M_i,\qquad F^{(r)}_i = f_t\big(F^{(r-1)}_i;\theta_t\big),\quad r=1,\dots,R,$$7, early stopping with patience 20, $$F^{(0)}_i = M_i,\qquad F^{(r)}_i = f_t\big(F^{(r-1)}_i;\theta_t\big),\quad r=1,\dots,R,$$8 validation split, minibatches with equal counts of flashcards and current-task samples, typical $$F^{(0)}_i = M_i,\qquad F^{(r)}_i = f_t\big(F^{(r-1)}_i;\theta_t\big),\quad r=1,\dots,R,$$9K, tanh activations in AE hidden layers, and hardware reported as 64GB RAM with an NVIDIA RTX Titan GPU.

5. Bidirectional Flashback Learning for stability-plasticity balance

"Flashbacks to Harmonize Stability and Plasticity in Continual Learning" introduces Flashback Learning (FL), which is also described as FlashbackCL in a terminology-aligned synthesis (Mahmoodi et al., 31 May 2025). Unlike the flashcards mechanism, this method is not based on recursive pseudo-input generation. Its defining idea is bidirectional regularization through two knowledge bases: a Stable Knowledge Base (SKB), representing what the host continual-learning method ordinarily keeps to preserve prior knowledge, and a Plastic Knowledge Base (PKB), distilled from a short Phase 1 training run on the new task.

The method operates in two phases under a fixed training budget $$\mathbf{F}_{Ri} \equiv F^{(R)}_i = f_t^{(R)}(M_i;\theta_t).$$0. Phase 1 starts from the stable checkpoint $$\mathbf{F}_{Ri} \equiv F^{(R)}_i = f_t^{(R)}(M_i;\theta_t).$$1, trains for $$\mathbf{F}_{Ri} \equiv F^{(R)}_i = f_t^{(R)}(M_i;\theta_t).$$2 epochs using the host continual-learning loss plus stability regularization, and produces a primary model $$\mathbf{F}_{Ri} \equiv F^{(R)}_i = f_t^{(R)}(M_i;\theta_t).$$3 that captures new-task knowledge. PKB is then built from $$\mathbf{F}_{Ri} \equiv F^{(R)}_i = f_t^{(R)}(M_i;\theta_t).$$4. Phase 2 resets the model back to $$\mathbf{F}_{Ri} \equiv F^{(R)}_i = f_t^{(R)}(M_i;\theta_t).$$5 and trains for $$\mathbf{F}_{Ri} \equiv F^{(R)}_i = f_t^{(R)}(M_i;\theta_t).$$6 epochs with a bidirectional objective:

$$\mathbf{F}_{Ri} \equiv F^{(R)}_i = f_t^{(R)}(M_i;\theta_t).$$7

$$\mathbf{F}_{Ri} \equiv F^{(R)}_i = f_t^{(R)}(M_i;\theta_t).$$8

The paper provides category-specific instantiations. For distillation-based hosts, both $$\mathbf{F}_{Ri} \equiv F^{(R)}_i = f_t^{(R)}(M_i;\theta_t).$$9 and $$\gamma_1 \le \big\|F^{(r)}_i - F^{(r-1)}_i\big\|_1 \le \gamma_2,$$0 are mean-squared deviations from the stable and primary model outputs. For replay-based hosts, the two terms are mean-squared deviations from stored stable and plastic outputs on the same memory samples. For parameter-regularization hosts such as oEWC, the two penalties are Fisher-weighted quadratic distances to $$\gamma_1 \le \big\|F^{(r)}_i - F^{(r-1)}_i\big\|_1 \le \gamma_2,$$1 and $$\gamma_1 \le \big\|F^{(r)}_i - F^{(r-1)}_i\big\|_1 \le \gamma_2,$$2. For dynamic architectures, the terms are KL losses to stable and primary soft predictions.

The theoretical analysis states that the Phase 2 gradient pulls the model toward a weighted interpolation of stable and primary targets, thereby explicitly balancing retention and acquisition. This is accompanied by an explicit Stability-Plasticity Ratio,

$$\gamma_1 \le \big\|F^{(r)}_i - F^{(r-1)}_i\big\|_1 \le \gamma_2,$$3

with lower values indicating a better balance.

Empirically, FL is reported to improve average accuracy by up to $$\gamma_1 \le \big\|F^{(r)}_i - F^{(r-1)}_i\big\|_1 \le \gamma_2,$$4 in Class-Incremental and $$\gamma_1 \le \big\|F^{(r)}_i - F^{(r-1)}_i\big\|_1 \le \gamma_2,$$5 in Task-Incremental settings on standard image classification benchmarks, while remaining within the same training budget (Mahmoodi et al., 31 May 2025). It integrates with replay, parameter regularization, distillation, and dynamic architecture techniques, and was evaluated with hosts including LUCIR, LwF.MC, iCaRL, X-DER, oEWC, FOSTER, and BEEF. The principal cost is memory: FL requires two knowledge bases and approximately doubles the memory used by the host method.

6. FlashbackCL in federated learning under temporal drift

In federated learning, FlashbackCL denotes a specific extension of Flashback to non-stationary client distributions over time (Ojewale et al., 2 Jun 2026). The motivating claim is that Flashback’s original monotonically accumulating per-class label counts are suitable for cross-client spatial heterogeneity but become miscalibrated under temporal distribution shift. When a class disappears from the federation, its historical count remains large, so dynamic distillation continues to allocate weight to stale classes and anchors the global model to an outdated class balance.

The paper formalizes temporal forgetting through a per-phase metric. If $$\gamma_1 \le \big\|F^{(r)}_i - F^{(r-1)}_i\big\|_1 \le \gamma_2,$$6 is accuracy on a held-out phase-$$\gamma_1 \le \big\|F^{(r)}_i - F^{(r-1)}_i\big\|_1 \le \gamma_2,$$7 test set at round $$\gamma_1 \le \big\|F^{(r)}_i - F^{(r-1)}_i\big\|_1 \le \gamma_2,$$8, and $$\gamma_1 \le \big\|F^{(r)}_i - F^{(r-1)}_i\big\|_1 \le \gamma_2,$$9 is the last round of phase $F_t$0, then

$F_t$1

The reported scalar is the mean over valid $F_t$2 pairs, denoted average temporal forgetting.

FlashbackCL introduces three modifications. First, global label counts are exponentially decayed:

$F_t$3

with $F_t$4. Second, it adds device-aware replay buffers with Class-Balanced Reservoir Sampling (CBRS), where per-client capacity is

$F_t$5

so rarely participating clients receive larger buffers. Third, it performs server-side active coreset curation on the public distillation set using normalized per-class trend slopes and class-sampling weights

$F_t$6

The local training loss mixes current and replay minibatches at ratio $F_t$7:

$F_t$8

with $F_t$9 and replay batch size 32 in the reported implementation. Reduction to Flashback occurs when $\gamma_2 < \epsilon_1$0, $\gamma_2 < \epsilon_1$1, and $\gamma_2 < \epsilon_1$2.

On CIFAR-10 with 50 clients and three controlled temporal shift modes, FlashbackCL achieves $\gamma_2 < \epsilon_1$3 to $\gamma_2 < \epsilon_1$4 relative improvement relative to Flashback and reduces temporal forgetting by up to $\gamma_2 < \epsilon_1$5 (Ojewale et al., 2 Jun 2026). A five-variant ablation identifies CBRS replay as the critical component. The paper additionally reports that FlashbackCL improves Flashback by $\gamma_2 < \epsilon_1$6 points on stationary CIFAR-100, which suggests that class-balanced replay regularizes spatial heterogeneity as well as temporal shift.

7. Relation to instruction tuning and broader interpretation

The flashback motif also appears in continual adaptation of LLMs. "Joint Flashback Adaptation for Forgetting-Resistant Instruction Tuning" proposes introducing flashbacks, defined as a limited number of prompts from old tasks, when adapting to new tasks, and constraining deviations of the adapted model’s outputs relative to the original model (Zhao et al., 21 May 2025). The method further interpolates latent tasks between flashbacks and new tasks so that relevant latent tasks, new tasks, and flashbacks are learned jointly. The abstract states that the method requires only a limited number of flashbacks without access to replay data, is task-agnostic, and was evaluated across 1000+ instruction-following tasks, arithmetic reasoning tasks, and general reasoning tasks.

This usage is technically distinct from both flashcards-based FlashbackCL and federated FlashbackCL. The commonality lies not in identical losses or architectures, but in the retention strategy: a restricted retained trace of prior behavior is used to regularize adaptation to new data. This suggests that "flashback" is better treated as a recurrent continual-learning principle—compact old-knowledge retention coupled to current-task optimization—than as a uniquely defined procedure across subfields.

Across these variants, the central design question is consistent. One must decide what form of old knowledge is retained: pseudo-inputs generated from a trained model, stable and plastic target functions from separate phases, decayed label-count statistics plus replay buffers in federated optimization, or a limited prompt set in instruction tuning. The literature shows that different answers to that question produce different algorithms, but all are motivated by the same objective: reducing catastrophic forgetting while avoiding unrestricted replay of historical data.