Memory-Based Continual Learning

• Memory-based continual learning methods are algorithms that explicitly retain past data in bounded buffers to counteract catastrophic forgetting in non-stationary task streams.
• They employ advanced replay strategies such as reservoir sampling and synthetic replay via SVD, GMM, or GANs to efficiently integrate old and new information.
• Innovations like uncertainty-driven selection, prototype-guided sampling, and hierarchical memory systems enhance performance, robustness, and scalability in diverse application domains.

Memory-based continual learning methods constitute a class of algorithms designed to enable neural networks and related models to learn from a non-stationary stream of tasks while mitigating catastrophic forgetting. These methods explicitly maintain and update a replay memory—often referred to as a buffer or episodic memory—of past samples or related distributional summaries, interleaving them with current task data to stabilize long-term knowledge. The architecture, update policies, compression techniques, and replay strategies have evolved to address the constraints of bounded memory, computational efficiency, data privacy, and domain or label imbalances. Below, key conceptual and technical advances in memory-based continual learning are systematically reviewed.

1. Core Principles and Memory Buffer Management

Memory-based continual learning methods rely on the explicit retention of past data or statistics to counteract catastrophic forgetting. The fundamental objective is to ensure that models do not lose performance on previously learned tasks as they acquire new knowledge from a continuous, non-i.i.d. data stream. A prototypical framework involves:

• Buffer design: A bounded-size memory buffer (often denoted $\mathcal{M}$ or $R$) retains exemplars representative of previously encountered data distributions (Arani et al., 2022, Bang et al., 2021).
• Update policy: New samples are admitted to the buffer via reservoir sampling, class-balanced allocation, prototype proximity, or manifold expansion heuristics, often subject to a strict memory budget (Bai et al., 2023, Xu et al., 2023, Ho et al., 2021).
• Replay strategy: At each training step or epoch, mini-batches drawn from the buffer are interleaved with new task data, defining a joint loss. Replay can be implemented via raw examples [CLS-ER, DER], synthetic generation [GANs, SVD, GMMs], or compressed representations (Liu et al., 20 Apr 2025, Lamers et al., 24 Jun 2025).
• Replacement rules: Advanced policies may prioritize buffer diversity, cover uncertainty quantiles, maximize feature-manifold spread, or explicitly balance class or domain factors (Xu et al., 2023, Bang et al., 2021, Cheng et al., 4 Aug 2024).

Fixed per-class or per-task quota, reservoir sampling, and quantile-based uncertainty selection are widely adopted, with extensions for multi-label and multi-modal data (Cheng et al., 4 Aug 2024). In streaming settings, buffer management is vital, as early data can be irreversibly lost if not carefully handled (Lee et al., 2021, Arani et al., 2022, Savadikar et al., 2023).

2. Memory Compression and Synthetic Replay

To achieve high information content per bit under rigid memory budgets, recent methods replace raw exemplars with compressed or generative representations:

• SVD-based low-rank models: Raw past features are compressed using truncated singular value decomposition (SVD), storing only class-conditional bases and covariance in a subspace. Synthetic samples are generated online by sampling from the learned low-dimensional distribution (Lamers et al., 24 Jun 2025). This approach yields significant memory compression—up to $\sim20\times$—with minimal performance loss on MNIST-like and small-domain shifts.
• GMM-based pseudo-feature replay: Distribution-Level Memory Recall (DMR) models the embedding distribution of each old class by fitting a full anisotropic Gaussian Mixture Model and replaying high-fidelity pseudo-features in subsequent tasks (Cheng et al., 4 Aug 2024). Storage is optimized by using diagonal or scalar variances (DMR-Lite).
• Semi-parametric wake–sleep consolidation: Patterns are encoded as entropy-regularized binary codes, replayed through compact per-task decoders. This semi-parametric paradigm trades off exemplar fidelity and storage, mimicking hippocampal pattern separation and completion (Liu et al., 20 Apr 2025).
• GAN-based generative replay: In triple memory setups, a task-conditional generator synthesizes pseudo-inputs for old tasks, with consolidation imposed by elastic weight regularization in discriminators and classifiers (Wang et al., 2020).

The use of compression and synthetic replay becomes increasingly advantageous as data dimensionality or task count grows, especially under privacy or hardware constraints.

3. Episodic Memory Diversity and Selection Criteria

Maximal replay efficacy is attained when buffer exemplars are diverse and span the true distributional manifold of past data. Key strategies include:

• Uncertainty-driven diversity: Exemplars are ranked by classification uncertainty, estimated by stability of predictions under stochastic augmentations. Uniform quantile selection over the uncertainty axis ensures coverage of class centers and boundaries, maximizing long-range generalization (Bang et al., 2021).
• Prototype-guided selection: Samples closest to dynamically updated class prototypes (mean embeddings) are retained, leveraging meta-learning or prototypical networks (Ho et al., 2021). In text or low-resource settings, as few as 5 samples per class—selected by proximity to prototypes—yield strong forgetting mitigation at minuscule memory cost.
• Manifold expansion: Incoming samples are deterministically retained if they expand the diameter of the feature-space manifold spanned by current buffer examples, otherwise subject to reservoir sampling (Xu et al., 2023). This geometric heuristic increases buffer representativeness, improving performance and reducing class/boundary bias.
• Adversarial criteria: For adversarial robustness, samples are filtered into memory based on difficulty to perturb (distance to decision boundary), maintaining attack-resilient replay coverage (Mi et al., 2023).

Memory diversity plays a central role in OOD generalization, bias mitigation, and resilience to rapid drift in task/domain (Rio et al., 2023).

4. Architectural and Algorithmic Innovations

Beyond buffer content, several architectural and loss-design principles enhance memory-based continual learning:

• Multi-memory systems: Drawing from neuroscience, dual or triple memory systems—as in CLS-ER (episodic, short-term semantic, long-term semantic) (Arani et al., 2022), or triple memory networks (generator, discriminator, classifier) (Wang et al., 2020)—enable rapid adaptation and robust consolidation.
• Hierarchical and hybrid storage: Hierarchical memory pools (fast RAM + slow disk) support larger effective replay sets with bounded high-speed memory, asynchronously swapping between levels to minimize forgetting without blocking compute (Lee et al., 2021, Wang et al., 2023).
• Replay scheduling and loss design: Wasserstein-distance-based distillation ensures feature-level preservation (Xu et al., 2023), while memory consistency regularization (e.g., CLS-ER's MSE to semantic memory outputs) aligns new decision boundaries with old knowledge (Arani et al., 2022).
• Adversarially-calibrated replay: Logit calibration and robust sample selection harmonize catastrophic forgetting avoidance and adversarial robustness (Mi et al., 2023).
• Memory-free and extreme-constraint regimes: Under MC-OCL constraints, replay is replaced with batch-level logit distillation and probability banks that consume <100 kB of memory (Fini et al., 2020).

Selection among these architectural innovations is task- and resource-dependent, with documented trade-offs in memory, computation, and performance.

5. Applications and Empirical Findings

Applications span image classification (MNIST, CIFAR, ImageNet), text classification (AGNews, Yelp, Amazon), multi-modal learning (vision, IMU), and video object segmentation (Nazemi et al., 2023, Cheng et al., 4 Aug 2024). Key empirical conclusions:

• State-of-the-art performance: Approaches such as CLS-ER, MaER, DMR, and semi-parametric wake–sleep replay routinely outperform classical rehearsal (ER, iCaRL, DER) and regularization-based baselines (EWC, MAS) across accuracy and forgetting metrics under class- and domain-incremental protocols (Arani et al., 2022, Xu et al., 2023, Liu et al., 20 Apr 2025, Cheng et al., 4 Aug 2024).
• Memory efficiency: Even under strict constraints (300–500 exemplars), modern diversity-driven or generative-compressed replay can achieve accuracy close to, or exceeding, joint-training upper bounds.
• Robustness and calibration: CLS-ER, semi-parametric memory, and DMR offer improved calibration (ECE), reduced recency bias, and more robust OOD/shifted-domain generalization (Arani et al., 2022, Liu et al., 20 Apr 2025, Cheng et al., 4 Aug 2024).
• Resource-constrained and edge deployment: Hierarchical and semi-supervised methods such as EdgeHML and CarM enable high accuracy at drastically reduced computation and memory, addressing embedded and mobile application scenarios (Wang et al., 2023, Lee et al., 2021).

A summary comparison is provided below:

Method (Reference)	Memory Policy	Replay Type	Key Innovation
Rainbow Memory (Bang et al., 2021)	Quantile/uncertainty	Exemplars	Uncertainty-driven selection
BrainCL (Liu et al., 20 Apr 2025)	Compressed codes + decoders	Generative	Semi-parametric wake-sleep
DMR (Cheng et al., 4 Aug 2024)	GMM per class	Synthetic features	Distribution-level recall
CarM (Lee et al., 2021)	Hierarchical pool (RAM/disk)	Exemplars	Asynchronous carousel swap
MaER (Xu et al., 2023)	Manifold expansion	Exemplars + distill	Geometric coverage + $W_2$
SVD generator (Lamers et al., 24 Jun 2025)	SVD low-rank per class	Synthetic features	Lightweight buffer
PMR (Ho et al., 2021)	Prototype proximity	Exemplars/meta	Memory-efficient, meta-replay

Empirical validation consistently demonstrates strong performance gains, with proper memory policy critical to robustness and scalability.

6. Limitations, Challenges, and Future Directions

Despite advances, several challenges remain:

• Out-of-distribution overfitting: Standard replay can overfit to memory distribution, yield poor OOD generalization, and reinforce spurious feature correlations, especially with small buffers and linear classifiers (Rio et al., 2023).
• Scalability: Large task/domain spaces strain buffer management. Compressing or augmenting memory via learned generative models, SVD, or hierarchical pools mitigates but does not fully resolve the memory-computation-accuracy trade-off (Lamers et al., 24 Jun 2025, Liu et al., 20 Apr 2025).
• Multi-label and multi-modal contexts: Standard buffer policies face combinatorial scaling and imbalance issues in multi-label and multi-modal settings. Recent work adapts buffer construction and pseudo-feature replay to these scenarios with adaptive class/modal selection and fusion (Cheng et al., 4 Aug 2024).
• Extreme memory/compute constraints: For MC-OCL settings, distillation and logit snapshotting methods remain notably below full-replay methods in accuracy, indicating an unresolved stability-plasticity bottleneck when replay is impossible (Fini et al., 2020).

Future directions include: active and distribution-aware replay scheduling, causal and modular representation learning to foster buffer-efficient generalization, integrating biological insights for architectural innovation, and adaptive memory management under resource-constrained, streaming, and privacy-sensitive regimes.

7. Position in the Continual Learning Taxonomy and Research Landscape

Memory-based continual learning constitutes a core pillar of continual learning strategies, contrasting and complementing parameter-isolation (EWC, SI), regularization (MAS), and architecture-growing (PNN, CHEEM) approaches (Savadikar et al., 2023). While replay is highly effective for class- and domain-incremental learning, its integration with generative, compression, and modular techniques shapes the frontier of scalable, robust, and biologically inspired continual learning systems.

Key representative works include the development of Rainbow Memory (Bang et al., 2021), CLS-ER (Arani et al., 2022), Manifold Expansion Replay (Xu et al., 2023), semi-parametric memory consolidation (Liu et al., 20 Apr 2025), and modern memory compression/generative replay (Lamers et al., 24 Jun 2025, Cheng et al., 4 Aug 2024). The precise choice of memory policy, buffer representation, and replay strategy is increasingly driven by application requirements and resource budgets, with ongoing research focused on attaining domain-independent, efficient, and robust continual learning.