Continual Learning with Deep Generative Replay (1705.08690v3)

Abstract: Attempts to train a comprehensive artificial intelligence capable of solving multiple tasks have been impeded by a chronic problem called catastrophic forgetting. Although simply replaying all previous data alleviates the problem, it requires large memory and even worse, often infeasible in real world applications where the access to past data is limited. Inspired by the generative nature of hippocampus as a short-term memory system in primate brain, we propose the Deep Generative Replay, a novel framework with a cooperative dual model architecture consisting of a deep generative model ("generator") and a task solving model ("solver"). With only these two models, training data for previous tasks can easily be sampled and interleaved with those for a new task. We test our methods in several sequential learning settings involving image classification tasks.

Continual Learning with Deep Generative Replay

Continual learning in artificial neural networks aims at incrementally acquiring new skills while retaining previously learned knowledge. The challenge of "catastrophic forgetting," where the model's performance on prior tasks rapidly degrades upon learning new tasks, persists as a significant barrier in enhancing the capability of deep neural networks in solving diverse tasks. Generative Replay (GR), drawing inspiration from the brain's complementary learning systems (CLS) theory, proposes a dual-model architecture involving a generative model and a task-solving model to tackle this problem.

Main Contributions

The paper introduces Deep Generative Replay (DGR), a holistic framework integrating a generator (a deep generative model) and a solver (a task-solving model). The generator synthesizes pseudo-samples that resemble old task data, allowing seamless integration and rehearsal with new data. Key contributions and empirical findings outlined in the paper include:

1. Generative Replay Mechanism: Leveraging generative adversarial networks (GANs), the framework periodically "replays" synthesized data from prior tasks. This mitigates the need for extensive memory storage of past data, addressing both ethical and technical constraints.
2. Solver Training: As the solver encounters new tasks, it is concurrently exposed to pseudo-samples generated by the GAN. This interleaved training enables the solver to maintain and build upon its existing knowledge base without catastrophic forgetting.
3. Empirical Evaluations: The framework's efficacy was tested across various sequential learning settings involving image classification tasks. Metrics reported showcased substantial retention of performance on previous tasks, substantially outperforming naive sequential training methodologies.

Theoretical Underpinnings and Related Work

Drawing inspiration from biological neural systems, specifically the interplay between the hippocampus and neocortex in CLS theory, the proposed framework ensures that memory consolidation-like mechanisms are mirrored in artificial systems. Previous approaches predominantly involved:

• Regularization Techniques: Methods such as Elastic Weight Consolidation (EWC) aim to preserve critical weights through weight-specific penalties during new training endeavors.
• Memory-Augmenting Networks: Architectures leveraging episodic memory systems to replay stored samples regularly. Learning without Forgetting (LwF), for instance, minimizes parameter alterations during new task acquisition but imposes significant limitations in neural architecture adaptability.
• Pseudorehearsal Techniques: Models employing pseudo-inputs to mitigate the necessity of original data also faced scalability issues in representing high-dimensional inputs. The proposed GR-based approach overcomes this limitation by synthesizing data mirroring actual high-dimensional distributions using GANs.

Experimental Highlights

Learning Independent Tasks:

GR significantly mitigates catastrophic forgetting when training on independent tasks, exemplified by experiments on permuted MNIST datasets. Solver performance, routinely interleaved with generative pseudo-replay, retained high accuracy on earlier tasks, while a naive approach led to immediate performance attrition.

Domain Transfer Learning:

The GR framework's flexibility was demonstrated by training on datasets spanning different domains (MNIST and SVHN). The solver trained with generative replay maintained higher accuracy in both domains compared to models trained naively or with random noise replay.

Incremental Class Learning:

When tasked with learning disjoint subsets of MNIST, the GR-enabled solver substantially outperformed others by effectively integrating knowledge across subsequent tasks. Here, the GAN's role in pseudo-sample generation was pivotal in maintaining balanced performance across all learned classes.

Implications and Future Directions

The implications of Deep Generative Replay extend beyond image classification, suggesting broader applicability across reinforcement learning and other AI domains requiring incremental learning. The fidelity of the generator crucially dictates solver performance, stressing the need for ongoing advancements in generative model capabilities.

Future research must evaluate larger-scale applications and explore integrative frameworks combining DGR with regularization-based approaches or dynamic network architectures for holistic continual learning solutions. The potential to extend DGR to complex, multimodal tasks presents an exciting frontier for enhancing AI systems' adaptability and robustness.

In summary, the introduction of Deep Generative Replay offers a promising pathway in overcoming catastrophic forgetting, leveraging generative models to sustain continuous learning without the constraints of extensive memory demands. The paper's empirical results underscore DGR's potential in revolutionizing the domain of continual learning.