Three scenarios for continual learning (1904.07734v1)

Abstract: Standard artificial neural networks suffer from the well-known issue of catastrophic forgetting, making continual or lifelong learning difficult for machine learning. In recent years, numerous methods have been proposed for continual learning, but due to differences in evaluation protocols it is difficult to directly compare their performance. To enable more structured comparisons, we describe three continual learning scenarios based on whether at test time task identity is provided and--in case it is not--whether it must be inferred. Any sequence of well-defined tasks can be performed according to each scenario. Using the split and permuted MNIST task protocols, for each scenario we carry out an extensive comparison of recently proposed continual learning methods. We demonstrate substantial differences between the three scenarios in terms of difficulty and in terms of how efficient different methods are. In particular, when task identity must be inferred (i.e., class incremental learning), we find that regularization-based approaches (e.g., elastic weight consolidation) fail and that replaying representations of previous experiences seems required for solving this scenario.

An Essay on "Three Scenarios for Continual Learning"

Continual learning is a crucial challenge in the advancement of artificial intelligence, as it involves incrementally learning different tasks without forgetting previously acquired knowledge. The paper "Three scenarios for continual learning" by Gido M. van de Ven and Andreas S. Tolias addresses the issue of catastrophic forgetting in standard artificial neural networks and proposes structured scenarios to evaluate methods for continual learning more comprehensively.

The authors introduce three distinct scenarios for continual learning to standardize evaluation protocols: Task-Incremental Learning (Task-IL), Domain-Incremental Learning (Domain-IL), and Class-Incremental Learning (Class-IL). Each scenario represents a different level of difficulty based on the availability and inference of task identity at test time.

1. Task-Incremental Learning (Task-IL):
   • The simplest scenario where models are provided with task identity at test time.
   • It allows the utilization of task-specific components, often implemented with a multi-headed output layer.
2. Domain-Incremental Learning (Domain-IL):
   • A more challenging scenario where task identity is not provided at test time.
   • Models do not need to infer the task but must solve the tasks they encounter based purely on the input data.
3. Class-Incremental Learning (Class-IL):
   • The most challenging scenario where models must both solve tasks and infer task identity at test time.
   • This scenario encapsulates real-world problems requiring incremental learning of new classes.

To compare the performance of different continual learning methods, the paper employs the split MNIST and permuted MNIST task protocols. The split MNIST protocol involves classifying MNIST digits across a sequence of five tasks, while the permuted MNIST protocol involves classifying digits with different pixel permutations across ten tasks.

Evaluation of Continual Learning Methods

The paper assesses several recently proposed continual learning methods, which are categorized into three primary strategies:

1. Task-specific Components:
   • XdG (Context-dependent Gating): Selects random subsets of network nodes uniquely for each task, used only in Task-IL.
2. Regularized Optimization:
   • EWC (Elastic Weight Consolidation): Introduces a quadratic penalty to critical network parameters to retain knowledge from previous tasks.
   • Online EWC: A variant of EWC with a running sum of Fisher Information matrices to make the approach scalable.
   • SI (Synaptic Intelligence): Similar to EWC, but estimates parameter importance via their contribution to loss changes, normalized by their total change.
3. Modifying Training Data (Replay-Based Methods):
   • LwF (Learning without Forgetting): Replays current task inputs labeled with soft targets to mitigate forgetting.
   • DGR (Deep Generative Replay): Utilizes a generative model to generate and replay samples from previous tasks.
   • DGR+distill: Combines generative replay with soft targets for replayed data.
4. Replay + Exemplars:
   • iCaRL (Incremental Classifier and Representation Learning): Integrates exemplar representation with replaying stored data and a nearest-class-mean classification rule.

Numerical Results and Comparisons

The extensive experiments reveal that replay-based methods significantly outperform regularization-based methods in the more challenging Domain-IL and Class-IL scenarios. Specifically, methods such as DGR, DGR+distill, and iCaRL demonstrate superior performance, particularly in the Class-IL scenario where both task-solving and task identity inference are required.

For the split MNIST protocol, replay-based methods achieved over 90% accuracy in all scenarios, whereas regularization methods struggled significantly when task identity had to be inferred (e.g., Class-IL scenario with around 20% accuracy). Similar trends were observed for the permuted MNIST protocol, though the Task-IL and Domain-IL scenarios displayed smaller differences in difficulty, partly due to the architecture's design focusing primarily on the output layer.

The use of replay methods shows that mitigating catastrophic forgetting might necessitate incorporating some form of replay, potentially through generated or stored samples, even in the face of growing data complexity.

Implications and Future Directions

The findings underscore the inadequacy of regularization-based methods in addressing the complexities of more challenging continual learning scenarios. The success of replay-based methods in maintaining high accuracy across tasks signifies a promising pathway for future exploration.

However, the scalability of generative replay techniques to more complex datasets and real-world applications remains an open question. As generative models improve, their ability to support continual learning will likely enhance, potentially leading to more robust AI systems capable of lifelong learning.

Moreover, the classification and storage-based replay approach in iCaRL suggests additional directions for integrating memory-efficient exemplar strategies with generative models for a more nuanced understanding of task hierarchies and data dependencies.

Overall, this paper establishes a structured framework for evaluating continual learning methods, providing critical insights into the effectiveness and limitations of existing approaches. It sets the stage for future research to refine these methods further and address the continuing challenge of lifelong learning in artificial intelligence.