A Neural Dirichlet Process Mixture Model for Task-Free Continual Learning (2001.00689v2)

Abstract: Despite the growing interest in continual learning, most of its contemporary works have been studied in a rather restricted setting where tasks are clearly distinguishable, and task boundaries are known during training. However, if our goal is to develop an algorithm that learns as humans do, this setting is far from realistic, and it is essential to develop a methodology that works in a task-free manner. Meanwhile, among several branches of continual learning, expansion-based methods have the advantage of eliminating catastrophic forgetting by allocating new resources to learn new data. In this work, we propose an expansion-based approach for task-free continual learning. Our model, named Continual Neural Dirichlet Process Mixture (CN-DPM), consists of a set of neural network experts that are in charge of a subset of the data. CN-DPM expands the number of experts in a principled way under the Bayesian nonparametric framework. With extensive experiments, we show that our model successfully performs task-free continual learning for both discriminative and generative tasks such as image classification and image generation.

• The paper proposes CN-DPM, a novel continual learning model that uses a Bayesian nonparametric framework to dynamically expand neural experts.
• It integrates classifiers with density estimators and a short-term memory system to trigger model expansion during a sleep phase.
• Experiments on MNIST, SVHN, and CIFAR demonstrate that CN-DPM outperforms traditional baselines in preventing catastrophic forgetting.

A Neural Dirichlet Process Mixture Model for Task-Free Continual Learning

This paper presents an innovative approach to continual learning termed the Continual Neural Dirichlet Process Mixture (CN-DPM), addressing the limitations posed by traditional task-based continual learning methodologies. Continual learning seeks to emulate the human cognitive model by learning from a sequentially presented non-iid dataset without experiencing catastrophic forgetting. The challenge is heightened in a task-free scenario where task boundaries and definitions are unidentified during both training and inference phases.

Methodology

The CN-DPM is an expansion-based model leveraging a Mixture of Experts (MoE) approach. It comprises several neural network experts—each modeled under the Bayesian nonparametric framework. This framework is particularly suited to task-free continual learning due to its innate ability to determine model complexity based on data, unlike parametric methods which require predefined complexity.

Each expert within the CN-DPM possesses a classifier and a generative density estimator. These jointly capture both generative and discriminative data properties, enabling the model to operate effectively across diverse continual learning tasks. The model's expansion capability permits the introduction of additional experts to accommodate novel data, thus circumventing catastrophic forgetting by preserving existing components.

A significant element of the CN-DPM's architecture is its short-term memory (STM). When a new task is identified (i.e., when a new expert is warranted), the STM collects relevant data, subsequently initiating model expansion during a 'sleep phase.' This ensures robust training of new experts, minimizing overfitting risks.

Experimental Results

The paper thoroughly evaluates the CN-DPM model in various task-free continual learning scenarios using datasets such as MNIST, SVHN, and CIFAR 10/100. The results demonstrate CN-DPM's capacity to outperform traditional baselines (e.g., reservoir sampling) significantly, especially when the task scenarios extend over numerous epochs. Notably, CN-DPM maintains its performance without succumbing to task forgetting, a common challenge in other continual learning approaches.

Furthermore, CN-DPM's generative capabilities are validated through sample generation tasks, confirming that the model effectively retains learned distributions.

Implications and Future Directions

The introduction of CN-DPM and its promising results suggest several implications for the future of AI developments in continual learning:

1. Scalability: CN-DPM's design and framework could prove pivotal in developing scalable AI systems capable of lifelong learning without human intervention, even in task-free environments.
2. Application: Expanding the application of CN-DPM into areas like reinforcement learning and NLP could yield models that adaptively learn task representations on-the-fly, integrating knowledge across various domains.
3. Enhancements: Future research could focus on addressing CN-DPM's components selection accuracy, potentially improving expert assignment and overall model performance.

These insights open avenues for integrating such Bayesian nonparametric approaches within AI systems requiring autonomous, adaptive learning capabilities adhering to realistic continual learning scenarios.