Unlocking Continual Learning Abilities in Language Models (2406.17245v2)

Abstract: LLMs (LMs) exhibit impressive performance and generalization capabilities. However, LMs struggle with the persistent challenge of catastrophic forgetting, which undermines their long-term sustainability in continual learning (CL). Existing approaches usually address the issue by incorporating old task data or task-wise inductive bias into LMs. However, old data and accurate task information are often unavailable or costly to collect, hindering the availability of current CL approaches for LMs. To address this limitation, we introduce $\textbf{MIGU}$ ($\textbf{M}$agn$\textbf{I}$tude-based $\textbf{G}$radient $\textbf{U}$pdating for continual learning), a rehearsal-free and task-label-free method that only updates the model parameters with large magnitudes of output in LMs' linear layers. MIGU is based on our observation that the L1-normalized magnitude distribution of the output in LMs' linear layers is different when the LM models deal with different task data. By imposing this simple constraint on the gradient update process, we can leverage the inherent behaviors of LMs, thereby unlocking their innate CL abilities. Our experiments demonstrate that MIGU is universally applicable to all three LM architectures (T5, RoBERTa, and Llama2), delivering state-of-the-art or on-par performance across continual finetuning and continual pre-training settings on four CL benchmarks. For example, MIGU brings a 15.2% average accuracy improvement over conventional parameter-efficient finetuning baselines in a 15-task CL benchmark. MIGU can also seamlessly integrate with all three existing CL types to further enhance performance. Code is available at https://github.com/wenyudu/MIGU.

• The paper introduces MIGU, a gradient update method that prevents catastrophic forgetting in language models without relying on past task data.
• The method employs L1-normalized output magnitudes to select parameter updates, achieving a 15.2% accuracy improvement across multiple benchmarks.
• It enables efficient model adaptation in continual learning frameworks by reducing dependency on supervised task labels and historical datasets.

Exploring Continual Learning in LLMs through Magnitude-based Gradient Updating

The research paper, "Unlocking Continual Learning Abilities in LLMs," introduces a significant contribution to the field of NLP, particularly addressing the challenge of catastrophic forgetting in LLMs (LMs). The authors present "MIGU" (Magnitude-based Gradient Updating), a rehearsal-free and task-label-free continual learning (CL) method designed to enhance LMs' adaptability without relying on past task data or task-specific biases.

The primary challenge facing LMs in CL scenarios is catastrophic forgetting, a prevalent issue where models forget previously learned tasks upon acquiring new information. Although various strategies have been proposed—such as rehearsal-based approaches, architecture modifications, and parameter-specific adjustments—these often require extensive data handling or task-specific annotations, which are not always feasible. MIGU aims to circumvent these constraints by harnessing inherent model features, specifically focusing on the magnitude of outputs in LMs' linear layers.

Methodology and Innovation

MIGU leverages the distribution of L1-normalized output magnitudes in LMs' linear layers during training. The key insight is that the magnitude distribution varies systematically across different tasks. By observing this variation, MIGU uses a threshold-based gradient update mechanism. During the forward pass, the magnitude of layer outputs is normalized and cached. During the backward pass, parameter updates are restricted to those with the largest magnitudes, as determined by a pre-set threshold. This selective updating purportedly aligns better with task-specific characteristics without external task labels.

Empirical Validation

The effectiveness of MIGU was empirically validated across multiple benchmarks and architectures, including T5, RoBERTa, and Llama2 models. Notably, MIGU achieved a 15.2% average accuracy improvement over conventional parameter-efficient fine-tuning baselines in a 15-task CL benchmark. Additionally, MIGU's compatibility with existing CL frameworks enhances its utility, as it can be integrated with established approaches to further improve performance.

The authors' experiments indicate that this approach is widely applicable and can function as a universal enhancer of continual learning capabilities. Importantly, MIGU shows substantial improvements in both continual fine-tuning and pre-training contexts, highlighting its adaptability and generalizability across various CL tasks.

Implications and Future Directions

From a theoretical perspective, MIGU suggests a novel pathway for mitigating task interference in continual learning, unlocking potential efficiencies in model adaptation without necessitating comprehensive task-specific data handling. The observed success across different model architectures underscores its robustness as a general method for enhancing CL in LMs.

Practically, MIGU paves the way for more efficient and cost-effective model retraining protocols, emphasizing reduced dependency on expansive and often inaccessible historical datasets. This development could significantly streamline processes in both academic and industrial settings, where continual adaptation to new data is paramount.

Future research could explore the potential of extending this magnitude-based approach to other neural architectures or even different modalities beyond NLP. Additionally, investigating the underlying theoretical reasons for observed behaviors in magnitude distributions could shed further light on intrinsic model properties that facilitate continual learning. This pursuit could also lead to refined methods for setting optimal thresholds dynamically, adapting more seamlessly to real-time changes in task requirements and data availability.

In conclusion, MIGU represents a practical and innovative step forward in addressing the debilitating issue of catastrophic forgetting within the framework of continual learning for LLMs, offering promising avenues for advancing state-of-the-art model adaptability in dynamic data environments.