CITB: A Benchmark for Continual Instruction Tuning (2310.14510v1)

Abstract: Continual learning (CL) is a paradigm that aims to replicate the human ability to learn and accumulate knowledge continually without forgetting previous knowledge and transferring it to new tasks. Recent instruction tuning (IT) involves fine-tuning models to make them more adaptable to solving NLP tasks in general. However, it is still uncertain how instruction tuning works in the context of CL tasks. This challenging yet practical problem is formulated as Continual Instruction Tuning (CIT). In this work, we establish a CIT benchmark consisting of learning and evaluation protocols. We curate two long dialogue task streams of different types, InstrDialog and InstrDialog++, to study various CL methods systematically. Our experiments show that existing CL methods do not effectively leverage the rich natural language instructions, and fine-tuning an instruction-tuned model sequentially can yield similar or better results. We further explore different aspects that might affect the learning of CIT. We hope this benchmark will facilitate more research in this direction.

Introduction to Continual Instruction Tuning

Continual learning (CL) represents an essential paradigm within AI research, primarily focused on developing models that can learn continuously, accumulate knowledge over time, and avoid the degeneration of previously learned information—a phenomenon known as catastrophic forgetting. Despite significant progress in LLM research, specifically through the implementation of instruction tuning (IT), questions remain about their performance within the CL context. Traditional models are adept at learning from static datasets but tend to underperform when required to adapt dynamically to new tasks without retraining. A paper introduces a novel benchmark named CITB which aims to tackle the unique challenges posed by Continual Instruction Tuning (CIT) to better understand and improve upon these issues.

The CITB Framework

CITB breaks new ground as a benchmark for evaluating the performance of LLMs under CL settings. The framework consists of two meticulously curated task streams named InstrDialog and InstrDialog++. Following a systematic approach, these streams allow for an in-depth investigation of existing CL methods in handling a sequence of NLP tasks with diverse characteristics. Pioneering experiments reveal that current CL techniques to prevent catastrophic forgetting and facilitate cross-task knowledge transfer are lacking, concluding with the strong suggestion that fine-tuning sequentially tuned instruction models can bring forth equal or superior outcomes.

Empirical Evaluation and Findings

Rigorous experimentation highlights a critical insight: current CL methodologies may not be fully leveraging natural language instructions to mitigate forgetting or aid in the transfer of knowledge. One of the most crucial findings suggests that the rich instructions embedded within tasks can enable better knowledge transfer and reduce the impact of catastrophic forgetting, a result that counters conventional wisdom in CL studies. The paper also embarks on several ablation studies, exploring the effects of instruction templates, task types, and training instance numbers on CIT.

Future Directions and Limitations

The conclusions drawn from the CITB benchmark present a compelling argument for revising the approach to CL in LLMs. This research underscores the need for innovative methods specifically engineered for the CIT paradigm. Moving forward, it is also crucial to explore models' performance with various languages and delve into the characteristics of task types in more detail, as these factors play an essential role in the effectiveness of CL methodologies. Additionally, researchers must weigh the measure used for evaluation, ensuring it accurately represents the model's capabilities for specific tasks. The paper concludes with a conviction that substantial progress in this arena will significantly advance the field.

In closing, the research calls for a shift in the development of CL methodologies that can make full use of the wealth of natural language instructions. This benchmark opens up new opportunities for the AI and ML communities to explore, develop, and refine techniques that reflect the dynamic nature of real-world task adaptation and continuous learning.