Chunking: Continual Learning is not just about Distribution Shift (2310.02206v2)

Abstract: Work on continual learning (CL) has thus far largely focused on the problems arising from shifts in the data distribution. However, CL can be decomposed into two sub-problems: (a) shifts in the data distribution, and (b) dealing with the fact that the data is split into chunks and so only a part of the data is available to be trained on at any point in time. In this work, we look at the latter sub-problem, the chunking of data. We show that chunking is an important part of CL, accounting for around half of the performance drop from offline learning in our experiments. Furthermore, our results reveal that current CL algorithms do not address the chunking sub-problem, only performing as well as plain SGD training when there is no shift in the data distribution. Therefore, we show that chunking is both an important and currently unaddressed sub-problem and until it is addressed CL methods will be capped in performance. Additionally, we analyse why performance drops when learning occurs on identically distributed chunks of data, and find that forgetting, which is often seen to be a problem due to distribution shift, still arises and is a significant problem. We also show that performance on the chunking sub-problem can be increased and that this performance transfers to the full CL setting, where there is distribution shift. Hence, we argue that work on chunking can help advance CL in general.

• The paper formulates and quantifies the chunking problem, showing it accounts for nearly half of the performance drop in Continual Learning.
• It demonstrates that state-of-the-art CL methods, including DER++ and ER, perform similarly to plain SGD when faced with chunked data.
• The authors propose per-chunk mean weight averaging, which significantly improves accuracy and transfers benefits to the standard CL setting.

Chunking: Continual Learning is not just about Distribution Shift

The paper "Chunking: Continual Learning is not just about Distribution Shift" by Thomas L. Lee and Amos Storkey investigates a key aspect of Continual Learning (CL) which has been largely overlooked in the literature—the chunking sub-problem. Continual Learning traditionally deals with shifts in data distribution, commonly known as task shift. However, the authors highlight that another critical component is the division of data into chunks, where each chunk of data is available to the learner only once, a situation referred to as the "chunking problem".

Core Contributions

The main contributions of the paper are threefold:

1. Formulation and Significance of the Chunking Problem: The authors decompose CL into two sub-problems: learning amidst shifts in data distribution and learning with data that is split into sequential chunks. Their experiments show that chunking itself accounts for approximately half of the performance drop observed when comparing CL to offline learning, which questions the community’s focus solely on distribution shift.
2. Performance Analysis of Existing CL Methods: Through rigorous experimentation, the paper demonstrates that current state-of-the-art CL methods do not adequately address the chunking problem. Methods tested include DER++, ER, and several others that perform comparably to plain SGD training under the chunking setting, indicating a significant gap in the current CL algorithms.
3. Improvement via Per-Chunk Weight Averaging: The paper introduces a per-chunk weight averaging method as a solution to mitigate the performance drops caused by chunking. This method shows considerable performance improvement in the chunking setting and the benefits transfer to the standard CL setting. The method particularly highlighted is mean weight averaging, which performs better than exponential moving average (EMA) and is also shown to outperform a traditional CL weight averaging approach, IMM.

Experimental Methodology and Results

The experiments cover three commonly used datasets: CIFAR-10, CIFAR-100, and Tiny ImageNet, using a ResNet18 backbone model. The authors designed their experiments carefully to distinguish the impact of chunking from that of distribution shift. By creating identical chunks where data distributions remained constant across chunks, the authors isolated the chunking problem.

Key numerical results include:

• In CIFAR-100, chunking is responsible for 50.05% of the performance drop from offline learning to CL.
• In Tiny ImageNet, chunking contributes 46.69% to the performance drop.
• Introducing per-chunk mean weight averaging can improve accuracy by significant margins, e.g., on CIFAR-100 it improved accuracy by up to +12.46% in standard CL settings.

Theoretical and Practical Implications

On a theoretical level, the paper’s findings call for a reconsideration of how we model and solve CL problems. The chunking problem suggests that forgetting in neural networks is not solely induced by task shift but also by the inability of CL algorithms to handle chunked data efficiently.

Practically, this work implies that future advancements in CL could borrow insights from online learning and improve learning efficiency through better handling of chunked data. Techniques like weight averaging, especially the mean weight approach, could become more prevalent in CL methodologies and might offer a new direction of research for reducing catastrophic forgetting and enhancing positive transfer across chunks.

Speculations on Future AI Developments

In the field of advancing artificial intelligence, addressing the chunking problem might lead to the development of more robust and adaptable CL systems. As most real-world applications involve learning from data streams that are inherently chunked (e.g., sensor data from autonomous vehicles, user interactions in web services), solutions to the chunking problem could significantly enhance the practical applicability of CL, leading to more effective, efficient, and flexible AI systems.

Overall, this research by Lee and Storkey provides critical insights and methods that question existing paradigms in CL and suggests a richer understanding and framework necessary for mastering continual learning. The exploration of the chunking sub-problem opens numerous avenues for developing more resilient and capable learning algorithms that better mimic human learning capabilities.