Soup to go: mitigating forgetting during continual learning with model averaging (2501.05559v1)

Abstract: In continual learning, where task data arrives in a sequence, fine-tuning on later tasks will often lead to performance degradation on earlier tasks. This is especially pronounced when these tasks come from diverse domains. In this setting, how can we mitigate catastrophic forgetting of earlier tasks and retain what the model has learned with minimal computational expenses? Inspired by other merging methods, and L2-regression, we propose Sequential Fine-tuning with Averaging (SFA), a method that merges currently training models with earlier checkpoints during the course of training. SOTA approaches typically maintain a data buffer of past tasks or impose a penalty at each gradient step. In contrast, our method achieves comparable results without the need to store past data, or multiple copies of parameters for each gradient step. Furthermore, our method outperforms common merging techniques such as Task Arithmetic, TIES Merging, and WiSE-FT, as well as other penalty methods like L2 and Elastic Weight Consolidation. In turn, our method offers insight into the benefits of merging partially-trained models during training across both image and language domains.

• The paper introduces Sequential Fine-tuning with Averaging (SFA) to mitigate catastrophic forgetting by averaging model parameters during training intervals.
• The method eliminates the need for data buffers, reducing memory overhead while maintaining robust performance across diverse tasks.
• Empirical results demonstrate that SFA outperforms both model merging and penalty-based approaches in continual learning benchmarks.

Overview of "Soup to Go: Mitigating Forgetting During Continual Learning with Model Averaging"

The paper "Soup to Go: Mitigating Forgetting During Continual Learning with Model Averaging" tackles the significant challenge of catastrophic forgetting in continual learning (CL). The authors address the degradation in model performance on previously learned tasks when a model is fine-tuned with new data from diverse domains. They propose a method called Sequential Fine-tuning with Averaging (SFA) as an efficient alternative to address this issue while minimizing computational costs.

Proposed Methodology: Sequential Fine-tuning with Averaging (SFA)

Sequential Fine-tuning with Averaging is inspired by model merging techniques and L2-regression. Rather than relying on existing State-Of-The-Art (SOTA) methods that maintain a data buffer or apply penalties at each gradient step, SFA involves averaging model parameters during training. This method merges the weights of the model currently being fine-tuned with those from prior checkpoints after certain training intervals, thus ensuring that the information from previous tasks is retained throughout the training process.

The SFA method only requires parameter averaging during fine-tuning, avoiding the need for storing past data or maintaining multiple sets of parameters, effectively reducing memory overhead. The authors introduce a new concept termed "averaging frequency," denoted as $p$, which controls how frequently this averaging occurs during training, allowing a balance between past and new task performance.

Performance and Results

The authors compare SFA against existing approaches using two extensive experimental settings: classical continual learning in image classification tasks, and fine-tuning models on language domains such as Law, Math, and Code. The evaluation demonstrates that SFA consistently outperforms several model merging techniques, such as Task Arithmetic, TIES Merging, and WiSE-FT, as well as penalty-based methods like L2 and Elastic Weight Consolidation.

Key findings include:

1. Image Classification Tasks: In settings like Food-101 and CIFAR-100 with 20 task sequences, SFA performed comparably with methods involving a data buffer.
2. Language Domain Tasks: When tested on disparate language domains, SFA showed robust performance and was able to maintain task performance without a data buffer.

The results indicate that SFA achieves superior performance in preventing forgetting, even outperforming classical methods that incorporate penalties.

Theoretical Insights and Implications

The authors provide a conceptual link between SFA and L2-regression. They demonstrate that SFA can roughly approximate classical CL methods that utilize penalties, suggesting that SFA offers a computationally lighter yet effective alternative to complex CL algorithms. By aligning model merging with L2-regression, the paper bridges traditional techniques with contemporary model merging methods, offering insight into their underlying merits.

Practical and Theoretical Implications

Practically, SFA offers significant computational efficiency by eliminating the need for large data buffers or maintaining multiple model parameter sets. Theoretically, the approach introduces a novel paradigm for parameter averaging that can influence future research directions in AI, particularly in the context of singular model architectures that need to adapt continually to varying tasks without performance degradation. This suggests potential advancements in crafting adaptive AI models that can seamlessly transition between diverse tasks with minimized forgetting.

Conclusion

This paper contributes a significant tool to the field of continual learning, showcasing the efficacy of model averaging in retaining task performance. The scalable and efficient nature of SFA makes it a valuable consideration for future developments in machine learning, particularly for applications requiring continual adaptation across various domains without compromising on past learning.

The paper's emphasis on balancing model parameter updates through novel merging strategies opens avenues for future research in optimizing continual learning frameworks, both in reducing computational burdens and enhancing model robustness across tasks.