Validation of ConPET under diverse continual learning scenarios and expert splitting strategies

Demonstrate the effectiveness of Continual Parameter-Efficient Tuning (ConPET) across more diverse continual learning scenarios and develop improved strategies for splitting tasks among expert modules.

Background

ConPET combines PET with continual learning, using static and dynamic expert modules to reduce tuning costs while aiming to preserve performance.

The review notes that broader validation and refined task-to-expert partitioning strategies are still needed.

References

However, this remains to be verified using more diverse continual learning scenarios with further improvement of task split strategies among experts.

— Towards Incremental Learning in Large Language Models: A Critical Review  (2404.18311 - Jovanovic et al., 2024) in Section 2.3 (Parameter-Efficient Learning) – ConPET

Search thus locates an intermediate optimum, on the slice of contiguous windows (the best of all 495 four-block allocations is unknown) and at the cost of 27 runs per backbone.

— Two Routes to the Middle: Placement Search and Brain Readouts Converge on Where Continual Learners Should Specialize  (2610.01590 - Huang et al., 1 Oct 2026) in Section 5.1, “The algorithmic route: search finds the middle, internal criteria miss it” (Results and Discussion)