Scalability and generalizability of INCPrompt across continual learning scenarios

Determine whether the INCPrompt task-aware prompt generation approach scales with the number of tasks and data complexity and whether it generalizes to continual learning scenarios with different data distributions.

Background

INCPrompt generates task-aware prompts via attention-based transformations and demonstrated improvements on Split CIFAR-100 and Split ImageNet-R.

The authors highlight that its scalability with growing task counts and data complexity, as well as its generalizability to other continual learning settings, has not yet been established.

References

However, its scalability concerning task number and data complexity and generalizability to other CL scenarios with different data distributions remain unclear.

— Towards Incremental Learning in Large Language Models: A Critical Review (2404.18311 - Jovanovic et al., 28 Apr 2024) in Section 2.1 (Continual Learning) – INCPrompt