Generality and adaptability across multiple tasks in LLM-based systems

Establish methods that enable Large Language Model (LLM)-based systems to achieve both generality and adaptability across multiple tasks, allowing continual acquisition of new abilities without degrading previously learned capabilities.

Background

The review concludes that despite significant progress on specialized LLM-based systems, achieving broad generality and adaptability across many tasks remains unresolved. Incremental learning is proposed as a means to bridge this gap by enabling models to acquire new knowledge over time while preserving existing capabilities.

The authors argue that current approaches often rely on periodic batch updates and external system components rather than true, real-time incremental updates to the core models, underscoring the open nature of this objective.

References

However, the goals of generality and adaptability across multiple tasks remain an open problem.

— Towards Incremental Learning in Large Language Models: A Critical Review  (2404.18311 - Jovanovic et al., 2024) in Section 4 (Conclusion)

However, existing methods typically employ fixed frequency selection strategies and focus primarily on single-task adaptation. Their applicability to continual learning scenarios remains largely unexplored.

— Frequency-Aware Continual Learning for Smart Contract Vulnerability Detection with Large Language Models  (2608.19680 - Huang et al., 20 Aug 2026) in Section 2, subsection “Parameter-Efficient Fine-Tuning for LLMs”

How to adapt such models sequentially without deforming their learned geometry therefore remains open.

— Hyperbolic Multimodal Continual Learning: A Closest-Admissible Solution  (2609.29329 - Liu et al., 24 Sep 2026) in Introduction, subsection “Hyperbolic Multimodal Learning”

Using contrastively derived embeddings for general task adaptation is relatively underexplored, and systematic comparisons with non-contrastive approaches remain an open area for further study.

— Efficient Task Adaptation in Large Language Models: A Survey of Weight-Based, Prompt-Based, and Embedding-Based Adaptations  (2610.00928 - Park et al., 1 Oct 2026) in Section 5, subsection “ICL-Derived Task Embeddings,” paragraph “Contrastively Derived”