Existence of a memory-efficient procedure that satisfies Cartridge desiderata

Determine whether there exists a training procedure for Cartridges—trainable key–value cache adapters that augment a frozen large language model—that simultaneously achieves generality across diverse user prompts and structural awareness of document dependencies while providing strong memory efficiency relative to placing the entire corpus in the context window.

Background

The paper introduces Cartridges, which are trained key–value cache representations of a corpus that can be loaded at inference time to emulate in-context learning while using far less memory. To be practically useful, Cartridges must replicate the breadth of tasks supported by in-context learning (generality) and maintain awareness of document structure and long-range relations (structural awareness), all under strict memory constraints.

Before proposing Self-Study, the authors explicitly state uncertainty about whether any training procedure can meet these desiderata without sacrificing memory efficiency, motivating the development and evaluation of their approach.