Capturing long‑range dependencies in Cartridge parameters Z

Develop a method to encode long‑range, order‑dependent relationships from a corpus C into the Cartridge parameters Z—trainable key–value cache vectors attached to the frozen large language model—so that the augmented model can reason over and retrieve dependencies spanning distant sections of C.

Background

The authors formalize requirements for Cartridges to match in-context learning behavior, including the need to capture long-range dependencies present in the corpus. Many tasks require understanding the order and relationships of information across distant parts of a document, which typical compression methods often degrade.

They explicitly note that it is not clear how to encode such dependencies into the Cartridge parameters Z, highlighting a focused technical challenge distinct from generality or composition.