Composition of operation records across multiple updates

Determine whether the operation-span records identified in the frozen-writer, isolation-trained-reader framework compose across multiple state updates, extending the single-step analysis to longer sequences of updates.

Background

The paper studies single-step records: a frozen base model writes key/value representations while processing one operation, and a separately trained reader accesses the operation span while the state-description lines are masked. The experiments show that these records support native routing and, after isolation training, direct access to operation-local payloads.

The authors explicitly limit the analysis to mostly synthetic, single-step tasks and leave unresolved whether records generated by successive updates can be combined reliably. Resolving this issue would test whether the observed localized records support longer, composed state transformations rather than only isolated operations.

References

We study single-step records on mostly synthetic tasks; composition across updates remains open, and ToMi and GSM8K test the access gap but not the mechanism.

— Written as a Record, Read as an Address: What a Forward Pass Leaves in an Operation's KV Cache  (2609.24635 - Wu et al., 21 Sep 2026) in Section 7, Limitations