Assess how private residual communication scales with model size

Determine how much residual-stream capacity remains available for private communication channels in transformer models as model scale increases, and whether the available free subspace shrinks with scale.

Background

The paper constructs cross-layer circuits by writing intermediate information into a random residual direction that existing components are unlikely to read. This works in the studied model because the model’s effective working dimensionality occupies only a fraction of the residual width.

The authors identify the amount of unused residual capacity and its scaling behavior as unresolved questions relevant to whether such private channels can support programmable or editable transformers at larger scales.

References

The second is cheap for a reason worth generalizing, since the space a trained model operates in occupies a modest fraction of its width, leaving room for channels that collide with nothing. How much room, and whether it shrinks with scale, is left for future work.

— Through the Looking Glass: Directly Reading and Writing Transformers  (2609.10210 - Oskin, 9 Sep 2026) in Section “Discussion,” paragraph “Writing”