Assess how private residual communication scales with model size

Determine how much residual-stream capacity remains available for private communication channels in transformer models as model scale increases, and whether the available free subspace shrinks with scale.

Background

The paper constructs cross-layer circuits by writing intermediate information into a random residual direction that existing components are unlikely to read. This works in the studied model because the model’s effective working dimensionality occupies only a fraction of the residual width.

The authors identify the amount of unused residual capacity and its scaling behavior as unresolved questions relevant to whether such private channels can support programmable or editable transformers at larger scales.

References

The second is cheap for a reason worth generalizing, since the space a trained model operates in occupies a modest fraction of its width, leaving room for channels that collide with nothing. How much room, and whether it shrinks with scale, is left for future work.

— Through the Looking Glass: Directly Reading and Writing Transformers  (2609.10210 - Oskin, 9 Sep 2026) in Section “Discussion,” paragraph “Writing”

A data-driven estimate of $\lambda_{\mathrm{occ}}$, $\lambda_{\mathrm{spare}}$, and $f_{\mathrm{occ}}$ from the readout-layer covariance spectrum would tighten this bound; we leave this to future work.

— Your Model Is Leaking: Covert Information Transfer through LLM Residual Streams  (2609.27996 - Li et al., 23 Sep 2026) in Appendix, Section 'Gaussian Surrogate and Defense Floor' (Appendix~\ref{sec:discussion:commtheory})

First, we evaluate models up to 27B parameters ($d{=}5120$); production-scale models (70B+, $d \geq 8192$) further increase the processing gain (Appendix~\ref{app:commtheory}) and are expected to be more vulnerable, but this remains unverified.

— Your Model Is Leaking: Covert Information Transfer through LLM Residual Streams  (2609.27996 - Li et al., 23 Sep 2026) in Appendix, Section 'Scope and Limitations' (\S\ref{app:limitations})