Extend the dependency graph across the full context

Construct a dependency graph that follows attention-channel aggregation across all source positions in the context, rather than stopping at the value read at the position being explained.

Background

The paper’s dependency graph follows feed-forward and attention components backward, but for an attention channel it traces only the value read at the attended position. It does not follow the attention pattern that aggregates values across the sequence.

The authors identify a full-context graph as the next instrument needed to analyze circuits whose computation depends on information originating at multiple positions.

References

The dependency graph follows an attention channel back to what it reads at the position being explained and no further (Section~\ref{sec:naming-attn}). What a channel contributes is that value aggregated over the sequence, and following the pattern that does the aggregating leaves the position being explained and opens a graph over the whole context. Every circuit in this paper is for one position; the graph over the context is the next instrument.

— Through the Looking Glass: Directly Reading and Writing Transformers  (2609.10210 - Oskin, 9 Sep 2026) in Section “Is this as good as it gets?”