Mechanistic role of layer 1 in causal demographic identity processing

Determine what demographic-identity information is mechanistically encoded at layer 1 of Mistral-7B, including whether its apparent causal effect reflects information distinct from the degenerate layer-0 read-out or merely the near-identical geometry of layers 1 and 2.

Background

The exhaustive causal patching sweep identified a statistically robust effect for RACE×POLIDEOLOGY at layer 11 and a suggestive, selection-sensitive effect at layer 1 for RACE×RELIG. However, the authors could not establish what layer 1 represents mechanistically because the layer-0 read-out is constant across cells under the prompt design, preventing a direct comparison with pure embeddings.

Layers 1 and 2 have highly similar geometry, while both differ appreciably from layer 11. Consequently, the paper treats the mechanistic interpretation of the layer-1 effect as unresolved rather than claiming that layer 1 constitutes an independent demographic-identity mechanism.

References

What layer 1 encodes mechanistically remains open: a check on already-collected residual streams was inconclusive because the layer-0 read-out is degenerate in our prompt design (the final token is shared across cells), so we cannot contrast layer 1 against pure embeddings; layers 1 and 2 are near-identical in geometry ($=0.90$) and both differ appreciably from layer 11 ($=0.78$).

— Readable, Faithful, Used: Three Dissociable Properties of Demographic Identity in a Language Model  (2608.18768 - Robbani, 19 Aug 2026) in Section 5.3, paragraph “Caveats”; Section 10, Limitations