Detecting computed but un-emitted distinctions

Determine whether distinctions computed by a transformer but never emitted into the output vocabulary basis are common, and develop methods to detect such distinctions without relying on unembedding-matrix logit-lens readouts.

Background

Contrastive projection reads differences between hidden states by projecting them through the unembedding matrix. The method can reveal only distinctions that have a component aligned with the model’s output vocabulary basis. The paper therefore distinguishes between a distinction that is absent from the computation and one that is computed internally but never emitted in a direction visible to the unembedding matrix.

The unresolved issue is whether such computed-but-unemitted distinctions occur frequently and how they could be identified by methods that do not depend on the same output-space projection. Resolving it would establish the ceiling of contrastive logit-lens methods and clarify whether an apparently absent feature reflects a limitation of the representation or merely a limitation of the readout.

References

A distinction the model computes but never emits into the output basis would be invisible to any $W_U$ readout, however clean the contrast. Whether such computed-but-unemitted distinctions are common, and how to detect them without the readout, is an open question.

— Contrastive Projection: Reading Transformer Internals by Differencing Logit Lenses  (2609.09902 - Tuomi, 9 Sep 2026) in Section 6, Discussion, paragraph “A hypothesis: metaphor as a linear conceptual mapping” preceding; specifically the final paragraph of the main Discussion section

Naming those quantities is the open problem; what follows is what is known about them.

— Through the Looking Glass: Directly Reading and Writing Transformers  (2609.10210 - Oskin, 9 Sep 2026) in Section “Where the components have no name”

Whether the rest is morphological, orthographic, or positional needs a method that names features rather than tokens, which is what a sparse autoencoder provides \citep{cunningham2023sparse,bricken2023monosemanticity}.

— Through the Looking Glass: Directly Reading and Writing Transformers  (2609.10210 - Oskin, 9 Sep 2026) in Section “Where the components have no name”