Establish whether Transcoders learn mechanistically faithful latents

Establish whether Transcoders learn more mechanistically faithful latent features than methods that reconstruct only dense activations.

Background

The paper contrasts Sparse Autoencoders, which reconstruct target activations, with Transcoders, which approximate a module’s full input-output function. The stronger functional objective is hypothesized to encourage recovery of features that correspond more faithfully to the underlying computation, but the claim is presented as a conjecture rather than an established result.

References

Because the Transcoder must reproduce the full input-output mapping rather than merely reconstruct activations, it is conjectured to learn more mechanistically faithful latents~\citep{paulo2025transcoders}.

— Sparse Transformer: a Controlled Testbed for Mechanistic Interpretability  (2610.19907 - Korznikov et al., 14 Oct 2026) in Section 2, Related Work, paragraph “SAEs and Transcoders”