Tracking Equivalent Mechanistic Interpretations Across Neural Networks

Abstract: Mechanistic interpretability (MI) is an emerging framework for interpreting neural networks. Given a task and model, MI aims to discover a succinct algorithmic process, an interpretation, that explains the model's decision process on that task. However, MI is difficult to scale and generalize. This stems in part from two key challenges: there is no precise notion of a valid interpretation; and, generating interpretations is often an ad hoc process. In this paper, we address these challenges by defining and studying the problem of interpretive equivalence: determining whether two different models share a common interpretation, without requiring an explicit description of what that interpretation is. At the core of our approach, we propose and formalize the principle that two interpretations of a model are equivalent if all of their possible implementations are also equivalent. We develop an algorithm to estimate interpretive equivalence and case study its use on Transformer-based models. To analyze our algorithm, we introduce necessary and sufficient conditions for interpretive equivalence based on models' representation similarity. We provide guarantees that simultaneously relate a model's algorithmic interpretations, circuits, and representations. Our framework lays a foundation for the development of more rigorous evaluation methods of MI and automated, generalizable interpretation discovery methods.

• The paper introduces interpretive equivalence, defining when different neural network interpretations yield equivalent mechanistic explanations.
• It presents the CONGRUITY algorithm, which employs causal interventions and representation similarity to estimate interpretation congruence efficiently.
• Empirical results on synthetic tasks and varying model scales provide evidence for robust model reduction and enhanced evaluation of interpretation quality.

Tracking Equivalent Mechanistic Interpretations Across Neural Networks

Introduction and Motivation

Mechanistic interpretability (MI) provides a formal framework for extracting algorithmic descriptions from neural networks, aiming to yield succinct, human-interpretable explanations for model decisions. Despite its promise, MI faces significant scalability and generalization challenges, compounded by the absence of a precise criterion for valid interpretations and the ad hoc nature of interpretation generation. This paper addresses these issues by focusing on interpretive equivalence: the formal problem of determining whether two models share a common interpretation, without necessitating an explicit description of the interpretation itself.

The central premise is that two mechanistic interpretations are equivalent if all their possible implementations are equivalent. This implementation-centric perspective bridges the gap between top-down and bottom-up MI approaches, offering operational reductions: (1) reductions to simpler models to facilitate scalable MI analysis, and (2) reductions to simpler tasks to enable decomposition of complex tasks via interpretive equivalence.

Formalization of Interpretive Equivalence

The authors develop a formal framework grounded in causal abstraction theory. They define mechanistic interpretations, circuits, and representations, using deterministic causal models as the abstraction backbone. A circuit is conceptualized as a minimal subset of the model's computation graph that suffices to reproduce its functional behavior. Interpretations are symbolic abstractions—causal models that abstract circuits and approximate the output with bounded error.

Definition of Interpretive Equivalence: Two interpretations are (approximately) equivalent if their sets of possible implementations (i.e., all circuits that abstract to them via permissible alignment maps) have small Hausdorff distance under a natural metric induced by circuit variables. This formulation readily accommodates abstraction-dependent interpretations and circumvents the non-identifiability issues highlighted by previous literature.

Interpretive Compression: A complementary metric, interpretive compression, is introduced as the diameter of the set of implementations for a given interpretation, quantifying the degree of abstraction and diversity within an interpretation's instantiations.

Algorithmic Framework

An explicit algorithm, termed CONGRUITY, is presented for estimating interpretive equivalence. The procedure is as follows:

• Enumerate implementations of interpretations via causal interventions—perturbing or ablating components in a model that are unrelated to core functional behavior.
• For each implementation, compute hidden representations.
• Measure pairwise linear representation similarity between representations ($\text{drepr}$), quantifying the bidirectional linear approximation error.
• Interpretations are considered congruent if, averaged over implementations, representation similarity cannot differentiate them.

The algorithm circumvents the computational barrier of strict circuit discovery (NP-hard for minimal circuits) by relaxing the requirement to identifying unimportant components, leveraging efficient scoring (e.g., activation patching).

Empirical Results

Calibration on Synthetic Tasks

On n-Permutation Detection using hard-coded RASP Transformers, CONGRUITY reliably distinguishes interpretations clustered into algorithmic families (sorting-based vs. counting-based), with high within-group congruence and low cross-group congruence. The framework captures nuanced, graded differences between interpretations, supporting its necessity-and-sufficiency characterization for interpretive equivalence.

Reductions Across Model Scale and Family

On the Indirect Object Identification (IOI) task, CONGRUITY detects interpretive equivalence within model families (GPT2 and Pythia) across scales, confirming that smaller models can faithfully represent larger ones (e.g., Pythia-160M is interpretively equivalent to Pythia-2.8B for IOI). Conversely, significant interpretive differences are found across families, aligned with documented circuit reuse patterns. These results bolster the claim that interpretive equivalence enables reduction of MI analysis to tractable subproblems.

Decomposition to Simpler Tasks

Applying CONGRUITY to next-token prediction and parts-of-speech (POS) identification reveals that GPT2's next-token prediction process is interpretively equivalent to POS identification for syntactic tokens (terminal punctuation and closing brackets), but not for semantically driven tokens (articles, prepositions). This offers a principled partitioning of next-token prediction via interpretive equivalence, advancing potential for mechanistic decomposition.

Theoretical Guarantees

Main Results (Sec. 6):

• Upper bound: Representation similarity (linear bidirectional approximation error) upper bounds interpretive equivalence, modulated by interpretive compression and representation quality.
• Lower bound: Approximate interpretive equivalence implies representation similarity must be close (up to compression and Lipschitz constant), establishing a necessary condition.
• CONGRUITY is shown to be tightly related to these bounds via hypothesis testing on representation similarity distributions over implementation sets.
• Interpretive equivalence can therefore be robustly estimated in practice using representation similarity—even without explicit enumeration or description of high-level algorithms.

Notably, abstraction-dependent interpretations (rather than abstraction-free algorithms) are necessary for non-pathological, meaningful equivalence, as proven in Appendix E.

Implications and Future Directions

The framework provides a rigorous foundation for comparing and tracking mechanistic interpretations across neural networks, enabling scalable MI analyses and reductions. Practical implications include:

• Automated MI evaluation: By estimating interpretive equivalence via representations and interventions, practitioners can sidestep full symbolic circuit discovery, accelerating MI workflows.
• Model validation and transfer: Ensuring interpretive equivalence allows interpretability analyses on efficient proxies, supporting safe deployment and transfer learning scenarios.
• Interpretation quality metrics: The formalism introduces interpretable measures (compression, congruity) to rigorously quantify abstraction and faithfulness.
• Generalization across modalities: The approach is agnostic to input modality, suggesting applicability to both language and vision models.

Theoretical implications are profound for AI safety and MI identifiability, substantiating that equivalence and compression duality is fundamental, and that representation similarity is a justified surrogate for algorithmic comparison.

Areas for further investigation include broader similarity metrics (beyond linear), deeper exploration of intervention-induced implementation enumeration, and tighter bounds dependent on restricted alignment classes and abstraction frameworks.

Conclusion

This paper establishes interpretive equivalence as the operational criterion for MI and provides an efficient algorithmic and theoretical apparatus for its estimation. By connecting circuits, representations, and interpretations through causal abstraction, the work advances the evaluability and scalability of mechanistic interpretability, charting a path toward more automated, generalizable interpretability discovery methods and robust evaluation frameworks for complex AI systems (2603.30002).