---
title: 'CircuitLens: Circuit-Based Neural Interpretability'
url: https://www.emergentmind.com/topics/circuitlens
type: topic
---

# CircuitLens: Circuit-Based Neural Interpretability

CircuitLens refers to a circuit-based mechanistic interpretability method for large neural networks that systematically uncovers the causal “wiring diagrams” underlying sparse feature activations, with an emphasis on transcoder-equipped transformer models. CircuitLens advances the landscape of explainable AI by moving beyond activation-centric analysis—addressing not only “what” triggers a feature, but “how” signals propagate and interact along attention and transcoder pathways to produce both local and downstream effects. The technique leverages explicit feature attributions, Jacobian-based signal tracing, and clustering of upstream and downstream contributors to reveal polysemantic subcircuits, thereby enriching the global interpretability and robustness of model analysis [2510.14936].

## 1. Motivating Background and Conceptual Foundations

Standard activation-based interpretability methods (e.g. MaxAct, Neuronpedia) are constrained to identifying inputs that maximally activate individual features or neurons, but they fail to capture the circuit-level interactions between network components, especially in the presence of attention mechanisms or complex transcoder features. Automated approaches based on isolated activations often depend on the quality of external LLMs and datasets, missing synergistic or causal pathways. CircuitLens addresses this by isolating not only which tokens and attention heads feed into a given activation (“input-centric” analysis), but also how these features influence downstream logits and outputs (“output-centric” analysis). It then aggregates these interactions across samples, employing clustering algorithms to delineate coherent subcircuits—subsets of pathways with distinct mechanistic functionality—unlocking richer interpretability and revealing polysemantic roles not accessible via conventional activation analysis [2510.14936].

## 2. Formal Attribution Framework

CircuitLens builds upon the transcoder attribution framework embedded within transformer models augmented with sparse linear encoders and decoders. For a given layer $\ell$, the residual stream of dimension $d_{\mathrm{model}}$ is mapped to a sparse feature space of dimension $d_{\mathrm{features}}$ via an encoder matrix $W_{\mathrm{enc}}^{(\ell)} \in \mathbb{R}^{d_\mathrm{model} \times d_\mathrm{features}}$ and decoder $W_{\mathrm{dec}}^{(\ell)} \in \mathbb{R}^{d_\mathrm{features} \times d_\mathrm{model}}$.

### Upstream Attribution
The contribution from an earlier feature $(\ell', i')$ to a later feature $(\ell, i)$ at the same token $t$ is quantified by:
$$
\text{activation}^{(\ell', i')}\big[t\big] \times \left( f^{(\ell', i')}_\mathrm{dec} \cdot J^{(\ell \rightarrow \ell')}\big[t\big] \cdot f^{(\ell, i)}_\mathrm{enc} \right)
$$
Here, the Jacobian $J^{(\ell \rightarrow \ell')}$ accounts for inter-layer interference, and the calculation traces causal signal propagation across circuit components, treating non-linearities as locally constant.

### Attention-Head Attribution
Given attention matrices, CircuitLens computes which previous tokens $s$ (via head $h$ in layer $\ell'$) affect feature $(\ell, i)$ at token $t$:
$$
\text{score}^{(\ell', h)}(r_{\mathrm{pre}}^{(\ell')}\big[t\big], r_{\mathrm{pre}}^{(\ell')}\big[s\big])\times \left( (W_{\mathrm{OV}}^{(\ell', h)})^T f_\mathrm{enc}^{(\ell, i)} \cdot r_{\mathrm{pre}}^{(\ell')}\big[s\big] \right)
$$
with $W_{\mathrm{OV}}$ the value-output matrix and $r_{\mathrm{pre}}$ denoting the residual stream before attention.

### Downstream Attribution
The influence of a feature activation on the model's output logits for the next token is:
$$
\text{activation}^{(\ell, i)}[t] \times \left( f^{(\ell, i)}_\mathrm{dec}\cdot J^{(y \rightarrow (\ell, i))}[t] \cdot W_U[:, y[t]] \right)
$$
where $W_U$ is the unembedding matrix. This allows rigorous tracking of feature contributions to model outputs.

## 3. CircuitLens Algorithmic Pipeline

The typical workflow proceeds as follows (assuming transcoder weights and Jacobian computations are available):

- **Step 0: Activation Sampling**  
  Partition the feature activation distribution into $B = 20$ quantile bins. Weight each activation in bin $j$ by $w_j = 1/(n_\mathrm{bin})^{0.9}$, where $n_{\rm bin}$ is the bin count. Sample $N=100$ token-contexts proportional to these weights to reduce top-$k$ bias.
- **Step 1: Input-centric Attribution**  
  For each sampled activation, compute head-token contribution scores. Select outlier head-token pairs (z-score above threshold); mask contexts to retain only strongly contributing tokens and the activating token, producing succinct “pattern” fragments.
- **Step 2: Output-centric Attribution (Optional)**  
  For each sample, autoregressively generate up to 15 new tokens. Use output-centric attribution to determine which generated tokens were notably promoted, based on z-score outlier thresholds.
- **Step 3: Circuit-based Clustering**  
  For each sample $j$, record the set $S_j$ of contributing elements: upstream features and significant head-token pairs. Frequency filter: retain elements present in at least a proportion $\rho$ (default 0.1). Compute pairwise Jaccard similarities, and perform DBSCAN clustering to delineate coherent input subcircuits.
- **Step 4: Cluster-level Interpretation**  
  For each cluster, present masked input/output fragments to a small LLM (e.g. GPT-4-mini) for literal pattern extraction and one-line cluster summary.
- **Step 5: Merge and Synthesize Explanations**  
  Aggregate or concatenate per-cluster labels into a unified interpretive statement describing feature $(\ell, i)$.

## 4. Experimental Results and Empirical Findings

All evaluations utilize the Gemma-2-2B transcoder on a held-out Pile subset, with the FADE framework providing Clarity (C), Responsiveness (R), Purity (P), and Faithfulness (F) scores. Comparative results by layer are summarized:

| Layer | CircuitLens-Full (C/R/P/F) | Neuronpedia | MaxAct* |
|-------|----------------------------|-------------|---------|
|   0   | 0.66 / 0.24 / 0.04 / 0.02  | 0.51/0.23/0.09/0.03 | 0.50/0.21/0.09/0.04 |
|   7   | 0.61 / 0.18 / 0.07 / 0.04  | 0.44/0.17/0.11/0.03 | 0.48/0.15/0.09/0.03 |
|  12   | 0.26 / 0.10 / 0.07 / 0.01  | 0.15/0.10/0.11/0.00 | 0.17/0.10/0.10/0.00 |
|  21   | 0.56 / 0.73 / 0.62 / 0.03  | 0.36/0.71/0.64/0.02 | 0.39/0.71/0.64/0.02 |

Key conclusions are: CircuitLens attains up to 30–40 points higher Clarity in early layers. Responsiveness and Purity improve with broad sampling of activation distribution. Faithfulness is universally low for all methods (≤0.05), underlining the distributed, polysemantic nature of features. Combining WeightLens-derived context-independent tokens with CircuitLens clusters (“WL + CircuitLens-Full”) yields further gains in robustness [2510.14936].

## 5. Qualitative Case Studies and Interpretive Insights

Case analysis demonstrates the method’s capacity for revealing polysemantic subcircuits:
- **Feature L12F619 (layer 12, feature 619):** Standard top-activated sentences show no clear semantic commonality. CircuitLens uncovers three distinct subcircuits, each focusing on tokens that indicate definite or demonstrative references (e.g., “the”, “this”, “that”). Cluster description: “Feature activates on phrases where a previously mentioned entity is referred to by a definite or demonstrative article.”
- **Feature L21F91 (layer 21, feature 91):** Input clusters correspond to prepositions and select verbs. Output-centric analysis shows that after activation, the feature promotes causal linker phrases (“based on”, “the basis of”). This pinpoints functional roles not accessible with activation-only pipelines.

## 6. Strengths, Limitations, and Applicability

**Strengths:**
- Reveals the explicit causal “wiring” of network features, tracking contributions along both attention and transcoder axes.
- Identifies polysemantic subcircuits via clustering of causal elements, not merely co-activations.
- Enhances clarity and responsiveness of explanations compared to activation-only approaches.
- Sampling across the full activation spectrum mitigates top-$k$ activation sampling bias.
- Can be combined with weight-based WeightLens to comprehensively address both context-independent and context-dependent interpretability.

**Limitations:**
- Downstream (output-centric) attribution demands a computationally intensive backward pass, impacting scalability.
- Faithfulness for single-feature interventions is low; circuit-level (multifeature) analyses may be required.
- Clustering (DBSCAN) is sensitive to hyperparameters $(\rho, \epsilon)$, risking over- or under-partitioning.
- The final natural language descriptions of feature clusters rely on a prompt to a small LLM.
- Efficacy degrades in highly context-dependent mid-model layers.

**Scope of Applicability:**
CircuitLens is designed for models equipped with transcoders or linear sparse decompositions, but its attribution and clustering routines can in principle generalize to any transformer model that permits Jacobian-tracked signal tracing through attention and MLP sublayers [2510.14936].

## 7. Future Directions and Open Problems

Emerging research themes and open problems include:
- Quantitative circuit-level interventions: developing “Faithfulness” metrics that probe the causal efficacy of entire discovered subcircuits.
- Automated hyperparameter tuning for robust and accurate subcircuit clustering (e.g., adaptive $\rho$, $\epsilon$ for DBSCAN).
- Scaling to larger models (e.g., Gemma-3, Llama-3) and other modalities such as vision transformers.
- Integrating sophisticated complexity-control techniques to reduce spurious cluster elements.
- Unification of WeightLens and CircuitLens into a seamless end-to-end framework for both context-invariant and highly context-dependent features, ideally without reliance on LLM prompts for cluster explanation.
- Experimenting with alternative or differentiable clustering objectives (e.g., spectral clustering, graph-based algorithms).
- Application in model steering tasks—bias mitigation, hallucination control—through targeted intervention on mechanistically interpreted subcircuits rather than isolated neurons.

In summary, CircuitLens operationalizes a new mode of mechanistic circuit analysis for deep networks, systematically capturing and interpreting the distributed causal pathways that underlie both context-rich and context-invariant behaviors, and thereby extends the frontier of robust, scalable, and granular interpretability in modern AI systems [2510.14936].

Source: https://www.emergentmind.com/topics/circuitlens