Circuit-Tracer Framework for VLM Interpretability

• The Circuit-Tracer Framework is a method for mechanistic interpretability in vision-language models that extracts a minimal causal circuit from transformer networks.
• It employs transcoders to convert MLP activations into sparse, nearly monosemantic features using top-k truncation and reconstruction loss minimization.
• The framework integrates attribution graphs and attention-based modules to pinpoint pixel-level visual contributions and validate causal circuits through feature steering and patching interventions.

Searching arXiv for the primary paper and closely related circuit-tracing work. The Circuit-Tracer Framework is an end-to-end methodology for mechanistic interpretability of vision-LLMs (VLMs) that aims to extract a minimal, sparse, human-interpretable “circuit” from the full transformer for a given image-text prompt (Yang et al., 23 Feb 2026). In this formulation, a circuit is a causal subgraph spanning raw pixels, visual tokens, text tokens, sparse internal features, and output logits. The framework is organized around three tightly integrated components: Transcoders, which expose sparse feature spaces inside MLP blocks; Attribution Graphs, which compute exact additive activation contributions under a prompt-specific local linearization; and Attention-Based Modules, which recover spatial semantics for visual features through self-attention analysis and rollout (Yang et al., 23 Feb 2026). The resulting pipeline is intended to reverse-engineer how a VLM hierarchically integrates visual feature embeddings with semantic token representations, and it is validated by feature steering and circuit patching interventions that test whether discovered circuits are causal and controllable (Yang et al., 23 Feb 2026).

1. Conceptual scope and terminology

Within the framework, the central objective is to determine how a VLM integrates visual and semantic information so that, for any image-text prompt, one can isolate a sparse subgraph that explains the model’s prediction (Yang et al., 23 Feb 2026). The method is defined at the level of internal computation rather than solely at the level of neurons, heads, or attention maps. Its unit of analysis is the prompt-specific circuit assembled from active sparse features, input tokens, and output logits.

The framework is presented as the first end-to-end methodology for mechanistic interpretability of vision-LLMs (Yang et al., 23 Feb 2026). Its novelty, as stated in the source description, lies in unifying sparse-feature extraction, exact attribution tracing, and spatial interpretation of visual features in a single multimodal pipeline. This distinguishes it from prior work on text-only circuit tracing, which focused primarily on unimodal LLMs and relied primarily on neuron- or head-level features (Yang et al., 23 Feb 2026).

A recurring source of terminological ambiguity is that the phrase “circuit-tracer” appears in unrelated literatures. In the present sense, the term refers specifically to a VLM interpretability framework built from transcoders, attribution graphs, and attention-based visual analysis (Yang et al., 23 Feb 2026). By contrast, “Circuit-Tracer” in constrained quantum systems denotes a tracer mapping for late-time spin correlations and transport (Feldmeier et al., 2022). This suggests that the label is polysemous across fields and should be interpreted from context.

2. Core architecture of the framework

The framework rests on three components that are described as tightly integrated (Yang et al., 23 Feb 2026). Each addresses a different obstruction to mechanistic analysis in multimodal transformers.

Transcoders are inserted into each MLP block in the transformer decoder. For an MLP sublayer input $x \in \mathbb{R}^{d_{model}}$ and output $y = MLP(x) \in \mathbb{R}^{d_{model}}$, the original mapping is replaced by an encoder-decoder pair:

$$z(x)=ReLU(W_{enc}\cdot x + b_{enc}), \qquad z\in\mathbb{R}^{d_{feat}},$$

$$TC(x)=W_{dec}\cdot z(x) + b_{dec}\in\mathbb{R}^{d_{model}}.$$

The latent dimension satisfies $d_{feat}\gg d_{model}$, and a $\mathrm{TopK}(z(x),k)$ operation zeroes out all but the $k$ largest activations (Yang et al., 23 Feb 2026). The purpose is to lift black-box MLP activations into sparse, approximately monosemantic feature spaces.

Attribution Graphs exploit the fact that, with every nonlinearity frozen at its prompt-specific value, the transcoder-augmented model is locally linear (Yang et al., 23 Feb 2026). Nodes include input image patch tokens, text tokens, active transcoder features, and output logits. For any source node $s$ and target node $t$, the framework defines an attribution

$A_{s\to t}=a_s\cdot w_{s\to t},$

where $y = MLP(x) \in \mathbb{R}^{d_{model}}$0 is the activation magnitude of $y = MLP(x) \in \mathbb{R}^{d_{model}}$1 and $y = MLP(x) \in \mathbb{R}^{d_{model}}$2 is a virtual weight induced by the local Jacobian (Yang et al., 23 Feb 2026). Summing these attributions yields an exact additive graph of causal influence.

Attention-Based Modules address the opacity of visual tokens emitted by the SigLIP vision encoder (Yang et al., 23 Feb 2026). The framework performs an attention-rollout over the last $y = MLP(x) \in \mathbb{R}^{d_{model}}$3 self-attention layers of the vision tower and selects the $y = MLP(x) \in \mathbb{R}^{d_{model}}$4 fraction of heads with lowest entropy to construct spatial heat maps for each feature (Yang et al., 23 Feb 2026). This attaches pixel-space semantics to internal features.

Together these components define a path from raw pixels to text logits and make possible what the paper describes as causal circuit discovery in VLMs (Yang et al., 23 Feb 2026).

3. Transcoders and sparse feature construction

The transcoder is the framework’s mechanism for replacing opaque MLP activity with sparse feature activations that can be inspected and intervened upon (Yang et al., 23 Feb 2026). Rather than adding an explicit $y = MLP(x) \in \mathbb{R}^{d_{model}}$5 sparsity penalty, the method fixes the sparsity level through a top-$y = MLP(x) \in \mathbb{R}^{d_{model}}$6 truncation:

$y = MLP(x) \in \mathbb{R}^{d_{model}}$7

Training minimizes the mean-squared reconstruction loss

$y = MLP(x) \in \mathbb{R}^{d_{model}}$8

with no additional regularizer because $y = MLP(x) \in \mathbb{R}^{d_{model}}$9 is fixed (Yang et al., 23 Feb 2026).

Reconstruction quality is monitored by the Fraction of Variance Unexplained (FVU):

$$z(x)=ReLU(W_{enc}\cdot x + b_{enc}), \qquad z\in\mathbb{R}^{d_{feat}},$$0

The paper specifies a training configuration of 30 K steps, batch size 12, warmup 1 K steps, LR $$z(x)=ReLU(W_{enc}\cdot x + b_{enc}), \qquad z\in\mathbb{R}^{d_{feat}},$$1 (Yang et al., 23 Feb 2026). After convergence, the transcoder replaces the original MLP in a replacement model, and the residual error

$$z(x)=ReLU(W_{enc}\cdot x + b_{enc}), \qquad z\in\mathbb{R}^{d_{feat}},$$2

is cached as an explicit error node in the final circuit graph (Yang et al., 23 Feb 2026).

The interpretive claim attached to this design is that, by retaining only the top-$$z(x)=ReLU(W_{enc}\cdot x + b_{enc}), \qquad z\in\mathbb{R}^{d_{feat}},$$3 latent activations, each feature approximates a monosemantic concept such as “digit-three shape” or “sea-otter appearance” (Yang et al., 23 Feb 2026). A plausible implication is that the framework treats monosemanticity as an operational approximation induced by sparsity and reconstruction fidelity rather than as a guaranteed ontological property of individual features.

4. Attribution graphs and exact causal decomposition

The attribution graph provides the formal mechanism for converting the prompt-conditioned model into a sparse directed graph of additive influences (Yang et al., 23 Feb 2026). Once nonlinearities such as ReLU, attention softmax, and LayerNorm are frozen at their values on the prompt, the model becomes piecewise linear (Yang et al., 23 Feb 2026). The graph $$z(x)=ReLU(W_{enc}\cdot x + b_{enc}), \qquad z\in\mathbb{R}^{d_{feat}},$$4 contains input image patch tokens and text tokens, active transcoder features $$z(x)=ReLU(W_{enc}\cdot x + b_{enc}), \qquad z\in\mathbb{R}^{d_{feat}},$$5 at each layer and position, and output logits for each vocabulary token (Yang et al., 23 Feb 2026).

For any source node $$z(x)=ReLU(W_{enc}\cdot x + b_{enc}), \qquad z\in\mathbb{R}^{d_{feat}},$$6 and target node $$z(x)=ReLU(W_{enc}\cdot x + b_{enc}), \qquad z\in\mathbb{R}^{d_{feat}},$$7, the framework defines

$$z(x)=ReLU(W_{enc}\cdot x + b_{enc}), \qquad z\in\mathbb{R}^{d_{feat}},$$8

where $$z(x)=ReLU(W_{enc}\cdot x + b_{enc}), \qquad z\in\mathbb{R}^{d_{feat}},$$9 are the decoder and encoder vectors and $$TC(x)=W_{dec}\cdot z(x) + b_{dec}\in\mathbb{R}^{d_{model}}.$$0 is the residual-stream Jacobian of the original transformer with fixed nonlinearities (Yang et al., 23 Feb 2026). The edge weight is then

$$TC(x)=W_{dec}\cdot z(x) + b_{dec}\in\mathbb{R}^{d_{model}}.$$1

and exact linearity gives

$$TC(x)=W_{dec}\cdot z(x) + b_{dec}\in\mathbb{R}^{d_{model}}.$$2

Edges with $$TC(x)=W_{dec}\cdot z(x) + b_{dec}\in\mathbb{R}^{d_{model}}.$$3 are pruned, with a typical $$TC(x)=W_{dec}\cdot z(x) + b_{dec}\in\mathbb{R}^{d_{model}}.$$4, to keep the graph sparse (Yang et al., 23 Feb 2026).

This formulation differs from approximate saliency methods because the framework states that the summed attributions form an exact, additive graph of causal influence under the frozen-nonlinearity local linearization (Yang et al., 23 Feb 2026). The significance of that claim is methodological: it supports a graph-based decomposition in which hidden states are not merely ranked by relevance but assembled into a compositional account of the model’s computation.

5. Attention-based tracing of visual features

Because the visual tokens entering the LLM remain opaque without additional processing, the framework includes a dedicated module for recovering their semantics (Yang et al., 23 Feb 2026). It computes an attention-rollout over the SigLIP vision encoder’s last $$TC(x)=W_{dec}\cdot z(x) + b_{dec}\in\mathbb{R}^{d_{model}}.$$5 layers. If $$TC(x)=W_{dec}\cdot z(x) + b_{dec}\in\mathbb{R}^{d_{model}}.$$6 denotes the $$TC(x)=W_{dec}\cdot z(x) + b_{dec}\in\mathbb{R}^{d_{model}}.$$7 attention matrix for head $$TC(x)=W_{dec}\cdot z(x) + b_{dec}\in\mathbb{R}^{d_{model}}.$$8 in layer $$TC(x)=W_{dec}\cdot z(x) + b_{dec}\in\mathbb{R}^{d_{model}}.$$9, the method selects the $d_{feat}\gg d_{model}$0 heads with lowest entropy, averages them to obtain $d_{feat}\gg d_{model}$1, adds identity, row-normalizes to $d_{feat}\gg d_{model}$2, and then multiplies the resulting matrices:

$d_{feat}\gg d_{model}$3

The $d_{feat}\gg d_{model}$4 submatrix corresponding to the visual tokens entering the LLM is then reshaped and upsampled to image resolution to yield a heat map for each feature (Yang et al., 23 Feb 2026).

In this framework, the function of attention rollout is not to explain the whole model directly but to localize the receptive field of a transcoder feature in pixel space (Yang et al., 23 Feb 2026). The resulting visual-semantic link is important because the causal circuit is meant to extend from raw pixels to text logits rather than terminate at uninterpreted vision embeddings.

The source description states that these procedures reveal which pixels drove each transcoder feature (Yang et al., 23 Feb 2026). This suggests that the framework treats visual feature interpretation as a localization problem coupled to sparse feature activations, rather than as a standalone property of attention maps.

6. Circuit discovery pipeline and intervention-based validation

The paper provides a six-stage Circuit Discovery Pipeline (Yang et al., 23 Feb 2026). First, transcoders are trained for each decoder MLP layer. Second, for a prompt $d_{feat}\gg d_{model}$5, the model runs a forward pass, records all active $d_{feat}\gg d_{model}$6, and caches residual error nodes. Third, the attribution graph is constructed in topological order by computing $d_{feat}\gg d_{model}$7 and $d_{feat}\gg d_{model}$8 and retaining edges that exceed the threshold $d_{feat}\gg d_{model}$9 (Yang et al., 23 Feb 2026). Fourth, individual feature nodes are interpreted using top-$\mathrm{TopK}(z(x),k)$0 activating examples from a held-out dataset and, for image-token features, attention-rollout heat maps. Fifth, human-guided circuit collapsing groups features with similar semantics into macro-nodes and sums the corresponding edges. Sixth, intervention experiments alter selected feature groups and compare resulting logits and sampled text (Yang et al., 23 Feb 2026).

The framework is validated by two intervention paradigms.

Feature Steering modifies a feature by setting a target value $\mathrm{TopK}(z(x),k)$1 and applying

$\mathrm{TopK}(z(x),k)$2

$\mathrm{TopK}(z(x),k)$3

The source states that the method systematically zeroes out (“ablate”) or amplifies (“steer”) individual features and measures the change in the model’s prediction distribution (Yang et al., 23 Feb 2026).

Circuit Patching transfers an entire subcircuit from one prompt into another by overwriting $\mathrm{TopK}(z(x),k)$4-values at matched layers and positions (Yang et al., 23 Feb 2026). The paper reports the following quantitative example for a Mars$\mathrm{TopK}(z(x),k)$5Earth transfer (Yang et al., 23 Feb 2026):

Prompt	Original P(correct)	Ablation P(correct)	Patching P(correct)
“This is Mars”	0.92	0.04	0.87
“This is Earth”	0.94	0.03	0.89

The source interprets these results as confirming that the transplanted circuit module preserves its function in a new context, thereby proving causal modularity (Yang et al., 23 Feb 2026). Within the stated methodology, that conclusion is central: circuit identification is not treated as complete until the candidate structure can be manipulated and transferred while preserving its function.

7. Limitations, comparison points, and related multimodal tracing work

The paper lists several limitations of the framework (Yang et al., 23 Feb 2026). Per-layer transcoders cannot disentangle cross-layer superposition, and many near-duplicate features are observed in long visual attention chains. Low-level visual patterns such as textures and colors remain hard to annotate, which slows human-guided circuit collapsing. The computational cost is described as high: building a single attribution graph (~2 minutes/GPU) and computing feature activations (~20 GPU-hours for ~2000 features) (Yang et al., 23 Feb 2026). The method is also stated to have been validated only on Gemma3-4B(it), and adaptation may be required for other VLM architectures such as Flamingo and LLaVA (Yang et al., 23 Feb 2026).

The same source outlines several potential extensions: cross-layer or hierarchical transcoders, automated feature interpretation through contrastive retrieval or caption-based clustering, integration with fine-tuning or prompt-tuning to upgrade circuits rather than merely inspect them, and application to other multimodal architectures such as diffusion-based image-text models (Yang et al., 23 Feb 2026). These are presented as directions rather than completed results.

In comparison with unimodal circuit tracing, the paper emphasizes that VLMs introduce heterogeneous residual streams—image patches versus text tokens—and therefore require attention-rollout modules to recover spatial semantics (Yang et al., 23 Feb 2026). It also states that multimodal circuits exhibit richer associative pathways, exemplified by “Mars$\mathrm{TopK}(z(x),k)$6Space Shuttle” purely from visual co-activation, and that causal interventions in VLMs often reveal interplay between perceptual priors, such as six-finger hallucination, and semantic circuits (Yang et al., 23 Feb 2026).

Related work on multimodal mechanistic interpretability highlights adjacent but distinct emphases. CircuitProbe studies spatiotemporal semantics in LVLMs through a three-circuit decomposition consisting of a visual auditing circuit, semantic tracing circuit, and attention flow circuit, and reports that removing specific object tokens can degrade performance by up to 92.6% (Zhang et al., 25 Jul 2025). The two frameworks are not identical: CircuitProbe centers token ablation, logit-lens decoding, and attention masking, whereas the Circuit-Tracer Framework centers transcoders, exact attribution graphs, and feature-level interventions (Zhang et al., 25 Jul 2025). A plausible implication is that these works occupy complementary positions within multimodal interpretability: one at the level of token-localized semantic emergence, the other at the level of sparse causal feature circuits spanning the full VLM computation.

More broadly, the framework belongs to a family of circuit-tracing approaches applied beyond VLMs. Decoder-only transformers have been analyzed through attribution graphs to study graph reasoning mechanisms such as token merging and structural memorization (Dai et al., 24 Sep 2025), while unrelated uses of similar terminology appear in hybrid quantum-classical tracing for factual recall circuits (Pan, 6 Feb 2026). These parallels indicate that “circuit tracing” has become a general mechanistic lens, but the VLM-specific framework in (Yang et al., 23 Feb 2026) is distinguished by its explicit treatment of multimodal residual streams and its use of attention-based visual localization.