---
title: Circuit-Tracer Framework for VLM Interpretability
url: https://www.emergentmind.com/topics/circuit-tracer-framework
type: topic
---

# Circuit-Tracer Framework for VLM Interpretability

Searching arXiv for the primary paper and closely related circuit-tracing work.
The **Circuit-Tracer Framework** is an end-to-end methodology for mechanistic interpretability of vision-language models (VLMs) that aims to extract a minimal, sparse, human-interpretable “circuit” from the full transformer for a given image-text prompt [2602.20330]. In this formulation, a circuit is a causal subgraph spanning raw pixels, visual tokens, text tokens, sparse internal features, and output logits. The framework is organized around three tightly integrated components: **Transcoders**, which expose sparse feature spaces inside MLP blocks; **Attribution Graphs**, which compute exact additive activation contributions under a prompt-specific local linearization; and **Attention-Based Modules**, which recover spatial semantics for visual features through self-attention analysis and rollout [2602.20330]. The resulting pipeline is intended to reverse-engineer how a VLM hierarchically integrates visual feature embeddings with semantic token representations, and it is validated by feature steering and circuit patching interventions that test whether discovered circuits are causal and controllable [2602.20330].

## 1. Conceptual scope and terminology

Within the framework, the central objective is to determine how a VLM integrates visual and semantic information so that, for any image-text prompt, one can isolate a sparse subgraph that explains the model’s prediction [2602.20330]. The method is defined at the level of internal computation rather than solely at the level of neurons, heads, or attention maps. Its unit of analysis is the prompt-specific circuit assembled from active sparse features, input tokens, and output logits.

The framework is presented as the first end-to-end methodology for mechanistic interpretability of vision-language models [2602.20330]. Its novelty, as stated in the source description, lies in unifying sparse-feature extraction, exact attribution tracing, and spatial interpretation of visual features in a single multimodal pipeline. This distinguishes it from prior work on text-only circuit tracing, which focused primarily on unimodal language models and relied primarily on neuron- or head-level features [2602.20330].

A recurring source of terminological ambiguity is that the phrase “circuit-tracer” appears in unrelated literatures. In the present sense, the term refers specifically to a VLM interpretability framework built from transcoders, attribution graphs, and attention-based visual analysis [2602.20330]. By contrast, “Circuit-Tracer” in constrained quantum systems denotes a tracer mapping for late-time spin correlations and transport [2205.07901]. This suggests that the label is polysemous across fields and should be interpreted from context.

## 2. Core architecture of the framework

The framework rests on three components that are described as tightly integrated [2602.20330]. Each addresses a different obstruction to mechanistic analysis in multimodal transformers.

**Transcoders** are inserted into each MLP block in the transformer decoder. For an MLP sublayer input $x \in \mathbb{R}^{d_{model}}$ and output $y = MLP(x) \in \mathbb{R}^{d_{model}}$, the original mapping is replaced by an encoder-decoder pair:
$$
z(x)=ReLU(W_{enc}\cdot x + b_{enc}), \qquad z\in\mathbb{R}^{d_{feat}},
$$
$$
TC(x)=W_{dec}\cdot z(x) + b_{dec}\in\mathbb{R}^{d_{model}}.
$$
The latent dimension satisfies $d_{feat}\gg d_{model}$, and a $\mathrm{TopK}(z(x),k)$ operation zeroes out all but the $k$ largest activations [2602.20330]. The purpose is to lift black-box MLP activations into sparse, approximately monosemantic feature spaces.

**Attribution Graphs** exploit the fact that, with every nonlinearity frozen at its prompt-specific value, the transcoder-augmented model is locally linear [2602.20330]. Nodes include input image patch tokens, text tokens, active transcoder features, and output logits. For any source node $s$ and target node $t$, the framework defines an attribution
$$
A_{s\to t}=a_s\cdot w_{s\to t},
$$
where $a_s$ is the activation magnitude of $s$ and $w_{s\to t}$ is a virtual weight induced by the local Jacobian [2602.20330]. Summing these attributions yields an exact additive graph of causal influence.

**Attention-Based Modules** address the opacity of visual tokens emitted by the SigLIP vision encoder [2602.20330]. The framework performs an attention-rollout over the last $K$ self-attention layers of the vision tower and selects the $q$ fraction of heads with lowest entropy to construct spatial heat maps for each feature [2602.20330]. This attaches pixel-space semantics to internal features.

Together these components define a path from raw pixels to text logits and make possible what the paper describes as causal circuit discovery in VLMs [2602.20330].

## 3. Transcoders and sparse feature construction

The transcoder is the framework’s mechanism for replacing opaque MLP activity with sparse feature activations that can be inspected and intervened upon [2602.20330]. Rather than adding an explicit $\ell_1$ sparsity penalty, the method fixes the sparsity level through a top-$k$ truncation:
$$
TopK(z(x),k).
$$
Training minimizes the mean-squared reconstruction loss
$$
L_{transcoder}
= \mathbb{E}_{x\sim D}\bigl\|MLP(x)-TC(x)\bigr\|_2^2
$$
with no additional regularizer because $k$ is fixed [2602.20330].

Reconstruction quality is monitored by the **Fraction of Variance Unexplained (FVU)**:
$$
FVU = \frac{\frac1n\sum_i\|y_i-\hat y_i\|^2}{\mathrm{Var}(y)}.
$$
The paper specifies a training configuration of **30 K steps, batch size 12, warmup 1 K steps, LR $\sim 2e^{-4}$** [2602.20330]. After convergence, the transcoder replaces the original MLP in a replacement model, and the residual error
$$
e(x)=MLP(x)-TC(x)
$$
is cached as an explicit error node in the final circuit graph [2602.20330].

The interpretive claim attached to this design is that, by retaining only the top-$k$ latent activations, each feature approximates a monosemantic concept such as “digit-three shape” or “sea-otter appearance” [2602.20330]. A plausible implication is that the framework treats monosemanticity as an operational approximation induced by sparsity and reconstruction fidelity rather than as a guaranteed ontological property of individual features.

## 4. Attribution graphs and exact causal decomposition

The attribution graph provides the formal mechanism for converting the prompt-conditioned model into a sparse directed graph of additive influences [2602.20330]. Once nonlinearities such as ReLU, attention softmax, and LayerNorm are frozen at their values on the prompt, the model becomes piecewise linear [2602.20330]. The graph $G=(V,E)$ contains input image patch tokens and text tokens, active transcoder features $z_{\ell,t,i}>0$ at each layer and position, and output logits for each vocabulary token [2602.20330].

For any source node $s$ and target node $t$, the framework defines
$$
w_{s\to t}=f_{dec}^{(s)\top}J^{\nabla}_{(s)\to(t)}f_{enc}^{(t)},
$$
where $f_{dec}^{(s)}, f_{enc}^{(t)}\in\mathbb{R}^{d_{model}}$ are the decoder and encoder vectors and $J^{\nabla}_{(s)\to(t)}$ is the residual-stream Jacobian of the original transformer with fixed nonlinearities [2602.20330]. The edge weight is then
$$
A_{s\to t}=a_s\,w_{s\to t},
$$
and exact linearity gives
$$
h_t=\sum_{s\in pred(t)}A_{s\to t}.
$$
Edges with $|A_{s\to t}|<\epsilon$ are pruned, with a typical $\epsilon\approx 10^{-4}$, to keep the graph sparse [2602.20330].

This formulation differs from approximate saliency methods because the framework states that the summed attributions form an **exact, additive graph of causal influence** under the frozen-nonlinearity local linearization [2602.20330]. The significance of that claim is methodological: it supports a graph-based decomposition in which hidden states are not merely ranked by relevance but assembled into a compositional account of the model’s computation.

## 5. Attention-based tracing of visual features

Because the visual tokens entering the language model remain opaque without additional processing, the framework includes a dedicated module for recovering their semantics [2602.20330]. It computes an attention-rollout over the SigLIP vision encoder’s last $K$ layers. If $A^{(\ell,h)}$ denotes the $T\times T$ attention matrix for head $h$ in layer $\ell$, the method selects the $q\cdot H$ heads with lowest entropy, averages them to obtain $\bar A^{(\ell)}$, adds identity, row-normalizes to $\tilde U^{(\ell)}$, and then multiplies the resulting matrices:
$$
R = \tilde U^{(\ell_1)}\tilde U^{(\ell_1+1)}\cdots \tilde U^{(\ell_K)}.
$$
The $N_v\times N_v$ submatrix corresponding to the visual tokens entering the language model is then reshaped and upsampled to image resolution to yield a heat map for each feature [2602.20330].

In this framework, the function of attention rollout is not to explain the whole model directly but to localize the receptive field of a transcoder feature in pixel space [2602.20330]. The resulting visual-semantic link is important because the causal circuit is meant to extend from raw pixels to text logits rather than terminate at uninterpreted vision embeddings.

The source description states that these procedures reveal which pixels drove each transcoder feature [2602.20330]. This suggests that the framework treats visual feature interpretation as a localization problem coupled to sparse feature activations, rather than as a standalone property of attention maps.

## 6. Circuit discovery pipeline and intervention-based validation

The paper provides a six-stage **Circuit Discovery Pipeline** [2602.20330]. First, transcoders are trained for each decoder MLP layer. Second, for a prompt $(I,T)$, the model runs a forward pass, records all active $z_{\ell,t,i}>0$, and caches residual error nodes. Third, the attribution graph is constructed in topological order by computing $w_{s\to t}$ and $A_{s\to t}=a_s w_{s\to t}$ and retaining edges that exceed the threshold $\epsilon$ [2602.20330]. Fourth, individual feature nodes are interpreted using top-$k$ activating examples from a held-out dataset and, for image-token features, attention-rollout heat maps. Fifth, human-guided circuit collapsing groups features with similar semantics into macro-nodes and sums the corresponding edges. Sixth, intervention experiments alter selected feature groups and compare resulting logits and sampled text [2602.20330].

The framework is validated by two intervention paradigms.

**Feature Steering** modifies a feature by setting a target value $v_{\ell,t,i}$ and applying
$$
\Delta z_{\ell,t,i}=v_{\ell,t,i}-z_{\ell,t,i}(x),
$$
$$
h_{\ell,t}\leftarrow h_{\ell,t}+\Delta z_{\ell,t,i}\cdot d_{\ell,i}.
$$
The source states that the method systematically zeroes out (“ablate”) or amplifies (“steer”) individual features and measures the change in the model’s prediction distribution [2602.20330].

**Circuit Patching** transfers an entire subcircuit from one prompt into another by overwriting $z$-values at matched layers and positions [2602.20330]. The paper reports the following quantitative example for a **Mars$\to$Earth transfer** [2602.20330]:

| Prompt | Original P(correct) | Ablation P(correct) | Patching P(correct) |
|---|---:|---:|---:|
| “This is Mars” | 0.92 | 0.04 | 0.87 |
| “This is Earth” | 0.94 | 0.03 | 0.89 |

The source interprets these results as confirming that the transplanted circuit module preserves its function in a new context, thereby proving causal modularity [2602.20330]. Within the stated methodology, that conclusion is central: circuit identification is not treated as complete until the candidate structure can be manipulated and transferred while preserving its function.

## 7. Limitations, comparison points, and related multimodal tracing work

The paper lists several limitations of the framework [2602.20330]. Per-layer transcoders cannot disentangle cross-layer superposition, and many near-duplicate features are observed in long visual attention chains. Low-level visual patterns such as textures and colors remain hard to annotate, which slows human-guided circuit collapsing. The computational cost is described as high: **building a single attribution graph (~2 minutes/GPU)** and **computing feature activations (~20 GPU-hours for ~2000 features)** [2602.20330]. The method is also stated to have been validated only on **Gemma3-4B(it)**, and adaptation may be required for other VLM architectures such as Flamingo and LLaVA [2602.20330].

The same source outlines several potential extensions: cross-layer or hierarchical transcoders, automated feature interpretation through contrastive retrieval or caption-based clustering, integration with fine-tuning or prompt-tuning to upgrade circuits rather than merely inspect them, and application to other multimodal architectures such as diffusion-based image-text models [2602.20330]. These are presented as directions rather than completed results.

In comparison with unimodal circuit tracing, the paper emphasizes that VLMs introduce heterogeneous residual streams—image patches versus text tokens—and therefore require attention-rollout modules to recover spatial semantics [2602.20330]. It also states that multimodal circuits exhibit richer associative pathways, exemplified by “Mars$\to$Space Shuttle” purely from visual co-activation, and that causal interventions in VLMs often reveal interplay between perceptual priors, such as six-finger hallucination, and semantic circuits [2602.20330].

Related work on multimodal mechanistic interpretability highlights adjacent but distinct emphases. **CircuitProbe** studies spatiotemporal semantics in LVLMs through a three-circuit decomposition consisting of a visual auditing circuit, semantic tracing circuit, and attention flow circuit, and reports that removing specific object tokens can degrade performance by up to **92.6%** [2507.19420]. The two frameworks are not identical: CircuitProbe centers token ablation, logit-lens decoding, and attention masking, whereas the Circuit-Tracer Framework centers transcoders, exact attribution graphs, and feature-level interventions [2507.19420]. A plausible implication is that these works occupy complementary positions within multimodal interpretability: one at the level of token-localized semantic emergence, the other at the level of sparse causal feature circuits spanning the full VLM computation.

More broadly, the framework belongs to a family of circuit-tracing approaches applied beyond VLMs. Decoder-only transformers have been analyzed through attribution graphs to study graph reasoning mechanisms such as token merging and structural memorization [2509.20336], while unrelated uses of similar terminology appear in hybrid quantum-classical tracing for factual recall circuits [2602.06852]. These parallels indicate that “circuit tracing” has become a general mechanistic lens, but the VLM-specific framework in [2602.20330] is distinguished by its explicit treatment of multimodal residual streams and its use of attention-based visual localization.

Source: https://www.emergentmind.com/topics/circuit-tracer-framework