CLT-Forge: A Scalable Library for Cross-Layer Transcoders and Attribution Graphs

Abstract: Mechanistic interpretability seeks to understand how LLMs represent and process information. Recent approaches based on dictionary learning and transcoders enable representing model computation in terms of sparse, interpretable features and their interactions, giving rise to feature attribution graphs. However, these graphs are often large and redundant, limiting their interpretability in practice. Cross-Layer Transcoders (CLTs) address this issue by sharing features across layers while preserving layer-specific decoding, yielding more compact representations, but remain difficult to train and analyze at scale. We introduce an open-source library for end-to-end training and interpretability of CLTs. Our framework integrates scalable distributed training with model sharding and compressed activation caching, a unified automated interpretability pipeline for feature analysis and explanation, attribution graph computation using Circuit-Tracer, and a flexible visualization interface. This provides a practical and unified solution for scaling CLT-based mechanistic interpretability. Our code is available at: https://github.com/LLM-Interp/CLT-Forge.

• The paper introduces CLT-Forge as a unified framework for scalable cross-layer transcoder training and efficient attribution graph analysis.
• The methodology leverages activation caching, distributed GPU sharding, and low-rank finetuning to reduce computational costs while preserving interpretability.
• Experiments on GPT-2 demonstrate high explained variance (~0.8), strong replacement scores (~0.8), and near-complete attribution graphs (~0.95), confirming its robustness and scalability.

CLT-Forge: Unified Infrastructure for Scalable Cross-Layer Transcoder Training and Attribution Graph Analysis

Background and Motivation

Mechanistic interpretability seeks to expose the internal feature representations and causal pathways within LLMs, fundamentally relying on the hypothesis that semantic information is encoded as linear directions in activation space. Dictionary learning and transcoder-based approaches have enabled the identification of monosemantic, interpretable features and their computation as sparse linear transformations in transformer MLPs, facilitating the construction of attribution graphs. These graphs form nodes from such features and define edges as causal influence pathways, providing structured explanations of the underlying neural mechanisms.

However, classical transcoders lead to large, redundant graphs, primarily because similar features repeatedly emerge across layers. Cross-Layer Transcoders (CLTs) are designed to share features across the network while retaining layer-specific decoding, therefore collapsing redundant nodes and yielding more compact, interpretable graphs. Despite their promise, CLTs have been hindered by significant computational costs, large parameter counts, and the absence of integrated training and interpretability pipelines in open-source research.

Figure 1: Overview of the CLT-Forge framework illustrating its modular architecture for training, interpretability, attribution analysis, and visualization.

Framework Architecture

CLT-Forge is proposed as the first comprehensive and scalable library integrating distributed CLT training, efficient activation caching, automated interpretability analysis, attribution graph computation, and interactive visualization.

Activation Caching utilizes symmetric per-layer quantization and zstd encoding, supporting int8, int4, and int2, which reduces storage demands by up to 12× with minimal reconstruction loss (<3%). Chunks are managed independently, streamlining distributed loading and eliminating memory bottlenecks during large-scale reconstruction tasks.

Distributed Training is accomplished by feature-wise GPU sharding, enabling expansion factors of 48 (∼1.5M features) on LLaMA 1B architectures across 8×80GB GPUs, far outstripping traditional DDP and FSDP in both memory efficiency and scalability. Training follows the compounded objective:

$\mathcal{L} = \sum_{\ell'} \|\hat{m}_{\ell'} - m_{\ell'}\|_2^2 + \lambda_0 \sum_{\ell} \tanh\!\big(C \, (z_\ell \odot \|\mathbf{W}_{\text{dec}^{\ell}\|)\big) + \lambda_1 \sum_{\ell} \mathrm{ReLU}\!\Big(\exp(\tau) - h_\ell^{\text{pre}\Big)\|\mathbf{W}_{\text{dec}^{\ell}\|}$

with multiple sparsity schedulers based on JumpReLU activations. Low-rank finetuning is also incorporated to mitigate computational costs of downstream adaptation.

Automated Interpretability is implemented with single-pass, parallelized pipelines, tracking top-K activating sequences at scale (10M tokens) and producing summary statistics for LLM-based explanations, all written to unified databases facilitating rapid feature querying.

Attribution Graph Computation and Visualization

Feature-level attribution scores are calculated via integration with Circuit-Tracer, mapping CLT parameters and observed activations into sparse graphs. Nodes are formed from monosemantic features, and edges are weighted by linear Jacobian pathways frozen via stop-gradient procedures.

Figure 2: Attribution graph visualization for the prompt "The opposite of 'large' is", with a high ($0.77$) replacement score, demonstrating monosemantic feature connectivity.

The Python-based Dash interface provides interactive exploration capabilities, supporting clustering, interventions, and explanatory prompt construction. It streamlines experimental workflows compared to Neuronpedia, with extensibility for circuit-level analysis.

Numerical Results and Evaluation

Experiments on GPT-2 models report:

• Explained variance $\sim$0.8 at $L_0 = 100$
• Replacement score $\sim$0.8
• Graph completeness $\sim$0.95 on standard interpretability benchmarks

These metrics closely match previously published results, confirming the robustness and reproducibility of the CLT-Forge pipeline (2603.21014). Feature sharding outperforms DDP with respect to memory efficiency, validating the scalability claims. Qualitative analyses demonstrate attribution graphs consistent with known LLM circuit motifs, and multilingual experiments corroborate the generality of the framework.

Figure 3: The evolution of $L_0$ sparsity regularization during CLT training, reflecting the compactness of feature selection as training progresses.

Practical Implications and Theoretical Impact

CLT-Forge substantially lowers the barrier to conducting mechanistic interpretability research at scale, enabling academia and open labs to reconstruct high-capacity attribution graphs previously restricted to proprietary industry tools. This opens extensive possibilities for:

• Automated auditing of emergent LLM behaviors via attribution intervention
• Comparative analysis of feature circuits across languages and tasks
• Rapid prototyping of new architectures (e.g., more memory-efficient CLTs)
• Systematic benchmarking of interpretability workflows

The unified design directly aligns with ongoing research on scalable dictionary learning, circuit tracing, and feature-based intervention ([ameisen2025circuit], [lindsey2025biology], [dunefsky2024transcoders]). Future theoretical advances will likely target more efficient objective optimization (e.g., directly targeting replacement score stability), improved attribution for attention, and alternative architectures for CLT parameter reduction.

Limitations and Future Directions

The paper notes several open challenges: attention attribution is currently omitted within attribution graph analysis, suggesting future work to integrate attention circuit tracing [kamath2025tracing]. CLT parameter counts remain large, motivating research into parameter-efficient or compressed architectures. Optimization for replacement score is currently unstable, indicating a need for more robust objectives.

Conclusion

CLT-Forge provides a scalable solution for end-to-end CLT training and interpretability analysis, integrating distributed computation, automated feature extraction, attribution graph computation, and intuitive visualization. Its modular design supports rapid iteration and reproducible mechanistic interpretability, facilitating widespread adoption and future innovation in feature circuit analysis. Further development should prioritize architectural efficiency and attribution scope expansion, cementing CLT-Forge as a foundation for large-scale mechanistic interpretability in LLMs.