Domain-Filtered Knowledge Graphs from Sparse Autoencoder Features

Abstract: Sparse autoencoders (SAEs) extract millions of interpretable features from a LLM, but flat feature inventories aren't very useful on their own. Domain concepts get mixed with generic and weakly grounded features, while related ideas are scattered across many units, and there's no way to understand relationships between features. We address this by first constructing a strict domain-specific concept universe from a large SAE inventory using contrastive activations and a multi-stage filtering process. Next, we build two aligned graph views on the filtered set: a co-occurrence graph for corpus-level conceptual structure, organized at multiple levels of granularity, and a transcoder-based mechanism graph that links source-layer and target-layer features through sparse latent pathways. Automated edge labeling then turns these graph views into readable knowledge graphs rather than unlabeled layouts. In a case study on a biology textbook, these graphs recover coherent chapter and subchapter-level structure, reveal concepts that bridge neighboring topics, and transform messy sentence-level activity containing thousands of features into compact, readable views that illustrate the model's local activity. Taken together, this reframes a flat SAE inventory as an internal knowledge graph that converts feature-level interpretability into a global map of model knowledge and enables audits of reasoning faithfulness.

• The paper introduces a multi-stage contrastive filtering pipeline to extract domain-specific features from LLM sparse autoencoders.
• It constructs structured co-occurrence and mechanism graphs using Jaccard normalization and k-NN based abstraction to reveal latent conceptual hierarchies.
• Empirical results on a biology textbook corpus validate the method’s ability to recover interpretable, activation-grounded domain structures for audit and analysis.

Domain-Filtered Knowledge Graphs from Sparse Autoencoder Features: A Comprehensive Review

Introduction and Motivation

Sparse autoencoders (SAEs) have enabled extraction of large-scale interpretable feature inventories from LLMs, but these inventories inherently lack structured organization. The raw SAE features include domain-specific, generic, and weakly grounded activations intermingled, obscuring both the organization of domain-relevant concepts and their functional relationships. The paper "Domain-Filtered Knowledge Graphs from Sparse Autoencoder Features" (2604.23829) systematically addresses this shortcoming by establishing a pipeline that filters for domain-specific concepts and constructs two aligned graph-based knowledge representations: a co-occurrence graph and a mechanism (transcoder) graph. This reframes a flat feature inventory into a structured, multiscale knowledge graph, exposing both conceptual organization and interpretable latent mechanisms within LLM activations.

Multi-Stage Contrastive Domain Filtering

SAE-derived features from LLMs, even when extracted using large public inventories such as Gemma Scope and Llama Scope, are dominated by generic, punctuation, or spurious signals. The authors implement a multi-stage, contrastive filtering procedure:

1. Contrastive Shortlisting: Features are selected based on support, enrichment, and localization scores compared to broad contrastive corpora, removing generic activations while preserving domain recall.
2. Packet-Level Adjudication: Each feature's evidence is reviewed, consulting representative contexts and contrast windows, to ensure visibility, distinctiveness, and regional appropriateness.

The result is a strict, domain-specific universe $\mathcal{V}^{\star}$, which is used as the basis for downstream structural modeling.

Knowledge Graph Construction and Organization

Co-Occurrence Graphs Across Granularity Levels

A multi-granular co-occurrence graph summarizes statistical co-activation relationships across sentences, paragraphs, subchapters, and chapters. Especially notable is that presence is defined at the sentence level, and higher-level units are constructed deterministically. Jaccard normalization is used to symmetrize and sparsify the graph, exposing stable conceptual clusters, bridges, and motifs specific to the domain corpus.

Figure 1: Shared-coordinate activation maps recover textbook structure at multiple scales, revealing distinct chapter-level basins and fine-grained, subchapter-specific activation modes.

Construction of the Abstraction Hierarchy

To handle redundancy and density, the authors propose an abstraction tree—a hierarchy built by recursively partitioning feature-description embeddings via $k$-NN graphs, with semantic summarization at each internal node. This structure supports both interpretability and aggressive compression of dense sentence-level graphs.

Figure 2: Construction of the abstraction tree by recursively splitting mutual-$k$NN graphs of feature-description embeddings and summarizing internal nodes with representative and anchor features.

Mechanism Graphs via Interpretable Transcoders

The mechanism graph links source and target SAE features across layers, using a transcoder trained between the corresponding residual streams. Each directed feature edge reflects the dynamic activation of sparse latent pathways conditioned on sentence input. The pipeline overlays source and target SAE dictionaries, enabling mechanistic pathways to be expressed directly in terms of interpretable features rather than opaque residual directions.

Figure 3: Transcoder mechanisms visualized through sparse SAE dictionaries, showing mappings from source to target features via interpretable, latent-mediated mechanisms, both in static profiles and dynamic, sentence-conditioned execution.

The static mechanism library defines latent-to-feature alignments, while the input-conditioned dynamic mechanism graph is derived by integrating activity, functional support, and mutual presence. Hierarchy-aware compression is performed to collapse coherent regions into supernodes unless blocked by active edges, resulting in highly readable, information-preserving graphs for human audit.

Figure 4: Hierarchy-respecting compression reduces dense mechanism graphs to succinct representations, aggregating edges between supernodes and preserving critical internal connections.

Edge-Labeled Knowledge Graphs

To provision readable semantics to the graph, the authors design an automated edge-labeling (auto-relate) pipeline:

• Co-Occurrence Edges: Labeled with evidence-based relation phrases via joint and contrastive context analysis, elucidating statistical conceptual relationships.
• Mechanism Edges: Labeled with directional relation phrases based on strong latent and sentence-level evidence, grounding cross-layer interactions in observable computation.

  Figure 5: Edge-labeled knowledge-graph views, including co-occurrence bridge corridors and sentence-conditioned, directed mechanism graphs with relation-labeled edges.

Experimental Validation

Corpus and Sparse Stack Selection

A biology textbook corpus (target) is used, with contrast corpora from world history and physical geology ensuring specificity. SAEs (width 65k) are extracted from Gemma Scope residuals at adjacent transformer layers, with a transcoder (width 16k) mapping between those layers.

Evaluation

The strict domain filter and resulting graphs are assessed on:

• Global Structure Recovery: Sentence-level and aggregated co-occurrence graphs are tested for alignment with human-authored chapter and subchapter segmentation. Mutual information-based alignment and intra/inter-chapter distances quantify structure preservation.
• Local Mechanism Readability: Sentence-conditioned graphs efficiently compress thousands of active features and their mechanistic flows into tractable subgraphs, retaining interpretability and relevance.

  Figure 6: Capability progression from diffuse sentence-level feature activity, to a concise co-occurrence graph, to a compressed transcoder-based mechanism view.

Strong empirical results: The retained universe and induced graph consistently recover distinct chapter and subchapter-level basins, bridge concepts, and support inspection-ready explanations for sentence-level reasoning in practical, highly compressed forms.

Implications

Theoretical

• The organization of SAE features into domain-filtered, edge-labeled knowledge graphs demonstrates that the internal representation in LLMs exhibits interpretable and structured conceptual topologies not accessible from feature lists or unstructured dictionaries.
• This approach suggests that internal, activation-derived knowledge graphs can serve as more faithful or granular explanations for model reasoning than surface-level chain-of-thought or attention-based rationales, though empirical faithfulness studies remain for future work.

Practical

• The proposed method enables systematic audits of LLMs, facilitating both high-level corpus analysis and fine-grained, input-conditioned reasoning inspections.
• The open-source browser artifact expands utility for interactive exploration of large-scale SAE knowledge graphs, supporting research into model behavior, domain transfer, and interpretability toolchains.

Future Directions

Extensions include the integration of attribution-based pathways (e.g., attention mediation or recursive graph tracing) to supplement projected transcoder edges, as well as scaling to multi-layer or whole-network path tracing. Comparative studies of activation-derived graphs versus external, language-model-generated rationales will elucidate the faithfulness gap in model explanations.

Conclusion

"Domain-Filtered Knowledge Graphs from Sparse Autoencoder Features" (2604.23829) provides a rigorous framework for organizing SAE-derived feature inventories in LLMs. By combining strict contrastive filtering, hierarchy-based semantic grouping, multi-granular co-occurrence and mechanism graph construction, and automated edge semantic labeling, the pipeline yields readable, activation-grounded internal knowledge graphs. These graphs enable new audits of model conceptual organization, interpretability, and reasoning faithfulness, setting a foundation for future mechanistic and explanatory research in LLMs.