From Insight to Action: A Novel Framework for Interpretability-Guided Data Selection in Large Language Models

Abstract: While mechanistic interpretability tools like Sparse Autoencoders (SAEs) can uncover meaningful features within LLMs, a critical gap remains in transforming these insights into practical actions for model optimization. We bridge this gap with the hypothesis that data selection guided by a model's internal task features is a effective training strategy. Inspired by this, we propose Interpretability-Guided Data Selection (IGDS), a framework that first identifies these causal task features through frequency recall and interventional filtering, then selects ``Feature-Resonant Data'' that maximally activates task features for fine-tuning. We validate IGDS on mathematical reasoning, summarization, and translation tasks within Gemma-2, LLaMA-3.1, and Qwen3 models. Our experiments demonstrate exceptional data efficiency: on the Math task, IGDS surpasses full-dataset fine-tuning by a remarkable 17.4% on Gemma-2-2B while using only 50% of the data, and outperforms established baselines focused on data quality and diversity. Analysis confirms a strong positive correlation between feature amplification and task performance improvement. IGDS thus provides a direct and effective framework to enhance LLMs by leveraging their internal mechanisms, validating our core hypothesis.

• The paper introduces IGDS, a novel framework that leverages mechanistic interpretability to drive targeted data selection for LLM optimization.
• It demonstrates that using causally-validated feature activations can yield significant performance gains, such as a +17.4% accuracy improvement with only half the training data.
• The framework is validated across diverse tasks including mathematical reasoning, translation, and summarization, offering a scalable and cost-effective approach to fine-tuning LLMs.

Interpretability-Guided Data Selection for LLM Optimization

Introduction

"From Insight to Action: A Novel Framework for Interpretability-Guided Data Selection in LLMs" (2604.25167) introduces Interpretability-Guided Data Selection (IGDS), a data-centric pipeline that operationalizes mechanistic interpretability to enhance supervised fine-tuning of LLMs. The central premise is to move beyond passive analysis of interpretable features to an actionable framework that uses these features for targeted data selection, yielding superior data efficiency and, in key cases, surpassing full-dataset fine-tuning. This essay critically reviews the IGDS paradigm, experimental methodology, empirical findings, and implications for model interpretability and optimization.

IGDS: Closing the Interpretability–Optimization Loop

Conventional mechanistic interpretability, primarily leveraging Sparse Autoencoders (SAEs), has produced a taxonomy of neurons or features with human-interpretable roles. Application of this knowledge for practical training regime improvements has remained largely unexplored. IGDS systematically closes this gap by introducing a two-stage loop: task feature identification and feature-resonant data selection.

Figure 1: Conceptual illustration of the IGDS paradigm. The diagram depicts the closed loop from internal insight to optimization action, showing how model features are leveraged to guide data selection.

The IGDS pipeline is depicted in a closed-loop configuration (Figure 1), where interpretability insights causally drive data selection, creating a feedback cycle between internal model analysis and external intervention.

Figure 2: An overview of the Interpretability-Guided Data Selection (IGDS) framework.

Methodologically, IGDS comprises:

1. Task Feature Identification: Extraction of a candidate set of high-frequency features using SAEs, followed by causal filtering via targeted activation interventions to establish a subset with reliable performance impact on the downstream task.
2. Feature-Resonant Data Scoring: Computation of a Feature-Resonant Score (FRS) per datum as the aggregate activation of validated task features at crucial prompt locations; this scoring ranks a large pool for efficient selection.

Empirical Evaluation and Data Efficiency

IGDS was evaluated on mathematical reasoning, summarization, and translation tasks using Gemma-2, LLaMA-3.1, and Qwen3-8B families. Task feature identification displays strong selectivity: only a minuscule fraction of the total feature space is recalled as both correlated and causally instrumental. Amplification of such features yields substantial isolated performance improvements (+12 on Math for Gemma-2-2B; +8.34 in Translation for Qwen3-8B).

A pronounced positive monotonic correlation exists between the degree of activation of task-validated features and downstream accuracy, as confirmed via activation analysis post-fine-tuning.

Figure 3: Correlation between feature activation and task performance on the Math task for Gemma-2-2B.

Notably, IGDS achieves a +17.4% accuracy improvement over full-data supervised fine-tuning for Gemma-2-2B on Math using only 50% of the training data, and consistently outperforms all baselines (Random, Loss, IFD, ZIP) in all main tasks. This result violates the prevailing hypothesis that data selection at scale yields negligible gains and that random selection is sufficient for most large models.

IGDS demonstrates robust superiority across a spectrum of data sampling budgets:

Figure 4: Performance comparison of different data selection strategies under varying sampling rates (20%, 50%, 80%) with Gemma-2-2B model.

Analysis of Feature Structure and Task Localization

The spatial analysis of validated features highlights that distinct tasks exhibit sharply different feature topologies. Mathematical reasoning features in Gemma-2-2B are globally distributed across nearly all layers, indicating full-stack utilization, whereas summarization and translation features are concentrated in middle to late transformer layers.

Figure 5: Distribution of positive features across layers and positions for Gemma-2-2B model. Each point represents a feature, plotted by its layer (x-axis) and position (y-axis, log scale).

Visualization of the fine-grained topology for Math tasks reveals that feature activations are not locked to narrow structural intervals but are distributed over the entire network depth, implying distributed credit assignment and robust compositionality.

Figure 6: Topological signature of features for Math Task, which exhibits a globally distributed topology. Features are activated across the entire depth (L0-L25).

Translation features, in contrast, display a more localized and sparse activation pattern, primarily in mid-to-deep layers, aligning with the requirements for context management and target language generation.

Figure 7: Topological signature of features for the Translation Task, with localized feature activation in the mid-to-late layers.

Ablation and Robustness

Ablation studies establish that both the frequency-based recalling step and subsequent causal intervention filtering are essential; omission of either degrades downstream accuracy. Data efficiency and impact are also sensitive to the focus of the feature set: scoring with a small number of top-ranked features yields the highest utility.

IGDS maintains competitive computational runtime. Data selection using IGDS adds minimal overhead compared to widely-used perplexity- or loss-based selection, as the SAE feature activations can be extracted within the main forward pass, making the pipeline scalable for large data pools.

Implications and Future Directions

IGDS sets a formal, reproducible precedent for leveraging mechanistic interpretability as an actionable tool for data-centric optimization. By proving that causally-validated internal features can serve as highly informative utility functions for data selection, IGDS motivates renewed research in (1) scaling the granularity and reliability of SAEs, (2) extending the method to more complex multi-task and multi-lingual settings, and (3) integrating feature-based signals into alternative optimization schemes (e.g., RLHF, preference modeling).

Given the dependency on SAE quality and coverage, especially for newer model families, community efforts in public SAE pretraining and expansion will serve as a practical bottleneck. Further, as models integrate broader modalities and grow in complexity, feature grounding and abstraction hierarchies will become critical for robust causality-driven data selection.

Conclusion

Interpretability-Guided Data Selection establishes that model-intrinsic causal features are a uniquely potent axis for efficient and effective data curation in LLM supervised fine-tuning. The framework achieves significant performance gains with data pruning, delivers strong robustness across models and tasks, and demonstrates that mechanistic interpretability can transition from post hoc analysis to systematic, large-scale model improvement (2604.25167). This paradigm offers a new path for research integrating model internals and optimization strategy, with clear ramifications for data-centric AI and transparent LLM improvement.