I Have Covered All the Bases Here: Interpreting Reasoning Features in Large Language Models via Sparse Autoencoders (2503.18878v1)

Abstract: LLMs have achieved remarkable success in natural language processing. Recent advances have led to the developing of a new class of reasoning LLMs; for example, open-source DeepSeek-R1 has achieved state-of-the-art performance by integrating deep thinking and complex reasoning. Despite these impressive capabilities, the internal reasoning mechanisms of such models remain unexplored. In this work, we employ Sparse Autoencoders (SAEs), a method to learn a sparse decomposition of latent representations of a neural network into interpretable features, to identify features that drive reasoning in the DeepSeek-R1 series of models. First, we propose an approach to extract candidate ''reasoning features'' from SAE representations. We validate these features through empirical analysis and interpretability methods, demonstrating their direct correlation with the model's reasoning abilities. Crucially, we demonstrate that steering these features systematically enhances reasoning performance, offering the first mechanistic account of reasoning in LLMs. Code available at https://github.com/AIRI-Institute/SAE-Reasoning

• The paper presents a method using sparse autoencoders to identify and validate specific internal features within large language models that are responsible for reasoning.
• By actively manipulating these identified reasoning features, the authors demonstrate that performance on reasoning tasks is consistently improved, providing evidence of their function.
• Identifying these reasoning features offers a step towards understanding how LLMs reason internally, potentially enabling the creation of more reliable, controllable, and interpretable AI systems.

Here's a summary of the paper "I Have Covered All the Bases Here: Interpreting Reasoning Features in LLMs via Sparse Autoencoders" (2503.18878):

Rationale

• Problem: While LLMs like DeepSeek-R1 show impressive reasoning abilities, how they perform this reasoning internally is largely unknown. Understanding these mechanisms is crucial for improving model performance, reliability, and safety.
• Goal: To identify and understand the specific internal components ("features") within the DeepSeek-R1 model that are responsible for its reasoning capabilities.

Method and Data

• Technique: The researchers used Sparse Autoencoders (SAEs). SAEs are trained to break down the complex internal representations (activations) of an LLM into simpler, more interpretable "features".
• Model Analyzed: DeepSeek-R1 series models, known for strong reasoning performance.
• Data: The specific datasets used to train the SAEs and evaluate the reasoning features are not detailed in the abstract but likely involved text requiring logical deduction or problem-solving.

Approach

• Feature Extraction: They developed a method to identify potential "reasoning features" from the features learned by the SAEs.
• Validation: The importance of these candidate features for reasoning was confirmed through analysis and standard interpretability techniques.
• Intervention: Crucially, they showed that by actively manipulating or "steering" these identified reasoning features, they could consistently improve the model's performance on reasoning tasks.

Key Findings

• It's possible to isolate specific, interpretable features within an LLM that are directly linked to its reasoning abilities.
• Activating these "reasoning features" enhances the model's reasoning performance, providing direct evidence of their function.
• This work offers one of the first mechanistic explanations for how reasoning occurs inside advanced LLMs.

Implications and Applications

• Improved Models: Understanding reasoning mechanisms can lead to designing more capable and efficient LLMs specifically for tasks requiring complex thought.
• Reliability & Safety: Identifying how models reason can help detect flaws or biases in their reasoning processes, leading to safer AI.
• Model Control: The ability to "steer" reasoning features opens possibilities for controlling and guiding LLM behavior during complex tasks.
• Interpretability: Provides tools and methods to better understand the internal workings of otherwise "black box" AI models.

In conclusion, this research takes a significant step towards demystifying how advanced LLMs perform reasoning. By using sparse autoencoders, the authors successfully identified and validated specific internal features responsible for reasoning, even demonstrating that manipulating these features can enhance the model's capabilities. This opens avenues for building more understandable, controllable, and effective reasoning AI systems.