Fantastic Reasoning Behaviors and Where to Find Them: Unsupervised Discovery of the Reasoning Process

Abstract: Despite the growing reasoning capabilities of recent LLMs, their internal mechanisms during the reasoning process remain underexplored. Prior approaches often rely on human-defined concepts (e.g., overthinking, reflection) at the word level to analyze reasoning in a supervised manner. However, such methods are limited, as it is infeasible to capture the full spectrum of potential reasoning behaviors, many of which are difficult to define in token space. In this work, we propose an unsupervised framework (namely, RISE: Reasoning behavior Interpretability via Sparse auto-Encoder) for discovering reasoning vectors, which we define as directions in the activation space that encode distinct reasoning behaviors. By segmenting chain-of-thought traces into sentence-level 'steps' and training sparse auto-encoders (SAEs) on step-level activations, we uncover disentangled features corresponding to interpretable behaviors such as reflection and backtracking. Visualization and clustering analyses show that these behaviors occupy separable regions in the decoder column space. Moreover, targeted interventions on SAE-derived vectors can controllably amplify or suppress specific reasoning behaviors, altering inference trajectories without retraining. Beyond behavior-specific disentanglement, SAEs capture structural properties such as response length, revealing clusters of long versus short reasoning traces. More interestingly, SAEs enable the discovery of novel behaviors beyond human supervision. We demonstrate the ability to control response confidence by identifying confidence-related vectors in the SAE decoder space. These findings underscore the potential of unsupervised latent discovery for both interpreting and controllably steering reasoning in LLMs.

• The paper introduces RISE, a sparse autoencoder framework that unsupervisedly discovers latent reasoning behaviors in large language models and enables targeted intervention.
• The methodology segments chain-of-thought activations and employs sparsity-constrained reconstruction to map distinct reasoning strategies like reflection and backtracking.
• Intervention experiments demonstrate that manipulating decoder vectors can steer behavior, improving accuracy by up to 4.66 percentage points and reducing token cost by 13.69%.

Unsupervised Discovery of Reasoning Behaviors in LLMs

Introduction

This paper introduces RISE (Reasoning behavior Interpretability via Sparse auto-Encoder), an unsupervised mechanistic interpretability framework which identifies and steers distinct reasoning behaviors within LLMs by applying Sparse Autoencoders (SAEs) to intermediate activations in chain-of-thought (CoT) traces. Moving beyond human-labeled, token-level concepts, RISE uncovers latent behavioral vectors that represent complex reasoning strategies such as reflection and backtracking, as well as more abstract or difficult-to-label cognitive properties including confidence and inference length. These vectors are shown to be interpretable, highly disentangled, and causally manipulable at inference—enabling precise control over the LLM’s reasoning trajectory without retraining.

Figure 1: The RISE pipeline: training an SAE on step-level activations and intervening in the original model using decoder vectors to amplify or suppress specific reasoning behaviors.

Methodology: Sparse Autoencoder for Unsupervised Reasoning Mining

The methodology builds on the Linear Representation Hypothesis, positing that cognitive behaviors are encoded as directions in the residual activation space. RISE partitions model outputs into step-level representations—obtained at predefined delimiters in CoT reasoning traces—and trains an SAE to reconstruct these while enforcing sparsity in latent activations. Each decoder column in the SAE thus aligns with an atomic, interpretable behavioral vector.

Formally, for model activations $h \in \mathbb{R}^d$ at each step, the SAE reconstructs them via

$$\hat{h} = W_\mathrm{decoder}^\top \, \sigma(z) + b_\mathrm{decoder}$$

with latent codes $$z = \sigma(W_\mathrm{encoder}^\top h + b_\mathrm{encoder})$$ and an objective combining reconstruction loss and $\ell_0$ sparsity on $z$. Theoretical analysis (Theorem 1 in the paper) establishes that, under standard incoherence and sparsity assumptions, the SAE recovers a dictionary of reasoning directions (up to scaling and permutation) in the decoder space.

The empirical pipeline encompasses: (1) extraction and segmentation of reasoning step activations per sample, (2) SAE training at a fixed layer with relatively low dimensionality to target behavior-level (rather than surface linguistic) structure, and (3) post hoc analysis and intervention using decoder vectors.

Emergence and Visualization of Disentangled Reasoning Behaviors

A core result is the empirical validation that distinct reasoning behaviors are mapped onto geometrically distinct clusters in the SAE decoder column space. Human-labeled step categories—reflection, backtracking, and other—were mapped back to their latent activations, with clear spatial separation observed under UMAP visualization.

Figure 2: Decoder columns projected to two dimensions; columns associated with human-defined behaviors form concentrated clusters, evidencing alignment between SAE structure and reasoning categories.

Quantitative analysis using normalized Silhouette scores demonstrates increasing separation of behavioral subspaces toward model mid-to-late layers, peaking before the last few layers where oversmoothing is more pronounced. Notably, reflection and backtracking occupy overlapping but non-identical subspaces, sharply distinct from the residual “other” behaviors.

Figure 3: Normalized Silhouette scores across model layers reveal the layerwise evolution of behavioral subspace separation.

Causal Intervention and Steering of Reasoning Trajectories

The paper demonstrates strong evidence that interventions along SAE-derived directions have predictable, robust causal effects on the LLM’s reasoning process. By injecting the average decoder vector of reflection-labeled columns at each step (normalized), the authors are able to amplify or suppress reflection in the model's reasoning trace while maintaining answer correctness. This process is sample- and test-time agnostic: the steering operation is applied with fixed vectors and requires no further fine-tuning.

Figure 4: Shifts in reflection/backtracking behavior frequency across tasks and models due to SAE interventions.

Figure 5: Model outputs with positive and negative reflection intervention; steps associated with reflection are highlighted.

Memory activation manipulation along the identified axes generalizes to different tasks (e.g., from mathematics to commonsense and logic reasoning) and across models, such as R1-1.5B and Qwen3-8B. Steered behaviors follows predictable monotonic trends with intervention magnitude: for reflection on the AIME25 benchmark, for instance, the number of reflection steps shifted from 58.6 to 166.9 as the intervention scalar varied from $-1.5$ (suppression) to $+1.5$ (amplification).

Discovery of Emergent Cognitive Behaviors: Confidence and Length

RISE further reveals that abstract reasoning properties, notably confidence and response length, are also encoded as distinct, interpretable directions.

To identify “confidence vectors,” the authors optimize for decoder columns associated with reduced output entropy across training samples. These vectors localize in the SAE space and, when injected, produce responses with (i) fewer metacognitive tokens and phrases (e.g., fewer “wait”, less backtracking), and (ii) a notable reduction in reflective/backtracking reasoning steps (e.g., on AIME25, reflection steps drop from 90.53 to 33.77 after intervention, and backtracking from 35.50 to 5.93).

Figure 6: Confidence-related decoder columns cluster distinctly; interventions alter frequent step-initial tokens, replacing metacognitive cues with confident calculation sequences.

Length-related behaviors are similarly captured: columns corresponding to long- and short-response clusters emerge, especially in the later layers.

Figure 7: SAE decoder columns (highlighted) associated with short (green) and long (yellow) responses.

RISE for Steering Reasoning Behavior and Efficiency

The practical utility of RISE is highlighted via experiments on accuracy and token efficiency by combining confidence vectors at inference in a sample-specific manner. Compared to previous steering techniques (e.g., SEAL, TIP), RISE-based steering achieves up to 4.66 percentage points improvement in accuracy and 13.69% reduction in token cost on the evaluation set.

Figure 8: Accuracy and token cost improvements under various steering methods.

Theoretical and Practical Implications

This work establishes that LLM cognitive behaviors—previously accessible only via coarse human annotation or surface artifact analysis—can be unsupervisedly discovered and precisely manipulated via highly structured, sparse subspaces in the activation geometry. It provides both rigorous theoretical underpinning and extensive empirical validation that high-level reasoning styles are encoded linearly, extend to novel domains, and can enable dynamic test-time adaptation without model retraining.

Practically, this enables LLMs to be fine-tuned at inference to match desired human behavior or application requirements (conciseness, deliberation, confidence), advancing transparency, safety, and resource efficiency. Conceptually, it suggests a unifying mechanistic basis for cognitive phenomena in large-scale sequence models, with clear directions for improved explainability and control.

Future work could expand on automating latent space exploration for domain-specific behaviors, formalizing connections to neuroscience models of cognition, or developing user-facing tools for interactive CoT manipulation.

Conclusion

The RISE framework represents a robust and scalable paradigm for unsupervised discovery and control of model-internal reasoning behaviors. By leveraging sparse autoencoders to uncover and steer atomic reasoning directions, RISE advances both the mechanistic interpretability and operational controllability of LLMs. These findings open new avenues for aligning reasoning style, enhancing efficiency, and developing explainable AI systems with fine-grained and dynamic behavior adaptation.