Reasoning Models Don't Just Think Longer, They Move Differently

Abstract: Reasoning-trained LLMs often spend more tokens on harder problems, but longer chains of thought do not show whether a model is merely computing for more steps or following a different internal trajectory. We study this distinction through hidden-state trajectories during chain-of-thought generation across competitive programming, mathematics, and Boolean satisfiability. Raw trajectory geometry is strongly shaped by generation length: longer generations mechanically alter path statistics, so difficulty-dependent comparisons are misleading without adjustment. After residualizing trajectory statistics on length, difficulty remains systematically coupled to corrected trajectory geometry across all domains studied. The clearest reasoning-specific separation appears in the code domain, where harder problems show more direct corrected trajectories and less heterogeneous local curvature in reasoning-trained models than in matched instruction-tuned baselines. Corrected difficulty-geometry coupling is weaker, but still present, in mathematics and Boolean satisfiability. Prompt-stage linear probes do not mirror the code-domain separation, and behavioral annotations show that stronger corrected coupling co-occurs with strategy shifts and uncertainty monitoring. Together, these findings establish length correction as a prerequisite for generation-time trajectory analysis and show that reasoning training can be associated with distinct corrected trajectory geometry, with the strength of the effect depending on the domain.

• The paper demonstrates that reasoning-trained models show significantly more direct hidden-state trajectories on harder problems, with a corrected median shift from -0.73 to +0.41 in code tasks.
• It employs a rigorous matched-pair design with continuous difficulty calibration via Rasch IRT to isolate genuine computational dynamics from length-related confounds.
• Findings indicate that trajectory geometry encodes adaptive behaviors, such as dynamic strategy shifting and uncertainty monitoring, not captured by linear probes.

Distinguishing Trajectory Geometry in Reasoning-Trained LLMs

Introduction

This work rigorously interrogates whether reasoning-trained LLMs merely extend the duration of computation for harder problems, or whether they traverse fundamentally different activation-space trajectories during chain-of-thought (CoT) generation. The critical claim is that while response length monotonically increases with difficulty, the geometry of the trajectory in the hidden state space carries additional, functionally meaningful distinctions, contingent on length-correction. The study systematically analyzes hidden-state trajectories across competitive programming, mathematics, and Boolean satisfiability (SAT), using IRT-calibrated problem difficulties and cross-model matched pairs to disentangle the mechanics of test-time compute allocation from genuine changes in latent computational dynamics.

Figure 1: Hidden-state trajectory geometry during chain-of-thought generation. Trajectories are extracted for matched reasoning and non-reasoning models; raw geometry is dominated by length, while length-corrected correlations reveal separation between reasoning and baseline models.

Methodology: Matched-Pair Design and Length Correction

The experimental setup incorporates three domains—code, math, SAT—with 500 items each, continuous difficulty calibration via Rasch (1PL) IRT models, and 32-model calibration pools validated against external labels (Glicko-2 ratings, MATH tiers, SAT clause counts). For trajectory extraction, hidden activations are sampled at five evenly spaced layers and stride 10. Directness (endpoint efficiency) and curvature variability (heterogeneity of local turning) quantify primary geometric properties of trajectories.

Crucially, raw geometry metrics are severely confounded by generation length, as longer sequences are mechanically less direct—a phenomenon well-documented in geometric statistics—but neglected in LLM trajectory literature. The analysis thus residualizes trajectory statistics (e.g., directness $D$) against $\log N$ (where $N$ is token length), yielding difficulty-geometry coupling coefficients such as $\rho_\perp^D = \rho_S(b_i, D_{\perp,im})$, where $b_i$ is IRT-calibrated difficulty and $D_{\perp,im}$ is the length-residualized directness.

Figure 2: Length correction reveals a sign reversal across all three domains. Raw correlations with difficulty are negative; after length correction, reasoning models (especially in code) show positive coupling of difficulty and directness.

The matched model design controls for family, parameter count, and instruction tuning, varying only the presence or absence of explicit reasoning-oriented training (e.g., R1 distillation or SFT+RL). This structure enables robust attribution of geometric effects to reasoning training rather than other architectural or optimization confounders.

Main Results: Domain-Dependent Effects and Geometric-Emergent Differences

The central empirical result is as follows: after controlling for length, harder problems systematically elicit more direct hidden-state trajectories in reasoning-trained models, with the effect manifesting most strongly in competitive programming (Codeforces), more weakly in math, and present but less reasoning-specific in SAT.

• In the code domain, the transition from raw to length-corrected directness-difficulty correlation is stark: reasoning models’ median $\rho_\perp^D$ moves from $-0.73$ (raw) to $+0.41$ (corrected), while matched baselines remain roughly centered at zero or slightly negative.
• For math and SAT, the length-corrected coupling persists but is attenuated and less differentiated by model class.
• Curvature variability complements directness: reasoning models, after length correction, exhibit less heterogeneous local curvature on harder code problems (median $\rho_\perp^V = -0.50$).

  Figure 3: Corrected geometry gaps are not mirrored by stronger linear difficulty decodability. Geometry metrics distinguish reasoning from baseline models, but prompt-stage linear probes do not.

Prompt-stage linear probes for difficulty decodability do not mirror the corrected geometry gaps, particularly in the code domain, suggesting that the trajectory geometry encodes information not accessible to simple linear predictors over hidden states, and that the effect is not merely a byproduct of more informative representations at the outset of generation.

Temporal and Behavioral Analyses

Temporal-prefix analysis demonstrates that, for code, the separation in corrected geometry appears early in the reasoning trace and persists throughout. Behavioral annotation—performed by independent LLM judges at the sentence level—reveals that denser strategy shifting and uncertainty monitoring co-occur with stronger corrected directness-difficulty coupling. This correlation is most pronounced in code, especially among R1-distilled models.

Figure 4: Where reasoning behaviors occur in DeepSeek-R1-7B traces. More difficult problems see an increased density and overlap of annotated behaviors such as self-correction and strategy shifting.

Such co-variation substantiates the claim that length-corrected trajectory geometry is not epiphenomenal, but reflects recognizable adaptive computation (e.g., dynamic strategy changes and meta-cognitive uncertainty monitoring) during hard problem solving. Notably, verification behaviors cluster toward the end of traces, and harder problems display richer, temporally distributed behavioral markers.

Robustness and Auxiliary Analysis

Extensive robustness checks confirm that the main effect is not an artifact of boundary segmentation, choice of correction family (log-N vs. $\log N$0 or binning), or layer selection. Auxiliary metrics of intrinsic dimensionality (TwoNN, PCA90) also exhibit length confounds, but their corrected patterns are weaker and less consistent with the reasoning-baseline separation.

Figure 5: Length-correction robustness: primary log-N and $\log N$1 corrections yield consistent geometric separation; alternative corrections show variable stability.

Figure 6: Residual length diagnostics show that directness, TwoNN, and PCA90, after primary correction, have negligible remaining length dependence, while curvature variability retains some residual correlation.

Correctness-conditioned analyses validate that the main trajectory effects are preserved within both correct and incorrect solution subsets, further bolstering the claim that the geometric signals are not simply artifacts of outcome composition.

Implications and Future Directions

This work operationalizes length correction as a necessary methodological prerequisite for trajectory-geometry analysis in LLMs: naive comparison of geometric quantities is fundamentally confounded by generation length, especially as it co-varies with difficulty and model family.

The data suggest that, at least in the code domain, reasoning training engenders not only longer traces but also different modes of internal computation, characterized by more direct and less tortuous hidden-space paths as problem difficulty increases. This has ramifications for the study of adaptive computation, reasoning control, and the development of theoretical models for how LLMs meta-adaptively manage inference-time dynamics.

Future avenues may include high-resolution interventions targeting trajectory geometry, causal manipulations of hidden state directions, and further decomposition of geometric-statistical signals according to fine-grained behavioral markers, as well as domain transfer from code to math or logic. A published trajectory archive provides a resource for subsequent studies.

Conclusion

The central contribution is a rigorous demonstration that generation-time trajectory geometry, when appropriately corrected for length, encodes domain- and training-dependent patterns of adaptation to problem difficulty. For competitive programming, reasoning-trained models do more than allocate additional compute: they traverse more direct, systematically organized representational paths, correlated with dynamic behavioral strategies. Failure to correct for generation length can obscure or invert these substantive effects. These findings underscore the importance of conditional analysis in the study of LLM internals and suggest that reasoning-oriented training shapes internal computational landscapes in a measurable, domain-contingent manner. Establishing mechanisms for causal control of such trajectories remains an important objective for understanding and engineering future reasoning systems.