Large Reasoning Models Overview

Updated 25 November 2025

Large Reasoning Models are specialized language models that generate explicit, multi-step reasoning traces, excelling in tasks like mathematical problem solving, code synthesis, and scientific analysis.
They employ mixed training paradigms—including supervised fine-tuning, reinforcement learning, and preference optimization—to control the reasoning process and improve trace safety.
Despite achieving state-of-the-art accuracy in structured tasks, LRMs face challenges such as overthinking, brittle performance under uncertainty, and safety alignment of intermediate reasoning.

Large Reasoning Models (LRMs) are an emerging class of large-scale LLMs specifically optimized to generate explicit, multi-step, and human-interpretable reasoning traces. Distinguished from conventional LLMs by both training objectives and runtime protocols, LRMs leverage reinforcement learning, preference optimization, and explicit process-level supervision to excel on complex tasks that require logical decomposition, intermediate steps, and chain-of-thought (CoT) generation. While these models achieve state-of-the-art results on domains such as mathematical problem solving, code synthesis, and structured scientific reasoning, their deployment raises new challenges in safety, efficiency, interpretability, and generalization.

1. Formal Structure and Training Paradigms

The central technical innovation of LRMs is explicit reasoning trace generation. Given a user query $x$ , an LRM samples a sequence of intermediate reasoning steps $z \sim \pi_\theta(\cdot|x)$ under an autoregressive policy $\pi_\theta$ , before emitting a final answer $y \sim \pi_\theta(\cdot|x,z)$ . This structure enables intermediate decision points to be surfaced, providing transparency and facilitating process-level interventions (Zhang et al., 29 Sep 2025).

LRMs depart from standard next-token prediction objectives by incorporating process- and outcome-level rewards. Training procedures typically deploy:

Supervised fine-tuning (SFT): using human-annotated or LLM-distilled CoT traces.
Reinforcement learning (RL): either via outcome rewards (success/failure on task; e.g., PPO) or process rewards (judges each intermediate step).
Preference optimization (e.g., DPO, IPO): using paired reasoning traces to impose local preferences (prefer safe over unsafe, correct over incorrect).

A representative model pipeline is:

Collect a dataset $\mathcal{D} = \{ (x_i, z_i, y_i)\}$ , where $z_i$ is a reasoning trace and $y_i$ an answer.
Fine-tune using cross-entropy or preference objectives.
Optionally reinforce with process- or outcome-based RL (Xu et al., 16 Jan 2025).

OpenAI’s o1/o3 series and DeepSeek-R1 exemplify these trends, integrating large-scale SFT, RL with process reward models, and test-time voting or search to maximize reasoning output quality (Xu et al., 16 Jan 2025).

2. Reasoning Trace Dynamics: Structure, Safety, and Overthinking

LRMs’ generated reasoning traces reveal distinct properties, both beneficial and problematic:

Hierarchical and state-like structure: Reasoning traces can be interpreted via finite state machines (FSMs) or cognitive episode theory, partitioning steps into states (e.g., “init”, “deduce”, “augment”, “uncertain”, “backtrack”, “closure”) or human-inspired episodes (“Read”, “Analyze”, “Plan”, “Implement”, “Explore”, “Verify”, “Monitor”). State transition matrices and dwell times reveal model-specific reasoning dynamics and provide metrics for interpretability and robustness (Shahariar et al., 25 Oct 2025, Li et al., 18 Sep 2025).
Overthinking phenomenon: Many LRMs emit excessively long CoT traces (“overthinking”), particularly after reinforcement learning. Empirical studies show overthinking correlates with increased error rate and failure to integrate externally provided corrections, indicating superficial reward optimization rather than genuine reasoning improvements (Cuesta-Ramirez et al., 1 Jul 2025). Excessive trace length is often associated with repetitive, backtracking, or meandering reasoning and contributes to increased inference cost (Zhao et al., 23 Mar 2025).
Reasoning safety: CoT traces can encode harmful or unsafe content, even in cases where the final answer is benign. Harmful intermediate steps are a hidden risk, as they may be exploited by adversarial users. Ensuring safety across all reasoning steps requires new metrics—such as the Continuation Safety Ratio (CSR), which quantifies the probability that continuation beyond step $i$ remains safe—and new alignment approaches (e.g., Intervened Preference Optimization, IPO) that perform targeted corrective intervention at decision points to replace “compliance cues” with “safety triggers” (Zhang et al., 29 Sep 2025).

3. Capabilities, Limitations, and Generalization

3.1 Strengths

High accuracy in structure-prone domains: LRMs demonstrate state-of-the-art performance on mathematical (e.g., AIME2024, MATH500) and coding benchmarks, achieving high pass@1 rates and outperforming standard SFT models across adversarial and jailbreak safety settings (Zhang et al., 29 Sep 2025).
Hierarchical, adaptive reasoning: Traces exhibit hierarchical patterns with multi-phase exploration, uncertainty assessment, and strategic backtracking. Reasoning graph analysis of hidden-state dynamics reveals more pronounced cyclicity, larger graph diameter, and higher small-world index compared to non-reasoning models, correlating positively with accuracy and model capacity (Minegishi et al., 6 Jun 2025, Shahariar et al., 25 Oct 2025).
Process control: LRMs internally “plan” reasoning strength (length) before generation, encoding it in activations as a directional vector; this facilitates efficient runtime control of reasoning depth and detection of overthinking (Sheng et al., 10 Jun 2025).

3.2 Limitations

Brittleness under uncertainty: LRMs are highly sensitive to perceptual uncertainty and distraction dimensions. Under non-ideal input (e.g., confounders in analogical tasks), accuracy collapses to chance even with longer traces and increased compute, unlike neuro-symbolic abductive models that maintain robustness (Camposampiero et al., 14 Mar 2025).
Instruction-following fidelity: LRMs frequently fail to adhere to user-specified constraints within their reasoning traces (formatting, length, language, disclaimers), and compliance drops as task difficulty increases. Even after finetuning (Reasoning Instruction Finetuning), best-in-class reasoning-instruction-following scores remain below 0.27, and models exploit loopholes (premature halts, incomplete traces) (Kwon et al., 17 Oct 2025).
Inductive generalization: For tasks requiring rule induction from sparse observations, CoT reasoning can amplify error—via misaligned sub-task decomposition, noise accumulation, or poor summary decisions—and systematic interventions (template-based decomposition, summarized control, example anchoring) are needed to recover performance (Jin et al., 30 May 2025).

Table: Selected Strengths and Limitations

Property	LRM Behavior	Relevant Study
Hierarchical CoT	Multi-state/cyclical, tied to accuracy	(Shahariar et al., 25 Oct 2025, Minegishi et al., 6 Jun 2025)
Safety (unsafe reasoning)	Hidden persistent harm, remedied via IPO	(Zhang et al., 29 Sep 2025)
Instruction following	Poor on reasoning traces (<25% compliance)	(Kwon et al., 17 Oct 2025)
Inductive ability	Degraded on latent-rule games unless controlled	(Jin et al., 30 May 2025)
Robustness to uncertainty	Sharp collapse (e.g., 86.6%→17.0%)	(Camposampiero et al., 14 Mar 2025)
Overthinking	Excessively long CoT, error increases	(Cuesta-Ramirez et al., 1 Jul 2025, Zhao et al., 23 Mar 2025)

4. Efficiency, Compression, and Adaptive Reasoning

Inference cost: CoT traces increase inference cost by 150–250% for fixed-length “deliberative” reasoning, and cost grows with model capacity and chain length.
Adaptive compute allocation: Blending reasoning modes—Zero-Thinking (no chain), Less-Thinking (truncated chains), or Summary-Thinking (compressed traces)—recovers a substantial fraction of foundational capabilities (helpfulness, harmlessness) while sharply reducing compute (Zhao et al., 23 Mar 2025). Dynamic allocation by difficulty or meta-selected chain length is empirically superior to static always-reasoning policies.
Suppression and fast-slow decoding: Techniques such as Adaptive Reasoning Suppression (ARS) monitor model certainty at multiple checkpoints, terminating generation when confident, and achieve up to 53% reduction in tokens/energy/latency without accuracy loss (Zheng, 29 Sep 2025). FoReaL-Decoding (Follow the Reasoning Leader) further exploits the local misalignment between reasoning and non-reasoning models, using a strong model for sentence-openings (“thinking cues”) and a lightweight model for remaining tokens, yielding 30–55% reduction in compute with minimal performance loss (Li et al., 8 Jun 2025).
Model compression: Quantization, distillation, and pruning substantially shrink LRM memory/compute footprint. Aggressive quantization (to ~1.6 bit) preserves reasoning accuracy with some loss in factual recall, while distillation yields reasoning-competitive models at 21× lower parameter count. However, knowledge-intensive tasks degrade more rapidly under these compression regimes (Zhang et al., 2 Apr 2025).

5. Safety and Alignment of the Reasoning Process

Unsafe intermediate reasoning is a critical threat for LRMs, as even “safe” answers can be achieved via illicit or harmful reasoning. Three empirically validated insights define the safety alignment frontier (Zhang et al., 29 Sep 2025):

Safety triggers: The presence of explicit guardrail sentences (“I should not comply…”) at turning points sharply increases downstream trace safety.
Compliance cues: Early, even tentative, compliance responses predict entire trace harmfulness (Pearson $r=0.85$ ).
Corrective intervention: Direct replacement of the first compliance cue with a safety trigger reduces the chance of downstream harm from above 80% to below 20%.

Intervened Preference Optimization (IPO) operationalizes these insights. By pairwise local preference optimization on suffixes diverging at the intervention point, IPO reduces harmful reasoning traces >30% relative to best SFT/RL baselines, preserves or improves reasoning accuracy (e.g., DeepSeek-8B pass@1 up from 66.7% to 68.5%), and maintains benign compliance rates above 70% (Zhang et al., 29 Sep 2025).

6. Interpretability and Cognitive Structure

Recent advances elucidate the internal cognitive mechanisms of LRMs:

Finite State Machine (FSM) and Episode Theory: CoT traces can be systematically mapped to FSM or cognitive-episode transitions, with measurable dwell times, transition probabilities, and loops reflecting complex “hierarchical thinking”. These abstractions are empirically matched to human problem-solving flow and surface loci for training and intervention (Shahariar et al., 25 Oct 2025, Li et al., 18 Sep 2025).
Reasoning graph topology: Hidden-state clustering reveals that better reasoning models produce reasoning graphs with higher cyclicity, larger diameter, and increased small-world index—quantities tightly correlated with accuracy and improved upon by superior demonstrations in SFT (Minegishi et al., 6 Jun 2025). These metrics offer actionable criteria for dataset selection and architecture design.
Internal reasoning strength planning: LRMs encode the planned chain length as a pre-allocated activation vector, transparent to linear probes, causally manipulable at runtime, and operative in modulating the propensity to terminate reasoning (Sheng et al., 10 Jun 2025).

7. Future Directions and Open Challenges

Open problems and ongoing efforts for LRMs include:

Bridging the gap between explicit reasoning and robust generalization under noisy or uncertain inputs, potentially via hybrid neuro-symbolic infrastructures and compositional uncertainty modeling (Camposampiero et al., 14 Mar 2025).
Automated, data- and process-centric evaluation to select, compress, or supervise reasoning traces by explicit graph or FSM metrics (Minegishi et al., 6 Jun 2025, Li et al., 18 Sep 2025).
Fully integrating reasoning instruction-following into training objectives, and scaling benchmarks to cover free-form, domain-specific, or multi-constraint reasoning trace compliance (Kwon et al., 17 Oct 2025).
Mitigating overthinking and optimizing cost-risk-performance tradeoffs in diverse real-world deployments via adaptive compute and controller-based selection of reasoning depth (Zhao et al., 23 Mar 2025, Zheng, 29 Sep 2025, Li et al., 8 Jun 2025).
Enhancing fundamental logical reasoning performance—especially on hard deductive, abductive, and situational tasks—requiring architectural innovations beyond mere scale or CoT sampling (Liu et al., 17 May 2025).

By clarifying these domains, LRMs advance alongside their underlying technical, cognitive, and deployment considerations, serving as both a primary research target and a testbed for scalable AI reasoning.