OpenAI's O1: A Large Reasoning Model

• OpenAI's o1 is a large-scale reasoning model that features explicit chain-of-thought reasoning, enabling multi-step problem solving with reinforcement learning.
• It uses a two-phase training process combining initial CoT supervision and dense reward-based fine-tuning to achieve state-of-the-art results in planning and scheduling tasks.
• Despite its advanced reasoning capabilities, o1 faces challenges in efficiency, domain generalization, and long-step planning, driving further research in robust AGI systems.

OpenAI’s o1 is a large-scale reasoning model (“Large Reasoning Model,” LRM) developed and released in late 2024 as a successor to previous autoregressive LLMs. In contrast to conventional LLMs, o1 is specifically engineered to internalize chain-of-thought reasoning and leverages advanced reinforcement learning techniques during both pre-training and fine-tuning. As a result, o1 sets new standards in reasoning, planning, and complex multi-step problem solving across a wide array of domains, while also presenting challenges related to efficiency, safety, and domain generalization.

1. Architectural Foundations and Training Paradigm

OpenAI o1 departs from standard LLM frameworks by positioning explicit multi-step reasoning—in the form of internalized chain-of-thought (CoT) mechanisms—at the core of its training and inference. Unlike traditional models that operate primarily as text retrievers or next-token predictors, o1 is trained through a reinforcement learning (RL) curriculum that rewards the construction of coherent, high-quality reasoning traces.

O1’s training integrates a two-phase process:

• Policy Initialization and CoT Supervision: Model parameters are first pre-trained on large corpora and then fine-tuned using supervised instruction data, which introduces early reasoning behaviors.
• Reinforcement Learning with Dense Rewards: The model undergoes RL fine-tuning where it learns to assign q-values to intermediate reasoning steps, using both process-level rewards (for stepwise quality) and outcome rewards (for final correctness). This process is formalized by optimizing a value function

$$Q(s, a) \approx \mathbb{E}\Bigl[\sum_{t=0}^{T} r_t \,\Big|\, s_0 = s,\, a_0 = a\Bigr]$$

so as to reinforce effective reasoning actions.

Additionally, o1 employs adaptive inference: at deployment time, it dynamically determines the number of reasoning (“thinking”) tokens internally generated, scaling its computation to problem complexity rather than responding with a fixed-length output (Valmeekam et al., 20 Sep 2024, Zeng et al., 18 Dec 2024).

2. Planning, Scheduling, and System 2 Reasoning Capabilities

The o1 model series demonstrates unprecedented performance on classical planning and scheduling benchmarks. On standardized tasks such as Blocksworld, o1 achieves near-perfect accuracy of 97.8% in zero-shot evaluations, dramatically surpassing earlier LLMs, which plateau at 28–62% accuracy in similar settings (Valmeekam et al., 20 Sep 2024, Valmeekam et al., 3 Oct 2024). In obfuscated or “mystery” variant problems—where surface forms are randomized—o1 maintains significantly above-random performance (52.8% accuracy), while LLMs nearly fail entirely.

However, on longer planning tasks, the advantage diminishes: for plans exceeding 20 steps, o1’s performance degrades to approximately 23.6%. In domains requiring adherence to intricate constraints and robust state tracking (e.g., Tyreworld, Termes), o1-preview demonstrates improved feasibility and constraint satisfaction relative to GPT-4, but still exhibits shortcomings in optimality (redundant steps), memory management, and generalization when faced with unfamiliar abstractions or spatial complexity (Wang et al., 30 Sep 2024).

In scheduling applications, o1-mini achieves up to 96% accuracy on graph coloring but only marginal improvements or inconsistencies in more complex travel planning and calendar scheduling scenarios (Valmeekam et al., 3 Oct 2024).

3. Reasoning Patterns: Divide-and-Conquer, Self-Refinement, and Test-Time Compute

Analyses of o1’s reasoning processes identify six distinct reasoning patterns that underlie its superior performance:

• Systematic Analysis: Decomposing problems by explicitly analyzing structure and constraints before responding.
• Method Reuse: Mapping novel tasks onto known strategies.
• Divide and Conquer: Splitting complex problems into sub-problems and hierarchically recombining solutions.
• Self-Refinement: Iterating over intermediate solutions, correcting errors through internal critique.
• Context Identification: Summarizing necessary context, especially for tasks requiring external knowledge.
• Emphasizing Constraints: Explicitly reinforcing formatting and operational requirements.

Empirical studies indicate that o1’s integration of divide-and-conquer and self-refinement is a major driver of its reasoning gains, enabling it to outperform both simple best-of-N and agent workflow strategies on complex math, coding, and commonsense reasoning tasks (Wu et al., 17 Oct 2024). The “thinking-before-responding” paradigm, realized via increased inference computation, is a notable shift away from the “one-shot” approaches of past LLMs.

4. Comparative Benchmark Performance and Domain-Specific Deployments

O1 consistently delivers leading performance across diverse domains:

• Mathematics: O1-preview scores near 97.8th percentile on Dutch national exams, outperforming both GPT-4o and most human candidates (Winter et al., 19 Sep 2024). On International Mathematics Olympiad (IMO) and lesser-known Chinese National Team datasets, o1’s consistent accuracy demonstrates true problem-solving over memorization (Li et al., 9 Nov 2024).
• Medicine: O1 achieves higher accuracy than prior models (average +6.2% over GPT-4) on datasets derived from NEJM and The Lancet quizzes, as well as improved multilingual performance (85.2% on XMedBench vs. 75.7% for GPT-4) (Xie et al., 23 Sep 2024). Nonetheless, areas such as hallucination, multilingual agent tasks, and decoding speed remain open challenges.
• Ophthalmology: O1 leads in accuracy (0.88) and macro-F1 (0.70) among LLMs on MedMCQA, but ranks third (after GPT-4o and GPT-4) in reasoning metrics that assess text-generation quality, indicating a gap between answer selection and explanation fidelity (Srinivasan et al., 20 Jan 2025).
• Higher-Order Cognition: O1-preview demonstrably outperforms human baselines in critical thinking, systematic thinking, data literacy, and scientific reasoning, but underperforms in certain types of logic and abstract/adaptive reasoning (Latif et al., 11 Oct 2024, Latif et al., 7 Dec 2024).
• Other Domains: O1-preview exhibits strong performance in chip design, robotics planning, quantitative investing, sentiment analysis, and table-to-text generation (Zhong et al., 27 Sep 2024).

Despite its breadth, o1’s reasoning prowess does not universally transfer to highly specialized domains without domain-specific adaptation, and verbose or rigid chain-of-thought outputs may lower performance on metrics sensitive to brevity or phrasing.

5. Efficiency, Scalability, and Inference Optimization

The “long thought” paradigm in o1, while boosting problem-solving ability, incurs substantial computational overhead. The model frequently generates hidden reasoning tokens beyond user output, causing high inference latency and financial cost—up to orders of magnitude greater than classical planners or standard LLMs for comparable tasks (Valmeekam et al., 20 Sep 2024, Valmeekam et al., 3 Oct 2024). For example, the cost-per-100 instances can exceed \$42 for o1-preview, whereas classical planners complete the same benchmarks in milliseconds at negligible cost (Valmeekam et al., 3 Oct 2024).

The inefficiency is addressed in part by post-hoc methods such as O1-Pruner (Luo et al., 22 Jan 2025), which applies length-harmonizing fine-tuning via RL to reduce redundant reasoning steps without sacrificing accuracy:

$$R_{LH}(x, y) = \frac{\bar{L}_{ref}(x)}{L(y)} - 1 + \lambda \bigl(A(x, y) - \bar{A}_{ref}(x)\bigr)$$

This approach yields up to 40% reduction in answer length on benchmarks like MATH, occasionally with improved accuracy.

6. Safety, Alignment, and Robustness

O1’s explicit incorporation of chain-of-thought facilitates enhanced safety protocols through “deliberative alignment,” in which the model reasons about its safety constraints before finalizing outputs (OpenAI et al., 21 Dec 2024). On adversarial tests—including disallowed content, jailbreak resistance, and refusal robustness—o1 achieves state-of-the-art scores, with “not_unsafe” metrics approaching or equaling 1. Statistical evaluations and red teaming confirm improvements over predecessors such as GPT-4o, particularly in withstanding complex jailbreak attempts and contextual safety scenarios (Wang et al., 26 Nov 2024). Nevertheless, o1 retains residual vulnerabilities: adversaries may exploit intermediate reasoning states (“attack surface”) or employ mathematically encoded prompts that evade conventional safety mechanisms.

Mitigation strategies include enhanced prompt engineering, supervised fine-tuning with detailed chain-of-thought responses, and reinforcement learning with process supervision (rewarding each reasoning step for safety and correctness). Balance between safety and overrefusal remains a focus of ongoing research.

7. Limitations, Open Research Directions, and Implications

Despite substantial advances, o1 is not a universal solution:

• Its performance degrades on long, highly compositional, or spatially complex planning tasks due to memory and state-tracking bottlenecks (Wang et al., 30 Sep 2024).
• “Quantum improvements” on standard benchmarks do not guarantee robustness or guarantees in unstructured, adversarial, or real-world settings; classical planners retain provable correctness and efficiency advantages.
• The chain-of-thought reasoning, while effective, exposes sensitivities to probability and distributional frequency—the so-called “embers of autoregression”—limiting performance when outputs are statistically rare or low-probability (McCoy et al., 2 Oct 2024).
• Domain transfer to specialized fields (e.g., ophthalmology) shows that o1’s reasoning improvements may require further refinement and fine-tuning for optimal results (Srinivasan et al., 20 Jan 2025).

Open research directions include:

• Enhanced memory management and decision-making modules for sustained multi-step reasoning.
• Adaptive computation policies to dynamically balance efficiency and reasoning depth (Luo et al., 22 Jan 2025).
• Unified evaluation protocols for reasoning quality, factuality, and hallucination mitigation in complex domains (Xie et al., 23 Sep 2024).
• Safety alignment approaches that secure each step of the reasoning process against adversarial manipulation (OpenAI et al., 21 Dec 2024, Wang et al., 26 Nov 2024).
• Exploration of journey learning paradigms and open science methodologies for transparent and community-driven refinement (Qin et al., 8 Oct 2024).

In summary, OpenAI’s o1 marks an epochal shift toward LLMs capable of explicit reasoning via reinforcement learning–trained chain-of-thought. While it establishes new state-of-the-art results for planning, problem solving, and cognitive benchmarking, o1’s limitations in cost, generalization, efficiency, and safe deployment underscore the ongoing challenges in advancing toward robust and trustworthy artificial general intelligence.