ORION: Teaching Language Models to Reason Efficiently in the Language of Thought (2511.22891v1)

Abstract: Large Reasoning Models (LRMs) achieve strong performance in mathematics, code generation, and task planning, but their reliance on long chains of verbose "thinking" tokens leads to high latency, redundancy, and incoherent reasoning paths. Inspired by the Language of Thought Hypothesis, which posits that human reasoning operates over a symbolic, compositional mental language called Mentalese, we introduce a framework that trains models to reason in a similarly compact style. Mentalese encodes abstract reasoning as ultra-compressed, structured tokens, enabling models to solve complex problems with far fewer steps. To improve both efficiency and accuracy, we propose SHORTER LENGTH PREFERENCE OPTIMIZATION (SLPO), a reinforcement learning method that rewards concise solutions that stay correct, while still allowing longer reasoning when needed. Applied to Mentalese-aligned models, SLPO yields significantly higher compression rates by enabling concise reasoning that preserves the benefits of detailed thinking without the computational overhead. Across benchmarks including AIME 2024 and 2025, MinervaMath, OlympiadBench, Math500, and AMC, our ORION models produce reasoning traces with 4-16x fewer tokens, achieve up to 5x lower inference latency, and reduce training costs by 7-9x relative to the DeepSeek R1 Distilled model, while maintaining 90-98% of its accuracy. ORION also surpasses Claude and ChatGPT-4o by up to 5% in accuracy while maintaining 2x compression. These results show that Mentalese-style compressed reasoning offers a step toward human-like cognitive efficiency, enabling real-time, cost-effective reasoning without sacrificing accuracy.

• The paper introduces Mentalese, a symbolic reasoning format that compresses verbose chain-of-thought traces by 10–20× while retaining interpretability.
• It leverages SLPO and GRPO reinforcement strategies to balance brevity and correctness, recovering 90–98% of baseline accuracy under a compressed token regime.
• Empirical results demonstrate 5× shorter reasoning traces, reduced latency and cost, and enhanced safety auditing in deployment scenarios.

ORION: Teaching LLMs to Reason Efficiently in the Language of Thought

Motivation and Problem Context

Current large reasoning models (LRMs) outperform prior generation LLMs on mathematics and program synthesis via explicit, long-form "chain-of-thought" (CoT) reasoning. However, even the best models—such as OpenAI o1 and DeepSeek R1—demonstrate significant inefficiencies: their outputs are dominated by verbose, redundant, natural language reasoning traces that balloon inference latency, operational cost, and complicate real-time deployment. These chains diverge strongly from the concise, symbolic reasoning deployed by humans. Addressing these limitations, ORION investigates cognitively-motivated approaches for efficient model reasoning, drawing from the Language of Thought Hypothesis (LOTH), and proposes a full stack—representation, optimization, dataset, and evaluation—for compressing and aligning LLM reasoning to human-like, minimal, and interpretable chains.

Mentalese: Symbolic Compression of Reasoning

The concept of Mentalese introduced by ORION encapsulates concise, structured, symbolic chains that abstract away verbose natural language, yielding reasoning languages that are closer to human cognitive processes. Each step in a Mentalese chain is a (OPERATION:EXPRESSION;) tuple, with a restricted set of logical operations over expressions—such as SET, ANS, SOLVE, EQ, CALC—systematically covering the core operations underlying mathematical reasoning.

This format provides:

• Massive token compression: Typical traces are 10–20x shorter than comparable CoT outputs while retaining complete logical structure and interpretability.
• Faithfulness and necessity: Each step is essential and sufficient, eliminating redundant rephrasing endemic to long-form LLM outputs.
• Cognitive plausibility: More closely aligns model reasoning to symbolic cognitive frameworks postulated in psycholinguistics.

  Figure 1: Comparison demonstrating the verbose, redundant nature of LLM outputs versus the concise, minimal symbolic chains of human (or Mentalese-style) reasoning.

  

  Figure 2: Example definition and structure of the Mentalese reasoning format, including sample steps and rules.

Optimization: SLPO and Group-Relative Reinforcement

While supervised fine-tuning (SFT) on the Mentalese corpus yields substantial compression, it destroys much of the performance benefits conferred by traditional long-form CoT: empirical results show a consistent 30–40 percentage point degradation in Pass@1 upon direct SFT. This is primarily due to the rigid, single-path nature of Mentalese traces versus the exploratory, self-corrective possibilities of forking and multi-hop reasoning in DeepSeek/O1 style chains.

To recover performance without sacrificing efficiency, ORION introduces two critical optimization strategies:

• Shorter Length Preference Optimization (SLPO): A group-based RL reward design that flexibly adapts the brevity/correctness trade-off at the group level, soft-preferencing shorter correct solutions whenever multiple paths exist while never penalizing necessary extended chains. Unlike previous hard-length-penalized objectives (e.g. L1, LCPO), SLPO dynamically adapts to per-instance complexity, avoiding under- or over-reasoning collapse modes.
• Group Relative Policy Optimization (GRPO): Structures RL objectives to normalize rewards among groups of sampled rollouts for the same prompt, improving the reward landscape and removing dependence on possibly pathological value baselines.

Empirically, application of SLPO after SFT on Mentalese recovers most of the accuracy loss from compression, producing models that operate on the Pareto frontier of the accuracy/efficiency trade-off in mathematical reasoning.

Figure 3: Performance–efficiency Pareto frontier across model-method families. ORION models with Mentalese+SLPO compression consistently achieve optimal accuracy at drastically reduced token costs.

Experimental Results

The paper provides extensive evaluation on both in-domain (AIME, AMC, MATH, Minerva, OlympiadBench) and out-of-domain (GPQA, LSAT, MMLU) reasoning benchmarks. Core findings include:

• Token Compression: Across all benchmarks, ORION models achieve 4–16× response length compression relative to DeepSeek R1, and 10–20× over typical baseline CoT models.
• Accuracy-Efficiency Tradeoff: At compression rates exceeding 10×, ORION Mentalese-aligned + SLPO models retain 90–98% of DeepSeek R1’s Pass@1, outperform Claude and ChatGPT-4o by up to 5% Pass@1 at double the compression.
• Latency and Cost: Reasoning traces are 5× shorter, yielding 7–9× reduction in both inference latency and RL training cost.
• Training Stability: Direct application of SLPO or group RL to base models without prior Mentalese alignment results in optimization divergence and catastrophic collapse. Insertion of an SFT-on-Mentalese phase stabilizes learning and enables large-batch distributed training.

  Figure 4: Training dynamics tracing response length, reward stability, and training cost for direct SLPO versus Mentalese-aligned RL. Only the Mentalese path avoids gradient instability and divergence.

• Agentic Implications: In real-world agentic deployments, compressed models avoid overfitting to communication bandwidth, improving interpretability of intermediate steps, facilitating easier safety auditing of intermediate chains, and enabling practical reasoning deployment in latency-constrained, multi-agent environments.

  Figure 5: Reasoning trace comparison for solving AIME-style math: Agentica-24k (base RL model) needs ~7800 tokens, ORION-AG/ORION-AG-SLPO solve the same instance with 150/300 tokens respectively, at equivalent accuracy.

  

  Figure 6: Violin plots indicating distribution of output lengths before and after Mentalese and RLVR pipeline—token usage consistently collapses and clusters around theoretical minimums aligned to logical complexity.

Theoretical and Practical Implications

ORION’s demonstration that Mentalese-style symbolic compression is recoverable with negligible accuracy loss, conditioned on appropriate RL reward design (SLPO), has both theoretical and practical impacts:

• Theoretically: It supports a revised view that model size and reasoning depth alone do not fully determine LLM performance; instead, the quality and structure of intermediate representations (and their alignment with human symbolic reasoning) is a dominant axis.
• Practically: Deploying models with 10–20× speedup and cost reduction with minimal accuracy penalty makes agentic LLMs viable on edge devices or in orchestrated simulation fleets. Furthermore, auditing and interpretability are improved due to factorized symbolic traces.

Future Directions

The findings suggest several research directions:

• Generalization of symbolic format: Extending Mentalese representations, with problem-specific operator sets, to complex domains beyond mathematics (e.g., science, code, planning).
• Latent symbolic reasoning: Investigating hybrid or latent symbolic/implicit intermediate representations for more complex tasks.
• Composition with policy search/planning: Integration with sampling-based planners, latent space search, and verification-in-the-loop RL methods for robust and trustworthy reasoning chains.

Conclusion

The ORION framework establishes that high-level symbolic compression via Mentalese, reinforced with tailored RL reward (SLPO), enables LLMs to approach the efficiency and compactness of human reasoning without sacrificing accuracy, validating cognitive hypotheses regarding reasoning efficiency. This approach yields practical improvements in latency, cost, and interpretability and offers a promising paradigm for next-generation large reasoning models prioritizing both cognitive alignment and deployment practicality.

Reference: "ORION: Teaching LLMs to Reason Efficiently in the Language of Thought" (2511.22891)