Efficient Agentic Reasoning Through Self-Regulated Simulative Planning

Abstract: How should an agent decide when and how to plan? A dominant approach builds agents as reactive policies with adaptive computation (e.g., chain-of-thought), trained end-to-end expecting planning to emerge implicitly. Without control over the presence, structure, or horizon of planning, these systems dramatically increase reasoning length, yielding inefficient token use without reliable accuracy gains. We argue efficient agentic reasoning benefits from decomposing decision-making into three systems: simulative reasoning (System II) grounding deliberation in future-state prediction via a world model; self-regulation (System III) deciding when and how deeply to plan via a learned configurator; and reactive execution (System I) handling fine-grained action. Simulative reasoning provides unified planning across diverse tasks without per-domain engineering, while self-regulation ensures the planner is invoked only when needed. To test this, we develop SR$^2$AM (Self-Regulated Simulative Reasoning Agentic LLM), realizing both as distinct stages within an LLM's chain-of-thought, with the LLM as world model. We explore two instantiations: recording decisions from a prompted multi-module system (v0.1) and reconstructing structured plans from traces of pretrained reasoning LLMs (v1.0), trained via supervised then reinforcement learning (RL). Across math, science, tabular analysis, and web information seeking, v0.1-8B and v1.0-30B achieve Pass@1 competitive with 120-355B and 685B-1T parameter systems respectively, while v1.0-30B uses 25.8-95.3% fewer reasoning tokens than comparable agentic LLMs. RL increases average planning horizon by 22.8% while planning frequency grows only 2.0%, showing it learns to plan further ahead rather than more often. More broadly, learned self-regulation instantiates a principle we expect to extend beyond planning to how agents govern their own learning and adaptation.

• The paper demonstrates that decomposing agentic LLM reasoning into reactive execution, simulative planning, and self-regulation improves both accuracy and efficiency.
• It employs a hybrid supervised and reinforcement learning approach to generate structured plans that achieve up to 95.3% reduced token consumption while maintaining high pass rates.
• The approach enhances interpretability with explicit plan annotations, facilitating debugging and adaptive decision-making for complex, long-horizon tasks.

Efficient Agentic Reasoning via Self-Regulated Simulative Planning: An Expert Review

Problem Setting and Motivation

The current paradigm for agentic LLMs predominantly treats the agent as a reactive policy with chain-of-thought (CoT) or latent-conditioned adaptive computation, aiming for planning to emerge implicitly from large-scale end-to-end optimization. However, this approach results in unbounded increases in reasoning trajectory length, leading to significant token inefficiency and often fails to provide explicit control over the planning process. The lack of mechanisms to modulate the presence, structure, or temporal horizon of planning hinders both efficiency and interpretability, while longer traces do not guarantee accuracy improvements. These limitations are particularly acute for interactive reasoning tasks that benefit from dynamic, context-sensitive planning and execution, as observed in benchmarks covering math, STEM, data analysis, and real-world web information seeking.

The Three-System Decomposition

This work introduces a compositional agentic framework---SR2AM (Self-Regulated Simulative Reasoning Agentic LLM)---that factorizes the decision process into three interacting systems:

• System I (Reactive Execution): Handles step-level, fine-grained reasoning and tool-augmented actions; suitable for low-uncertainty, high-frequency decisions.
• System II (Simulative Reasoning): Explicitly constructs structured plans via forward simulation using a world model in language space; supports high-level, multi-step goal-directed planning.
• System III (Self-Regulation): A learned configurator governs whether, when, and how extensively System II is invoked at each decision point, fostering conditional invocation of deliberation based on situational complexity and uncertainty.

Unlike prior work, this decomposition unifies simulative planning with internal, learned regulation of planning frequency and depth, implemented natively as distinct chain-of-thought stages within the LLM.

Methodology

Two instantiations of SR2AM are developed:

• SR2AM-v0.1: Supervises the decomposition using data collected from a multi-module prompted system, where the configurator, planner, and reflection modules are realized as explicit LLM tool calls.
• SR2AM-v1.0: Scales the approach by reconstructing structured plans and regulatory decisions from traces of pretrained reasoning LLMs, preserving free-form CoT while adding explicit plan and configurator annotations.

Training proceeds with a supervised learning phase, followed by reinforcement learning (RL) using a composite reward (correctness, structure, extractability) and group-normalized advantages (GRPO) for joint optimization of the three system components. The LLM itself functions as the world model, supporting simulative reasoning directly in language.

The agent is evaluated across 11 benchmarks encompassing four domains (math, science, tabular, web) in realistic, tool-augmented interaction settings.

Empirical Findings

Accuracy and Efficiency

• SR2AM-v0.1-8B achieves Pass@1 competitive with agentic LLMs at 30-32B and pretrained tool-using LLMs with 120-355B parameters.
• SR2AM-v1.0-30B matches or exceeds the pass rates of baselines at up to 1T scale, with Pass@1 = 71.3, outperforming strong 30-32B agentic LLMs and competing with proprietary models at significantly larger scale (e.g., DeepSeek-V3.2 at 685B, Kimi-K2.5 at 1.0T).

Crucially, SR2AM-v1.0-30B achieves these results while consuming 25.8–95.3% fewer reasoning tokens than comparable agentic LLMs, establishing a new efficiency frontier for long-horizon agentic tasks. Unlike unregulated deliberation, which incurs adverse token growth without proportional accuracy, SR2AM's token use is tightly controlled by the self-regulatory configurator.

Planning Horizon and Regulation

Analysis of post-RL models reveals that:

• RL increases the average planning horizon by 22.8%, while planning frequency rises only marginally (+2.0%), demonstrating that the model learns to plan further ahead—not simply to plan more frequently.
• Component ablation demonstrates each system's unique contribution: removing selective planning (System III) increases token use, removing simulative planning (System II) or its structure reduces accuracy, and ablating free-form reasoning (System I) results in the largest performance degradation.
• Over-planning on simple tasks is rare, but indicates outstanding opportunity for configurator calibration on when to terminate planning.

Generality of Decomposition

Supervised initialization with the three-system decomposition outperforms unregulated reasoning even when both use the same teacher LLM and data volume, confirming that structural content rather than raw LLM capability is the dominant factor. RL training of the structured decomposition exhibits higher accuracy, greater reasoning efficiency, and lower out-of-context rates compared to unregulated baselines, with improvements scaling with both data quantity and teacher quality.

Implications

Theoretical Implications

• The explicit separation of planning, acting, and regulation orthogonalizes agentic reasoning capabilities, facilitating targeted improvements and interpretability.
• The configurator’s learned self-regulatory capacity enables the emergence of adaptive metacognition, suggesting a route toward more autonomous and efficient artificial agents.
• The decomposition is general and could extend to settings like embodied multi-agent systems, or inform bootstrapping schedules for autonomous learning and adaptation, potentially yielding agents that regulate their own curriculum or revise internal models in light of persistent uncertainty.

Practical Implications

• Substantially reduced token consumption mitigates compute costs and context-window overflows for long-horizon agentic applications.
• Explicit, structured simulative planning provides a unified, domain-agnostic grounding for decision making, obviating the need for per-domain heuristics.
• The interpretability of plan and configurator outputs enables both debugging (in downstream pipelines) and automated error correction via reflection or fallback strategies.

Limitations and Future Work

The evaluation is currently restricted to language-mediated, tool-augmented agentic reasoning. Extensions are required for embodied agents with richer perceptual state representations, and for multi-agent coordination settings. The current world model is limited to text; integrating multimodal or symbolic world models would increase the realism and generality. Diagnostic studies isolating configurator/world model accuracy and richer context management could further improve performance. Lastly, moving beyond planning—using learned regulation to govern other forms of agent autonomy—remains an open frontier.

Conclusion

"Efficient Agentic Reasoning Through Self-Regulated Simulative Planning" (2605.22138) provides evidence that decomposing agentic LLMs into explicit, learned modules for planning, regulation, and reactive action yields better accuracy-efficiency tradeoffs and facilitates the emergence of deep, anticipatory planning with controlled resource consumption. These findings challenge the implicit-planning paradigm of chain-of-thought models and suggest a path toward more autonomous, interpretable, and scalable agentic intelligence. The self-regulatory principle articulated here holds substantial promise for the continued evolution of capable, efficient, and adaptable AI agents.