Autonomous Multi-Turn Tool Invocation

• Autonomous multi-turn tool invocation is a field that uses LLM-driven planning, embodied agents, and external APIs to dynamically select and execute tools in iterative turns.
• It leverages formal frameworks like MDPs, POMDPs, and DAG-based tool graphs to manage dialogue histories, tool outputs, and plan adaptation under uncertainty.
• Hybrid memory structures, reinforcement learning, and graph-structured orchestration enable robust and efficient workflows across dialog systems, program synthesis, and enterprise automation.

Autonomous multi-turn tool invocation is a research area at the intersection of LLMs, embodied agents, and external API interaction, wherein an agent dynamically and repeatedly selects, configures, and executes external tools and APIs over multiple conversational or reasoning turns without direct human-in-the-loop supervision or pre-specified pipeline control. This capability underpins advanced agentic workflows in domains ranging from complex dialog systems, program synthesis, enterprise automation, and embodied reasoning to multi-agent and multimodal environments. Current state-of-the-art systems combine model-driven planning, execution feedback integration, memory management, preference alignment, and graph-structured tool orchestration to achieve robust autonomy under real-world uncertainty.

1. Formal Problem Definition and Computational Frameworks

Autonomous multi-turn tool invocation is typically formalized via sequential decision processes—most commonly Markov Decision Processes (MDPs), Partially Observable MDPs (POMDPs), or planning-over-structured representations (such as DAGs or tool graphs).

• In the POMDP or MDP view, the agent’s state incorporates dialogue history, tool outputs, and internal environment state. The action space is partitioned into {tool calls, parameterizations, or textual responses}, transitions reflect query resolution and environment updates, and rewards are linked to task completion, correct tool usage, and preference satisfaction (Jung et al., 2 Apr 2025, Zhao et al., 26 Aug 2025, Xu et al., 29 Oct 2025, Li et al., 8 Dec 2025).
• In DAG-based or tool graph formulations, agent reasoning and invocation are modeled as generating and traversing a directed acyclic graph, where each node is a tool or subtask, and edges encode data or control dependencies (Lu et al., 28 Oct 2025).
• Modern systems maintain rich histories: at each turn, the agent context includes $(h_t)$—pairs of observations (user utterances, tool outputs) and actions (tool invocations or messages), frequently augmented with explicit state or memory structures (Li et al., 8 Dec 2025, Maben et al., 29 Jun 2025).

State and action models are further complicated by the need for slot filling, parameter inference, and dynamically adapting plans based on tool execution outcomes and user clarifications (Jung et al., 2 Apr 2025, Maben et al., 29 Jun 2025, Li et al., 29 Dec 2025).

2. Tool Selection, Memory, and Context Management

Effective multi-turn invocation must address the combinatorial challenge of tool retrieval, selection, and short-term memory management:

• Statistical and Graph-Based Selection: Trajectory mining and directed tool graphs encode empirical transition probabilities and parameter dependencies, allowing agents to predict next best tools from historical inertia (tool usage inertia). Frameworks such as AutoTool build explicit $G=(V,E,W)$ graphs, with inertia-driven selection reducing LLM inference cost by up to 30% without sacrificing completion rates (Jia et al., 18 Nov 2025).
• Hybrid Memory Structures: State-Integrated Tool Graphs (SIT-Graph) attach compact state summaries (episodic fragments) alongside procedural transition weights, enabling agents to adapt between routine tool chains and context-dependent recall, significantly boosting process accuracy over both pure graph and pure memory methods (Li et al., 8 Dec 2025).
• Memory Pruning and Scalability: MemTool provides agentic and workflow-based modes to avoid context overflow as the number of dynamically introduced tools grows. Autonomous agent mode, workflow mode, and hybrid mode offer tradeoffs in removal efficiency and adaptive control, with proper pruning necessary to keep tool selection within system constraints (tool-count limits, context size) (Lumer et al., 29 Jul 2025).
• Proactive Retrieval: Rather than pre-injecting schemas, proactive agent frameworks (MCP-Zero) empower LLMs to request only needed schemas at subtask boundaries, employing hierarchical coarse-to-fine retrieval to keep token costs sub-linear in toolset size and supporting iterative invocation with on-the-fly schema updates (Fei et al., 1 Jun 2025).

3. Learning Paradigms: Supervised, Reinforcement, and Preference Optimization

Multi-turn tool invocation policies are acquired and refined via staged learning pipelines:

• Supervised Fine-Tuning (SFT): Models are bootstrapped using expert or synthetic trajectories, often generated and verified by LLMs, to encode plan–action–reflect cycles or tool-augmented chain-of-thought traces (Jiang et al., 8 Sep 2025, Qian et al., 21 May 2025).
• Reinforcement Learning (RL, GRPO, RLHF): Group Relative Policy Optimization (GRPO) and related policy gradient techniques are extensively used to fine-tune stepwise tool selection and adaptive replanning under execution-aware reward functions. Credit signals reward not just end-to-end success but also per-step execution correctness, tool formatting compliance, self-correction, and efficiency (Zeng et al., 2 Apr 2025, Xu et al., 29 Oct 2025, Zhao et al., 26 Aug 2025, Qian et al., 21 May 2025, Li et al., 29 Dec 2025).
• Direct Preference Optimization (DPO): Contrastive preference learning as in DiaTool-DPO converts expert/human or synthetic preference pairs between correct and incorrect multi-turn trajectories into an alignment loss, directly training models to favor robust dialogue state control, accurate slot-filling, and safe tool rejection (Jung et al., 2 Apr 2025).
• Closed-Loop Roleplaying Synthesis: InfTool and similar approaches synthesise high-coverage datasets without human annotation, using multi-agent roleplaying (user, assistant, server) with self-reflection, stratified sampling, and iterative RL fine-tuning. This approach has demonstrated over 250% accuracy improvements (to 70.9% on BFCL) in tool use compared to static SFT alone (Li et al., 29 Dec 2025).

4. Orchestration, Planning, and Complex Multi-Tool Control

Advanced frameworks address the orchestration of multiple tools, plan adaptation, and complex dependency management:

• Graph-Structured Orchestration: Plan generation and execution as DAGs or intent trees enables robust multi-turn orchestration, supports correct data binding, concurrent/sub-intent scheduling, and fine-grained failure recovery. OrchDAG formalizes tool orchestration as topological traversal and proposes graph edit distance–based rewards for intermediate credit assignment, substantially improving partial-correctness learning (Lu et al., 28 Oct 2025).
• Intent Parsing and Subtask Decomposition: Multi-agent systems like Z-Space rely on structured semantic parsing, hierarchical sub-intent decomposition, and execution DAG construction for concurrent multi-turn planning, supported by high-granularity semantic intent–tool alignment via fused embedding schemas (FSWW) (He et al., 23 Nov 2025).
• Proactive and Reactive Refinement: Iterative, feedback-driven tool invocation cycles, integrated with execution outcome checking and adaptive plan refinement, are crucial for context-sensitive, self-correcting behavior, as demonstrated in frameworks like MCP-Zero and TableMind (Fei et al., 1 Jun 2025, Jiang et al., 8 Sep 2025).

5. Temporal Dynamics, Preference Alignment, and Evaluation

The challenge of temporal context in multi-turn settings—when to refresh tool calls versus rely on cached observations—has emerged as a limiting factor in current agent performance:

• Temporal Blindness: Most LLM agents are time-static: without explicit elapsed-time cues, models perform only marginally above random in human-aligned tool-calling (PAR ≈ 50–60%). Even with explicit timestamps, top models yield at most ≈65% alignment with human preferences (Cheng et al., 27 Oct 2025).
• Preference Data and Metrics: Evaluations employ preference-alignment rates, per-class attempt rates, and DAG- and step-wise accuracy metrics. Direct preference optimization and custom metrics for multi-turn tool use, including tool usage appropriateness, chain coherence, and perception-guided alignment, are increasingly central (Qian et al., 21 May 2025, Xu et al., 29 Oct 2025, Jung et al., 2 Apr 2025, Cheng et al., 27 Oct 2025).
• Prompt Engineering Limitations: Prompt-based interventions—time reminders, explicit “rules,” or extended chain-of-thought—have only modest effect on strong models and negligible on open-source or smaller models, highlighting the need for targeted post-training alignment (Cheng et al., 27 Oct 2025).

6. Practical Architectures, Applications, and Empirical Benchmarks

Current state-of-the-art systems instantiate these principles in end-to-end pipelines for dialogue agents, program synthesis, vision-language reasoning, and enterprise automation:

• Voice-Driven Agents: Cascaded speech-to-speech systems (AURA) integrate ASR, dialog state tracking, LLM action policy, tool wrappers, and TTS, achieving 90% multi-turn success in complex spoken scenarios (Maben et al., 29 Jun 2025).
• Autonomous Coding and Table Reasoning: Agents like TableMind use plan–action–reflection loops with RL fine-tuning, code execution in secure sandboxes, and self-reflective correction to achieve >90% accuracy on table- and code-intensive benchmarks (Jiang et al., 8 Sep 2025).
• Enterprise Orchestration: Z-Space deploys multi-agent architectures for dynamic test-data generation, demonstrating 96% reduction in token inference cost and 92% tool-invocation accuracy in production environments (He et al., 23 Nov 2025).
• Benchmarks: Empirical evaluation leverages complex, multi-turn, and multi-tool datasets and split metrics: ScaleMCP, StableToolBench, TicToc-v1, BFCL, TableMind, AgentThink, and domain-specific enterprise datasets (Cheng et al., 27 Oct 2025, Lumer et al., 29 Jul 2025, Jung et al., 2 Apr 2025, Jiang et al., 8 Sep 2025, Qian et al., 21 May 2025, Li et al., 29 Dec 2025).

Framework/Approach	Core Principle	Notable Metric/Result
AutoTool (Jia et al., 18 Nov 2025)	Tool-inertia graph/select	–30% LLM calls at equal or ↑completion
MemTool (Lumer et al., 29 Jul 2025)	Short-term memory pruning	0.9–0.94 tool-removal ratio in top models
AURA (Maben et al., 29 Jun 2025)	Cascaded S2S agentic pipeline	90% multi-turn speech task success
AgentThink (Qian et al., 21 May 2025)	CoT + tool-augmented RL	+51.9% reasoning/+33.5% accuracy
Z-Space (He et al., 23 Nov 2025)	Multi-agent, FSWW filtering	96% token ↓, 92% tool accuracy
InfTool (Li et al., 29 Dec 2025)	Closed-loop self-synthesis	70.9% BFCL, outperforms 10× larger models

7. Current Challenges and Research Directions

Despite rapid progress, several open challenges persist:

• Temporal Alignment: Temporal blindness remains a fundamental obstacle—explicit elapsed-time awareness and post-training alignment to human preferences are necessary for contextually correct tool use over extended dialogs (Cheng et al., 27 Oct 2025).
• Complex Multi-Tool Planning: Accurate orchestration of complex, dependent tool execution and recovery from partial failures require advanced graph-based planning and execution tracking, which is only partially addressed in current DAG-based frameworks (Lu et al., 28 Oct 2025, Li et al., 29 Dec 2025).
• Generalization and Scalability: Achieving robust generalization to new tools, domains, and stateful environments, while maintaining efficiency in search, memory, and orchestration, is an active research area (Jia et al., 18 Nov 2025, Li et al., 8 Dec 2025).
• Preference Transfer and Human Alignment: Contrasting preference-aligned learning and standard supervised or RL objectives, ongoing work investigates the limits and transferability of preference models for autonomous multi-turn invocation in multilingual, multi-agent, and high-stakes settings (Jung et al., 2 Apr 2025, Qian et al., 21 May 2025).
• Fully Autonomous Data Synthesis: Synthetic, closed-loop multi-agent self-play has proven effective for bootstrap data generation and continual improvement, but sim-to-real gaps and the injection of human-in-the-loop correction remain open issues (Li et al., 29 Dec 2025).

Autonomous multi-turn tool invocation is thus a multidimensional field fusing structured decision process modeling, memory optimization, agentic RL and preference optimization, and scalable deployment practicalities. Recent advances across these axes have already enabled substantial gains in robustness, efficiency, and autonomy, but new methods for temporal reasoning, generalization, and flexible orchestration are required to reach human-level and enterprise-grade performance.