End-to-End Tool-Use Trajectories

• End-to-end tool-use trajectories are defined as a complete sequence of tool interactions where an agent plans, selects, parameterizes, and executes multiple actions.
• These trajectories incorporate both parallel and sequential patterns, enabling independent execution of tools and chaining outputs as inputs in complex workflows.
• Diagnostic metrics such as exact-match, tool usage, and trajectory satisfaction reveal key failure modes like tool confusion and parameter errors, guiding system improvements.

End-to-end tool-use trajectories refer to the entire process by which an intelligent system—robotic or agent-based—plans, selects, parameterizes, sequences, and executes a series of tool interactions to achieve a complex objective. Unlike approaches that evaluate only the final outcome, trajectory-aware methodologies track each step in these multi-turn workflows, allowing for comprehensive analysis of tool selection, argument correctness, and dependency satisfaction. This concept is central to agentic AI systems and robotics, where successful completion of tasks often requires not just the right result but correct intermediate decisions over diverse tools and APIs. Recent advances have highlighted the need for precise metrics and diagnostic frameworks to robustly benchmark and analyze end-to-end agentic tool use.

1. Definition and Scope of Tool-Use Trajectories

End-to-end tool-use trajectories encompass the ordered sequence of tool invocations—including selection, parameterization, and execution—performed by an agent or system to accomplish a multi-step task. In real-world settings, such trajectories typically involve high-fidelity tool environments: production APIs (e.g., travel, finance, education), physical robotic tools, or simulated control interfaces. Trajectory breadth refers to parallel tool calls (multiple independent actions within a task), whereas trajectory depth captures sequential dependencies (where output from one tool feeds into subsequent tool invocations).

TRAJECT-Bench (He et al., 6 Oct 2025) formalizes this by constructing tasks grounded in executable APIs, synthesizing both parallel and sequential multi-call trajectories. Each trajectory is analyzed for agentic planning ability, from tool selection to argument binding and ordered execution, emphasizing comprehensive evaluation over short, mid-length, and long-horizon workflows.

2. Evaluation Metrics for Trajectory Assessment

Precise evaluation of end-to-end tool-use trajectories necessitates multi-dimensional metrics beyond simple final accuracy. TRAJECT-Bench introduces a suite of diagnostic scores, summarized as follows:

Metric Name	Assessment Focus	Diagnostic Purpose
Final Answer Acc	Match to ground-truth output	Task outcome correctness
Trajectory EM	Exact tool sequence match (by name)	Planning fidelity
Inclusion	Proportion of ground-truth tools used	Coverage assessment
Tool Usage	Parameter correctness	Argument configuration errors
Traj-Satisfy	LLM judge trace satisfaction	Stepwise execution validity
Retrieval Rate	Gold tool retrieval via modules	Selection recall in retrieval mode

Exact-match (EM) and Inclusion metrics quantify the agent’s ability to select and order tools as in the reference trajectory, while Tool Usage pinpoints errors in parameter values or types, uncovering failures such as parameter-blind selection. Traj-Satisfy uses LLM-based evaluation (e.g., Claude-4) for cases without explicit gold traces, providing coverage of implicit dependencies within a trajectory. Retrieval Rate is crucial in retrieval-augmented architectures, measuring whether latent intent is correctly mapped to available tool sets, especially on hard queries.

3. Trajectory Construction: Parallel and Sequential Patterns

TRAJECT-Bench synthesizes trajectories exhibiting both parallel and sequential structural patterns. Parallel trajectories involve multiple tools invoked independently, each with fully specified inputs; for example, initiating separate calls for hotel search, flight search, and weather retrieval. Breadth here relates to the set of invoked tools.

Sequential trajectories model interdependent tool calls, with outputs from one feeding as parameters into the next. A directed graph $G_t = (V, E)$ formalizes this, where edges $t_1 \to t_2$ capture that $t_1$'s output is used as $t_2$'s input. Sequential depth captures increasing planning complexity, as agents must resolve argument binding and ordering against ground-truth dependencies, a setting where many LLM-based agents reveal bottlenecks especially as chaining length rises.

The benchmark stresses both parallel and sequential queries to rigorously evaluate agents’ capacity for reasoning, planning, and execution over real production-style tool sets.

4. Diagnostics: Failure Modes and Bottlenecks

Trajectory-level diagnostics in TRAJECT-Bench reveal systematic failure modes in current agentic architectures:

• Similar Tool Confusion: Agents often misselect tools with overlapping semantics but differ in API scope or argument specification (e.g., Spotify Search vs. YouTube Music Search). This points to weak API disambiguation in current LLMs.
• Parameter-Blind Selection: Correct tool types may be chosen, yet arguments are malformed or missing, leading to downstream trajectory errors.
• Redundant Tool Calling: Empirical analyses show agents sometimes hallucinate extra tool calls ("cover all bases"), introducing noise and increasing latency.
• Implied Queries: Hard queries with indirect requirements result in poor tool selection and parameter inference, as retrieval modules often fail to map query intent to set of appropriate tools.
• Scaling Bottlenecks: Exact-match and trajectory satisfaction metrics reveal steep performance degradations as the number of tools rises (especially 3 → 5 calls), with trajectory length and dependency order serving as principal bottlenecks. Retrieval rates also diminish significantly under increased tool library diversity and complexity.

5. Methods for Enhancing Trajectory Fidelity

The observed limitations motivate targeted improvements:

• Training on Sequential, Dependency-Aware Planning: Emphasizing trajectory-level supervision during model fine-tuning and RL promotes learning both selection and ordering of tools.
• Enhanced Parameter Sensitivity: Training should explicitly focus on correct argument values/types to mitigate parameter-blind errors.
• API Disambiguation: Including more nuanced tool descriptions or signatures aids agents in distinguishing overlapping tools, reducing confusion.
• Advanced Retrieval: Improving the semantic depth of retrieval mechanisms is recommended, especially for complex, indirect queries.
• Iterative Agentic Frameworks: Methods such as ReAct (reasoning-then-acting iteratively with dynamic retrieval per turn) improve performance on complex, long-horizon trajectories by integrating reasoning and execution in each cycle.

6. Implications, Scaling Behavior, and Future Directions

TRAJECT-Bench analyses reveal that as task complexity scales, current LLM agents face persistent bottlenecks, particularly in mid-length trajectories with interdependent tool calls, and with large tool libraries. This suggests a pressing need for agent architectures and training paradigms capable of handling long-context reasoning and sophisticated dependency resolution.

Actionable guidance includes intensive focus on training trajectories involving dependency chains, refining argument configuration, better retrieval strategies for latent intent, and integrating iterative reasoning-execution frameworks. A plausible implication is that future agent frameworks will require hybrid supervision and RL tailored to trajectory-aware diagnostics, with emphasis on scaling behavior and error analysis.

7. Conclusions

End-to-end tool-use trajectories, as elucidated by TRAJECT-Bench (He et al., 6 Oct 2025), represent a rigorous and comprehensive approach to evaluating and understanding agentic tool use. By going beyond final answers and focusing on the entirety of the tool-use trajectory—including selection, argument correctness, dependency, and ordering—researchers can diagnose failure modes, track scaling bottlenecks, and derive actionable methods for improvement. This trajectory-centric perspective forms the foundation for advancing agentic reasoning and planning in increasingly complex, multi-tool environments, both in LLM agents and embodied robotic systems.