ToolSandbox: A Stateful, Conversational, Interactive Evaluation Benchmark for LLM Tool Use Capabilities

Abstract: Recent LLMs advancements sparked a growing research interest in tool assisted LLMs solving real-world challenges, which calls for comprehensive evaluation of tool-use capabilities. While previous works focused on either evaluating over stateless web services (RESTful API), based on a single turn user prompt, or an off-policy dialog trajectory, ToolSandbox includes stateful tool execution, implicit state dependencies between tools, a built-in user simulator supporting on-policy conversational evaluation and a dynamic evaluation strategy for intermediate and final milestones over an arbitrary trajectory. We show that open source and proprietary models have a significant performance gap, and complex tasks like State Dependency, Canonicalization and Insufficient Information defined in ToolSandbox are challenging even the most capable SOTA LLMs, providing brand-new insights into tool-use LLM capabilities. ToolSandbox evaluation framework is released at https://github.com/apple/ToolSandbox

• The paper introduces ToolSandbox, a dynamic benchmark incorporating stateful execution and implicit dependencies to assess LLM tool-use capabilities.
• It employs Milestones and Minefields to evaluate multi-turn, conversational tasks and reveals performance differences between proprietary and open-source models.
• The evaluation highlights LLM limitations in reasoning and state tracking, urging improvements in model architectures and training strategies.

ToolSandbox: A Comprehensive Evaluation Benchmark for LLM Tool-Use Capabilities

ToolSandbox introduces a novel benchmark for evaluating the tool-use capabilities of LLMs. It addresses key limitations in existing benchmarks by incorporating stateful tool execution, implicit dependencies, and a dynamic evaluation strategy. The following sections explore the design, implementation, and implications of ToolSandbox, focusing on its various components and evaluation metrics.

Introduction to ToolSandbox

ToolSandbox aims to evaluate the real-world tool-use capabilities of LLMs, an area of growing research interest. Unlike previous benchmarks that rely on stateless interactions or off-policy dialogues, ToolSandbox introduces a stateful environment with implicit state dependencies. It evaluates LLMs in a conversational and interactive setting, providing insights into how these models handle complex, multi-turn tasks.

Figure 1: An example evaluation trajectory from ToolSandbox, illustrating the interaction between User, Agent, and Execution Environment.

Architecture and Design of ToolSandbox

Core Components

ToolSandbox is a Python-native testing environment, featuring the following components:

• Execution Context: Acts as the world state abstraction, containing mutable database snapshots and a Message Bus for role communication.
• Tools: Highly composable Python functions that interact with the world state. They are designed to test complex reasoning scenarios and depend on world states, requiring the LLM to manage state changes effectively.
• Roles and Message Bus: Includes User, Agent, and Execution Environment roles, with interactions facilitated through a message passing system. Each role has specific visibility into the Message Bus, influencing its interaction capabilities.

  Figure 2: Interaction between the User, Agent, and the Execution Environment.

Evaluation Strategy

ToolSandbox employs an innovative evaluation strategy based on Milestones and Minefields:

• Milestones: Represent critical steps that must occur to achieve a goal. They allow for evaluating the sequence and correctness of tool interactions in a task.
• Minefields: Define events that must not happen, preventing illogical actions and hallucinations in task completion.

This approach provides rich intermediate and final execution signals, enabling a deep understanding of model performance.

Evaluation Results and Model Comparisons

ToolSandbox highlights a significant performance gap between proprietary and open-source models. Proprietary models like GPT-4o excel in complex tasks but still face challenges in scenarios like State Dependency and Canonicalization. The benchmark reveals that LLMs often struggle with tasks involving implicit dependencies and require improved reasoning and state-tracking capabilities.

Figure 3: Example GPT-4 trajectory for Insufficient Information category Minefield Evaluation.

Implications and Future Directions

ToolSandbox's detailed evaluation metrics and diverse test scenarios offer new insights into the capabilities and limitations of LLMs in real-world tool use. The challenging nature of state dependencies and the requirement for accurate canonicalization highlight the need for improved LLM architectures and training strategies. ToolSandbox serves as a valuable resource for advancing research in LLM tool-use capabilities and can inspire future developments in this rapidly evolving field.

Conclusion

ToolSandbox redefines the evaluation of LLM tool-use capabilities by introducing stateful, conversational, and interactive test scenarios. It provides a comprehensive framework for understanding the strengths and weaknesses of current models and sets the stage for future advancements in this area. Through its innovative design and robust evaluation metrics, ToolSandbox contributes significantly to the field of AI research, promoting the development of more capable and efficient tool-use LLMs.