Advanced Tool Learning and Selection System (ATLASS): A Closed-Loop Framework Using LLM (2503.10071v1)

Abstract: The combination of LLM agents with external tools enables models to solve complex tasks beyond their knowledge base. Human-designed tools are inflexible and restricted to solutions within the scope of pre-existing tools created by experts. To address this problem, we propose ATLASS, an advanced tool learning and selection system designed as a closed-loop framework. It enables the LLM to solve problems by dynamically generating external tools on demand. In this framework, agents play a crucial role in orchestrating tool selection, execution, and refinement, ensuring adaptive problem-solving capabilities. The operation of ATLASS follows three phases: The first phase, Understanding Tool Requirements, involves the Agents determining whether tools are required and specifying their functionality; the second phase, Tool Retrieval/Generation, involves the Agents retrieving or generating tools based on their availability; and the third phase, Task Solving, involves combining all the component tools necessary to complete the initial task. The Tool Dataset stores the generated tools, ensuring reusability and minimizing inference cost. Current LLM-based tool generation systems have difficulty creating complex tools that need APIs or external packages. In ATLASS, we solve the problem by automatically setting up the environment, fetching relevant API documentation online, and using a Python interpreter to create a reliable, versatile tool that works in a wider range of situations. OpenAI GPT-4.0 is used as the LLM agent, and safety and ethical concerns are handled through human feedback before executing generated code. By addressing the limitations of predefined toolsets and enhancing adaptability, ATLASS serves as a real-world solution that empowers users with dynamically generated tools for complex problem-solving.

• The paper presents a closed-loop framework (ATLASS) that dynamically analyzes, generates, and selects external tools to perform complex tasks using LLMs.
• It integrates components like Task Analyzer, Tool Master, and Tool Selector to reduce inference cost and ensure tool reuse via a structured JSON database.
• Experimental analysis shows that ATLASS improves task execution efficiency and adaptability compared to similar frameworks by minimizing redundancy.

Advanced Tool Learning and Selection System (ATLASS): A Closed-Loop Framework Using LLM

Overview of ATLASS Framework

ATLASS is designed to enhance the capabilities of LLMs by allowing them to dynamically generate and select external tools to solve complex tasks. This framework is divided into three critical phases: Tool Requirements Analysis, Tool Retrieval/Generation, and Task Execution. The integration of a tool database facilitates the reuse of generated tools, minimizing inference cost and enhancing efficiency.

Figure 1: Overview of ATLASS workflow, with Tool Requirements Analysis, Tool Retrieval/Generation, and Task Execution.

Tool Requirements Analysis

The first phase involves determining the necessity of external tools through two key components: Task Analyzer and Tool Master.

Task Analyzer

The Task Analyzer component breaks down user queries into actionable subtasks, enabling efficient problem-solving by identifying specific tool requirements. This stage ensures that all potential needs for external tools are recognized at an early stage.

Figure 2: Task Analyzer Pipeline

Tool Master

Tool Master evaluates the outputs from the Task Analyzer to decide whether external tools are required. It provides a structured JSON output specifying names and descriptions of necessary tools, which are then either retrieved from the existing database or generated anew.

Figure 3: Tool Master Pipeline

Tool Retrieval/Generation

Once the need for tools is identified, the tools are either retrieved from the Tool Dataset or generated using the Tool Generator component.

Tool Database

The Tool Database is an essential component of the ATLASS framework, storing all available tools in a JSON format, enabling efficient retrieval and reuse.

Tool Selector

The Tool Selector accesses the Tool Dataset to ascertain the availability of required tools. It distinguishes between tools that need to be generated and those that can be retrieved, thereby optimizing the tool generation process.

Figure 4: Tool Selector Pipeline

Tool Generator

Tool generation is carried out by two subcomponents, handling both non-API-based and API-based requirements. The non-API-based tool generation uses the Python environment for code execution, while API-based tools are generated by fetching current API documentation, ensuring up-to-date functionality.

Figure 5: Tool Generator Pipeline

Solving Tasks

The final phase focuses on task execution utilizing the previously retrieved or generated tools, managed by the Task Solver component.

Task Solver

Task Solver uses generated or retrieved tools to effectively execute the given tasks. When no tools are required, it relies on the LLM's internal knowledge base to provide solutions.

Figure 6: Task Solver Pipeline

Result Analysis

Extensive testing demonstrates that ATLASS significantly enhances task execution capabilities by dynamically generating reusable tools. Compared to frameworks like LATM, ATLASS supports more complex tasks, including those requiring API-based information retrieval and external library integration. The advanced tool selection mechanism also detects and reuses similar tools, enhancing adaptability.

Tool Selection and Generation

ATLASS's Tool Selector effectively identifies equivalent tools within the system, optimizing reuse and minimizing redundancy. Its Tool Generator component demonstrates high effectiveness in generating versatile tools integrated with external libraries and APIs.

Efficiency of Inference

The framework efficiently reduces inference cost by leveraging the Tool Database to minimize the tools generated for each query. Cost analysis indicates reduced token usage when pre-existing tools are available, demonstrating the economic advantage of the ATLASS framework.

Conclusion

ATLASS provides a robust solution for dynamically generating and selecting tools for complex task solving using LLMs. By incorporating real-time web search capabilities and promoting tool reusability, ATLASS improves efficiency, adaptability, and the practical application of autonomous agents across diverse domains.

Limitations and Future Work

ATLASS currently relies heavily on the OpenAI GPT-4.0 model, which may limit adaptability. Future enhancements could include optimizing the Tool Generator for increased complexity and security, as well as integrating open-source LLMs for less demanding tasks. Improvements in API integration and key security are also essential to ensure comprehensive system security.