AutoAgents: A Framework for Automatic Agent Generation (2309.17288v3)

Abstract: LLMs have enabled remarkable advances in automated task-solving with multi-agent systems. However, most existing LLM-based multi-agent approaches rely on predefined agents to handle simple tasks, limiting the adaptability of multi-agent collaboration to different scenarios. Therefore, we introduce AutoAgents, an innovative framework that adaptively generates and coordinates multiple specialized agents to build an AI team according to different tasks. Specifically, AutoAgents couples the relationship between tasks and roles by dynamically generating multiple required agents based on task content and planning solutions for the current task based on the generated expert agents. Multiple specialized agents collaborate with each other to efficiently accomplish tasks. Concurrently, an observer role is incorporated into the framework to reflect on the designated plans and agents' responses and improve upon them. Our experiments on various benchmarks demonstrate that AutoAgents generates more coherent and accurate solutions than the existing multi-agent methods. This underscores the significance of assigning different roles to different tasks and of team cooperation, offering new perspectives for tackling complex tasks. The repository of this project is available at https://github.com/Link-AGI/AutoAgents.

• The paper introduces a novel framework that automatically generates specialized agents for complex tasks through a two-stage process.
• It employs self-refinement and collaborative refinement mechanisms to continuously optimize agent performance.
• Experimental evaluations demonstrate that AutoAgents outperforms traditional multi-agent systems in benchmarks such as creative writing and software development.

AutoAgents: A Framework for Automatic Agent Generation

The paper "AutoAgents: A Framework for Automatic Agent Generation" introduces a novel framework designed to address the limitations of predefined multi-agent systems by dynamically generating and coordinating specialized agents to form an AI team tailored to specific tasks. This document provides a structured exploration of the AutoAgents framework, practical implementation advice, and an evaluation of its efficacy in real-world scenarios.

Framework Overview

AutoAgents is structured into two primary stages: the Drafting Stage and the Execution Stage. During the Drafting Stage, several predefined agents collaborate to generate a list of task-specific agents and a detailed execution plan. The Execution Stage involves the refined application of these plans by utilizing self-refinement and collaborative refinement mechanisms to further improve task performance.

Figure 1: A schematic diagram of AutoAgents. The system takes the user input as a starting point and generates a set of specialized agents for novel writing, along with a corresponding execution plan. The agents collaboratively carry out the tasks according to the plan and produce the final novel. Meanwhile, an observer monitors the generation and execution of the Agents and the plan, ensuring the quality and coherence of the process.

Figure 2: The execution process of AutoAgents. During the Drafting Stage, three predefined agents collaboratively determine the list of agents and the execution plan. During the Execution Stage, a predefined agent facilitates coordination and communication among the generated agent teams, and the individual generated agents enhance their execution efficiency through self-refinement.

Key Components and Design

Drafting Stage

The Drafting Stage emphasizes the dynamic generation of agents, the construction of execution plans, and the synthesis of heterogeneous information to handle complex tasks. By employing predefined agents like the Planner, Agent Observer, and Plan Observer, AutoAgents aims to create well-balanced agent teams capable of effective collaboration.

Execution Stage

The Execution Stage employs two significant actions:

• Self-refinement: It allows individual agents to improve task-specific competencies, enabling continuous enhancement of their abilities.
• Collaborative refinement: Agents engage in knowledge exchange, leveraging their interdisciplinary expertise to accomplish complex tasks collectively.

  Figure 3: Two types of actions for executing tasks: Self-refinement enables an individual agent to enhance its competence in performing some specialized tasks. Collaborative refinement facilitates knowledge exchange among multiple agents and accomplishes tasks that demand interdisciplinary expertise.

Knowledge Sharing Mechanisms

The framework employs a sophisticated memory system to optimize information handling and task execution, consisting of:

• Long-term memory: Maintains a collection of historical outputs from diverse actions.
• Short-term memory: Captures details of the immediate action phase, ensuring relevant information is retained throughout agents' self-refinement and collaborative efforts.
• Dynamic memory: Utilizes extracted critical data tailored to specific tasks, enabling efficient problem-solving.

  Figure 4: Legend of Three Knowledge Sharing Mechanisms. (a) Long-term memory focuses on chronicling the historical trajectory of multiple actions. (b) Short-term memory records the history of the self-refinement or collaborative refinement phases of an individual action. (c) Dynamic memory serves actions necessitating specialized attention extracted from the long-term memory.

Experimental Evaluation

AutoAgents demonstrated significant improvements over existing multi-agent frameworks across various benchmarks, including open-ended question answering and trivia creative writing tasks. Notably, AutoAgents consistently achieved higher win rates compared to GPT-4, Vicuna-13B, and ChatGPT. This superiority stems from the effective utilization of dynamic agent generation, self-refinement, and collaborative refinement strategies.

Case Study: Software Development

A practical application of AutoAgents is illustrated in the development of Python-based software for the Tetris game, where the system autonomously generated roles such as game design experts, programmers, and UI designers. The implementation exemplified AutoAgents' capacity to facilitate seamless collaboration and comprehensive problem-solving across diverse roles.

Figure 5: Comparison of whether there is a collaborative discussion in the Drafting Stage in the task that developing Python-based software for the Tetris game.

Conclusion

AutoAgents presents a significant advancement in multi-agent systems by offering a robust, adaptable framework capable of addressing complex tasks through automatic agent generation and coordination. By integrating dynamic roles and fostering collective agent refinement, AutoAgents enhances the capability and reliability of AI teams. Future research can focus on expanding the application domains of AutoAgents and potentially enhancing other frameworks with similar adaptive systems.