Agent-Oriented Planning in Multi-Agent Systems

Abstract: Through the collaboration of multiple LLM-empowered agents possessing diverse expertise and tools, multi-agent systems achieve impressive progress in solving real-world problems. Given the user queries, the meta-agents, serving as the brain within multi-agent systems, are required to decompose the queries into multiple sub-tasks that can be allocated to suitable agents capable of solving them, so-called agent-oriented planning. In this study, we identify three critical design principles of agent-oriented planning, including solvability, completeness, and non-redundancy, to ensure that each sub-task can be effectively resolved, resulting in satisfactory responses to user queries. These principles further inspire us to propose AOP, a novel framework for agent-oriented planning in multi-agent systems, leveraging a fast task decomposition and allocation process followed by an effective and efficient evaluation via a reward model. According to the evaluation results, the meta-agent is also responsible for promptly making necessary adjustments to sub-tasks and scheduling. Besides, we integrate a feedback loop into AOP to further enhance the effectiveness and robustness of such a problem-solving process. Extensive experiments demonstrate the advancement of AOP in solving real-world problems compared to both single-agent systems and existing planning strategies for multi-agent systems. The source code is available at https://github.com/lalaliat/Agent-Oriented-Planning

• The paper presents a novel framework using a meta-agent to effectively decompose user queries into independently solvable sub-tasks.
• It implements key design principles—solvability, completeness, and non-redundancy—to ensure optimal task allocation among specialized agents.
• Experimental results indicate improved task accuracy and efficiency compared to traditional single-agent and other multi-agent planning methods.

Agent-Oriented Planning in Multi-Agent Systems

The paper "Agent-Oriented Planning in Multi-Agent Systems" presents a novel approach to task planning within multi-agent systems where coordination and task decomposition are handled by a specialized control entity known as the meta-agent. The study introduces principles for effective task planning and proposes an innovative framework that addresses the identified challenges in agent-oriented planning.

Introduction

In modern AI systems, LLMs are leveraged for their advanced capabilities in natural language understanding and generation. When integrated into multi-agent systems, such models empower individual agents to solve specific tasks using their unique expertise. However, the core challenge in multi-agent systems lies in the planning and allocation of tasks by the meta-agent, which breaks down complex user queries into sub-tasks assigned to specialized agents.

Challenges

The process of agent-oriented planning faces two main challenges:

1. Decomposition Complexity: A user query needs to be broken down into sub-tasks effectively, ensuring that the associated agents can handle these sub-tasks considering their capabilities. For example, tasks such as querying material properties might require precise information extraction from web resources, beyond typical search queries.
2. Task Allocation: Determining which agent is best suited for a given sub-task is difficult, especially when the agents' descriptive capabilities are not sufficiently detailed, leading to suboptimal performance.

   Figure 1: Examples of agent-oriented planning in multi-agent systems, regarding two challenges (left side) and three design principles (right side).

Proposed Framework

The paper introduces a comprehensive framework that addresses the challenges by relying on three fundamental design principles: solvability, completeness, and non-redundancy.

Design Principles

• Solvability: Each sub-task must be independently solvable by at least one agent.
• Completeness: Task decomposition must include all relevant information of the original user query.
• Non-Redundancy: Sub-tasks should be unique and necessary, avoiding overlaps.

Fast Decomposition and Allocation

The framework begins with a fast decomposition and allocation process, where the meta-agent uses LLM capabilities to draft an initial plan by considering the described abilities of each agent. This plan is then subjected to further evaluation and refinement to ensure adherence to the design principles.

Figure 2: Overall architecture of the proposed agent-oriented planning framework.

Evaluation with Reward Models

A reward model is deployed for evaluating the sub-tasks to determine if they meet the solvability principle without executing all sub-tasks across multiple agents, thus saving resources.

Sub-Task Modifications

Sub-tasks requiring further refinement are processed using "re-describe" or "plan-in-detail" strategies to enhance task descriptions or further decompose them into manageable chunks. This step ensures that each component of the task is tailored to the agents’ capabilities.

Experiments and Results

The framework was benchmarked against existing approaches using a dataset requiring multi-agent collaboration for reasoning tasks. Results indicated superior performance in achieving higher task accuracy and efficiency compared to single-agent systems and other multi-agent strategies.

Ablation Studies

Additional experiments demonstrated the necessity of each framework component, highlighting the significant impact of the detector on ensuring completeness and avoidance of redundancy.

Conclusion

The paper elaborates on a sophisticated framework for managing tasks within multi-agent systems by employing a robust agent-oriented planning methodology that encompasses task decomposition, evaluation, and feedback mechanisms. This framework significantly enhances the effectiveness of multi-agent systems in solving complex real-world problems by leveraging the diversity and expertise of LLM-empowered agents. Future work may explore further optimizations in real-time system instances and scalability to accommodate increasing complexity and diversity in agent capabilities.