SMART-LLM: Smart Multi-Agent Robot Task Planning using Large Language Models (2309.10062v2)

Abstract: In this work, we introduce SMART-LLM, an innovative framework designed for embodied multi-robot task planning. SMART-LLM: Smart Multi-Agent Robot Task Planning using LLMs, harnesses the power of LLMs to convert high-level task instructions provided as input into a multi-robot task plan. It accomplishes this by executing a series of stages, including task decomposition, coalition formation, and task allocation, all guided by programmatic LLM prompts within the few-shot prompting paradigm. We create a benchmark dataset designed for validating the multi-robot task planning problem, encompassing four distinct categories of high-level instructions that vary in task complexity. Our evaluation experiments span both simulation and real-world scenarios, demonstrating that the proposed model can achieve promising results for generating multi-robot task plans. The experimental videos, code, and datasets from the work can be found at https://sites.google.com/view/smart-LLM/.

Overview of SMART-LLM: Smart Multi-Agent Robot Task Planning using LLMs

The paper introduces an innovative framework, SMART-LLM (Smart Multi-Agent Robot Task Planning using LLMs), for task planning in heterogeneous multi-robot systems. The primary objective of SMART-LLM is to leverage the capabilities of LLMs for converting high-level natural language instructions into executable multi-robot task plans. The framework is designed to perform three critical stages of task planning: task decomposition, coalition formation, and task allocation, utilizing LLM prompts within the few-shot prompting paradigm. This approach is particularly advantageous in dealing with the complexities and ambiguities typically encountered in task descriptions.

Key Contributions

1. Multi-Robot Task Planning Framework: The core contribution of the research is the development of a framework that integrates task decomposition, coalition formation, and skill-based task allocation for multi-robot systems. The framework's ability to generate task plans using LLMs demonstrates promising outcomes for both simulated and real-world experiments.
2. Benchmark Dataset: The research provides a benchmark dataset intended for evaluating multi-agent task planning systems. The dataset encompasses tasks of varying complexity, ranging from elemental tasks manageable by a single robot to complex instances requiring strategic collaborations among a team of heterogeneous robots.
3. Comprehensive Evaluation: The methodology is tested extensively in both simulated environments (using AI2-THOR) and real-world settings, ensuring the framework's reliability and scalability across different scenarios.

Technical Approach

The proposed approach utilizes LLMs to perform the three key stages:

• Task Decomposition: The task decomposition process involves breaking down high-level instructions into temporally ordered sub-tasks based on robot skill sets and environmental constraints. These sub-tasks are then mapped to executable robot actions.
• Coalition Formation: In this stage, a strategic assignment of robots to sub-tasks takes place, ensuring the utilization of the optimal team of robots according to their individual capabilities. The inclusion of programming language prompts allows for robust team formation policies based on a robot's skill set and constraints.
• Task Allocation: Finally, task allocation involves assigning decomposed sub-tasks to specific robots, ensuring the execution sequence honors any team-based and skill-based requirements formulated in coalition policies.

Numerical Results

Simulation experiments conducted using extensive tests across varied task complexities demonstrate the framework's robustness and adaptability. Notably, SMART-LLM achieves high performance across elemental, simple, compound, and complex task categories with different LLM backbones such as GPT-4, GPT-3.5, Llama2, and Claude3. Its performance is quantitatively evaluated using metrics such as Success Rate (SR), Task Completion Rate (TCR), Goal Condition Recall (GCR), Robot Utilization (RU), and Executability (Exe).

Implications and Future Directions

SMART-LLM's potential implications are vast, particularly in enhancing task planning efficiency in complex environments and facilitating seamless transitions between simulated and real-world robot applications. The formal structure of handling task planning using LLMs lays the groundwork for further advancements in multi-agent systems, allowing researchers to develop methods that generalize across various task scenarios without requiring extensive retraining or code modifications.

Future research could focus on optimizing the framework by integrating dynamic task allocation strategies, leveraging multi-agent cooperative architectures, and employing smaller models for localized computing needs. Investigations into expanding the utility of LLMs in more intricate robotic tasks, especially those demanding higher autonomy and reasoning, present promising directions for exploration. Furthermore, improving error handling and adaptability in unforeseen environments would be crucial for advancing the prototype towards broader applications in the domain of autonomous robotics.