A Dynamic LLM-Powered Agent Network for Task-Oriented Agent Collaboration

Abstract: Recent studies show that collaborating multiple LLM powered agents is a promising way for task solving. However, current approaches are constrained by using a fixed number of agents and static communication structures. In this work, we propose automatically selecting a team of agents from candidates to collaborate in a dynamic communication structure toward different tasks and domains. Specifically, we build a framework named Dynamic LLM-Powered Agent Network ($\textbf{DyLAN}$) for LLM-powered agent collaboration, operating a two-stage paradigm: (1) Team Optimization and (2) Task Solving. During the first stage, we utilize an $\textit{agent selection}$ algorithm, based on an unsupervised metric called $\textit{Agent Importance Score}$, enabling the selection of best agents according to their contributions in a preliminary trial, oriented to the given task. Then, in the second stage, the selected agents collaborate dynamically according to the query. Empirically, we demonstrate that DyLAN outperforms strong baselines in code generation, decision-making, general reasoning, and arithmetic reasoning tasks with moderate computational cost. On specific subjects in MMLU, selecting a team of agents in the team optimization stage improves accuracy by up to 25.0% in DyLAN.

• The paper introduces DyLAN, which leverages adaptive multi-round interactions and dynamic agent selection to enhance task-oriented collaboration.
• It implements inference-time agent selection and automatic team optimization using an unsupervised Agent Importance Score to efficiently allocate computational resources.
• The approach improves performance in reasoning and code generation tasks, achieving up to 35.7% accuracy on MATH and 82.9% Pass@1 on code generation benchmarks.

A Dynamic LLM-Powered Agent Network for Task-Oriented Agent Collaboration

Introduction

LLMs have demonstrated efficacy across various tasks, such as reasoning and code generation. Conventional approaches use static architectures with fixed roles for agent collaboration, limiting adaptability and requiring extensive human oversight. The paper introduces Dynamic LLM-Agent Network (DyLAN), which constructs adaptive, dynamic interaction architectures that enhance performance and efficiency. DyLAN incorporates inference-time agent selection and early-stopping mechanisms, allowing agents to interact over multiple rounds. It further employs an automatic agent team optimization algorithm based on Agent Importance Score, an unsupervised metric, selecting the most contributory agents.

Figure 1: Overview of DyLAN showing its feed-forward answer output mechanism.

Dynamic Interaction Architecture

DyLAN leverages a feed-forward network structure, treating agents and the messages they exchange as nodes and edges, respectively, in this formulation. Agents at different time steps serve as nodes, with their communications forming directed edges. The multi-round interactions are organized into multilayered architectures, allowing dynamic reconfiguration based on query-specific requirements. This enables task-agnostic and efficient interaction patterns, improving generalization and adaptability without relying on task-specific designs.

Inference-Time Agent Selection

DyLAN implements inference-time agent selection by employing an LLM-powered ranker to dynamically select top-performing agents based on their contributions during specified interaction rounds. Inefficient agents are deactivated to focus resources on promising responses, enhancing computational efficiency without sacrificing performance. This mechanism addresses the sensitivity issues of static agent setups and promotes consensus-reaching among agents, guided by Byzantine Consensus principles.

Agent Team Optimization

Agent team optimization in DyLAN relies on a three-step procedure involving propagation, aggregation, and selection. Agents rate predecessors' solutions, with these scores aggregated to quantify each agent's contribution over time. The computed Agent Importance Score facilitates selecting an optimal subset of agents, ensuring dynamic compositions tailored to specific tasks and domains. This approach circumvents manual role designation, allowing automatic adaptation and fostering robust team configuration.

Figure 2: Impact of optimized agent team size on MMLU dataset performance and efficiency.

Experimental Results

DyLAN demonstrates superior accuracy and efficiency in reasoning and code generation tasks compared to baseline methods. On MATH dataset, DyLAN achieves up to 35.7% overall accuracy, a significant improvement over single-execution baselines. In general reasoning with the MMLU dataset, accuracy benefits are notable, with DyLAN outperforming other approaches by margins reaching 4.1%. Code generation tasks display substantial gains, with DyLAN achieving 82.9% in Pass@1 metric, underscoring its proficiency in handling complex generative tasks.

Conclusion

DyLAN offers an innovative framework for collaborative agent networks powered by LLMs, resolving inefficiencies in static architectures and enhancing performance via dynamic interaction paradigms and optimized agent selection. It facilitates improved adaptability across diverse tasks, backed by empirical success demonstrated in varied applications. Future explorations may integrate DyLAN with open-source models and extend its utility to domains like software development or virtual interactions, further broadening its applicability and effectiveness.

Figure 3: DyLAN's robust performance across varying temperatures on MATH and HumanEval datasets.