AgentOhana: Design Unified Data and Training Pipeline for Effective Agent Learning (2402.15506v4)

Abstract: Autonomous agents powered by LLMs have garnered significant research attention. However, fully harnessing the potential of LLMs for agent-based tasks presents inherent challenges due to the heterogeneous nature of diverse data sources featuring multi-turn trajectories. In this paper, we introduce \textbf{AgentOhana} as a comprehensive solution to address these challenges. \textit{AgentOhana} aggregates agent trajectories from distinct environments, spanning a wide array of scenarios. It meticulously standardizes and unifies these trajectories into a consistent format, streamlining the creation of a generic data loader optimized for agent training. Leveraging the data unification, our training pipeline maintains equilibrium across different data sources and preserves independent randomness across devices during dataset partitioning and model training. Additionally, we present \textbf{xLAM-v0.1}, a large action model tailored for AI agents, which demonstrates exceptional performance across various benchmarks. Begin the exploration at \url{https://github.com/SalesforceAIResearch/xLAM}.

The Agent Collection: Designing Unified Data and Training Pipeline for Effective Agent Learning

The exploration of LLM-augmented Autonomous Agents (LAAs) presents an intriguing frontier in AI research, where the inherent language processing capabilities of LLMs are harnessed to enhance autonomous agents' performance in complex task environments. This paper offers a systematic evaluation of different agent architectures and the performance implications of integrating various LLM backbones, aiming to provide comprehensive insights into optimizing LAAs.

Overview of LLM-Augmented Autonomous Agents

LAAs represent a nascent domain where autonomous agents make decisions and interact with environments leveraging LLMs' capacity to process and generate language. These agents benefit from past interactions, synthesizing observations and actions to address complex sequences in decision-making tasks. The authors provide a thorough comparative analysis of different LAA architectures, particularly focusing on how different LLMs facilitate agent interaction efficacy. Additionally, they propose a novel multi-agent orchestration model, BOLAA, which distributes task responsibilities among specialized agents to enhance performance.

Agent Architectures

Several LAA architectures are systematically analyzed, each tailored to different task requirements:

1. Zero Shot (ZS) and Zero Shot Think (ZST) LAA: ZS-LAA initiates action generation with zero-shot prompting, while ZST-LAA enhances this with intermediate reasoning steps.
2. ReAct LAA: Employs few-shot prompting to contextualize action generation, improving interaction efficacy.
3. PlanAct and PlanReAct LAA: These incorporate planning steps before action execution, with PlanReAct integrating reasoning before action generation.
4. BOLAA: Distinguishes itself by orchestrating multiple LAAs, each focusing on specific action types, coordinated by a central controller managing task allocation and inter-agent communication.

Experimental Results

The paper reports experiments conducted in two complex environments: WebShop for decision-making and HotPotQA for knowledge reasoning. Performance is assessed via reward metrics and recall rates, providing quantitative insights into the suitability of certain LAA architectures when paired with various LLMs.

Decision-Making Environment

In WebShop, BOLAA consistently outperformed other architectures, demonstrating significant improvements in reward scores. The strategy of distributing task responsibilities among specialist agents appears instrumental in this superior performance. Selecting optimal LLMs and leveraging BOLAA's architecture enhances performance, particularly in complex task scenarios. For instance, OpenAI's GPT models showed enhanced action generation capabilities under simpler ZS architectures, while planning flow benefited models like LongChat’s 13b variant significantly.

Knowledge Reasoning Environment

In the HotPotQA setting, ReAct LAA exhibited superior performance, indicating the necessity of few-shot examples in augmenting LLMs for complex reasoning tasks. Planning flows, typically advantageous in decision-making environments, can introduce detrimental effects on reasoning tasks due to pre-established plans not adapting well to emergent contexts.

Implications and Future Directions

The findings of this research provide valuable guidance on designing and deploying LAAs effectively. The results stress the importance of aligning agent architectures with suitable LLM models, identifying context length and model size as influential factors. The introduction of specialized agents, as demonstrated with BOLAA, offers a viable path forward in managing complex tasks efficiently.

Looking ahead, the field can anticipate further advancements in LAA capabilities through fine-tuning specialized agents and developing comprehensive benchmarks across differing task settings. As AI strategies become more sophisticated, orchestrating multiple agents with autonomous controllers, potentially imbued with reinforcement learning capabilities, represents a fertile area for exploration and enhancement.

By systematically evaluating complex AI systems across various architectures and environments, this paper provides a foundational approach to structuring and optimizing LLM-driven autonomous agents, fostering advances in both theoretical understanding and practical application.