AgentLite: A Lightweight Library for Building and Advancing Task-Oriented LLM Agent System (2402.15538v1)

Abstract: The booming success of LLMs initiates rapid development in LLM agents. Though the foundation of an LLM agent is the generative model, it is critical to devise the optimal reasoning strategies and agent architectures. Accordingly, LLM agent research advances from the simple chain-of-thought prompting to more complex ReAct and Reflection reasoning strategy; agent architecture also evolves from single agent generation to multi-agent conversation, as well as multi-LLM multi-agent group chat. However, with the existing intricate frameworks and libraries, creating and evaluating new reasoning strategies and agent architectures has become a complex challenge, which hinders research investigation into LLM agents. Thus, we open-source a new AI agent library, AgentLite, which simplifies this process by offering a lightweight, user-friendly platform for innovating LLM agent reasoning, architectures, and applications with ease. AgentLite is a task-oriented framework designed to enhance the ability of agents to break down tasks and facilitate the development of multi-agent systems. Furthermore, we introduce multiple practical applications developed with AgentLite to demonstrate its convenience and flexibility. Get started now at: \url{https://github.com/SalesforceAIResearch/AgentLite}.

AgentLite: A Lightweight Library for Task-Oriented LLM Agent Systems

The paper "AgentLite: A Lightweight Library for Building and Advancing Task-Oriented LLM Agent System" introduces AgentLite, a streamlined framework designed to facilitate the rapid development and evaluation of task-oriented LLM agents. This document addresses the complexities involved in creating and refining reasoning strategies and agent architectures, an area currently impeded by intricate existing frameworks.

Overview of AgentLite

AgentLite is characterized by its lightweight architecture and flexibility, requirements vital for researchers who need to innovate quickly without the burden of extensive modifications. It operates within a task-oriented framework, split into two core components: the Individual Agent and the Manager Agent.

• Individual Agent: This is the foundational element of AgentLite. It incorporates modules for prompt generation, action execution, interaction with LLM, and memory management. The simplification of these components allows for the prototyping of new reasoning types—critical for exploring alternative cognitive processes in LLMs.
• Manager Agent: Extending from the Individual Agent, the Manager Agent is designed to orchestrate multi-agent systems, emphasizing task decomposition and collaboration among agents. It structures interactions through Task Packages, ensuring efficient agent communication.

Comparative Analysis

AgentLite is evaluated against libraries such as AutoGen and LangChain, highlighting its simplicity and efficiency. With under 1,000 lines of core code, it provides comprehensive functionality including task decomposition and multi-agent orchestration, making it versatile for diverse research scenarios.

Experimental Results

Quantitative analysis using AgentLite was conducted on benchmarks including HotPotQA for retrieval-augmented question answering and WebShop for decision-making in a simulated online shopping environment. The library demonstrated the ability to integrate with advanced models like GPT-4 and xLAM, showcasing flexibility across different model architectures and providing competitive performance metrics.

• HotPotQA Results: AgentLite enabled models like GPT-4-32k to achieve substantial F1-Scores and accuracy, particularly in medium complexity tasks. The xLAM model demonstrated the effectiveness of fine-tuning LLMs on action trajectories.
• WebShop Performance: The library facilitated agents to meet or exceed the performance of models like GPT-3.5, reaffirming its practical adaptability and lightweight design.

Practical Applications

AgentLite supports various applications ranging from creative projects like an Online Painter to interactive problem-solving in mathematics. These implementations underscore the library's capability to handle both single-agent and complex multi-agent systems. The flexibility of AgentLite is further demonstrated in dynamic applications like online chess games and philosophical discussions, illustrating its potential for educational and exploratory AI development.

Implications and Future Directions

AgentLite positions itself as a pivotal tool for researchers aiming to explore new frontiers in LLM agent architectures. Its lightweight nature allows for rapid iteration and exploration of reasoning strategies. Future enhancements mentioned by the authors include expanding agent communication methods and broadening the repertoire of reasoning types, developments which could significantly impact the evolution of AI agents.

In conclusion, AgentLite offers a significant contribution by meeting the dual demands of simplicity and functionality, fostering innovation in the rapidly evolving domain of task-oriented LLM agents.