KwaiAgents: Generalized Information-seeking Agent System with Large Language Models (2312.04889v3)

Abstract: Driven by curiosity, humans have continually sought to explore and understand the world around them, leading to the invention of various tools to satiate this inquisitiveness. Despite not having the capacity to process and memorize vast amounts of information in their brains, humans excel in critical thinking, planning, reflection, and harnessing available tools to interact with and interpret the world, enabling them to find answers efficiently. The recent advancements in LLMs suggest that machines might also possess the aforementioned human-like capabilities, allowing them to exhibit powerful abilities even with a constrained parameter count. In this paper, we introduce KwaiAgents, a generalized information-seeking agent system based on LLMs. Within KwaiAgents, we propose an agent system that employs LLMs as its cognitive core, which is capable of understanding a user's query, behavior guidelines, and referencing external documents. The agent can also update and retrieve information from its internal memory, plan and execute actions using a time-aware search-browse toolkit, and ultimately provide a comprehensive response. We further investigate the system's performance when powered by LLMs less advanced than GPT-4, and introduce the Meta-Agent Tuning (MAT) framework, designed to ensure even an open-sourced 7B or 13B model performs well among many agent systems. We exploit both benchmark and human evaluations to systematically validate these capabilities. Extensive experiments show the superiority of our agent system compared to other autonomous agents and highlight the enhanced generalized agent-abilities of our fine-tuned LLMs.

KwaiAgents: Generalized Information-Seeking Agent System with LLMs

The paper presents 'KwaiAgents', an advanced system leveraging LLMs to create a generalized information-seeking agent. This system navigates the complexity of user queries by integrating cognitive functionalities intrinsic to LLMs with a robust mechanism for information retrieval and memory storage. The system aims to replicate human-like inquiry processes through efficient planning and reflective actions, bolstering the capabilities of smaller, open-source models within autonomous agent frameworks.

System Architecture

The architecture of KwaiAgents is delineated into three primary components:

1. KAgentSys: The autonomous agent loop integrates memory banks, a toolkit library, and task modules to provide a cohesive environment for the agent's operation. The memory bank retains contextual interactions across a session, optimizing engagement by utilizing conversation memory, task history, and external knowledge sources. The tool library enriches this by offering both factual and time-aware toolsets, ensuring comprehensive information retrieval.
2. KAgentLMs with Meta-Agent Tuning: At its core, KwaiAgents investigates the efficacy of smaller LLMs in performing complex agent-related tasks traditionally reserved for larger models. It introduces a Meta-Agent Tuning (MAT) framework, applying refined template and prompt design for small-scale LLMs. This ensures these models exhibit competencies in planning, reflection, and tool utilization.
3. KAgentBench: This benchmark evaluates and verifies the agent's performance across tasks involving various agent system prompts, focusing on distinct capabilities. It systematically analyzes performance across several elements, from factual data retrieval to dynamic tool use.

Experimental Evaluation

The experiments conducted provide a rigorous evaluation of the system's components. The paper meticulously measures the impact of MAT on small open-source models, evidencing their enhanced capabilities in comparison to larger, commercial models. Metrics within KAgentBench serve as crucial indicators, emphasizing planning and tool-use proficiency.

The human evaluation further corroborates the system's efficiency. It demonstrates that the KwaiAgents system outperforms other models when applied to both standard and novel queries. The implicit advantage of MAT in adapting smaller models is evident, showcasing substantial performance improvements.

Implications and Future Directions

The implications of this work are manifold, emphasizing both practical applications in AI-driven assistance and theoretical advancements in LLM utilization. By enhancing the capabilities of smaller models, KwaiAgents addresses the resource constraints posed by larger systems, offering a scalable and adaptable solution. This advancement suggests a significant leap toward the development of highly efficient, resource-conscious AI systems.

The potential for future developments is vast. Enhancements could explore diverse data domains or integrate additional languages, expanding the scope and applicability of KwaiAgents. Further refinement in the tuning processes or integration of more sophisticated tools could push the boundaries of what these systems can achieve, leading us closer to realizing truly autonomous LLM-powered agents.

In conclusion, KwaiAgents stands as a testament to innovative approaches in harnessing LLMs for autonomous information-seeking tasks, providing a template for future research into efficient AI agent systems.