Investigate-Consolidate-Exploit: A General Strategy for Inter-Task Agent Self-Evolution (2401.13996v1)

Abstract: This paper introduces Investigate-Consolidate-Exploit (ICE), a novel strategy for enhancing the adaptability and flexibility of AI agents through inter-task self-evolution. Unlike existing methods focused on intra-task learning, ICE promotes the transfer of knowledge between tasks for genuine self-evolution, similar to human experience learning. The strategy dynamically investigates planning and execution trajectories, consolidates them into simplified workflows and pipelines, and exploits them for improved task execution. Our experiments on the XAgent framework demonstrate ICE's effectiveness, reducing API calls by as much as 80% and significantly decreasing the demand for the model's capability. Specifically, when combined with GPT-3.5, ICE's performance matches that of raw GPT-4 across various agent tasks. We argue that this self-evolution approach represents a paradigm shift in agent design, contributing to a more robust AI community and ecosystem, and moving a step closer to full autonomy.

• The paper introduces the ICE framework, which transfers diverse task experiences into streamlined, automated workflows.
• It consolidates successful planning and execution trajectories into finite automata, reducing API calls by up to 80%.
• Experimental results show that ICE, when combined with GPT-3.5, achieves performance on par with GPT-4, enhancing AI autonomy.

Introduction

A novel strategy named Investigate-Consolidate-Exploit (ICE) is introduced for improving the adaptability and flexibility of AI agents through inter-task self-evolution. This approach diverges from the prevalent intra-task learning methods and promotes the transfer of knowledge between tasks, aiming to emulate human experiential learning. ICE investigates planning and execution trajectories, consolidates them into simplified workflows and pipelines, and subsequently exploits these constructs to enhance task execution.

Investigate

The initial phase, Investigation, seeks out experiences worthy of reference, tracking the planning and execution status of each decomposed goal and extracting successful execution trajectories. As highlighted in the paper, traditional methods have centered on intra-task learning and the replication of execution processes, which prove to be largely ineffective in self-evolution scenarios. The ICE strategy instead emphasizes the significance of discontinuous experience for re-utilization, promoting a methodology akin to human cognitive flexibility and problem-solving skills obtained through diverse experiences.

Consolidate

Following the investigation, the Consolidation phase involves standardizing these experiences to automate and streamline future re-utilization. This entails pruning successful execution trajectories into a linear workflow and transforming them into finite automata for automated application. These structured experiences are then stored as the agent's memory, ready for future use. A key component of this strategy is the decoupling of the agent's planning and execution experiences, which maximizes their utility in a wide array of subsequent tasks.

Exploit

The final phase, Exploitation, leverages consolidated experiences, optimizing efficiency and effectiveness in the completion of new tasks. This stage equips the agent to expediently access and utilize prior consolidated experiences, with consolidated plans serving as in-context references for generating and refining new plans, and previously established trajectories directly applied for automated execution. Experimental evidence presented demonstrates the ICE strategy's capacity to reduce API calls by up to 80% and notably decrease the dependency on the model's inherent capabilities. Notably, when paired with GPT-3.5, the performance of ICE rivals that of GPT-4 across a spectrum of agent tasks. These results argue for a transformative shift in agent design, dimishing the barriers to deployment and moving toward greater autonomy.

Research in this domain suggests a potential paradigm shift in agent design, advancing the AI community and its ecosystem toward full autonomy. The ICE self-evolution approach exemplifies this shift, offering a heuristic for agents to evolve independently through accumulated experiences.