A Comprehensive Survey of Self-Evolving AI Agents: A New Paradigm Bridging Foundation Models and Lifelong Agentic Systems (2508.07407v1)

Abstract: Recent advances in LLMs have sparked growing interest in AI agents capable of solving complex, real-world tasks. However, most existing agent systems rely on manually crafted configurations that remain static after deployment, limiting their ability to adapt to dynamic and evolving environments. To this end, recent research has explored agent evolution techniques that aim to automatically enhance agent systems based on interaction data and environmental feedback. This emerging direction lays the foundation for self-evolving AI agents, which bridge the static capabilities of foundation models with the continuous adaptability required by lifelong agentic systems. In this survey, we provide a comprehensive review of existing techniques for self-evolving agentic systems. Specifically, we first introduce a unified conceptual framework that abstracts the feedback loop underlying the design of self-evolving agentic systems. The framework highlights four key components: System Inputs, Agent System, Environment, and Optimisers, serving as a foundation for understanding and comparing different strategies. Based on this framework, we systematically review a wide range of self-evolving techniques that target different components of the agent system. We also investigate domain-specific evolution strategies developed for specialised fields such as biomedicine, programming, and finance, where optimisation objectives are tightly coupled with domain constraints. In addition, we provide a dedicated discussion on the evaluation, safety, and ethical considerations for self-evolving agentic systems, which are critical to ensuring their effectiveness and reliability. This survey aims to provide researchers and practitioners with a systematic understanding of self-evolving AI agents, laying the foundation for the development of more adaptive, autonomous, and lifelong agentic systems.

• The paper introduces a unified framework outlining a four-stage evolutionary trajectory for self-evolving AI agents.
• It details training, prompt, memory, and tool optimization techniques to enhance adaptability and performance.
• The work highlights challenges in safety, alignment, and scalability while outlining future research directions.

Self-Evolving AI Agents: Bridging Foundation Models and Lifelong Agentic Systems

Introduction

The surveyed work provides a comprehensive synthesis of the emerging paradigm of self-evolving AI agents, which integrate the static capabilities of foundation models—primarily LLMs—with the continuous adaptability required for lifelong agentic systems. The survey formalizes the conceptual and technical landscape, introduces a unified framework for agent self-evolution, and systematically reviews the optimization techniques, evaluation methodologies, and open challenges in this domain. The analysis is grounded in the context of the transition from static, manually configured architectures to dynamic, autonomous, and self-improving agentic ecosystems.

Conceptual Framework and Evolutionary Trajectory

The paper delineates a four-stage evolutionary trajectory for LLM-centric agentic systems:

1. Model Offline Pretraining (MOP): Foundation models are pretrained on static corpora and deployed in a frozen state.
2. Model Online Adaptation (MOA): Post-deployment adaptation via supervised fine-tuning, adapters, or RLHF, enabling limited online learning.
3. Multi-Agent Orchestration (MAO): Coordination of multiple LLM agents through message exchange and workflow design, without modifying model parameters.
4. Multi-Agent Self-Evolving (MASE): Lifelong, closed-loop self-evolution where agent populations autonomously refine prompts, memory, tool-use, and interaction patterns based on environmental feedback and meta-rewards.

This progression is formalized in a unified conceptual framework comprising four core components: System Inputs, Agent System, Environment, and Optimisers. The framework abstracts the iterative feedback loop that underpins self-evolving agentic systems, supporting both single-agent and multi-agent settings.

Single-Agent Optimization

LLM Behaviour Optimization

• Training-based: Supervised fine-tuning (e.g., STaR, NExT, DeepSeek-Prover) and RL (e.g., DPO, self-rewarding, Absolute Zero, R-Zero) are leveraged to enhance reasoning, planning, and tool-use capabilities. Notably, RL-based approaches such as DeepSeek-R1 and Absolute Zero demonstrate that self-play and meta-reward mechanisms can drive significant improvements in reasoning without external supervision.
• Test-time: Search-based strategies (e.g., CoT-SC, Tree-of-Thoughts, Graph-of-Thoughts) and feedback-based methods (e.g., verifier modules, process reward models) enable inference-time optimization without parameter updates, scaling reasoning via increased compute and structured exploration.

Prompt Optimization

• Edit-based: Local search via token/phrase edits (e.g., GRIPS, TEMPERA).
• Generative: LLM-driven prompt generation guided by optimization signals, meta-prompts, or search strategies (e.g., PromptAgent, OPRO, MIPRO).
• Text gradient-based: Natural language feedback as "text gradients" for iterative prompt improvement (e.g., ProTeGi, TextGrad).
• Evolutionary: Population-based search with mutation/crossover (e.g., EvoPrompt, Promptbreeder).

Memory Optimization

• Short-term: Summarization, selective retention, and context filtering (e.g., COMEDY, ReadAgent, MemoryBank) address context window limitations and support local coherence.
• Long-term: Retrieval-augmented generation (RAG), agentic retrieval, and structured memory (e.g., A-MEM, MemGPT, GraphReader) enable persistent knowledge retention and cross-session generalization. Memory control mechanisms (e.g., reinforcement learning, prioritization) are critical for scalable, adaptive memory management.

Tool Optimization

• Training-based: SFT and RL on tool-use trajectories (e.g., ToolLLM, Confucius, ReTool, ToolRL, Nemotron-Research-Tool-N1) embed tool-use policies in the LLM.
• Inference-time: Prompt-based (e.g., EASYTOOL, PLAY2PROMPT) and reasoning-based (e.g., ToolChain, Tool-Planner) methods optimize tool documentation and selection at test time.
• Tool creation: Agents autonomously generate or adapt tools (e.g., CREATOR, LATM, CRAFT, AgentOptimizer, Alita), extending the action space and supporting open-ended problem solving.

Multi-Agent Optimization

Workflow and Topology Optimization

• Manual design: Parallel, hierarchical, and debate-based workflows encode domain knowledge but are brittle and costly to maintain.
• Self-evolving systems: Automated search over prompts, topologies, and agent roles using RL, MCTS, evolutionary algorithms, and learning-based controllers (e.g., AutoFlow, AFlow, ScoreFlow, GPTSwarm, DynaSwarm, G-Designer, EvoAgent, EvoFlow, MASS, MaAS, ANN).
• Unified optimization: Joint search over prompts and topologies, often using code as a universal representation (e.g., ADAS, FlowReasoner), or explicit evolutionary/learning-based coordination.
• LLM backbone optimization: Multi-agent fine-tuning and RL (e.g., Sirius, MALT, MaPoRL, MARFT, MARTI) enhance reasoning and collaboration capabilities, with evidence of emergent cooperative behaviors.

Domain-Specific Optimization

Biomedicine

• Medical diagnosis: Multi-agent simulation and collaboration (e.g., MedAgentSim, PathFinder, MDAgents, MDTeamGPT) support multi-modal, interactive, and evidence-based reasoning.
• Molecular discovery: Integration of cheminformatics tools, memory-enabled reasoning, and multi-agent coordination (e.g., CACTUS, LLM-RDF, ChemAgent, OSDA Agent, DrugAgent, LIDDIA) enable interpretable and scalable symbolic reasoning.

Programming

• Code refinement: Self-feedback, collaborative workflows, and tool integration (e.g., Self-Refine, AgentCoder, CodeAgent, CodeCoR, OpenHands) support iterative code improvement and maintainability.
• Code debugging: Modular agent architectures and runtime feedback (e.g., Self-Debugging, Self-Edit, PyCapsule, RGD, FixAgent) enable autonomous, execution-aware fault correction.

Finance and Law

• Financial decision-making: Modular, multi-agent architectures (e.g., FinCon, PEER, FinRobot) integrate expert knowledge, tool use, and sentiment analysis for robust, context-aware decision support.
• Legal reasoning: Role-based, collaborative frameworks (e.g., LawLuo, AgentCourt, LegalGPT, AgentsCourt) simulate judicial processes and enforce rule-grounded, interpretable reasoning.

Evaluation Methodologies

• Benchmark-based: Diverse benchmarks assess tool use, web navigation, multi-agent collaboration, GUI/multimodal environments, and domain-specific tasks (e.g., ToolBench, WebArena, MultiAgentBench, SWE-bench, AgentClinic).
• LLM-based evaluation: LLM-as-a-Judge and Agent-as-a-Judge paradigms provide scalable, flexible evaluation of outputs and reasoning trajectories, with multi-agent deliberation frameworks (e.g., CollabEval) improving reliability.
• Safety, alignment, and robustness: Continuous, evolution-aware evaluation is required to ensure compliance with safety constraints, legal standards, and alignment objectives. Risk-focused benchmarks (e.g., AgentHarm, RedCode, MACHIAVELLI) and meta-evaluation approaches (e.g., AgentEval, R-Judge) are critical for monitoring and mitigating emergent risks.

Open Challenges and Future Directions

The survey identifies several persistent challenges:

• Safety and alignment: Dynamic evolution undermines static legal and safety frameworks, necessitating evolution-aware audits and adaptive regulatory mechanisms.
• Reward modeling and optimization stability: Scarcity and inconsistency of feedback signals can destabilize agent behavior.
• Evaluation in scientific and domain-specific contexts: Absence of reliable ground truth complicates optimization and assessment.
• Efficiency-effectiveness trade-offs: Large-scale MAS optimization incurs significant computational costs; explicit modeling of resource constraints is needed.
• Transferability and generalization: Optimized prompts and topologies often fail to generalize across backbones and domains.
• Multimodal and spatial reasoning: Most optimization algorithms are text-centric; real-world agents require perceptual and temporal reasoning.
• Tool co-evolution: Autonomous discovery and adaptation of tools remain underexplored.

Future research directions include the development of open-ended simulation environments for autonomous self-evolution, advanced tool use and creation strategies, real-world and longitudinal evaluation protocols, efficiency-aware MAS optimization algorithms, and domain-aware evolution for specialized applications.

Conclusion

This survey establishes a rigorous foundation for the paper and development of self-evolving AI agents, formalizing the transition from static foundation models to lifelong, adaptive agentic systems. By introducing a unified conceptual framework, systematically reviewing optimization and evaluation techniques, and articulating the core challenges and future directions, the work provides a roadmap for advancing the field toward scalable, resilient, and trustworthy autonomous agents. The emphasis on the triad of Endure (safety adaptation), Excel (performance preservation), and Evolve (autonomous optimization) as guiding principles ensures that future agentic systems will be not only performant and adaptive but also safe and aligned throughout their operational lifetimes.