Self-Improvements in Modern Agentic Systems: A Survey

Abstract: Self-improving autonomous agents are moving from research prototypes to deployed systems. The primary goal is controllable evolution, or adaptation, from experience with minimal or even no human input. This survey frames modern self-improving agents as adaptive systems that convert experience into accumulated capability gains. We offer a system-level framework that represents a modern agent as a configuration coupling a foundation model with an operational scaffold of prompts, memory, tools, and control logic. Within this framework, self-improvement is formalized as a self-induced update operator that obtains and commits updates to model parameters or scaffold components. We organize prior work by update target and by the signals that drive change, then review applications and discuss evaluation, before closing with open problems and future directions. For convenience, we track technical updates on https://github.com/selfimproving-agent/awesome-Self-Improving-Agents.

• The paper introduces a formal taxonomy distinguishing between parametric (foundation model) and non-parametric (scaffold) self-improvements in agentic systems.
• It details rigorous methodologies and evaluation protocols that quantify improvements via intrinsic and extrinsic learning signals across diverse applications.
• Empirical findings demonstrate measurable performance gains in domains like software engineering and robotics through integrated self-modifying AI architectures.

Self-Improvements in Modern Agentic Systems: An Expert Analysis

The surveyed paper, "Self-Improvements in Modern Agentic Systems: A Survey" (2607.13104), delivers a comprehensive, system-level articulation of self-improvement phenomena emerging within autonomous agent frameworks, particularly those grounded in Foundation Models (FMs). By presenting a rigorous taxonomy and formalism, the work crystallizes distinctions between parametric (model-level) and non-parametric (scaffold-level) self-modification, providing clarity amid rapidly proliferating and terminologically fragmented research. This essay synthesizes the major technical contributions, methodological advances, empirical findings, and overarching implications for the development and trajectory of self-improving AI agents.

---

Formalism and Taxonomy of Self-Improvement

The system-level agent formalism adopted centers a frozen or trainable FM ($\theta$) coupled to a modifiable scaffold ($\Sigma$) encompassing prompts, memory, tools, and control logic. Self-improvement is explicitly defined as the application of a self-induced update process, $\mathcal{U}$, driven by execution-derived learning signals, acting persistently on either FM weights or scaffold components. The resultant taxonomy classifies self-improvement mechanisms along two principal axes:

• Foundation Model Improvement: Parametric updates internalize behavioral improvements via supervisory signals generated intrinsically (e.g., self-generated demonstrations or evaluative feedback) or extrinsically (environment-grounded experience).
• Scaffolding Improvement: Non-parametric, often heuristic or structure-driven updates modify prompts, memory, tool sets, or control workflows, without changing model weights.

  Figure 1: Overview of self-improvement paradigms for modern AI agents, distinguishing between foundation model improvement (parametric) and scaffolding improvement (non-parametric) as dual axes of adaptation.

A historical and conceptual lineage traces these paradigms from early error-driven learning, symbolic self-modification, meta-learning, and G\"odel-machine-style recursion, culminating in modern FM-based architectures which leverage natural language as both substrate and vehicle for self-improvement.

Figure 2: Timeline and taxonomy of self-improvement in FM-based agents, highlighting the bifurcation into FM-level and scaffold-level adaptation over recent years.

---

Foundation Model Improvement: Learning Signal Modalities

Intrinsic Generative Demonstrations: Models synthesize their own training data—pseudo-labels, pseudo-interactions, or decomposed reasoning traces—bootstrapping advances from limited seeds. Sophisticated protocols introduce curriculum-based decompositions, self-consistency filtering, or on-the-fly test-time adaptation (e.g., LoRA per-instance finetuning) to improve sample utility and targeted remediation. However, the iterative use of model-generated data risks data drift and collapse, mandating safeguards such as anchor datasets, explicit diversity control, and robust validators.

Intrinsic Evaluative Feedback: Feedback signals such as rubric-based scores, preference pairs, self-consistency-derived confidence, or natural language revisions are synthesized by models themselves or auxiliary (anchor) models. The approach generalizes beyond RLHF to variants with dense AI feedback, pairwise comparison, and recursive critique-revision pipelines. Notably, robustness is a persistent challenge, as optimization against a brittle internal evaluator or shared model family can yield superficial or even adversarial 'improvements'.

Extrinsic Exploratory Experience: Agents interact with external task environments—physical simulators, programmatic sandboxes, or learned world models—collecting trajectories and outcome data for downstream RL, imitation, or hybrid optimization. World models, when properly calibrated, accelerate synthetic experience generation, but introduce failure modes through hallucinated rollouts and feedback misalignment.

Figure 3: Overview of foundation model improvement under agent-induced learning signals, illustrating demonstration, evaluative, and exploratory signal loops.

---

Scaffolding Improvement: Prompt, Memory, Tool, and Full Scaffold Dynamics

Prompt Optimization: Advances move from search-based tuning over scalar metrics to population-based, debate-driven, and 'textual gradient' approaches where natural language instructions encode structured feedback. Genetic programming and reflective evolution (e.g., GEPA) merge meta-level reflection with diversity maintenance, often outperforming pure RL-style prompt selection in sample efficiency and convergence.

Memory Governance: Scaffold memory moves from append-only storage to structured, multi-level, and graph-based regimes that dynamically manage salience, retention, and forgetting. Non-parametric, feedback-driven CRUD operations enable continual distillation of reusable episodic or semantic knowledge, as well as interpretability and resilience against contamination.

Tool Inventories and Routing: Dynamic tool orchestration, meta-cognitive tool evaluation, and agent-driven skill invention transform the static toolkit paradigm into an evolving, competence-amplifying resource managed via execution feedback, repair logs, and population-level reuse metrics.

Full Scaffold Self-Modification: The most radical pathway entails recursive, verifier-gated reconfiguration of the agent's own codebase or workflow, often framed through evolutionary search, program synthesis, or bounded G\"odel-machine analogues. Population-level archives, monotonicity constraints, and sandboxing are critical to control the risk surface.

Figure 4: Full scaffolding self-improvement across iterations, visualizing recursive and population-based update trajectories.

---

Empirical Applications and Numerical Results

The taxonomy is instantiated and validated across core domains:

• Software Engineering: Agents achieve measurable improvements by evolving both execution scaffolds (dynamic prompts, tool protocols, and modular codebases) and parameter weights using compiler/test-based feedback, with significant gains on benchmarks such as SWE-bench+ [aleithan2024swebenchenhancedcodingbenchmark].
• Web Navigation/Automation: Scaffold-level advances—especially memory and grounding—yield notably higher generalization and survivability under distributional drift, while FM improvement from click/transition signals provides incremental robustness (though overfitting to static layouts remains a concern).
• Strategy Games: FM-parameter improvement via self-play and cooperative/competitive self-curricula demonstrates robust, non-trivial transfer to held-out agent pools and unobserved opponent strategies, especially when scaffold-level memory and role composition are jointly updated.
• Scientific Discovery and Robotics: Full workflow self-improvement and curriculum evolution enable agents to autonomously discover, validate, and preserve scientific hypotheses and embodied skill repertoires, with measured improvement in reproducibility, coverage, and adaptation rates.

  Figure 5: Representative application domains for self-improving agents, situating empirical advances within the system-level taxonomy.

Empirical protocols emphasize trajectory-based, not just endpoint, reporting—learning curves, regression rates, and overhead measurement—requiring rigorous resource cost accounting and judge independence in human-aligned tasks.

---

Evaluation Protocols and Benchmarks

Evaluation frameworks distinguish mechanism benchmarks (isolating FM vs. scaffold update effects) from domain benchmarks (measuring applied improvement with domain constraints). Rigor demands:

• Strict held-out validation, adversarial/hidden test sets, and temporal split to prevent feedback leakage;
• Explicit operational budget accounting (compute, tokens, wall-clock, human interventions);
• Reporting safety and regression rates, not merely mean success.

Language-judge pipelines (LLM-as-a-Judge) and trajectory retention audits are increasingly used in domains lacking hard programmatic oracles, though agent over-optimization to judges remains a risk, necessitating judge diversity and cross-model validation [zhuge2024agent].

Figure 6: Paper--benchmark incidence matrix for self-improving agent methods and benchmark coverage, delineating FM/scaffold update contributions and interface dependencies.

---

Theoretical and Practical Implications

The paradigm presented fundamentally shifts agent development toward dynamical, self-governing systems, where improvement is continuous, multiplexed, and distributed across architecture and learning substrate. Key technical implications include:

• The necessity for layered safety gating, particularly as scaffold and model updates become more tightly coupled and recursively self-referential.
• The emergence of mixed improvement loops (fast scaffold adjustment, slow parameter consolidation), reflecting dual system-theoretic timescales.
• The critical role of curated, robust feedback signals—with critics/judges occupying a central, but attackable, position in the process.
• Practical adoption within AI research workflows, software engineering, scientific research, robotics, and open-ended game settings, all reflecting measurable, attributed capability expansion.

Open problems highlighted include robust test-time continual adaptation, curiosity-driven and verifiable exploration, credit assignment across substrate boundaries, and stable evaluation under domain non-stationarity and adversarial drift.

---

Conclusion

This survey formalizes and structures the domain of self-improving agentic systems, clarifying both the empirical and theoretical dimensions of agent self-improvement. Foundation model and scaffold duality, combined with rigorous, methodological evaluation, sets a unified research agenda. As the field approaches recursively self-modifying agent architectures, future advances will demand sophisticated safety apparatus, layered governance, and integrated evaluation infrastructures. The trajectory toward increasingly autonomous, self-referential AI will depend not only on scaling model capacity, but on constructing measurable, attributable, safely bounded systems that progressively and robustly expand their own competencies in concert with their environment and scaffolding.