SAGE: Self-Evolving Reflective Memory Agents

Updated 14 November 2025

SAGE agents are self-evolving systems that integrate explicit memory modules and reflective self-improvement to overcome the limitations of static large language models.
They employ structured memory architectures—including working, episodic, and semantic memories—to store, retrieve, and update experiences for continual adaptation in dynamic tasks.
This paradigm enhances performance, reduces catastrophic forgetting, and enables test-time adaptation across multimodal and long-horizon environments.

Self-evolving agents with reflective and memory-augmented abilities (often abbreviated as SAGE) represent a paradigmatic advance in the design and deployment of artificial agents, especially those leveraging LLMs or multimodal neural architectures. These agents overcome the fundamental limitations of static LLMs by integrating explicit memory architectures, reflective reasoning mechanisms, and continual adaptation processes, thereby exhibiting lifelike capabilities for learning from experience, reducing catastrophic forgetting, and achieving robust performance in complex, long-horizon, and open-ended environments. The SAGE paradigm brings together research threads on agent memory, self-reflection, lifelong or continual learning, and policy evolution.

1. Foundational Principles and Motivations

The SAGE class is motivated by two principal deficits in classical LLM and embodied agent systems: (1) inability to retain and integrate experience, and (2) lack of mechanisms for introspective self-improvement. SAGE agents are distinguished by:

Memory Augmentation: The explicit separation of working, episodic, and semantic memories, supporting both short-term contextualization and long-term knowledge retention (Liang et al., 1 Sep 2024, Jin et al., 13 Oct 2025, Hassell et al., 22 Oct 2025).
Reflective Self-improvement: Agents perform explicit self-critique or abstraction over their own trajectories, extracting patterns (“lessons learned”) that are re-injected as guiding context for subsequent reasoning (Feng et al., 9 Feb 2025, Hayashi et al., 8 Nov 2025).
Self-evolutionary Loops: Closed cycles in which environment interaction, reflection, and memory update are tightly interleaved for continual, unsupervised policy refinement (Feng et al., 9 Feb 2025, Gao et al., 28 Jul 2025, Cai et al., 26 Aug 2025).
Test-time Adaptation: Agents adapt to new tasks at inference without parameter-level fine-tuning, through memory retrieval and in-context learning mechanisms (Hayashi et al., 8 Nov 2025, Hassell et al., 22 Oct 2025).

This approach supports adaptation across dynamic tasks and scenarios, allows for interpretability and inspection of the agent’s evolving policy, and systematically mitigates catastrophic forgetting encountered in lifelong learning.

2. Core Architectural Modules

The canonical SAGE agent comprises several interacting components, which may be instantiated with different computational architectures across research efforts:

Memory Modules
- Working Memory (WM): Temporary scratchpad within the LLM context window.
- Short/Long-term Memory (STM/LTM): Separate stores for recent episode-specific reflections (STM) and consolidated, experience-aggregated insights (LTM), often structured as key-value embeddings with time- and salience-based retention (Liang et al., 1 Sep 2024, Jin et al., 13 Oct 2025).
- Episodic and Semantic Memory: Instance-level records (e.g., input, agent output, critique) vs. distilled summaries or procedural abstractions (Hassell et al., 22 Oct 2025).
- Multimodal Experience Pool (MEP): Recordings of trajectories in fully embodied or visually grounded settings (Feng et al., 9 Feb 2025).
Reflective Mechanisms
- Self-Reflection: At regular intervals or upon task/episode completion, the agent synthesizes textual, symbolic, or abstract representations of what succeeded, failed, or was novel—either via LLM prompt chains or dedicated reflection modules (Liang et al., 1 Sep 2024, Feng et al., 9 Feb 2025, Lippmann et al., 4 Nov 2024, Hayashi et al., 8 Nov 2025).
- Plan Abstraction: Induction of concise, high-level procedural guides from noisy trajectories for re-use as heuristic scaffolds (Hayashi et al., 8 Nov 2025).
- Meta-Reflectors: Secondary reflection systems able to recalibrate the agent’s own assessment or feedback quality (e.g., Checker-of-Checker in MARS (Liang et al., 25 Mar 2025)).
Memory Management and Optimization
- Ebbinghaus-based Forgetting: Decay-based memory retention rules balancing agility and resource efficiency, implemented as function of time, usage frequency, and importance scoring (Liang et al., 1 Sep 2024, Liang et al., 25 Mar 2025).
- Curriculum-guided and Priority-based Sampling: Selection of experiences or reflections for model update based on relevance, subgoal priority, and utility (Feng et al., 9 Feb 2025).
Plan-Driven and Model-Based Control
- LLM-driven Planners: High-level task decomposition into subtasks via LLMs conditioned on current state, episode memory, and multimodal context (Feng et al., 9 Feb 2025).
- World Model-based Controllers: Model-based reinforcement learning controllers (e.g., RSSM variants) that plan low-level actions subject to experience and learned dynamics (Feng et al., 9 Feb 2025).
Knowledge Updaters and Abstraction-induced Refinement
- Policy Update via Soft Memory: Use of retrieved or abstracted plans as in-context modifications of the base policy, formalizable via Bayesian or option-based perspectives (Hayashi et al., 8 Nov 2025).
- Skill Abstraction and Internalization: Clustering of recurrent behavioral motifs to establish reusable skill modules and knowledge distillation into base policy parameters (Cai et al., 26 Aug 2025).
Multi-agent and Social Reasoning Extensions
- Social Reasoners, Negotiation and Goal Modelling: In Diplomacy-style applications, integration of multi-agent theory-of-mind modules, reflective negotiation, and episodic memory for tracking and revising joint intent (Guan et al., 9 Jul 2024, Yu et al., 20 Apr 2025).

3. Memory Structures, Reflection Algorithms, and Update Rules

Memory in SAGE systems is parameterized explicitly, supporting storage, retrieval, and forgetting at different timescales and modalities.

Memory Type	Content / Role	Update / Retention
Working (WM)	Current context, dialogue	Flushed per task, ~context window
Episodic	Instance-level trajectories	Appended per episode/trial
Semantic	Abstracted critiques/plans	Summarized periodically
Short-Term (STM)	Recent reflections	Decay-selected, Ebbinghaus filter
Long-Term (LTM)	Archived experience	Salience- and time-thresholded

Decay-based retention: $M(\Delta t) = M_0 \exp(-\lambda \Delta t)$ for trace strength $M_0$ and decay $\lambda$ (Liang et al., 1 Sep 2024, Liang et al., 25 Mar 2025).
Reflection as function: $r_t = \mathrm{Ref}(o_{1:t}, R_{1:t})$ (Liang et al., 1 Sep 2024, Hassell et al., 22 Oct 2025).
Policy update: $\pi_{t+1} = \psi(\pi_t, \mathcal{E}^{t+1})$ with $\mathcal{E}^{t+1}$ an evolutionary goal comprising memory and self-adjustment directions (Liang et al., 1 Sep 2024).
Plan-augmented policy: $\pi_\theta^+(a \mid s) = \pi_\theta(a \mid [x_t, A])$ where $A$ is plan abstraction (Hayashi et al., 8 Nov 2025).

Reflection can focus on positive outcomes (Lippmann et al., 4 Nov 2024), failures, or synthesized lessons; the dual-buffer memory (STM, LTM) ensures both recency-sensitive recall and stable knowledge consolidation.

4. Empirical Evaluation and Quantitative Performance

SAGE systems have demonstrated significant performance gains across domains and benchmarks:

Minecraft Long-Horizon Tasks: EvoAgent (SAGE architecture) attains a 105.85% improvement in overall success rate and a 6x reduction in ineffective actions compared to prior baselines such as PPO, DreamerV3, and Optimus-1 (Feng et al., 9 Feb 2025). Module ablations reveal 25%+ success rate increases from reflective components, with continual world model updates providing further boosts.
Web Navigation: Reflection-Augmented Planning (ReAP) yields a +11 point success rate improvement on previously unseen tasks, and up to +29 points on hard failures, with reduced step and wall-time cost (Azam et al., 2 Jun 2025).
Text Environments: Incorporating positive-experience reflection, as in Sweet & Sour, achieves +10–17 point gains over failure-only baselines, with smaller LLMs benefiting most from positive memory (Lippmann et al., 4 Nov 2024).
Benchmarked QA/Agent Tasks: Memory-augmented critique-driven adaptation in SAGE yields up to 24.8% absolute accuracy gains; episodic memories outperform semantic summaries in 12/14 settings (Hassell et al., 22 Oct 2025).
Software Engineering Tasks: Plan abstraction SAGE raises resolution rates by up to 7.2% over strong Mini-SWE-Agent baselines, and up to 74.6% Pass@1 judged accuracy with ensemble LLM selection (Hayashi et al., 8 Nov 2025).

Evaluation metrics span success rate (SR), exploration efficiency (EE), forgetting (FGT), backward transfer (BWT), proactivity, and human trust/preference. Consistent findings include the importance of prioritized, curriculum-guided memory sampling, and the critical role of case-specific (episodic) reflection for effective in-context adaptation.

5. Variants and Mechanistic Innovations

Several mechanistic variants have been proposed and empirically compared:

Generative Latent Memory (MemGen): Latent-token memory preprended into the LLM’s reasoning loop via RL-trained trigger and generative LoRA-weaver modules, enabling emergent working, procedural, and planning memory faculties. MemGen outperforms retrieval-based (e.g., ExpeL, AWM) and parametric methods by up to 38.22% absolute (Zhang et al., 29 Sep 2025).
MetaAgent & Tool Meta-Learning: Unifies in-context self-reflection, verified memory, in-house tool construction, and persistent knowledge bases; ablation studies show that reflection, memory, and emergent utility composition are all critical to performance (Qian et al., 1 Aug 2025).
Dual Memory, Social Reasoning, Co-evolution: Multi-agent simulations (Richelieu for Diplomacy, AI-Agent School in educational settings) demonstrate that situated, role-based, or social reflection and memory further boost adaptation, negotiation, and complex group behavior (Guan et al., 9 Jul 2024, Jin et al., 13 Oct 2025).
Policy Evolution with Theory of Mind: Dynamic adaptation of tabular or parametric policies based on reflective memory and LLM-inferred higher-order intentionality, outperforming RL and prior LLM-based agents in strategic games (Yu et al., 20 Apr 2025).

6. Evaluation Frameworks, Benchmarks, and Open Challenges

The SAGE paradigm is systematically surveyed and benchmarked in (Gao et al., 28 Jul 2025), with the following organizing principles:

Dimension	Representative Methods/Benchmarks	Key Consideration
What To Evolve	Model, Context/Memory, Tools, Architecture	Which agent substrates are mutable
When To Evolve	Intra-/Inter-test time (ICL/SFT/RL/P-E)	Adaptation timing: online vs. offline
How To Evolve	Reward, textual/self feedback, imitation, Evo	Algorithms: Reflection, Imitation, Population
Metrics	SR, FGT, BWT, adaptivity, safety, efficiency	Retention, transfer, proactivity, computation

Core benchmarks: AgentBench, LifelongAgentBench, ScienceWorld, StuLife, SWE-Bench, WebArena, PlanBench, MemoryAgentBench.

Critical open challenges include:

Plasticity–stability dilemma: Retaining old task knowledge while integrating new experience (catastrophic forgetting vs. adaptability) (Feng et al., 9 Feb 2025, Gao et al., 28 Jul 2025).
Memory Scalability: Efficiently storing, summarizing, and retrieving relevant memories; leveraging hierarchical and decay-prioritized memory (Liang et al., 1 Sep 2024, Liang et al., 25 Mar 2025).
Safe Evolution: Preventing undesirable co-evolution, out-of-distribution behavior, and unsafe self-modification (Gao et al., 28 Jul 2025).
Computational cost: Balancing depth and frequency of reflection/memory retrieval with real-time performance constraints.
Interpretability and Auditing: Ensuring that reflective and memory-driven modifications can be traced and inspected, aiding troubleshooting and compliance (Hassell et al., 22 Oct 2025).

7. Significance, Implications, and Future Directions

Self-evolving agents with reflective and memory-augmented abilities are a unifying abstraction for constructing robustly adaptive, explainable, and resource-efficient AI systems across environments and modalities. The explicit separation of dialogue, memory, and reflection avoids the rigidity of purely parametric or stateless models, affording strong continual learning and systematic alignment with task objectives.

Trajectory abstraction, episodic reflection, curriculum-based experience selection, and memory retention mechanisms (Ebbinghaus curve, decay-based policies) are repeatedly validated across domains. Performance improvements accrue from synergy between in-context “soft” learning (prompting, reflection-guided action) and selective, resource-efficient memory organization.

Future work is explicitly oriented around:

Extending abstractions to multi-agent social environments with dynamic intent modeling (Guan et al., 9 Jul 2024).
Automating the meta-selection of what, when, and how to reflect/memorize based on agent state and environmental volatility (Liang et al., 25 Mar 2025).
Scaling structured memory to support proactive, long-term, and multi-modal planning (Feng et al., 9 Feb 2025, Jin et al., 13 Oct 2025).
Integration of intrinsic motivation and curiosity mechanisms with explicit memory (Cai et al., 26 Aug 2025).
Formalizing retrieval policies for plan/abstraction selection (Hayashi et al., 8 Nov 2025).
Advancing interpretability tools to support user trust and compliance (Hassell et al., 22 Oct 2025).

This body of research establishes SAGE as a rigorous, empirically validated framework for designing and understanding artificial agents capable of sustained, human-like adaptation grounded in the dynamic interplay between memory, reflection, and continual self-evolution.