Self-Evolving Agent Memory

• Self-evolving agent memory is an adaptive system that continuously updates stored interactions to improve decision-making in LLM- and MLLM-based agents.
• It employs diverse memory substrates—explicit, procedural, and latent—to support factual retention, revision, and transfer across tasks.
• Empirical benchmarks reveal significant gains in task success and efficiency while highlighting challenges in cost management and effective memory revision.

Self-evolving agent memory is a class of memory mechanisms for LLM- and MLLM-based agents in which memory is not treated as a passive archive, but as an adaptive substrate that is continually written, updated, reorganized, reused, and, in some systems, allowed to alter the agent’s effective policy during deployment. Recent work frames this capability along two axes—memory scope, from in-episode to cross-episode, and memory content, from knowledge-oriented to execution-oriented—so that memory must support not only factual retention but also revision, procedural reuse, and transfer across tasks (Wang et al., 18 May 2026). Across this literature, the central shift is from “remembering what was said” toward “remembering what was learned,” whether by ecological selection over trajectories, online fast-weight adaptation, skill induction, architecture search, or decentralized multi-agent memory evolution (Mi et al., 30 Jan 2026, Ren et al., 3 Jun 2026, Zhang et al., 21 Dec 2025).

1. Conceptual scope and problem regimes

A common formalization treats the agent at step $t$ as acting from current observation, history, and a memory state $m_t$, with memory updated after each interaction. EvoMemBench makes this explicit through

$$a_t^{e} \sim \pi(\cdot \mid h_{t-1}^{e}, o_t^{e}, m_t^{e}), \qquad m_{t+1}^{e} = \mathcal{U}(m_t^{e}, o_t^{e}, a_t^{e}, y_t^{e}),$$

thereby defining self-evolving memory as a capability to store, update, retrieve, revise, and transfer information over time rather than merely attach retrieved text to a prompt (Wang et al., 18 May 2026).

The problem settings in which this notion has become technically salient are diverse but structurally related. They include long-horizon, cross-application GUI automation with revisitation and procedural discovery (Mi et al., 30 Jan 2026); interactive medical decision support in which the agent must reuse prior reasoning across evolving cases (Sun et al., 8 Jun 2026); streaming prediction over a live 10-week horizon with continuous feedback (Zhang et al., 2 Feb 2026); and multi-agent systems in which role-specific experience must persist across tasks without collapsing into a centralized shared strategy (Hao et al., 21 May 2026). In all of these settings, failure is attributed not only to limited context windows, but also to coarse retrieval, stale memory, weak revision, and an inability to convert interaction outcomes into future control.

This has led to a broader distinction between memory-augmented agents and genuinely self-evolving memory systems. In the former, memory is often a fixed retrieval add-on. In the latter, memory contents, memory usage policy, and sometimes the memory architecture itself are allowed to change online or across evaluation rounds (Liu et al., 13 May 2026, Zhang et al., 2 Feb 2026, Zhang et al., 21 Dec 2025).

2. Memory substrates and representational choices

One major branch of the literature retains memory as explicit, external, and interpretable, but changes its granularity and governance. Darwinian Memory System stores reusable subtask-level units rather than whole workflows, with planner-generated subtasks $$p_i=\langle \text{Precondition}, \text{Goal}\rangle$$ serving as memory keys; a memory entry is $m=(p,\tau,s_{\text{meta}})$, where $\tau$ is the execution trajectory (Mi et al., 30 Jan 2026). SkeMex similarly rejects raw trajectory storage in favor of skill memories $m_i=(k_i,c_i,u_i)$, organized into general, task-level, and action-level branches and managed through a “Read–Write–Assess–Govern” lifecycle (Sun et al., 8 Jun 2026). WebCoach keeps condensed episodic summaries in an external memory store, using a WebCondenser and a Coach to turn prior web trajectories into selective runtime advice across sessions (Liu et al., 17 Nov 2025). SEDM pushes this further by turning each memory item into a verifiable, weighted, provenance-carrying snippet admitted only after reproducible replay (Xu et al., 11 Sep 2025).

A second branch makes memory procedural rather than purely textual. Metis explicitly separates text memory and code memory, organizing textual experience into execution plans, environment facts, and common pitfalls, while selectively crystallizing recurring plans into validated callable tools (Dai et al., 23 Jun 2026). The paper’s controlled comparison shows why this distinction matters: text memory is cheaper to construct and transfers more reliably, whereas code memory is more execution-efficient once available (Dai et al., 23 Jun 2026). DCPM adds a different hierarchy, ascending from raw inputs and atomic facts through identity items and supersedes chains to schemas, latent intentions, and cross-domain core schemas (Fei et al., 8 Jun 2026). In that formulation, memory is no longer only a set of retrievable facts; it becomes a structured cognitive graph capable of belief revision and abstraction.

A third branch internalizes memory into model-adjacent parameters or latent modules without updating the frozen backbone. TMEM maintains episode-local fast parametric memory $\Delta_t$ as LoRA weights and samples actions from $\pi_{\theta_0+\Delta_t}$, so that extraction actions produce supervision which updates $\Delta_t$ during the same rollout (Ren et al., 3 Jun 2026). MoLEM instead uses a frozen reasoner plus a dynamic mixture-of-experts latent-memory module, where experts generate latent memories, a router selects and weights them, and the aggregated latent memory is injected into reasoning while old stage modules are frozen to prevent forgetting (Yu et al., 21 May 2026). These systems redefine self-evolution as intra-episode or stage-wise policy alteration through memory-bearing modules rather than prompt-space lookup alone.

3. Evolution operators: selection, revision, mutation, and governance

A defining feature of the field is that memory quality is not assumed at write time. DMS operationalizes “survival of the fittest” through a survival value that combines utility, adaptive decay, and reliability, then ranks the memory bank, prunes long-tail entries, expands capacity when overflow is still valuable, and suppresses high-risk plans through Bayesian feedback regulation (Mi et al., 30 Jan 2026). Live-Evo uses a simpler but explicitly online weighting rule: for each retrieved experience, the update is

$m_t$0

while retrieval uses

$m_t$1

so experiences that improve outcome are reinforced and misleading ones are down-weighted and gradually forgotten (Zhang et al., 2 Feb 2026). SEDM formalizes write admission even more directly: a candidate memory is accepted iff

$m_t$2

where $m_t$3, and later retrieval uses $m_t$4 to combine relevance and empirically validated utility (Xu et al., 11 Sep 2025). SkeMex similarly updates skill utilities from category-normalized downstream advantage and uses the same value signal both for retrieval and for retention decisions such as promotion, deprecation, and deletion (Sun et al., 8 Jun 2026).

Revision is another central operator. DCPM preserves belief evolution through doubly linked supersedes chains, so old beliefs are not overwritten but versioned via Supersedes and SupersededBy pointers (Fei et al., 8 Jun 2026). Metis uses invalidate-and-replace rather than in-place editing for superseded textual memories (Dai et al., 23 Jun 2026). EvoMemBench identifies this as a major fault line in the field: current memory systems are markedly better at retention than at revision under contradiction (Wang et al., 18 May 2026).

Several systems add explicit mutation or failure-driven variation. DMS uses $m_t$5-mutation: with small probability the agent ignores retrieved memory, re-solves a subtask from scratch, and replaces the old trajectory if the new one is successful and shorter (Mi et al., 30 Jan 2026). FORGE turns failed trajectories into prompt-injected memory artifacts—Rules, Examples, or Mixed—and then uses population broadcast to propagate the best-performing memory state across instances between stages, with graduation freezing converged instances (Bogdanov et al., 15 May 2026). MemSkill makes the memory procedures themselves evolvable: a controller selects memory skills, an executor produces structured INSERT/UPDATE/DELETE actions under those skills, and a designer periodically revises or adds skills based on clustered hard cases (Zhang et al., 2 Feb 2026).

At the most expansive end of the design space, some systems evolve the memory architecture itself. MemEvolve treats a memory system as a four-module architecture $m_t$6 over encode, store, retrieve, and manage, then performs outer-loop diagnose-and-design evolution across candidate architectures while the inner loop evolves memory contents (Zhang et al., 21 Dec 2025). EvolveMem narrows that meta-evolution to retrieval infrastructure, exposing scoring functions, fusion modes, context budgets, decomposition, verification, and answer-generation policies as a structured action space optimized by an LLM diagnosis module with revert-on-regression and explore-on-stagnation safeguards (Liu et al., 13 May 2026).

4. Retrieval, policy coupling, and memory-conditioned control

Self-evolving memory systems differ sharply in how memory is coupled to decision-making. DMS uses dual-factor retrieval, multiplying precondition similarity and goal similarity so that a subtask memory is reused only when both starting state and intended objective align (Mi et al., 30 Jan 2026). SkeMex retrieves once at episode onset, balancing similarity, utility, and memory strength with temporal decay, and injects selected skills as a separate runtime block rather than appending them to rolling conversation memory (Sun et al., 8 Jun 2026). SEDM’s retrieval score $m_t$7 makes empirical usefulness first-class at read time (Xu et al., 11 Sep 2025). Live-Evo adds a compilation layer: retrieved experiences and a meta-guideline are turned into a task-specific guideline rather than concatenated verbatim (Zhang et al., 2 Feb 2026). DCPM keeps read latency low by performing vector search over facts and identity items, traversing supersedes chains, and then surfacing schema, intention, and core-schema nodes through evidence links, without invoking an LLM in the read path (Fei et al., 8 Jun 2026).

Other systems couple memory more directly to the policy substrate. TMEM is explicit that prompt-space memory alone cannot make the agent “learn from” experience during an episode; instead, extraction actions update fast weights $m_t$8, and later decisions are sampled from an adapted policy $m_t$9 (Ren et al., 3 Jun 2026). MoLEM likewise internalizes experiential knowledge into auxiliary modules: a router performs key-query matching over expert memories, aggregates latent memory, and injects it into a frozen pretrained reasoner (Yu et al., 21 May 2026). DecentMem extends this logic to multi-agent systems: each agent routes between an exploitation pool and an exploration pool according to online-updated weights, retrieves or generates a candidate trajectory, acts, receives stage-wise feedback from an LLM-as-a-judge, and then reweights exploitation probability accordingly (Hao et al., 21 May 2026).

This range of couplings illustrates a substantive divide in the field. Some systems treat memory as external control context; others use memory to alter the effective policy, latent state, or routing dynamics. A plausible implication is that “self-evolving memory” now names a continuum from adaptive retrieval to adaptive control.

5. Empirical evaluation and benchmark evidence

Benchmarking work has made clear that self-evolving memory is not a solved primitive. EvoMemBench, which compares 15 representative memory methods across in-episode and cross-episode, knowledge-oriented and execution-oriented regimes, finds that long-context baselines remain highly competitive, memory helps most when current context is insufficient or tasks are difficult, retrieval-based methods remain strong for knowledge-intensive settings, and procedural or long-term memory methods are more effective for execution-oriented tasks when stored experience matches task structure (Wang et al., 18 May 2026). Its core conclusion is negative but important: no single memory form works consistently across all settings (Wang et al., 18 May 2026).

Within domain-specific systems, however, large gains are repeatedly reported. On AndroidWorld, DMS improves average success rate by $$a_t^{e} \sim \pi(\cdot \mid h_{t-1}^{e}, o_t^{e}, m_t^{e}), \qquad m_{t+1}^{e} = \mathcal{U}(m_t^{e}, o_t^{e}, a_t^{e}, y_t^{e}),$$0 and execution stability by $$a_t^{e} \sim \pi(\cdot \mid h_{t-1}^{e}, o_t^{e}, m_t^{e}), \qquad m_{t+1}^{e} = \mathcal{U}(m_t^{e}, o_t^{e}, a_t^{e}, y_t^{e}),$$1, and raises Qwen2.5-VL-72B from $$a_t^{e} \sim \pi(\cdot \mid h_{t-1}^{e}, o_t^{e}, m_t^{e}), \qquad m_{t+1}^{e} = \mathcal{U}(m_t^{e}, o_t^{e}, a_t^{e}, y_t^{e}),$$2 to $$a_t^{e} \sim \pi(\cdot \mid h_{t-1}^{e}, o_t^{e}, m_t^{e}), \qquad m_{t+1}^{e} = \mathcal{U}(m_t^{e}, o_t^{e}, a_t^{e}, y_t^{e}),$$3 success (Mi et al., 30 Jan 2026). On WebVoyager, WebCoach improves a 38B browser-use agent from $$a_t^{e} \sim \pi(\cdot \mid h_{t-1}^{e}, o_t^{e}, m_t^{e}), \qquad m_{t+1}^{e} = \mathcal{U}(m_t^{e}, o_t^{e}, a_t^{e}, y_t^{e}),$$4 to $$a_t^{e} \sim \pi(\cdot \mid h_{t-1}^{e}, o_t^{e}, m_t^{e}), \qquad m_{t+1}^{e} = \mathcal{U}(m_t^{e}, o_t^{e}, a_t^{e}, y_t^{e}),$$5 success while reducing or maintaining the average number of steps (Liu et al., 17 Nov 2025). On AppWorld, Metis improves task accuracy by up to $$a_t^{e} \sim \pi(\cdot \mid h_{t-1}^{e}, o_t^{e}, m_t^{e}), \qquad m_{t+1}^{e} = \mathcal{U}(m_t^{e}, o_t^{e}, a_t^{e}, y_t^{e}),$$6 over ReAct while reducing execution cost by up to $$a_t^{e} \sim \pi(\cdot \mid h_{t-1}^{e}, o_t^{e}, m_t^{e}), \qquad m_{t+1}^{e} = \mathcal{U}(m_t^{e}, o_t^{e}, a_t^{e}, y_t^{e}),$$7 (Dai et al., 23 Jun 2026). On the live Prophet Arena benchmark over a 10-week horizon, Live-Evo improves Brier score by $$a_t^{e} \sim \pi(\cdot \mid h_{t-1}^{e}, o_t^{e}, m_t^{e}), \qquad m_{t+1}^{e} = \mathcal{U}(m_t^{e}, o_t^{e}, a_t^{e}, y_t^{e}),$$8 and increases market returns by $$a_t^{e} \sim \pi(\cdot \mid h_{t-1}^{e}, o_t^{e}, m_t^{e}), \qquad m_{t+1}^{e} = \mathcal{U}(m_t^{e}, o_t^{e}, a_t^{e}, y_t^{e}),$$9 (Zhang et al., 2 Feb 2026).

Parametric, latent, and multi-agent systems show similarly strong results. TMEM consistently outperforms summary-based and retrieval-based baselines on LoCoMo, LongMemEval-S, search, and CL-Bench, and benefits especially strongly from RL because extraction quality directly affects online adaptation (Ren et al., 3 Jun 2026). SkeMex improves DeepSeek-V3.2 from $$p_i=\langle \text{Precondition}, \text{Goal}\rangle$$0 to $$p_i=\langle \text{Precondition}, \text{Goal}\rangle$$1 in offline in-domain testing and remains above ReAct across all reported OOD benchmarks, indicating transferable skill memory (Sun et al., 8 Jun 2026). MoLEM reaches $$p_i=\langle \text{Precondition}, \text{Goal}\rangle$$2 average accuracy after a continual-learning sequence, improving by $$p_i=\langle \text{Precondition}, \text{Goal}\rangle$$3 over the vanilla pretrained baseline while keeping forget and backward transfer at $$p_i=\langle \text{Precondition}, \text{Goal}\rangle$$4 from Stage 2 onward in the default setting (Yu et al., 21 May 2026). FORGE improves average evaluation return by $$p_i=\langle \text{Precondition}, \text{Goal}\rangle$$5–$$p_i=\langle \text{Precondition}, \text{Goal}\rangle$$6 over zero-shot and by $$p_i=\langle \text{Precondition}, \text{Goal}\rangle$$7–$$p_i=\langle \text{Precondition}, \text{Goal}\rangle$$8 over Reflexion in all 12 reported model-representation conditions (Bogdanov et al., 15 May 2026). DCPM’s System 2 contributes most on PersonaMem-v2, where it yields gains up to $$p_i=\langle \text{Precondition}, \text{Goal}\rangle$$9, matching its design target of implicit cross-session inference rather than surface recall (Fei et al., 8 Jun 2026). In multi-agent settings, DecentMem improves average accuracy by up to $m=(p,\tau,s_{\text{meta}})$0 over the strongest centralized memory baseline and by up to $m=(p,\tau,s_{\text{meta}})$1 over the no-memory baseline, while reducing token usage by up to $m=(p,\tau,s_{\text{meta}})$2 (Hao et al., 21 May 2026).

6. Limitations, controversies, and open directions

The literature is unusually explicit about limitations. Many mechanisms depend on reliable verification or judging: DMS assumes a verifier accurate enough for strike counts, pruning, and Bayesian inhibition to reflect real utility (Mi et al., 30 Jan 2026); SEDM depends on reproducible replay and meaningful A/B marginal utility estimates (Xu et al., 11 Sep 2025); DecentMem relies on LLM-as-a-judge stage scoring to update routing weights (Hao et al., 21 May 2026). Extraction quality is another recurrent bottleneck: bad QA pairs lead to bad online adaptation in TMEM (Ren et al., 3 Jun 2026), weak distillation can produce harmful skills in SkeMex (Sun et al., 8 Jun 2026), and Meta-guideline or experience summarization errors can mislead Live-Evo (Zhang et al., 2 Feb 2026).

Cost and latency remain substantial. SkeMex increases average execution steps and wall-clock time relative to ReAct even while improving accuracy (Sun et al., 8 Jun 2026). Metis shows that code memory can reduce execution tokens and steps but incurs much higher construction cost if codification is too eager (Dai et al., 23 Jun 2026). EvolveMem requires repeated benchmark-scale evaluation rounds, diagnosis calls, and guarded reconfiguration, making its self-evolution process closer to autonomous retrieval research than lightweight online adaptation (Liu et al., 13 May 2026). MoLEM avoids catastrophic forgetting by appending stage-local modules, but memory grows linearly with stages and still assumes stage-triggered expansion rather than fully online domain discovery (Yu et al., 21 May 2026).

A deeper controversy concerns generality. EvoMemBench shows that long-context baselines can outperform explicit memory when raw evidence fits within context, and that cross-episode transfer is brittle when stored experience does not align with the target decision process (Wang et al., 18 May 2026). Domain-specific assumptions are pervasive: DMS is tightly matched to GUI subtasks and deterministic replay (Mi et al., 30 Jan 2026); DCPM is optimized for persona and diachronic preference reasoning (Fei et al., 8 Jun 2026); DecentMem targets decentralized multi-agent collaboration rather than single-agent control (Hao et al., 21 May 2026). A plausible implication is that the field is converging not on one universal memory substrate, but on a family of task-structure-sensitive designs.

Even so, the trajectory of the area is clear. Self-evolving agent memory now spans content evolution, utility-aware governance, revision and forgetting, fast parametric adaptation, latent-memory modularization, skill evolution, retrieval self-optimization, and meta-evolution of the memory architecture itself (Zhang et al., 2 Feb 2026, Liu et al., 13 May 2026, Zhang et al., 21 Dec 2025). The unifying principle is that memory must be selective, revisable, and action-relevant: it must not only preserve the past, but continuously test, reinterpret, and reorganize it in service of future behavior.