---
title: Self-Evolving Agent Memory
url: https://www.emergentmind.com/topics/self-evolving-agent-memory
type: topic
---

# Self-Evolving Agent Memory

Self-evolving agent memory is a class of memory mechanisms for LLM- and MLLM-based agents in which memory is not treated as a passive archive, but as an adaptive substrate that is continually written, updated, reorganized, reused, and, in some systems, allowed to alter the agent’s effective policy during deployment. Recent work frames this capability along two axes—memory scope, from in-episode to cross-episode, and memory content, from knowledge-oriented to execution-oriented—so that memory must support not only factual retention but also revision, procedural reuse, and transfer across tasks [2605.18421]. Across this literature, the central shift is from “remembering what was said” toward “remembering what was learned,” whether by ecological selection over trajectories, online fast-weight adaptation, skill induction, architecture search, or decentralized multi-agent memory evolution [2601.22528] [2606.04536] [2512.18746].

## 1. Conceptual scope and problem regimes

A common formalization treats the agent at step \(t\) as acting from current observation, history, and a memory state \(m_t\), with memory updated after each interaction. EvoMemBench makes this explicit through
\[
a_t^{e} \sim \pi(\cdot \mid h_{t-1}^{e}, o_t^{e}, m_t^{e}), \qquad
m_{t+1}^{e} = \mathcal{U}(m_t^{e}, o_t^{e}, a_t^{e}, y_t^{e}),
\]
thereby defining self-evolving memory as a capability to store, update, retrieve, revise, and transfer information over time rather than merely attach retrieved text to a prompt [2605.18421].

The problem settings in which this notion has become technically salient are diverse but structurally related. They include long-horizon, cross-application GUI automation with revisitation and procedural discovery [2601.22528]; interactive medical decision support in which the agent must reuse prior reasoning across evolving cases [2606.09365]; streaming prediction over a live 10-week horizon with continuous feedback [2602.02369]; and multi-agent systems in which role-specific experience must persist across tasks without collapsing into a centralized shared strategy [2605.22721]. In all of these settings, failure is attributed not only to limited context windows, but also to coarse retrieval, stale memory, weak revision, and an inability to convert interaction outcomes into future control.

This has led to a broader distinction between memory-augmented agents and genuinely self-evolving memory systems. In the former, memory is often a fixed retrieval add-on. In the latter, memory contents, memory usage policy, and sometimes the memory architecture itself are allowed to change online or across evaluation rounds [2605.13941] [2602.02474] [2512.18746].

## 2. Memory substrates and representational choices

One major branch of the literature retains memory as explicit, external, and interpretable, but changes its granularity and governance. Darwinian Memory System stores reusable subtask-level units rather than whole workflows, with planner-generated subtasks \(p_i=\langle \text{Precondition}, \text{Goal}\rangle\) serving as memory keys; a memory entry is \(m=(p,\tau,s_{\text{meta}})\), where \(\tau\) is the execution trajectory [2601.22528]. SkeMex similarly rejects raw trajectory storage in favor of skill memories \(m_i=(k_i,c_i,u_i)\), organized into general, task-level, and action-level branches and managed through a “Read–Write–Assess–Govern” lifecycle [2606.09365]. WebCoach keeps condensed episodic summaries in an external memory store, using a WebCondenser and a Coach to turn prior web trajectories into selective runtime advice across sessions [2511.12997]. SEDM pushes this further by turning each memory item into a verifiable, weighted, provenance-carrying snippet admitted only after reproducible replay [2509.09498].

A second branch makes memory procedural rather than purely textual. Metis explicitly separates text memory and code memory, organizing textual experience into execution plans, environment facts, and common pitfalls, while selectively crystallizing recurring plans into validated callable tools [2606.24151]. The paper’s controlled comparison shows why this distinction matters: text memory is cheaper to construct and transfers more reliably, whereas code memory is more execution-efficient once available [2606.24151]. DCPM adds a different hierarchy, ascending from raw inputs and atomic facts through identity items and supersedes chains to schemas, latent intentions, and cross-domain core schemas [2606.09483]. In that formulation, memory is no longer only a set of retrievable facts; it becomes a structured cognitive graph capable of belief revision and abstraction.

A third branch internalizes memory into model-adjacent parameters or latent modules without updating the frozen backbone. TMEM maintains episode-local fast parametric memory \(\Delta_t\) as LoRA weights and samples actions from \(\pi_{\theta_0+\Delta_t}\), so that extraction actions produce supervision which updates \(\Delta_t\) during the same rollout [2606.04536]. MoLEM instead uses a frozen reasoner plus a dynamic mixture-of-experts latent-memory module, where experts generate latent memories, a router selects and weights them, and the aggregated latent memory is injected into reasoning while old stage modules are frozen to prevent forgetting [2605.21951]. These systems redefine self-evolution as intra-episode or stage-wise policy alteration through memory-bearing modules rather than prompt-space lookup alone.

## 3. Evolution operators: selection, revision, mutation, and governance

A defining feature of the field is that memory quality is not assumed at write time. DMS operationalizes “survival of the fittest” through a survival value that combines utility, adaptive decay, and reliability, then ranks the memory bank, prunes long-tail entries, expands capacity when overflow is still valuable, and suppresses high-risk plans through Bayesian feedback regulation [2601.22528]. Live-Evo uses a simpler but explicitly online weighting rule: for each retrieved experience, the update is
\[
Weight_{new} = Weight_{old} + (score_{noexp} - score_{exp}),
\]
while retrieval uses
\[
Score = Weight * Sim(exp,query),
\]
so experiences that improve outcome are reinforced and misleading ones are down-weighted and gradually forgotten [2602.02369]. SEDM formalizes write admission even more directly: a candidate memory is accepted iff
\[
\mathrm{accept}(m) \Longleftrightarrow S \ge \eta,
\]
where \(S=\Delta R-\lambda_L\Delta L-\lambda_T\Delta T\), and later retrieval uses \(s(q,m)=\operatorname{sim}(q,m)\times w(m)\) to combine relevance and empirically validated utility [2509.09498]. SkeMex similarly updates skill utilities from category-normalized downstream advantage and uses the same value signal both for retrieval and for retention decisions such as promotion, deprecation, and deletion [2606.09365].

Revision is another central operator. DCPM preserves belief evolution through doubly linked supersedes chains, so old beliefs are not overwritten but versioned via `Supersedes` and `SupersededBy` pointers [2606.09483]. Metis uses invalidate-and-replace rather than in-place editing for superseded textual memories [2606.24151]. EvoMemBench identifies this as a major fault line in the field: current memory systems are markedly better at retention than at revision under contradiction [2605.18421].

Several systems add explicit mutation or failure-driven variation. DMS uses \(\epsilon\)-mutation: with small probability the agent ignores retrieved memory, re-solves a subtask from scratch, and replaces the old trajectory if the new one is successful and shorter [2601.22528]. FORGE turns failed trajectories into prompt-injected memory artifacts—Rules, Examples, or Mixed—and then uses population broadcast to propagate the best-performing memory state across instances between stages, with graduation freezing converged instances [2605.16233]. MemSkill makes the memory procedures themselves evolvable: a controller selects memory skills, an executor produces structured INSERT/UPDATE/DELETE actions under those skills, and a designer periodically revises or adds skills based on clustered hard cases [2602.02474].

At the most expansive end of the design space, some systems evolve the memory architecture itself. MemEvolve treats a memory system as a four-module architecture \(\Omega=(\mathcal{E},\mathcal{U},\mathcal{R},\mathcal{G})\) over encode, store, retrieve, and manage, then performs outer-loop diagnose-and-design evolution across candidate architectures while the inner loop evolves memory contents [2512.18746]. EvolveMem narrows that meta-evolution to retrieval infrastructure, exposing scoring functions, fusion modes, context budgets, decomposition, verification, and answer-generation policies as a structured action space optimized by an LLM diagnosis module with revert-on-regression and explore-on-stagnation safeguards [2605.13941].

## 4. Retrieval, policy coupling, and memory-conditioned control

Self-evolving memory systems differ sharply in how memory is coupled to decision-making. DMS uses dual-factor retrieval, multiplying precondition similarity and goal similarity so that a subtask memory is reused only when both starting state and intended objective align [2601.22528]. SkeMex retrieves once at episode onset, balancing similarity, utility, and memory strength with temporal decay, and injects selected skills as a separate runtime block rather than appending them to rolling conversation memory [2606.09365]. SEDM’s retrieval score \(s(q,m)=\operatorname{sim}(q,m)\times w(m)\) makes empirical usefulness first-class at read time [2509.09498]. Live-Evo adds a compilation layer: retrieved experiences and a meta-guideline are turned into a task-specific guideline rather than concatenated verbatim [2602.02369]. DCPM keeps read latency low by performing vector search over facts and identity items, traversing supersedes chains, and then surfacing schema, intention, and core-schema nodes through evidence links, without invoking an LLM in the read path [2606.09483].

Other systems couple memory more directly to the policy substrate. TMEM is explicit that prompt-space memory alone cannot make the agent “learn from” experience during an episode; instead, extraction actions update fast weights \(\Delta_t\), and later decisions are sampled from an adapted policy \(\pi_{\theta_0+\Delta_t}\) [2606.04536]. MoLEM likewise internalizes experiential knowledge into auxiliary modules: a router performs key-query matching over expert memories, aggregates latent memory, and injects it into a frozen pretrained reasoner [2605.21951]. DecentMem extends this logic to multi-agent systems: each agent routes between an exploitation pool and an exploration pool according to online-updated weights, retrieves or generates a candidate trajectory, acts, receives stage-wise feedback from an LLM-as-a-judge, and then reweights exploitation probability accordingly [2605.22721].

This range of couplings illustrates a substantive divide in the field. Some systems treat memory as external control context; others use memory to alter the effective policy, latent state, or routing dynamics. A plausible implication is that “self-evolving memory” now names a continuum from adaptive retrieval to adaptive control.

## 5. Empirical evaluation and benchmark evidence

Benchmarking work has made clear that self-evolving memory is not a solved primitive. EvoMemBench, which compares 15 representative memory methods across in-episode and cross-episode, knowledge-oriented and execution-oriented regimes, finds that long-context baselines remain highly competitive, memory helps most when current context is insufficient or tasks are difficult, retrieval-based methods remain strong for knowledge-intensive settings, and procedural or long-term memory methods are more effective for execution-oriented tasks when stored experience matches task structure [2605.18421]. Its core conclusion is negative but important: no single memory form works consistently across all settings [2605.18421].

Within domain-specific systems, however, large gains are repeatedly reported. On AndroidWorld, DMS improves average success rate by \(18.0\%\) and execution stability by \(33.9\%\), and raises Qwen2.5-VL-72B from \(41.0\%\) to \(66.4\%\) success [2601.22528]. On WebVoyager, WebCoach improves a 38B browser-use agent from \(47\%\) to \(61\%\) success while reducing or maintaining the average number of steps [2511.12997]. On AppWorld, Metis improves task accuracy by up to \(20.6\%\) over ReAct while reducing execution cost by up to \(22.8\%\) [2606.24151]. On the live Prophet Arena benchmark over a 10-week horizon, Live-Evo improves Brier score by \(20.8\%\) and increases market returns by \(12.9\%\) [2602.02369].

Parametric, latent, and multi-agent systems show similarly strong results. TMEM consistently outperforms summary-based and retrieval-based baselines on LoCoMo, LongMemEval-S, search, and CL-Bench, and benefits especially strongly from RL because extraction quality directly affects online adaptation [2606.04536]. SkeMex improves DeepSeek-V3.2 from \(48.20\%\) to \(56.08\%\) in offline in-domain testing and remains above ReAct across all reported OOD benchmarks, indicating transferable skill memory [2606.09365]. MoLEM reaches \(73.93\%\) average accuracy after a continual-learning sequence, improving by \(10.40\%\) over the vanilla pretrained baseline while keeping forget and backward transfer at \(0.00\) from Stage 2 onward in the default setting [2605.21951]. FORGE improves average evaluation return by \(1.7\)–\(7.7\times\) over zero-shot and by \(29\)–\(72\%\) over Reflexion in all 12 reported model-representation conditions [2605.16233]. DCPM’s System 2 contributes most on PersonaMem-v2, where it yields gains up to \(+5.20\), matching its design target of implicit cross-session inference rather than surface recall [2606.09483]. In multi-agent settings, DecentMem improves average accuracy by up to \(23.8\%\) over the strongest centralized memory baseline and by up to \(52.5\%\) over the no-memory baseline, while reducing token usage by up to \(49\%\) [2605.22721].

## 6. Limitations, controversies, and open directions

The literature is unusually explicit about limitations. Many mechanisms depend on reliable verification or judging: DMS assumes a verifier accurate enough for strike counts, pruning, and Bayesian inhibition to reflect real utility [2601.22528]; SEDM depends on reproducible replay and meaningful A/B marginal utility estimates [2509.09498]; DecentMem relies on LLM-as-a-judge stage scoring to update routing weights [2605.22721]. Extraction quality is another recurrent bottleneck: bad QA pairs lead to bad online adaptation in TMEM [2606.04536], weak distillation can produce harmful skills in SkeMex [2606.09365], and Meta-guideline or experience summarization errors can mislead Live-Evo [2602.02369].

Cost and latency remain substantial. SkeMex increases average execution steps and wall-clock time relative to ReAct even while improving accuracy [2606.09365]. Metis shows that code memory can reduce execution tokens and steps but incurs much higher construction cost if codification is too eager [2606.24151]. EvolveMem requires repeated benchmark-scale evaluation rounds, diagnosis calls, and guarded reconfiguration, making its self-evolution process closer to autonomous retrieval research than lightweight online adaptation [2605.13941]. MoLEM avoids catastrophic forgetting by appending stage-local modules, but memory grows linearly with stages and still assumes stage-triggered expansion rather than fully online domain discovery [2605.21951].

A deeper controversy concerns generality. EvoMemBench shows that long-context baselines can outperform explicit memory when raw evidence fits within context, and that cross-episode transfer is brittle when stored experience does not align with the target decision process [2605.18421]. Domain-specific assumptions are pervasive: DMS is tightly matched to GUI subtasks and deterministic replay [2601.22528]; DCPM is optimized for persona and diachronic preference reasoning [2606.09483]; DecentMem targets decentralized multi-agent collaboration rather than single-agent control [2605.22721]. A plausible implication is that the field is converging not on one universal memory substrate, but on a family of task-structure-sensitive designs.

Even so, the trajectory of the area is clear. Self-evolving agent memory now spans content evolution, utility-aware governance, revision and forgetting, fast parametric adaptation, latent-memory modularization, skill evolution, retrieval self-optimization, and meta-evolution of the memory architecture itself [2602.02474] [2605.13941] [2512.18746]. The unifying principle is that memory must be selective, revisable, and action-relevant: it must not only preserve the past, but continuously test, reinterpret, and reorganize it in service of future behavior.

Source: https://www.emergentmind.com/topics/self-evolving-agent-memory