MetaMem: Evolving Meta-Memory for LLMs

• MetaMem is a framework that treats memory as an active component, enabling systematic retrieval, organization, and utilization of stored information.
• It employs an iterative, self-reflective process with symbolic updates (ADD, DEL, MOD) to refine memory usage, resulting in measurable performance gains on benchmarks.
• MetaMem encompasses both narrow implementations and broad design paradigms, offering explicit control over memory versioning, rollback, and meta-level reasoning for long-horizon tasks.

MetaMem denotes a line of research that treats memory as an object of explicit control rather than a passive store. In its most specific use, MetaMem is a framework that augments memory systems with a self-evolving meta-memory, aiming to teach LLMs how to effectively utilize memorized knowledge (Xin et al., 27 Jan 2026). In broader usage, MetaMem refers to giving LLMs meta-level control over their memory—structured, inspectable memory representations and operations such as versioning, branching, rollback, and trace, as well as learned decisions about what is worth remembering, when to retrieve it, and how to organize what is known (Li, 23 Jun 2026, Wu et al., 1 Jul 2026).

1. Conceptual scope

MetaMem begins from a distinction between ordinary memory and meta-memory. In the MetaMem framework, the ordinary memory set is written as

$\mathcal{M} = \{m_1, \dots, m_n\},$

where each $m_i$ is a memory unit constructed by a base system such as LightMem, while the meta-memory is

$\mathcal{E}_t = \{e_t^1, \dots, e_t^n\},$

where each $e_t^i$ is a meta-memory unit: a textual rule or experience about how to use memories, not a fact about the world (Xin et al., 27 Jan 2026). The central claim is that many existing memory systems focus on how to store and retrieve memory, but not on how to use memory coherently once retrieved. Fragmented memory units, disrupted logical and temporal relationships, and inconsistent or contradictory evidence then degrade long-horizon reasoning (Xin et al., 27 Jan 2026).

This conception aligns with the cognitive-science notion of metamemory adopted by AutoMem: the learned skill of deciding what is worth remembering, when to retrieve it, and how to organize what is known, formalized as a monitor–control loop with an object level and a meta level (Wu et al., 1 Jul 2026). It also aligns with MemMA’s account of the memory cycle, in which construction, retrieval, and utilization are coupled along a forward path, while downstream failures should feed back into repairs of the memory bank along a backward path (Lin et al., 19 Mar 2026).

The term is therefore both narrower and broader than a single implementation. Narrowly, it names the self-evolving meta-memory framework of "MetaMem: Evolving Meta-Memory for Knowledge Utilization through Self-Reflective Symbolic Optimization" (Xin et al., 27 Jan 2026). Broadly, it names a design paradigm in which a LLM does not merely access memory, but reasons about memory state, retrieval strategy, version history, and editability (Li, 23 Jun 2026).

2. The MetaMem framework as self-evolving meta-memory

The specific MetaMem framework is designed as a layer on top of an ordinary memory system. User interactions are first stored by a base memory mechanism, and MetaMem then learns explicit knowledge utilization experiences that guide the LLM to systematically identify and integrate critical evidence from scattered memory fragments (Xin et al., 27 Jan 2026). During inference, the model answers with a prompt that includes the query $q$, retrieved memory $\mathcal{M}$, and the learned meta-memory $\mathcal{E}_T$: $$y = \text{LLM}(\text{Instruct}_{\text{Gen}}(q,\mathcal{M},\mathcal{E}_T)).$$ The learned meta-memory thus functions as a prompt-level meta-policy for reasoning over retrieved evidence (Xin et al., 27 Jan 2026).

Meta-memory optimization is implemented as an iterative symbolic edit process. For each mini-batch $\tilde{\mathcal{D}}_t$, MetaMem computes a set of update operations

$$\tilde{O}_t = \text{Update}(\tilde{\mathcal{D}}_t,\mathcal{E}_t),$$

and then executes them to obtain

$m_i$0

The permissible edit types are ADD, DEL, and MOD: add a new meta-memory unit, delete an existing one, or modify an existing one (Xin et al., 27 Jan 2026). These units are written in natural language and encode reusable strategies such as prioritizing explicit high-level summaries over fragmented partial details, or identifying inclusion relationships to avoid double-counting subsets already encompassed in a reported total (Xin et al., 27 Jan 2026).

The optimization loop relies on self-reflection. For each training instance $m_i$1, the model samples multiple candidate responses under the current meta-memory, a stronger judge determines whether each response is correct, and the model then self-reflects on why the response was right or wrong and how memory was used or misused (Xin et al., 27 Jan 2026). Those reflections are transformed into symbolic update proposals through Instruct_Action, and contradictory proposals are resolved through Instruct_Filter before execution. The result is a persistent, accumulating set of explicit rules about memory utilization rather than a transient chain-of-thought or a gradient update (Xin et al., 27 Jan 2026).

3. Optimization procedure and empirical behavior

MetaMem was evaluated primarily on LongMemEval, a benchmark with 500 samples spanning Single-Session User, Single-Session Assistant, Multi-Session, Temporal Reasoning, Knowledge Update, and Single-Session Preference; because of the benchmark’s size, the study used 5-fold cross-validation with 400 samples for meta-memory construction and 100 samples for testing in each fold (Xin et al., 27 Jan 2026). The backbone reasoners were Qwen3-30B-A3B-Instruct and Llama3.1-70B-Instruct, while Qwen3-235B-A22B served as the judge (Xin et al., 27 Jan 2026).

Against LightMem, MetaMem improved average accuracy from 67.50 to 71.90 on Qwen3-30B-A3B-Instruct and from 66.17 to 69.08 on Llama3.1-70B-Instruct (Xin et al., 27 Jan 2026). On Qwen3-30B-A3B-Instruct, Multi-Session rose from 62.86 to 69.24, Temporal Reasoning from 64.22 to 69.60, Knowledge Update from 75.76 to 79.18, and Single Preference from 91.67 to 94.16 (Xin et al., 27 Jan 2026). The framework also improved Full Text and RAG when layered on top of them, which the paper presents as evidence that MetaMem is plug-and-play rather than tightly coupled to LightMem (Xin et al., 27 Jan 2026).

Ablation results isolate the role of iterative evolution and reflection. Removing evolution reduced average accuracy from 71.90 to 68.33 on Qwen3-30B-A3B-Instruct and from 69.08 to 66.42 on Llama3.1-70B-Instruct, while removing self-reflection reduced the same averages to 70.02 and 68.33, respectively (Xin et al., 27 Jan 2026). The paper’s qualitative case study on cumulative road-trip distances illustrates the intended behavior: LightMem mis-aggregated 1,800 miles across three trips, 1,200 miles for Yellowstone, a 300-mile first-day segment already included in the Yellowstone total, and a 570-mile planned trip; MetaMem instead used meta-memory rules about prioritizing high-level totals, identifying inclusion relationships, and excluding future plans, yielding the correct total of 3,000 miles (Xin et al., 27 Jan 2026).

The evolution of the meta-memory itself was also analyzed. The proportion of “General” meta-memory units rose from about 65% to more than 80%, suggesting a shift from task-specific heuristics to transferable reasoning patterns (Xin et al., 27 Jan 2026). In data-scaling experiments on ShareGPT, accuracy gains appeared once training used about 400 samples, and perplexity decreased from 3.59 to 3.19 as the number of samples increased to 1000, with gains saturating around 800–1000 (Xin et al., 27 Jan 2026).

4. MetaMem as coordinated control of the memory cycle

MemMA generalizes the same intuition from explicit rule accumulation to coordinated multi-agent control over construction, retrieval, and utilization (Lin et al., 19 Mar 2026). Its diagnosis begins with two failures: strategic blindness on the forward path, where construction and retrieval are driven by local heuristics rather than explicit strategic reasoning, and sparse, delayed supervision on the backward path, where downstream failures rarely translate into direct repairs of the memory bank (Lin et al., 19 Mar 2026).

To address this, MemMA introduces a planner–worker decomposition. A Meta-Thinker $m_i$2 produces structured guidance for construction and retrieval; a Memory Manager $m_i$3 executes ADD, UPDATE, DELETE, or NONE over the current backend; a Query Reasoner $m_i$4 performs diagnosis-guided iterative retrieval; and an Answer Agent $m_i$5 generates the final response (Lin et al., 19 Mar 2026). The construction-side guidance extracts atomic facts and indicates importance or redundancy, while the retrieval-side guidance performs an answerability check, identifies key gaps, specifies missing speakers or temporal anchors, and supplies retrieval guidance for the next query rewrite (Lin et al., 19 Mar 2026). This is a direct implementation of a meta-memory controller that reasons about what memory is missing or misused.

MemMA’s backward path is its in-situ self-evolving memory construction. After each session $m_i$6, the system builds a provisional memory $m_i$7, synthesizes probe QA pairs $m_i$8, uses the current memory to answer them, and turns failures into repair proposals $m_i$9, which are then consolidated through SKIP, MERGE, or INSERT to produce a refined memory $\mathcal{E}_t = \{e_t^1, \dots, e_t^n\},$0 (Lin et al., 19 Mar 2026). This converts sparse end-task supervision into dense local supervision on memory quality.

On LoCoMo, MemMA consistently improved multiple backends. With GPT-4o-mini, LightMem improved from ACC 75.66 to 81.58, A-Mem from 52.63 to 78.29, and Single-Agent from 52.60 to 84.87 (Lin et al., 19 Mar 2026). Ablations showed that removing iterative retrieval caused the largest drop, while removing in-situ self-evolution caused the second-largest drop (Lin et al., 19 Mar 2026). This suggests that MetaMem can be understood not only as a set of textual rules, but also as explicit control over the entire memory cycle.

5. Broader MetaMem patterns: metamemory skills, autonomous memory, and anticipatory recall

AutoMem places MetaMem in a cognitive-science frame. It treats memory management as an active, trainable skill and promotes file-system operations to first-class memory actions alongside task actions, letting the model itself decide what to store, what to look up, and how to structure its records (Wu et al., 1 Jul 2026). Its first loop revises the memory structure that shapes how the agent interacts with its memory files, while its second loop uses the agent’s own good memory decisions as training signal to sharpen memory proficiency directly (Wu et al., 1 Jul 2026). Across Crafter, MiniHack, and NetHack, optimizing memory alone improved the base agent’s performance about $\mathcal{E}_t = \{e_t^1, \dots, e_t^n\},$1–$\mathcal{E}_t = \{e_t^1, \dots, e_t^n\},$2, and the final 32B system reached 51.36 on Crafter, 30.00 on MiniHack, and 1.85 on NetHack (Wu et al., 1 Jul 2026). In MetaMem terms, AutoMem makes the scaffold and the policy over memory actions jointly optimizable.

U-Mem extends the idea to autonomous memory agents. It models a frozen backbone $\mathcal{E}_t = \{e_t^1, \dots, e_t^n\},$3, a time-varying memory store $\mathcal{E}_t = \{e_t^1, \dots, e_t^n\},$4, and a learning-by-using loop in which retrieval, inference, feedback, and memory evolution are repeated over a task stream (Wu et al., 25 Feb 2026). Its cost-aware knowledge-extraction cascade escalates from cheap self or teacher signals to tool-verified research and then expert feedback only when needed, while semantic-aware Thompson sampling balances exploration and exploitation over memories (Wu et al., 25 Feb 2026). On Qwen2.5-7B, U-Mem improved HotpotQA from 37.80% to 52.40% and AIME25 from 6.67% to 13.33%; the abstract summarizes this as improving HotpotQA by 14.6 points and AIME25 by 7.33 points (Wu et al., 25 Feb 2026). This broadens MetaMem from memory utilization to cost-aware knowledge acquisition and memory curation.

T-Mem reframes long-term conversational memory in terms of reachability. It argues that similarity-based memory supports descriptive recall but misses associative recall, where query and memory are tied only by a latent semantic arc (Guo et al., 13 Jun 2026). Its solution is to generate triggers at write time: item-level Entity and Bridge triggers, and scene-level Scene and Horizon triggers, so that each memory remains reachable from both surface-similar and relevance-bound queries (Guo et al., 13 Jun 2026). On LoCoMo and LoCoMo-Plus, T-Mem reached 80.26 and 74.81, respectively, with a LoCoMo/LoCoMo-Plus gap of 5.45, whereas HyperMem’s gap was 28.38 and MemOS’s gap was 43.13 (Guo et al., 13 Jun 2026). This suggests a MetaMem system can operate not only by storing rules about utilization, but also by constructing anticipatory indexing structures at write time.

CALMem provides an application-layer variant of the same general agenda. It combines an episodic memory layer built on sliding-window vector embeddings of conversation history with a semantic memory layer of agent-writable structured facts, and uses a token-budget-adaptive injection mechanism called the MOIM (Message of Injected Memory) to retrieve and inject relevant past context each turn (Jena et al., 20 May 2026). A key contribution is intra-session retrieval: compacted-away turns from the current session remain searchable, which the paper describes as closing a gap unaddressed by prior work (Jena et al., 20 May 2026). While CALMem does not present a self-evolving meta-memory, it exemplifies MetaMem’s application-layer concern with how memory should be organized and injected rather than merely stored.

6. Explicit memory substrates, version-aware operations, and open problems

A separate but closely related branch treats explicit memory representations as the substrate on which MetaMem-style control can operate. MeMo, in "MeMo: Towards LLMs with Associative Memory Mechanisms," proposes a language modeling architecture that explicitly memorizes sequences of tokens in layered associative memories, using Correlation Matrix Memories, random Gaussian embeddings, and Johnson–Lindenstrauss transforms, with memorization, retrieval, and forgetting implemented by architectural operations rather than gradient-based learning (Zanzotto et al., 18 Feb 2025). Because memories are sums of outer products, inserting and deleting are direct linear operations, and the paper emphasizes transparency and the possibility of model editing, including forgetting texts (Zanzotto et al., 18 Feb 2025). This suggests a memory substrate on which a higher-level MetaMem controller could operate.

That possibility is made explicit in "Towards Version-aware Operations and Transaction Memories for Multi-layer MeMo," which states that in a broad sense, MetaMem refers to giving LLMs structured, inspectable memory representations and operations like versioning, branching, rollback, and trace (Li, 23 Jun 2026). The paper introduces a version-aware operation layer above MeMo’s primitive memo, forget, and retrieve interface; high-level operations such as replace, obsolete, keep-history, rollback, and trace are compiled into ordered transactions of primitive edits (Li, 23 Jun 2026). Two auxiliary CMMs are added: a Version CMM (V-CMM) for mapping version transitions to transaction handles, and a Transaction CMM (T-CMM) for storing reusable change contents and inverse programs (Li, 23 Jun 2026). Here MetaMem becomes literally memory about memory edits.

The term is bounded, however, by several unresolved problems. In the original MetaMem framework, performance depends on judge quality, long and nuanced reflections, and a careful stopping point for evolution; too many steps can overfit the meta-memory, and in-domain meta-memory still outperforms out-of-domain meta-memory (Xin et al., 27 Jan 2026). MemMA adds concerns about probe quality, session structure assumptions, and the cost of repeated LLM calls (Lin et al., 19 Mar 2026). AutoMem remains episodic-memory-only, game-only, and dependent on strong meta-LLMs, with separate scaffolds and specialists per environment (Wu et al., 1 Jul 2026). T-Mem depends on a strong construction LLM, currently uses an offline construction pipeline, and lacks a fact-level associative benchmark (Guo et al., 13 Jun 2026). Version-aware MeMo identifies further open questions around scaling to large knowledge bases and very long version histories, interference between versions, automation of change detection, and complex dependency management between transactions (Li, 23 Jun 2026).

Across these variants, a stable core remains. MetaMem denotes the move from memory as a passive artifact to memory as a controlled, inspectable, revisable, and in some cases self-improving component of the language-model system. The exact implementation may be a self-evolving set of knowledge utilization experiences, a planner that coordinates the memory cycle, a trainable memory-management skill, a trigger-generating anticipatory memory, or a transaction-aware explicit memory substrate; the common claim is that effective long-horizon intelligence requires not only memory, but explicit reasoning about memory itself (Xin et al., 27 Jan 2026, Lin et al., 19 Mar 2026, Wu et al., 1 Jul 2026, Guo et al., 13 Jun 2026, Li, 23 Jun 2026).