RecMem: Efficient Memory Consolidation for LLMs

Updated 5 July 2026

The paper introduces a cost-efficient framework that triggers LLM-based consolidation only when semantically recurring interactions reach a defined threshold.
RecMem is defined as a system that employs a three-tier architecture—subconscious, episodic, and semantic stores—to manage raw data, narrative events, and precise facts.
Empirical results demonstrate that RecMem reduces construction tokens by up to 87% while exceeding accuracy compared to other state-of-the-art memory systems.

RecMem, short for "Recurrence-based Memory Consolidation for Efficient and Effective Long-Running LLM Agents," is a long-term memory system for LLM agents that redefines when costly memory consolidation should occur. Instead of invoking an LLM on every incoming interaction, RecMem stores all user-agent interactions in a lightweight subconscious memory layer and triggers LLM-based extraction only when semantically similar interactions exhibit sustained recurrence. Its design combines a subconscious store for raw interaction units, an episodic store for time-anchored event narratives, and a semantic store for atomic facts. The system is motivated by the claim that eager consolidation is unnecessarily expensive for long-running agents, and it is evaluated as reducing the memory construction token cost of three SOTA memory systems by up to 87% while exceeding their accuracy (Dai et al., 15 May 2026).

1. Problem formulation and consolidation principle

RecMem addresses long-running conversational agents that accumulate an interaction stream

$\mathcal{O}_{1:t} = \{o_1, \ldots, o_t\}, \quad o_t = (s_t, x_t, \tau_t),$

where $s_t$ is the speaker, $x_t$ the text, and $\tau_t$ the timestamp. In such agents, external memory is used because LLMs have limited context windows and are “lost in the middle” on very long inputs. RecMem is explicitly training-free and text-based, and it treats memory as an external system rather than as parametric knowledge inside model weights (Dai et al., 15 May 2026).

The central contrast in RecMem is between eager consolidation and recurrence-based consolidation. Eager systems such as Mem0, A‑Mem, MemoryOS, Zep, and TreeMem invoke the LLM on every incoming interaction to extract facts, update notes, or rewrite memory structures. RecMem argues that this is token expensive, scales poorly, and performs unnecessary structuring on interactions that may never be queried. Its alternative is to buffer all interactions cheaply and invoke the LLM only when repeated, semantically similar content indicates that a topic has become memory-worthy.

The paper explicitly grounds this design in a cognitive analogy: only repeatedly activated experiences are consolidated into long-term memory, whereas isolated, non-recurring events remain transient. In RecMem, recurrence is implemented operationally rather than cognitively. A new interaction is compared with prior subconscious entries, and consolidation is triggered only when the new item and its similar predecessors form a sufficiently large semantic cluster. This suggests a design in which memory cost scales with the number of recurring topics rather than with the number of turns.

2. Three-tier memory architecture

RecMem organizes external memory into three layers. The architecture is multi-store, but the distinction is functional rather than merely storage-oriented: subconscious memory preserves raw evidence, episodic memory captures temporally organized narratives, and semantic memory stores fine-grained reusable facts (Dai et al., 15 May 2026).

Layer	Stored content	Construction trigger
Subconscious memory	Raw user–assistant interaction units with embeddings	Every turn
Episodic memory	Time-anchored event narratives	Merge-first update or sustained recurrence
Semantic memory	Atomic facts	Semantic refinement over episodes

The subconscious layer is the universal ingestion path. RecMem defines interaction units per user–assistant turn rather than per single message or large chunk: $\mathcal{H}_t = (u_1, \ldots, u_t), \qquad u_i = (m_i^{\mathrm{usr}}, m_i^{\mathrm{ast}}, \tau_i).$ Each unit is embedded as

$s_i = (v_i, u_i), \qquad v_i = \Phi(u_i),$

using a lightweight dense encoder. In the experiments, the embedding model is OpenAI’s text-embedding-3-small. The resulting store is faithful, cheap, and incrementally searchable.

The episodic layer stores high-level event narratives summarizing how a topic evolves over multiple turns. These are not mere summaries of arbitrary windows; they are topic-centered episodes, chronologically organized and intended to support temporal reasoning and coherent reconstruction of evolving states.

The semantic layer stores atomic facts such as user preferences, names, or updated states. Unlike the episodic layer, which preserves narrative structure, semantic memory is designed for precise fact retrieval. RecMem’s architecture therefore separates raw evidence retention, event-level abstraction, and fact-level precision.

A common misunderstanding is to treat episodic and semantic memory as redundant. The ablations contradict that view: removing episodic memory produces a small drop, whereas removing semantic memory produces a much larger one, indicating that event narratives do not fully substitute for fine-grained factual storage.

3. Recurrence detection and consolidation mechanics

The triggering mechanism is the defining algorithmic component of RecMem. For a new interaction $u_i$ with embedding $v_i$ , RecMem first retrieves the top- $k$ nearest subconscious entries: $\mathcal{N}_i = \text{TopK}_k(\mathcal{S}_{\text{sub}}, v_i).$ It then filters this neighborhood by a similarity threshold $s_t$ 0: $s_t$ 1 If the recurrence count satisfies

$s_t$ 2

RecMem declares sustained recurrence and forms a semantic cluster

$s_t$ 3

This criterion distinguishes one-off interactions from recurring topics. Interactions that fail the threshold remain retrievable in subconscious memory but are not summarized. The mechanism is therefore selective rather than lossy: non-consolidated material is not discarded, only left unstructured.

RecMem also uses merge-first episodic updates. Before cluster-based consolidation, the system retrieves the nearest existing episodic memory $s_t$ 4. If

$s_t$ 5

the new interaction is merged directly into the existing episode via

$s_t$ 6

The merge prompt is intended to preserve chronological order and update evolving details such as changing preferences. This avoids proliferation of small, fragmented episodes and maintains topic continuity.

The paper reports domain-specific thresholds. For LoCoMo, $s_t$ 7 and $s_t$ 8; for LongMemEval-S, $s_t$ 9 and $x_t$ 0. Hyperparameter analysis shows that increasing $x_t$ 1 reduces token cost but can hurt accuracy when set too high.

Once a cluster $x_t$ 2 is formed, RecMem extracts its underlying interaction units,

$x_t$ 3

sorts them by timestamp, formats them into a temporal sequence, and passes the resulting text to an episodic extraction LLM: $x_t$ 4 The output is a set of episodes. Each episode is a coherent narrative that may itself segment a cluster into subtopics if multiple themes appear.

Semantic memory is then produced by semantic refinement, which operates on top of newly created episodes rather than directly from raw text. For each episode $x_t$ 5, RecMem retrieves related existing semantic facts

$x_t$ 6

and applies

$x_t$ 7

The refiner is instructed to perform Detail Recovery and Fact Maintenance / Update. It recovers omitted names, times, numbers, and preferences from the raw evidence, while also de-duplicating against prior facts and handling evolving states. This refinement step is specifically motivated by the lossy nature of episodic summaries.

At query time, RecMem encodes the question $x_t$ 8 as $x_t$ 9 and retrieves from all three stores: $\tau_t$ 0

$\tau_t$ 1

$\tau_t$ 2

The default budgets are $\tau_t$ 3, $\tau_t$ 4, and $\tau_t$ 5, with the design choice $\tau_t$ 6. Retrieved raw interactions, episodic narratives, and semantic facts are combined into a structured prompt for the answer LLM.

A plausible implication is that RecMem’s answer-time behavior depends on complementary failure modes across stores. The subconscious layer protects against missed consolidation, episodic memory supports chronology and narrative continuity, and semantic memory supplies exact details. The ablations support this interpretation: subconscious memory is indispensable for completeness, whereas semantic memory is the main source of precision.

5. Empirical performance, ablations, and efficiency profile

RecMem is evaluated on LoCoMo and LongMemEval-S with GPT‑4o-mini and GPT‑4.1-mini backbones. The baselines are Full Context, Naive RAG, Mem0, A‑Mem, and MemoryOS, all of which use eager consolidation in some form (Dai et al., 15 May 2026).

On LoCoMo with GPT‑4.1‑mini, overall accuracies are reported as: Full Context 84.18, Naive RAG 54.42, MemoryOS 67.60, Mem0 62.92, A‑Mem 68.83, and RecMem 81.10. Construction tokens are 400.7K for MemoryOS, 1520.8K for Mem0, 1459.9K for A‑Mem, and 193.2K for RecMem. The paper therefore reports RecMem as the best among memory systems and second only to Full Context on this benchmark, while reducing construction tokens by 87.3% versus Mem0 and 86.8% versus A‑Mem.

On LongMemEval-S with GPT‑4.1‑mini, RecMem reaches 76.80 overall, compared with 74.40 for MemoryOS, 71.20 for Mem0, 71.60 for A‑Mem, 67.00 for Naive RAG, and 66.20 for Full Context. Construction tokens are 669.22K for MemoryOS, 1626.54K for Mem0, 1264.25K for A‑Mem, and 365.49K for RecMem. The reported savings are approximately 77.5% versus Mem0 and 71.1% versus A‑Mem.

The ablations are structurally revealing. On LoCoMo with GPT‑4.1‑mini, removing subconscious memory drops accuracy from 81.10 to 51.88, showing that the raw store is the only place where all interactions persist. Removing episodic memory yields 79.94, a small decline. Removing semantic memory yields 70.58, a much larger loss. A direct extraction variant, in which semantic facts are extracted from raw text without episodic-guided refinement, yields 74.22 versus 79.94 when episodic memory exists but semantic uses direct extraction. These results indicate that semantic refinement is not just a bookkeeping layer; it is the main mechanism that recovers omitted factual detail.

The efficiency argument is as central as the accuracy argument. Construction-time LLM calls are incurred only when merge-first updates or recurrence-triggered consolidation occurs. Query-time token usage remains comparable to baselines. RecMem is therefore explicitly presented as a system whose LLM usage scales with recurring topics rather than with total interaction volume.

6. Relation to adjacent memory research, evaluation, and limitations

RecMem’s main contribution is to the timing of consolidation, not to contextual validation after retrieval or reflective control during search. Subsequent work has made those distinctions explicit. RaMem characterizes “RecMem-style systems” as architectures centered on similarity-based retrieval and compression, and argues that retrieval must be supplemented by contextual verifiability so that a retrieved fragment can be checked against the correct episode rather than merely judged content-relevant (Yang et al., 22 Jun 2026). In that framing, RecMem answers when to consolidate, whereas RaMem answers how to verify that retrieved memory is valid evidence.

R^2-Mem operates at another layer. It is described as an experience layer for deep memory search systems, distilling reusable planning and reflection rules from prior search trajectories and then using them to guide future retrieval (Wang et al., 13 May 2026). A plausible synthesis is that RecMem can supply the long-term store, while reflective systems such as R^2-Mem regulate how that store is queried.

RealMem sharpens the evaluation demands placed on long-term memory systems. It introduces “long-term project-oriented” interactions with average per-user context of 269,190 tokens and approximately 205 sessions, emphasizing dynamic state evolution, interleaved projects, schedule management, and proactive alignment (Bian et al., 11 Jan 2026). These are settings in which recurrence alone may be insufficient as a salience signal, because rare but critical constraints can matter disproportionately.

That concern is closely related to one of RecMem’s stated limitations: recurrence is only a proxy for salience. Rare but important events may never reach the recurrence threshold and therefore remain unconsolidated, even though they are preserved in subconscious memory. Other limitations are the use of hand-tuned thresholds $\tau_t$ 7 and $\tau_t$ 8, sensitivity to embedding quality, and the absence of explicit salience cues beyond recurrence. The paper also notes that embedding noise or domain shift can cause missed or incorrect clusters, and that multimodal inputs are outside its scope (Dai et al., 15 May 2026).

Finally, MemDelta provides an evaluative caution that bears directly on RecMem-like systems. It argues that reported memory gains are often confounded by changes in embedding models, answer LLMs, or retrieval stacks, and shows that swapping only the embedding model in an otherwise identical pipeline shifts accuracy by +6.2 percentage points on LongMemEval-S (Wang, 29 Jun 2026). This suggests that RecMem’s empirical gains should be interpreted together with careful baseline control, especially when comparing against RAG or extraction-based systems.

RecMem is therefore best understood as a cost-aware memory architecture that separates cheap universal retention from expensive selective consolidation. Its conceptual move is simple but consequential: not every interaction deserves immediate LLM-mediated structuring. By using recurrence to gate memory formation, and by combining subconscious retention with episodic abstraction and semantic refinement, it offers a specific answer to the engineering problem of long-running agent memory: preserve everything cheaply, consolidate only what recurs, and recover precision through a fact-refinement stage when consolidation actually occurs (Dai et al., 15 May 2026).