NapMem: Navigation Over Pyramid Memory

Updated 8 July 2026

The paper introduces NapMem, a framework that treats memory as a structured action space, enabling LLM agents to select optimal memory abstractions.
It organizes user history into a pyramid of four layers—raw conversations, memory records, topic tracks, and user profiles—to support multi-level evidence retrieval.
Reinforcement learning optimizes the navigation policy, reducing tool calls and improving benchmark performance in long-term memory tasks.

Searching arXiv for the primary NapMem paper and closely related memory-system papers to ground the article. NapMem, short for “Navigate over Pyramid Memory,” is a framework for long-term user memory in LLM agents that treats memory as a structured action space rather than passively retrieved context. It organizes user history into a linked multi-granularity memory pyramid—raw conversations, typed memory records, topic tracks, and a user profile—and exposes these levels through memory tools. The agent is trained with memory-tool reinforcement learning to select memory according to the query and intermediate evidence, allowing it to inspect different memory granularities before answering (Xu et al., 7 Jul 2026).

1. Conceptual basis and problem setting

NapMem is motivated by a limitation in typical RAG and user-memory systems: memory is usually handled as a system-level retrieval module, so the LLM receives a bundle of text that an external component has already decided is relevant. In that setting, the model does not control which memory representation is used, whether to re-query or refine, or how many hops to take. NapMem instead makes memory access “agent-native”: the agent decides whether to use memory, at what abstraction level, and when it has enough evidence (Xu et al., 7 Jul 2026).

This design is formalized as a tool-use policy over a persistent memory bank. At step $t$ , the next action is sampled as

$a_t \sim \pi_\theta(\cdot \mid q, \mathcal{M}, a_{<t}, o_{<t}),$

where actions include memory-tool calls or a decision to stop and answer. The full trajectory is

$\tau = (a_1, o_1, \ldots, a_k, o_k, y),$

with $y$ the final textual answer. The key claim is not merely that memory should be retrieved, but that memory use itself should be learned as a sequence of decisions conditioned on the query and accumulated evidence (Xu et al., 7 Jul 2026).

A common misconception is to treat NapMem as a variant of passive retrieval with extra storage layers. The framework differs precisely at the interface between the agent and the memory system: the pyramid is not only a storage hierarchy, but a structured action space with explicit tools over multiple granularities. Another misconception is to equate it with profile summarization. The profile is only one layer; the framework also supports direct navigation over topic tracks, typed records, and raw message-level evidence.

2. Pyramid memory representation

NapMem builds a separate memory pyramid for each user. Construction is incremental and bottom-up over chronological sessions. The four layers are linked through provenance relations, enabling top-down or bottom-up traversal during inference (Xu et al., 7 Jul 2026).

Layer	Contents	Role
Raw conversations	Original user-assistant messages with raw text, speaker role, timestamp, session ID, message ID	Highest-fidelity evidence
Memory records	Typed memories: fact, event, instruction, preference	Main semantic unit
Topic tracks	Markdown files with topic narrative, summary, and evidence section with record IDs	Medium-range abstraction
User profile	Single Markdown file with basic information, long-term preferences, current context, interaction protocol, high-level patterns	Global compact summary

At the raw-conversation layer, messages are stored as an append-only log, for example as JSONL, together with an embedding-based index using Qwen3-Embedding-0.6B and a keyword index. This layer preserves the original evidentiary substrate: raw text, timestamps, session identifiers, and persistent message identifiers (Xu et al., 7 Jul 2026).

The memory-record layer converts conversation snippets into compact typed memories. NapMem defines four record types: fact, event, instruction, and preference. Each record stores a persistent record ID, type, content, source_message_ids, timestamps, session information, metadata, and a priority score. Records are created by an extraction step and then reconciled against existing records via hybrid retrieval. The reconciliation model chooses one of four actions—store, skip, update, or merge—and preserves provenance by unioning source_message_ids and timestamps when records are merged (Xu et al., 7 Jul 2026).

Extraction itself follows a staged flush schedule that grows from 1 to 2 to 4 to 5 turns. The extraction model receives newly appended messages, nearby background messages, and recent records as context, and outputs JSON containing a scene_name, message_ids, and a list of extracted memories. The extraction heuristics explicitly exclude trivial chit-chat, ephemeral one-offs, and repeated content, while retaining answerable details such as names, dates, and quantities (Xu et al., 7 Jul 2026).

Topic tracks provide medium-range abstraction across sessions. Each topic track is a Markdown file with metadata, a short summary, a narrative description of an ongoing theme, and an explicit evidence section referencing record IDs. A topic update is triggered when there are unprocessed records and a session or date boundary is crossed, when more than some threshold of new records—given as, for example, 20—have accumulated since the last update, or at end of replay or final flush. The topic writer can update an existing track, merge two tracks, or create a new one (Xu et al., 7 Jul 2026).

The user profile is a single compact Markdown summary built from topic tracks. It contains basic user information, long-term preferences, current context or state, interaction “protocol,” and high-level patterns and evolution. The profile is revised under a character budget of about 2000 characters and is required to remain grounded in topic tracks rather than inferred from file names or metadata (Xu et al., 7 Jul 2026).

The provenance structure is central. Profile statements are grounded in topic tracks, topic tracks reference memory records, and records reference source messages. This yields a downward path—profile $\rightarrow$ topics $\rightarrow$ records $\rightarrow$ raw conversations—which is the structural basis for NapMem’s navigation policy.

NapMem exposes the pyramid through five memory tools (Xu et al., 7 Jul 2026).

Tool	Input	Function
`search_records(query)`	Free-text query	Hybrid search over records; returns top-5 records
`search_conversation(query)`	Free-text query	Hybrid search over raw messages; returns top-5 snippets
`get_records(record_id_list)`	Record IDs	Fetch exact structured records
`get_conversation(message_id_list)`	Message IDs	Fetch exact raw messages
`read_file(file_name)`	File name	Read topic tracks or the profile

These tools support both top-down and bottom-up retrieval policies. Top-down navigation starts at read_file(profile) or a topic-track file, then descends to records or raw messages when verification is needed. Bottom-up navigation starts from search_records or search_conversation and may later consult higher abstractions if local evidence is insufficient. The model conditions each next tool call on the current query and all tool outputs collected so far, so the decision boundary is not fixed at the start of the episode (Xu et al., 7 Jul 2026).

Training uses Qwen3.5-9B as the base model. The RL data are memory-intensive queries drawn from LoCoMo and PersonaMem-v2, with user-level train/validation/test splits of 60/20/20. LongMemEval is never used in training and serves only as an out-of-domain test. The reported RL hyperparameters include GRPO, group size 4, batch size 8, learning rate $1 \times 10^{-6}$ , max assistant turns 5, KL coefficient 0.001, bfloat16, and 5 epochs (Xu et al., 7 Jul 2026).

NapMem uses terminal-only rewards. For a completed trajectory $\tau$ , the reward is

$r(\tau) = \begin{cases} -1, & F=0,\ 1, & F=1,\ C=1,\ U=1,\ 0, & F=1,\ C=1,\ U=0,\ -0.5, & F=1,\ C=0,\ U=1,\ -1, & F=1,\ C=0,\ U=0, \end{cases}$

where $a_t \sim \pi_\theta(\cdot \mid q, \mathcal{M}, a_{<t}, o_{<t}),$ 0 is format validity, $a_t \sim \pi_\theta(\cdot \mid q, \mathcal{M}, a_{<t}, o_{<t}),$ 1 is correctness, and $a_t \sim \pi_\theta(\cdot \mid q, \mathcal{M}, a_{<t}, o_{<t}),$ 2 indicates whether at least one memory tool was used. For open-ended tasks, correctness is judged by Claude-Opus-4.6 under a strict 0/1 prompt; on a 100-example validation against human majority labels, this judge achieved 99% accuracy and Cohen’s $a_t \sim \pi_\theta(\cdot \mid q, \mathcal{M}, a_{<t}, o_{<t}),$ 3 (Xu et al., 7 Jul 2026).

The policy optimization uses GRPO with group-relative advantage

$a_t \sim \pi_\theta(\cdot \mid q, \mathcal{M}, a_{<t}, o_{<t}),$ 4

and objective

$a_t \sim \pi_\theta(\cdot \mid q, \mathcal{M}, a_{<t}, o_{<t}),$ 5

with

$a_t \sim \pi_\theta(\cdot \mid q, \mathcal{M}, a_{<t}, o_{<t}),$ 6

The same trajectory-level advantage is applied to both tool-call tokens and final-answer tokens, so the learned policy jointly shapes navigation and answer production (Xu et al., 7 Jul 2026).

Inference uses a hard budget of at most 4 memory tool-calling turns per query in both training and evaluation; exceeding this budget counts as failure. This explicit cap is important because NapMem’s claim is not simply that more search improves answers, but that the model can learn concise, targeted navigation policies.

4. Empirical performance and observed behavior

NapMem is evaluated on three memory-intensive benchmarks: LoCoMo, LongMemEval, and PersonaMem-v2. LoCoMo is converted into speaker-specific user–assistant histories and yields a final test set of 1,315 questions. LongMemEval contributes 100 held-out test questions. PersonaMem-v2 uses 200 users and a test set of 911 multiple-choice questions with shuffled answer options (Xu et al., 7 Jul 2026).

Benchmark	NapMem-9B w/ RL	Comparison
LoCoMo	F1 41.28, L-J 59.92	Mem0: F1 41.19, L-J 60.86
LongMemEval	F1 57.41, L-J 80.33	Mem0: F1 53.86, L-J 78.00
PersonaMem-v2	Acc 47.97	Mem0: Acc 38.89
Average metric	62.74	NapMem-397B: 59.85; Mem0: 59.25

The average metric is defined as the mean of LoCoMo L-J, LongMemEval L-J, and PersonaMem-v2 accuracy. On this aggregate measure, NapMem-9B w/ RL achieves the best reported value, 62.74, exceeding both the untrained NapMem-397B variant at 59.85 and Mem0 at 59.25 (Xu et al., 7 Jul 2026).

A notable result is that a 9B model with learned navigation exceeds a 397B model using the same memory pyramid but no additional RL training. This suggests that query-time policy learning is not merely a marginal optimization over structure; it is a major determinant of memory utility. A plausible implication is that, for long-horizon user memory, architectural structure and policy learning can substitute for substantial increases in base-model scale.

NapMem is also evaluated on non-memory tasks. On GPQA-D, base Qwen3.5-9B scores 53.03% accuracy, NapMem-9B without RL scores 55.74% with a 34.51% memory call rate, and NapMem-9B w/ RL scores 57.58% with only a 6.90% memory call rate. On BFCL-v3, task accuracy remains 95.00 for both the base model and NapMem-9B w/ RL, with 0% memory call rate for the NapMem variants. On V*Bench, answer accuracy rises from 84.82 for the base model to 91.10 for NapMem-9B w/ RL, again with 0% memory call rate (Xu et al., 7 Jul 2026).

These non-memory results matter because they constrain a possible objection: that memory-tool RL might induce indiscriminate tool dependence. The reported behavior is the opposite. RL substantially reduces unnecessary memory usage on GPQA-D and leaves BFCL-v3 and V*Bench memory call rates at zero.

Ablation studies further isolate the source of the gains. Full NapMem scores 62.74 on the average metric. Removing RL reduces this to 48.39. Removing navigation while keeping the same memory sources yields 54.08. Restricting the system to records-only tools lowers the score to 44.93. Removing upper levels, so that only raw conversations and records remain, gives 54.11. These numbers identify two critical components: the learned navigation policy and the presence of higher abstractions such as topic tracks and the profile (Xu et al., 7 Jul 2026).

The system is also comparatively storage-efficient. Total memory-bank storage across PersonaMem-v2, LongMemEval, and LoCoMo is reported as 4.83 GiB for NapMem, versus 10.44 GiB for Mem0, 23.10 GiB for MemOS, 14.10 GiB for Zep, 6.68 GiB for MemoryOS, and 2.99 GiB for AgeMem. Only AgeMem is smaller, but it performs substantially worse on the main benchmarks (Xu et al., 7 Jul 2026).

Behavioral analyses show that NapMem-9B w/ RL averages 2.15 tool calls per query, uses multi-level navigation in 79.63% of cases, and attains an evidence-hit ratio of 34.92%. The corresponding no-RL variant averages 3.97 tool calls with a 20.66% evidence-hit ratio. Larger NapMem variants, including 122B and 397B, more often start with file reading at higher levels, whereas the 9B variants more often start at records or conversations. The reported interpretation is that larger models prefer higher-level user context, while smaller models rely more heavily on compact records and raw snippets (Xu et al., 7 Jul 2026).

A representative success case involves the question “What state did Nate visit?” Direct conversation and record search are noisy, but reading a topic track describing a Tampa beach visit surfaces the necessary evidence. The model then infers Florida from Tampa. This example captures the practical purpose of the pyramid: higher layers need not restate every answer verbatim, but can expose evidence at a level where downstream inference becomes straightforward (Xu et al., 7 Jul 2026).

5. Position within the long-term memory research landscape

NapMem occupies a specific point in a broader design space of LLM memory systems. Several nearby works clarify what NapMem does and does not attempt to solve.

D-MEM describes “NapMem-style systems” as systems that give an LLM agent a long-term memory that is not just a log, but a structured, periodically consolidated store, with “nap” phases where the agent reflects, compresses, and reorganizes experience. D-MEM then proposes an event-based alternative using Reward Prediction Error routing, so that only high-RPE inputs trigger deep restructuring (Song et al., 15 Mar 2026). Relative to that framing, NapMem is primarily a query-time memory-use framework: it emphasizes active navigation over a provenance-linked pyramid rather than RPE-gated consolidation.

The “Human-Inspired Memory Architecture for LLM Agents” proposes a full memory lifecycle with sleep-phase consolidation, interference-based forgetting, engram maturation, reconsolidation upon retrieval, entity knowledge graphs, and hybrid multi-cue retrieval (Kerestecioglu et al., 8 May 2026). That work is closest to a biological or lifecycle notion of “nap,” whereas NapMem concentrates on how an agent should use memory at inference time. The two are complementary rather than redundant.

T-Mem argues that standard memory systems are reachability-bounded by lexical or dense similarity and therefore cover descriptive recall but fail on associative recall. Its solution is write-time rehearsal through entity, bridge, scene, and horizon triggers, enabling both descriptive and associative recall across facts and full exchanges (Guo et al., 13 Jun 2026). NapMem addresses a different bottleneck: not the reachability of stored content under similarity search, but the agent’s ability to choose among multiple memory granularities and provenance levels during reasoning. This suggests that trigger-based reachability and action-space navigation are orthogonal axes.

MetaMem adds a self-evolving meta-memory of “knowledge utilization experiences,” represented as natural-language rules that teach an LLM how to interpret, filter, and combine scattered memory fragments (Xin et al., 27 Jan 2026). A plausible implication is that a MetaMem-style layer could sit above NapMem, learning usage policies such as when to privilege topic tracks over records or when to descend from profile-level evidence to source-message verification.

AMP, or memorywire, addresses a different layer of the stack: a vendor-neutral wire format for memory operations such as remember, recall, forget, merge, and expire, with memory types, backend adapters, RRF fusion, and optional HITL governance (Munirathinam, 31 May 2026). For NapMem-like systems, this suggests an interoperability and governance path rather than a change in reasoning policy.

MemCompiler is perhaps the closest conceptual relative at query time. It critiques “Ahead-of-time Monolithic Memory Injection” in embodied agents and replaces it with State-Conditioned Memory Compilation over a Brief State, with text and latent Soft-Mem channels (Ding et al., 8 May 2026). Both systems reject passive retrieval. The major difference is domain and interface: MemCompiler is a step-wise embodied control policy over runtime state, whereas NapMem is a long-term user-memory framework over raw conversations, records, topic tracks, and profiles.

Taken together, these comparisons indicate that NapMem’s distinctive contribution is not universal memory management, not anticipatory trigger generation, and not protocol standardization. It is the coupling of a linked multi-granularity memory pyramid with a learned policy that treats memory access itself as a structured action space.

6. Limitations, failure modes, and future directions

The reported limitations are explicit. First, evaluation is centered on benchmarked conversational settings—LoCoMo, LongMemEval, and PersonaMem-v2—rather than open-ended real-world personalization. Second, the framework does not yet address user-requested deletion, differential privacy, or policy-compliant forgetting or retention windows. Third, the paper identifies scaling and behavior as open areas, especially the interaction between model size and preferred memory granularity (Xu et al., 7 Jul 2026).

Additional trade-offs are also described. Scaling to extremely large histories or many users may require sharding the pyramid, more aggressive summarization, or stronger indexing and routing. Robustness depends on the quality of record extraction, reconciliation, and provenance links. If records are mis-typed or incorrectly merged, errors can propagate upward into topic tracks and the user profile. The RL stage is also compute-intensive, and the current setup uses only terminal rewards; more fine-grained credit assignment remains open (Xu et al., 7 Jul 2026).

A concrete failure case illustrates these risks. In a question about peaceful weekend activities, the model retrieves records about a fundraiser entertainment lineup switching to salsa because an earlier salsa set had been successful, then over-generalizes from this event and from evidence that the user felt energized by packed big events to conclude that the user “already loves spending weekends at lively salsa dance socials.” The judge marks this as incorrect. The failure is not simple retrieval miss. It is a misuse of evidence, specifically an event-to-preference overgeneralization (Xu et al., 7 Jul 2026).

The future directions named in the paper include richer real-world evaluations, explicit privacy, control, and forgetting tools, systematic model-scaling studies, improved credit assignment, multi-modal and cross-domain memory, and stronger robustness to noisy memories (Xu et al., 7 Jul 2026). These directions all preserve the central premise of the framework: memory should be externally structured, provenance-linked, and navigated by a learned policy rather than passively consumed as preselected context.