xMemory Framework

• xMemory is a hierarchical agent memory framework that decomposes dialogue into atomic semantic units and groups them into theme nodes for efficient retrieval.
• It employs a decoupling-to-aggregation paradigm with uncertainty-gated expansion to preserve critical dependency chains and reduce redundancy.
• Empirical evaluations demonstrate that xMemory outperforms traditional RAG methods by achieving higher evidence density and improved token efficiency across LLMs.

xMemory is a hierarchical agent memory framework designed to address the limitations of standard Retrieval-Augmented Generation (RAG) pipelines in settings where memories arise from bounded, coherent dialogue streams rather than large, diverse document corpora. In such agent memory contexts, the high correlation and redundancy among dialogue spans render fixed top-$k$ similarity retrieval approaches ineffective, leading to retrieval redundancy and potential loss of prerequisite information due to post-hoc pruning. xMemory introduces a decoupling-to-aggregation paradigm, where the dialogue is decomposed into reusable semantic components, grouped into themes, and retrieved using structure-driven, uncertainty-aware algorithms that target compactness, diversity, and the preservation of reasoning prerequisites. Empirical results show xMemory outperforms RAG and multiple structured-memory baselines across several LLMs and long-term QA benchmarks (Hu et al., 2 Feb 2026).

1. Motivation and Conceptual Overview

Standard RAG assumes a heterogeneous, large-scale corpus, where retrieval benefits from diversity among passages and top-$k$ similarity search is effective. However, agent memory is typically a bounded, coherent, and highly self-referential dialogue history. In this regime, fixed top-$k$ retrieval collapses into dense redundancy, magnifying near-duplicates and failing to recover temporally linked dependencies required for multi-turn or multi-hop reasoning. Post-hoc pruning or compression fails to address these limitations and may sever contextually critical prerequisite chains (e.g., co-reference or ellipsis-dependent turns).

xMemory replaces this paradigm by explicitly decoupling the dialogue stream into atomic semantic units distinct from episodic traces and then aggregating related semantic units into higher-level theme clusters. Retrieval operates over this hierarchy in a top-down manner—first selecting a compact, diverse set of high-level representations, then progressively expanding into more granular details when necessary for the downstream LLM's uncertainty reduction. This approach enforces redundancy control by construction and ensures intact evidence units are preserved for reasoning.

2. Hierarchical Memory Construction

xMemory employs a four-level memory hierarchy tailored to agent dialogue histories:

1. Original messages: Raw utterances representing primitive dialogue blocks with timestamps.
2. Episodes: Contiguous message summaries that preserve local context and temporal structure, each episode linked to its originating message(s).
3. Semantic nodes: Distilled, persistent facts extracted from episodes (e.g., “User prefers PyTorch over TensorFlow”). Semantic nodes are designed to disentangle repeated or co-referential traces, enabling fact reuse and reducing redundancy.
4. Theme nodes: Clusters of related semantic facts, each serving as a high-level index or topic theme (e.g., “User’s technical preferences”).

This hierarchy ensures every message maps to exactly one episode; episodes may generate multiple semantic nodes; each semantic node belongs to exactly one theme; and each theme contains multiple semantic nodes. This enforces preservation of prerequisite chains at each granularity.

The construction and maintenance of the hierarchy are governed by an objective composed of two terms:

• SparsityScore promotes balance in theme sizes:

$$\text{SparsityScore}(P) = N \cdot K \sum_{k=1}^K n_k^2$$

with $P = \{C_1, \ldots, C_K\}$, $|C_k| = n_k$.

• SemScore promotes semantic coherence while penalizing semantic redundancy (themes with high inter-theme proximity) and “semantic islands” (isolated themes):

$$\text{SemScore}(P) = \frac{1}{K} \sum_{k=1}^K \left[ \frac{1}{n_k} \sum_{i\in C_k} \cos(x_i, p_k) \right] \cdot g(s_k)$$

where $x_i$ are semantic embeddings, $p_k$ is the centroid of theme $k$, and $k$0 is a bell-shaped penalty based on deviation from median inter-theme similarity.

When semantic nodes are added, assignment (to nearest theme or new theme), splits (when a theme exceeds a size cap), and merges (for underpopulated themes) are guided by maximization of $k$1. Additionally, a $k$2-NN graph over themes and semantics is maintained to facilitate efficient search and to avoid formation of semantic islands.

3. Retrieval and Uncertainty-Gated Expansion

xMemory’s retrieval pipeline proceeds in two main stages, traversing the memory hierarchy top-down:

Stage I: High-level Representative Selection

• The query $k$3 is embedded.
• Candidate theme nodes $k$4 and semantic nodes $k$5 are selected via embedding similarity.
• To counteract dense-region collapse and enable effective set-level (multi-fact or multi-hop) evidence selection, a greedy representative selection is performed over the $k$6-NN graph. The selection criterion at each step is:

$k$7

where $k$8 is the covered set, $k$9 identifies uncovered candidates adjacent to $k$0, $k$1 are graph edge weights, $k$2 is query-node similarity, $k$3 balances coverage vs. relevance, and $k$4 is a normalization factor. This procedure is first applied over the theme layer, then over semantic nodes induced by the selected themes.

Stage II: Uncertainty-Gated Expansion

• Candidate episodes linked to selected semantic nodes are ranked by combined relevance and semantic coverage.
• The context is initialized with selected theme and semantic summaries.
• For each candidate episode $k$5, uncertainty is measured via LLM next-token entropy:

$k$6

If $k$7 (for small $k$8), $k$9 is included; otherwise, expansion stops.

• Optionally, the same gating is applied to inclusion of raw messages within accepted episodes.

The method halts once added detail ceases to reduce LLM uncertainty, thereby maximizing evidence density under a token budget.

Hierarchy Level	Node Type	Content Description
Level 1	Original message	Raw dialogue block, timestamped
Level 2	Episode	Local summary, contiguous span
Level 3	Semantic node	Distilled fact, persistent, de-duplicated
Level 4	Theme node	Cluster of related semantics (index)

4. Integration with LLMs and Token Efficiency

The constructed context $$\text{SparsityScore}(P) = N \cdot K \sum_{k=1}^K n_k^2$$0—comprising selected themes, semantic summaries, and, if required by uncertainty, episodes or raw messages—is concatenated with the query and passed to the LLM using a fixed prompt. No additional LLM fine-tuning is required.

Compactness and diversity are enforced through strict high-level representative selection and uncertainty-gated expansion. This hierarchical retrieval delivers shorter, high-evidence-density contexts, optimizing the use of inference tokens without compromising on answer accuracy.

The method’s effective control over redundancy by construction, rather than post-hoc pruning, distinguishes it from previous approaches.

5. Empirical Evaluation

xMemory is empirically benchmarked on two datasets:

• LoCoMo: 50 multi-session dialogues (~9K tokens, ~300 turns), covering single-hop, multi-hop, temporal, and open-domain QA. Metrics: BLEU-1, token-level F1, and tokens per query.
• PerLTQA: Personal long-term QA covering profiles, relationships, events, dialogue history. Metrics: BLEU-1, F1, ROUGE-L, tokens per query.

Backbone models include Qwen3-8B, Llama-3.1-8B-Instruct, and GPT-5 nano.

Key results:

Configuration	BLEU-1 ↑	F1 ↑	ROUGE-L ↑	Tokens/query ↓
LoCoMo/Qwen3-8B, RAG	27.92	36.42	–	6613
LoCoMo/Qwen3-8B xMemory	34.48	43.98	–	4711
PerLTQA/Qwen3-8B RAG	35.14	42.35	38.48	6499
PerLTQA/Qwen3-8B xMemory	36.24	47.08	42.50	5087

xMemory consistently outperforms Naive RAG and five structured-memory baselines (Nemori, LightMem, A-Mem, MemoryOS) in answer quality and token efficiency. It also increases the evidence density, as shown by higher “2-hit” and “multi-hit” block proportions, and achieves superior coverage efficiency (mean 5.66 blocks—974 tokens—to cover all answer tokens, compared to 10.81 blocks—1979 tokens—with Naive RAG).

Ablation experiments demonstrate that both Stage I (representative selection) and Stage II (uncertainty-gated expansion) offer complementary gains in both retrieval quality and efficiency.

6. Limitations and Prospective Extensions

xMemory’s effectiveness depends on the quality of the embeddings and the LLM-generated summaries; mis-assignments in the theme or semantic node hierarchy can propagate and degrade retrieval. Thresholds for split/merge and uncertainty gating are dataset-dependent and may require tuning.

The uncertainty measure currently relies on next-token entropy of the LLM; alternative confidence or calibration-aware scores might further optimize adaptive inclusion.

Several prospective research directions are enumerated:

• Extension to real-time, dynamic memory updates supporting incremental hierarchy maintenance during active agent interaction.
• Adaptation for multi-agent systems, where each agent’s memory must remain distinct while supporting efficient, non-redundant cross-agent retrieval.
• Generalization to multi-modal agent memories (e.g., dialogue, images, code), necessitating modality-agnostic decoupling and aggregation.
• Integration with reasoning-centric modules (e.g., planning, tool-use) that exploit the memory hierarchy for multi-step task decomposition and execution.
• Application to non-dialogue environments such as personal logs, wikis, or other timeline-oriented records, contingent on suitable semantic extraction and theme clustering methods.

These directions suggest the potential for broader impact of the xMemory framework in LLM-augmented agent memory systems operating over diverse long-term histories (Hu et al., 2 Feb 2026).