HyperMem: Hypergraph Memory Architecture

• HyperMem is a hypergraph-based hierarchical memory architecture that unifies dialogue elements into topic, episode, and fact nodes for coherent long-term memory retrieval.
• It employs a dual-indexing strategy with BM25 lexical matching and dense semantic embeddings to enhance precision in retrieving contextually rich conversation fragments.
• Empirical evaluations on the LoCoMo benchmark demonstrate its superior performance in multi-hop, temporal, and open-domain retrieval compared to traditional graph-based models.

HyperMem is a hypergraph-based hierarchical memory architecture developed for long-term conversational agents, explicitly designed to capture high-order associations among dialogue content. The architecture unifies temporally and topically scattered information into coherent memory units, supporting efficient, accurate retrieval across extended multi-turn dialogues. HyperMem achieves superior performance on the LoCoMo benchmark, establishing itself as a state-of-the-art system for managing long-term conversational memory (Yue et al., 9 Apr 2026).

1. Hierarchical Hypergraph Memory Structure

HyperMem organizes dialogue memory into a three-level hierarchy, in which each level is represented as a node type in a hypergraph:

• Topic Nodes ($V^T$): Represent overarching, persistent themes or situations that span long time intervals (e.g., "Nate’s tournaments").
• Episode Nodes ($V^E$): Encapsulate individual, temporally contiguous dialogue segments relating to a single coherent event (e.g., "June 3rd tennis match discussion").
• Fact Nodes ($V^F$): Encode atomic assertions extracted from specific episodes (e.g., "Nate has won seven tournaments").

Two primary families of hyperedges organize these nodes:

• Episode Hyperedges ($\mathcal{E}^E$): Group all episodes under the same topic, enabling explicit aggregation of topically related but temporally scattered segments.
• Fact Hyperedges ($\mathcal{E}^F$): Group all facts extracted from the same episode into a compact sub-memory for precise retrieval.

The formal memory structure is

$$H = ( V^T \cup V^E \cup V^F, \ \mathcal{E}^E \cup \mathcal{E}^F )$$

where each $e \in \mathcal{E}^E$ connects a topic node to a set of episodes with associated node weights, and each $e \in \mathcal{E}^F$ connects an episode node to a set of extracted facts.

2. Hypergraph Formulation and High-Order Associations

In contrast to traditional (pairwise) graph approaches, HyperMem exploits hypergraphs to explicitly model $j$-way associations:

• Nodes: $V = V^T \cup V^E \cup V^F$
• Hyperedges: $V^E$0, each $V^E$1, possibly connecting more than two nodes.
• Incidence Relation: A node $V^E$2 is incident to $V^E$3 if $V^E$4, with neighborhood $V^E$5.

This formalism enables explicit representation of relations among arbitrarily many episodes or facts. For instance, all episodes belonging to "sport" can be retrieved or reasoned over as a single logical unit, which is not feasible with simpler pairwise connections. This design preserves high-order coherence vital for accurate multi-hop and temporal reasoning in dialogue agents (Yue et al., 9 Apr 2026).

3. Hybrid Lexical–Semantic Indexing and Embedding Propagation

Each node $V^E$6 in HyperMem is indexed in two complementary ways:

• Sparse Lexical Index: BM25 over the node’s text for exact keyword retrieval.
• Dense Semantic Embedding: $V^E$7, constructed using Qwen3-Embedding-4B.

To enhance the semantic alignment of nodes connected by hyperedges, HyperMem performs lightweight embedding propagation:

1. Hyperedge Embedding:

$V^E$8

2. Node Embedding Refinement:

$V^E$9

where $V^F$0 controls propagation strength.

The refined embeddings $V^F$1 support improved semantic similarity retrieval, especially for high-order associations that span multiple conversational segments.

4. Coarse-to-Fine Retrieval Pipeline

HyperMem employs a hierarchical, coarse-to-fine retrieval strategy that aggressively prunes candidates at each layer:

1. Stage 1: Topic Retrieval
   • Compute combined lexical (BM25) and semantic (cosine similarity) scores for each topic node; select top $V^F$2 topics.
2. Stage 2: Episode Retrieval
   • For episodes under selected topics, compute and rank using the same dual scoring; select top $V^F$3 episodes.
3. Stage 3: Fact Retrieval
   • For facts under chosen episodes, again apply dual ranking; select top $V^F$4 facts.

Rank fusion uses Reciprocal Rank Fusion (RRF):

$V^F$5

A learned reranker can optionally refine the top-100 candidates at each stage. Memory complexity is substantially reduced since only a small, fixed number of nodes are considered at each hierarchical level, rather than the entire memory store.

5. Hyperedges for Coherent Evidence Aggregation

Traditional memory graphs, limited to binary relations, produce fragmented retrievals—connecting only pairs of episodes or facts. HyperMem’s hyperedges enable:

• Aggregation of Dispersed Content: Topic hyperedges assemble all episodes relating to a long-term theme, regardless of temporal separation, supporting complex multi-hop queries (e.g., aggregating "Nate tournaments" over weeks or months).
• Compact Evidence Packets: Fact hyperedges collate all assertions from an episode, yielding coherent evidence sets critical for precise, fine-grained question answering.

This explicit high-order grouping reduces result fragmentation, directly addressing issues observed with retrieval-augmented generation (RAG) and pairwise graph-based memories.

6. Memory Construction and Algorithmic Workflow

Memory construction proceeds in three algorithmic phases:

1. Episode Detection: Dialogue is buffered and processed by an LLM-based boundary detector to segment episodes.
2. Topic Aggregation: Each new episode is matched (via similarity) to existing topics or initiates a new topic node; related episodes are grouped via topic hyperedges.
3. Fact Extraction: LLMs extract atomic facts from each episode, which are then connected to their parent episode via fact hyperedges.

Pseudocode for both construction and online retrieval are explicitly provided in the original specification. The architecture supports efficient incremental updates and context-dependent similarity estimation (Yue et al., 9 Apr 2026).

7. Empirical Results and Comparative Evaluation

Experiments on the LoCoMo benchmark demonstrate that HyperMem significantly outperforms graph-based and RAG memory systems:

System	Overall Accuracy	Single-hop	Multi-hop	Temporal	Open-domain
HyperMem	92.73%	96.08%	93.62%	89.72%	70.83%
HyperGraphRAG	86.49%	90.61%	84.04%	85.36%	—
MIRIX	85.38%	—	—	—	—

Ablation analyses reveal that:

• Removing episode context reduces overall accuracy by −3.76%, and specifically degrades temporal reasoning by −5.61%.
• Flattening retrieval to facts only incurs a −5.68% drop on multi-hop queries.
• Dropping topic retrieval leads to a −0.72% performance decline.

These results validate the necessity of all three structural levels, the hypergraph formulation, and the hierarchical retrieval procedure for optimal long-term conversational reasoning. The architecture establishes new benchmarks for coherent, efficient, and high-fidelity memory retrieval in dialogue agents (Yue et al., 9 Apr 2026).