Improving Multi-step RAG with Hypergraph-based Memory for Long-Context Complex Relational Modeling

Abstract: Multi-step retrieval-augmented generation (RAG) has become a widely adopted strategy for enhancing LLMs on tasks that demand global comprehension and intensive reasoning. Many RAG systems incorporate a working memory module to consolidate retrieved information. However, existing memory designs function primarily as passive storage that accumulates isolated facts for the purpose of condensing the lengthy inputs and generating new sub-queries through deduction. This static nature overlooks the crucial high-order correlations among primitive facts, the compositions of which can often provide stronger guidance for subsequent steps. Therefore, their representational strength and impact on multi-step reasoning and knowledge evolution are limited, resulting in fragmented reasoning and weak global sense-making capacity in extended contexts. We introduce HGMem, a hypergraph-based memory mechanism that extends the concept of memory beyond simple storage into a dynamic, expressive structure for complex reasoning and global understanding. In our approach, memory is represented as a hypergraph whose hyperedges correspond to distinct memory units, enabling the progressive formation of higher-order interactions within memory. This mechanism connects facts and thoughts around the focal problem, evolving into an integrated and situated knowledge structure that provides strong propositions for deeper reasoning in subsequent steps. We evaluate HGMem on several challenging datasets designed for global sense-making. Extensive experiments and in-depth analyses show that our method consistently improves multi-step RAG and substantially outperforms strong baseline systems across diverse tasks.

• The paper introduces HGMEM, a hypergraph memory that dynamically evolves via update, insertion, and merging operations to capture complex, high-order relationships.
• It employs adaptive evidence retrieval combining local probing and global exploration to enhance comprehensive understanding over lengthy contexts.
• Empirical results show HGMEM outperforms traditional RAG systems, achieving significant accuracy gains on long-context and complex relational modeling benchmarks.

Hypergraph-based Memory for Multi-step Retrieval-Augmented Generation: Mechanisms and Empirical Analysis

Contextual Motivation: Limitations of Traditional Memory in Multi-step RAG

Multi-step Retrieval-Augmented Generation (RAG) constitutes a highly effective protocol for extending the reasoning and context-processing capabilities of LLMs in knowledge-intensive tasks. However, the predominant memory mechanisms in existing RAG architectures manifest as static, low-expressiveness structures—mostly serving as passive aggregators of primitive facts. Such designs typically summarize interaction histories into plaintext or elemental units (e.g., knowledge triples, relational tables), leading to weak support for modeling high-order dependencies and concept integration over long contexts. The lack of fine-grained memory evolution (i.e., operations that connect, abstract, and synthesize evidence dynamically) fragments reasoning and restricts LLMs’ ability to resolve global sense-making queries or perform complex relational modeling.

HGMEM: Hypergraph-based Dynamic Memory Structure

This paper introduces HGMEM, a hypergraph-based memory system extending the expressiveness of working memory in multi-step RAG. In HGMEM, memory is formalized as a hypergraph $(\mathcal{V}_M, \mathcal{E}_M)$, where vertices represent entities (drawn from an offline-built knowledge graph over the input corpus), and hyperedges correspond to memory points—each encapsulating a set of interconnected entities along with structured descriptions grounded in retrieved evidence.

Key features of HGMEM include:

• Evolving Structure: At each reasoning step, the LLM adaptively generates subqueries to retrieve new evidence from raw documents and the graph. Retrieved information is integrated via three explicit memory evolving operations: update (modifying hyperedges’ descriptions), insertion (adding new hyperedges for novel relationships), and merging (combining multiple points to form high-order correlations).
• Higher-order Expressiveness: Unlike graph-based memories limited to binary relations, hyperedges in HGMEM can connect arbitrary subsets of vertices, naturally capturing n-ary (n≥2) composite relationships vital to multi-faceted reasoning.
• Adaptive Evidence Retrieval: HGMEM supports both local investigation (anchoring queries to specific memory points and probing their neighborhoods) and global exploration (querying outside the existing memory scope), resulting in more accurate and comprehensive knowledge acquisition.

Experimental Framework and Benchmarking

Benchmarks span generative sense-making QA (Longbench V2) and long narrative understanding (NarrativeQA, NoCha, Prelude), each demanding holistic comprehension and evidence integration across large contexts (often exceeding 100k tokens per document).

Implementation details:

• Documents are chunked and processed via GPT-4o to construct entity-relation graphs.
• Qwen2.5-32B-Instruct and GPT-4o serve as representative LLMs, interfacing with the hypergraph memory via vector retrieval (bge-m3 embeddings) and the hypergraph-db backend.
• Baseline methods include NaiveRAG, GraphRAG, LightRAG, HippoRAG v2, DeepRAG, and ComoRAG—spanning single-step and multi-step, with/without structured memory.

Evaluation metrics center on comprehensiveness and diversity (for generative QA), and prediction accuracy (for narrative tasks), with scoring calibrated via LLM-based judgments.

Results: Empirical Advantages and Mechanistic Validation

HGMEM demonstrates consistent superiority across benchmarks and LLM backbones, outperforming both traditional and memory-augmented RAG architectures. Notable results:

Method	Longbench Comp.	NarrativeQA Acc.	NoCha Acc.	Prelude Acc.	Diversity
HGMEM (GPT-4o)	65.73	69.74	55.00	73.81	62.96
ComoRAG (GPT-4o)	62.18	65.82	54.00	63.49	54.07

• Numerical Gains: HGMEM achieves up to +4.92 points greater accuracy in NarrativeQA and up to +9.82 in Prelude versus ComoRAG baseline (Qwen2.5-32B-Instruct), with gains correlated to complex, high-relation queries.
• Robustness: HGMEM maintains or exceeds performance relative to systems powered by stronger LLMs (GPT-4o), suggesting high data/model efficiency.
• Cost Analysis: The merging operation (critical for high-order correlation modeling) introduces negligible computational overhead.

Ablation studies delineate mechanistic contributions:

• Adaptive Retrieval: Isolating evidence retrieval strategies—global exploration or local investigation alone—results in significant losses compared to the combined adaptive mode.
• Memory Merging: Disabling the merging operation (key for high-order concept formation) drops accuracy for sense-making queries by up to 10%, confirming the essentiality of relational abstraction.

Analyses by query type reveal:

• For primitive, fact-based queries, both full and ablated HGMEM reach similar accuracy; for sense-making queries demanding evidence integration, only HGMEM with merging achieves superior performance and higher average entities per hyperedge.

Mechanistic Implications and Case Studies

Case analyses illustrate the practical impact of HGMEM: the system not only synthesizes explicit entities and relationships, but also constructs abstract reasoning chains, enabling the resolution of queries that confound competing baselines. For example, it correctly infers causality in NarrativeQA (Xodar’s enslavement) and disambiguates entity-based facts in NoCha, by forming memory points that encode nuanced, multi-level relationships.

The integration of dynamic memory evolution (update, insertion, merging), guided by subquery-driven evidence acquisition, establishes HGMEM as a protocol for continual knowledge synthesis—moving beyond static accumulation to support compositional, context-aware reasoning.

Theoretical and Practical Implications

HGMEM signals a paradigm shift in LLM-based retrieval-augmented systems:

• Relational Compositionality: By leveraging hypergraph representations, memory modules can capture and reason over arbitrary n-ary associations.
• Scalable Reasoning: The protocol supports incremental knowledge construction without exponential cost, facilitating deployment in long-context, knowledge-intensive real-world tasks.
• Hybrid Memory Architectures: Future RAG systems can combine HGMEM’s dynamic working memory with persistent long-term memory or agentic memory schemes for even richer contextualization and multi-agent coordination.

Future Directions

Potential extensions include:

• Generalization to Dialog-based and Multi-agent Settings: Embedding HGMEM within multi-agent frameworks to enhance cross-step and cross-agent reasoning.
• Incorporation of Parametric and Contextual Memory: Integrating parametric models that combine persistent knowledge with dynamic hypergraph memory.
• Automated Schema Induction: Enabling HGMEM to infer new schemas, entity types, and relation abstractions autonomously from raw data.

Conclusion

The HGMEM hypergraph-based memory mechanism enables multi-step RAG systems to transcend passive fact accumulation, facilitating expressive, compositional memory evolution and high-order relational modeling. Empirical results validate its ability to enhance long-context reasoning, comprehensiveness, and accuracy, with practical efficiency and scalability. HGMEM provides a foundation for advancing LLM agents in tasks centered on global sense-making and complex narrative understanding (2512.23959).