MemRAG: Memory-Augmented RAG Systems

• MemRAG is a framework that integrates explicit dynamic memory mechanisms into retrieval-augmented generation pipelines, boosting retrieval accuracy and system adaptability.
• It employs dynamic, hierarchical, and quantized memory approaches to optimize query retrieval, reduce energy consumption, and improve task success rates.
• MemRAG architectures support diverse applications including mobile automation, long-context reasoning, and multi-agent knowledge graph construction with measurable performance gains.

MemRAG refers to a spectrum of retrieval-augmented generation (RAG) methodologies and system components with a central shared principle: explicit memory mechanisms, often dynamic or hierarchical, are integrated into the retrieval, storage, or action-reuse logic of RAG pipelines. MemRAG approaches enable more effective, efficient, and adaptive retrieval for LLM inference and downstream automation, across diverse environments including dynamic QA, mobile task completion, edge device LLMs, long-context reasoning, and multi-agent knowledge graph construction (Bursa, 4 Jan 2026, Liao et al., 31 Oct 2025, Loo et al., 4 Sep 2025, Qian et al., 2024, Wu et al., 30 May 2026).

1. Dynamic Memory Architectures in RAG

Several MemRAG designs extend the standard RAG stack—query encoder, ANN retriever, and LLM generator—through a dynamic, usage-sensitive embedding memory. As formalized in "A Dynamic Retrieval-Augmented Generation System with Selective Memory and Remembrance" (Bursa, 4 Jan 2026), dynamic memory stores each passage $i$ as a tuple $(E_i, c_i, \tau_i, \mathrm{remembered}_i)$ where $E_i \in \mathbb{R}^d$ is the embedding, $c_i$ tracks access count, $\tau_i$ is the last access timestamp, and $\mathrm{remembered}_i$ is a permanence flag.

Retrieval updates these tuples, and core memory dynamics follow biological principles:

• Items frequently retrieved are consolidated as 'remembered' ($c_i \geq \theta$).
• Stale, insufficiently accessed items decay multiplicatively ($E_j \leftarrow \alpha E_j$ if $\mathrm{remembered}_j = \mathrm{False}$ and $t-\tau_j > \gamma$).
• Items are pruned once their embedding norm falls below a threshold.

This architecture supports both static (unchanging index) and dynamic RAG, adapts at runtime via parameters $(E_i, c_i, \tau_i, \mathrm{remembered}_i)$0, $(E_i, c_i, \tau_i, \mathrm{remembered}_i)$1, and $(E_i, c_i, \tau_i, \mathrm{remembered}_i)$2, and exposes per-item statistics for system observability (Bursa, 4 Jan 2026).

2. Memory-Efficient and Hierarchical Retrieval

In edge and low-power settings, MemRAG can refer to two-stage or hierarchical retrieval architectures that prioritize both memory and energy efficiency. One prominent example (Liao et al., 31 Oct 2025) implements a candidate generation stage using 4 MSBs (INT4) per embedding dimension to select a shortlist (e.g., $(E_i, c_i, \tau_i, \mathrm{remembered}_i)$3); only the shortlisted candidates undergo full INT8 similarity computation for final ranking. Data is stored in a bit-planar DRAM layout for bandwidth efficiency, and all computations are performed using low-power processing elements.

This scheme halves memory traffic and reduces MAC computation by up to 75% with a minimal precision loss (e.g., SciFact P@1 only drops from 0.507 to 0.497 for INT4→INT8–two-stage). Energy cost is reduced from 337.7 μJ/query to 177.76 μJ/query on TSMC 28nm, enabling battery-efficient LLM operation on wearables (Liao et al., 31 Oct 2025). This architecture does not rely on tree/graph indexes and can be generalized to other quantization levels.

3. Memory-Based Learning for Mobile Agents

Within mobile agent frameworks, MemRAG is deployed as a key-value memory that stores each successful user query $(E_i, c_i, \tau_i, \mathrm{remembered}_i)$4, its decomposition into atomic actions $(E_i, c_i, \tau_i, \mathrm{remembered}_i)$5, and embedding $(E_i, c_i, \tau_i, \mathrm{remembered}_i)$6 (Loo et al., 4 Sep 2025). This supports direct action reuse or guided planning:

• A new query is embedded and compared to memory via cosine similarity.
• If similarity $(E_i, c_i, \tau_i, \mathrm{remembered}_i)$7, the historical action sequence is reused without LLM planning; if $(E_i, c_i, \tau_i, \mathrm{remembered}_i)$8 similarity $(E_i, c_i, \tau_i, \mathrm{remembered}_i)$9, retrieved steps are injected into the LLM prompt for guided completion.
• Memory is optionally capped and managed with FIFO eviction.

This regime significantly reduces planning latency, improves task success rate (from 86.7% to 93.3% TSR), and decreases required LLM calls and operational steps (average steps fall from 10.6 to 8.2 per task). MemRAG forms one component in a triad (with InterRAG and LocalRAG), tightly integrating immediate memory with external and local retrieval (Loo et al., 4 Sep 2025).

4. Long-Context and Global Memory-Enhanced RAG

In scenarios with extremely long input contexts (e.g., $E_i \in \mathbb{R}^d$0 tokens), MemRAG-type designs employ a learned global memory to enable efficient evidence location. MemoRAG (Qian et al., 2024) realizes this via:

• A lightweight memory model processes the lengthy context in sliding windows, compressing each window into a small set of memory tokens via dedicated projections, yielding a global memory $E_i \in \mathbb{R}^d$1 with size $E_i \in \mathbb{R}^d$2.
• Given a query $E_i \in \mathbb{R}^d$3, the memory model generates a "staging answer" $E_i \in \mathbb{R}^d$4—draft retrieval cues optimized during supervised training.
• Top-$E_i \in \mathbb{R}^d$5 passages are retrieved by passing $E_i \in \mathbb{R}^d$6 to a standard retriever; a high-capacity generator then synthesizes the final answer from $E_i \in \mathbb{R}^d$7 and retrieved evidence.

MemoRAG substantially improves F1 and ROUGE scores on both standard and ultra-long context QA/summarization tasks, outperforming both vanilla RAG and direct full-context LLMs (Qian et al., 2024). The memory is trained and updated end-to-end via feedback from downstream answer generation, ensuring it encodes retrieval-useful information.

5. Memory-Based Multi-Agent and Graph RAG Approaches

MemGraphRAG (Wu et al., 30 May 2026) introduces explicit memory to manage structural graph-based retrieval:

• A three-layer global memory $E_i \in \mathbb{R}^d$8 tracks sampled schemas, extracted triples, and provenance passages.
• Multi-agent collaboration (extraction, conflict detection, conflict resolution) maintains a globally consistent and non-redundant knowledge graph.
• Hierarchical retrieval first filters via vector similarity in memory layers, then projects to graph nodes, initializing a query-specific Personalized PageRank over the constructed graph for evidence retrieval.

Conflicts (e.g., semantic, temporal, granularity) are dynamically adjudicated by rule-based protocols referencing both symbolic and passage evidence. This guarantees the knowledge graph's structural connectivity and thematic consistency. Empirically, MemGraphRAG achieves superior retrieval accuracy (59.25% LLM-Acc vs. 57.15% for best baselines) and reduced per-query latency, demonstrating transferability to other graph-based RAG systems (Wu et al., 30 May 2026).

6. Comparative Performance and Efficiency

MemRAG systems are calibrated for various trade-off axes—accuracy, memory footprint, latency, and energy—with explicit hyperparameter settings (e.g., default $E_i \in \mathbb{R}^d$9, $c_i$0, $c_i$1 in (Bursa, 4 Jan 2026)). Ablation studies underscore the importance of remembering dynamics, decay rate, and hierarchical filtering for stable, high-performance retrieval. Several empirical results are summarized below.

Approach	NDCG@5	Embedding Params	Latency (s)	Task Success Rate
MemRAG (ARM, (Bursa, 4 Jan 2026))	0.940	~22M	8–13	–
MemRAG Hierarchical (Liao et al., 31 Oct 2025)	0.497	–	<0.1	–
Mobile MemRAG (Loo et al., 4 Sep 2025)	–	–	~0.005 per query	93.3%
MemGraphRAG (Wu et al., 30 May 2026)	–	–	0.061	59.25% LLM-Acc

Application-specific deployments inform the parameterization for minimal memory, aggressive forgetting, or persistent global context. Monitoring and configuration hooks are provided for production (e.g., memory growth, prune rate, runtime adjustment).

7. Use Cases, Deployment, and Practical Implications

MemRAG frameworks are deployed across diverse environments:

• Search and QA: Adaptive memory enables long-term relevance and self-regularizing resource consumption even under storage constraints (Bursa, 4 Jan 2026).
• Edge/Wearable Devices: Hierarchical retrieval architectures minimize energy and memory usage, supporting privacy-preserving on-device LLMs (Liao et al., 31 Oct 2025).
• Mobile Automation: Action-sequence memory enables rapid, error-resistant sequential task execution, reducing redundant planning (Loo et al., 4 Sep 2025).
• Long-Context Understanding: Memory-augmented models efficiently localize evidence in vast, semi-structured contexts, surpassing full-context LLMs (Qian et al., 2024).
• Knowledge Graph Construction: Memory-aware, multi-agent extraction and retrieval address fragmentation, logical inconsistency, and structural disconnects in knowledge graphs, yielding denser and more robust graph representations for complex reasoning (Wu et al., 30 May 2026).

This breadth demonstrates that MemRAG—at its core, the fusion of memory with retrieval mechanisms—forms a generalizable design axis for scalable, adaptive, and efficient RAG systems in both research and applied settings.