Beyond Fact Retrieval: Episodic Memory for RAG with Generative Semantic Workspaces (2511.07587v1)

Abstract: LLMs face fundamental challenges in long-context reasoning: many documents exceed their finite context windows, while performance on texts that do fit degrades with sequence length, necessitating their augmentation with external memory frameworks. Current solutions, which have evolved from retrieval using semantic embeddings to more sophisticated structured knowledge graphs representations for improved sense-making and associativity, are tailored for fact-based retrieval and fail to build the space-time-anchored narrative representations required for tracking entities through episodic events. To bridge this gap, we propose the \textbf{Generative Semantic Workspace} (GSW), a neuro-inspired generative memory framework that builds structured, interpretable representations of evolving situations, enabling LLMs to reason over evolving roles, actions, and spatiotemporal contexts. Our framework comprises an \textit{Operator}, which maps incoming observations to intermediate semantic structures, and a \textit{Reconciler}, which integrates these into a persistent workspace that enforces temporal, spatial, and logical coherence. On the Episodic Memory Benchmark (EpBench) \cite{huet_episodic_2025} comprising corpora ranging from 100k to 1M tokens in length, GSW outperforms existing RAG based baselines by up to \textbf{20\%}. Furthermore, GSW is highly efficient, reducing query-time context tokens by \textbf{51\%} compared to the next most token-efficient baseline, reducing inference time costs considerably. More broadly, GSW offers a concrete blueprint for endowing LLMs with human-like episodic memory, paving the way for more capable agents that can reason over long horizons.

• The paper presents the GSW framework, integrating episodic memory into RAG systems to enhance narrative reasoning.
• The framework employs an Operator to extract semantic roles and a Reconciler to integrate local workspaces into a coherent global memory.
• Experimental results show a 20% F1-score improvement and a 51% reduction in context tokens, demonstrating its efficiency in processing complex narratives.

"Beyond Fact Retrieval: Episodic Memory for RAG with Generative Semantic Workspaces"

Introduction

The paper investigates the limitations of current RAG (Retrieval-Augmented Generation) systems when applied to long-context reasoning tasks. Traditional RAG approaches, designed primarily for fact-based retrieval, often fall short in modeling complex narratives involving evolving roles, actions, and spatiotemporal contexts. This work introduces a novel framework, called Generative Semantic Workspace (GSW), which augments LLMs with biologically inspired episodic memory capabilities.

Generative Semantic Workspace Framework

The GSW framework is designed to bridge the gap between existing RAG methods and the requirements of narrative reasoning. It comprises two main components: the Operator and the Reconciler.

• Operator: Inspired by the neocortical functions, the Operator extracts semantic roles and builds local workspace instances for each chunk of input text. It creates semantic graphs that represent the roles, actions, and interactions within a given context, facilitating the construction of a coherent storyline.
• Reconciler: This component functions akin to the hippocampal complex, integrating and updating the semantic structures generated by the Operator into a unified Global Memory. It ensures temporal, spatial, and logical coherence across episodes, thus enabling the system to model and recall narrative knowledge effectively.

  Figure 1: Unifying Brain-Inspired and Generative Semantics for Episodic Memory Modeling.

Episodic Memory Creation and Question Answering

The GSW framework operates by segmenting large-scale text into semantically coherent chunks, processing each with the Operator to create local workspace instances. These instances are incrementally integrated by the Reconciler, producing a comprehensive Global Memory.

During the QA phase, the system retrieves relevant segments of this memory by matching named entities from the query to identifiers in the semantic network. It then reconstructs episodic summaries, which are passed to an LLM to generate the final answer, optimizing for query relevance and minimizing hallucination.

Figure 2: Episodic Memory Creation and QA: The end-to-end process of constructing a workspace and QA from the workspace.

Experimental Results

The GSW framework was evaluated on the Episodic Memory Benchmark (EpBench), significantly outperforming existing RAG-based baselines. The GSW demonstrated improvements in F1-score by 20% in complex multi-document synthesis tasks. Additionally, it reduced query-time context tokens by 51%, highlighting its efficiency in managing memory retrieval and minimizing inference costs.

The experimental results underscore the framework's ability to provide structured, interpretable representations that support LLMs in reasoning over long narrative contexts efficiently. The GSW framework offers a scalable solution for equipping LLMs with human-like episodic memory, allowing for more capable agents capable of reasoning over extended durations and complex narrative arcs.

Conclusion

The introduction of GSW marks a significant advance in the ability of LLMs to engage with complex narrative tasks. Its biologically-inspired design models the intricate processes of episodic memory, offering a structured approach to processing and recalling narrative information. Future developments could explore integrating GSW with diverse data modalities and test its adaptability across various domains, potentially transforming how AI systems understand and interact with complex information landscapes.