VikingMem: A Memory Base Management System for Stateful LLM-based Applications

Abstract: LLMs have revolutionized interactive applications; however, their finite context windows pose a critical data management challenge for maintaining stateful, long-term interactions. Existing memory approaches often rely on simplistic extraction methods that lead to incomplete memories or use rigid, single-purpose memory extraction prompts tailored to a single use case, such as chatbots. Consequently, they lack generalizability and perform poorly across diverse downstream tasks. To bridge this gap, we introduce the Memory Base, a novel data management paradigm for managing the persistent state of long-term interactions. It is characterized by three core principles: selective extraction of high-value memories from raw information streams; inherent statefulness and evolution, where memory content is progressively summarized, corrected, and temporally weighted to prioritize recent interactions; and a generalizable abstraction paradigm designed for robust transferability across diverse applications, including education, recommendation, and agent memory. Building on this foundation, we present VikingMem, an end-to-end Memory Base Management System implemented on the VikingDB vector engine. VikingMem materializes this paradigm through interconnected event and entity abstractions. It features event-centric memory extraction to selectively handle complex information streams, while entities are dynamically updated by events to achieve stateful evolution. Using temporal compression via a topic-wise timeline and time-weighted recall, the system progressively produces high-level summary memories, prioritizes recent items, and compresses and fades older ones. Extensive evaluations on long-term memory benchmarks demonstrate that VikingMem outperformes baselines by up to 30% in memory retrieval effectiveness while maintaining the low latency essential for interactive applications.

• The paper presents a novel Memory Base system that transforms raw session logs into structured, evolving memories for stateful LLM applications.
• It leverages a one-pass transformer and LLM-guided saliency filtering with an event-entity schema, achieving up to 38% higher retrieval accuracy.
• The system delivers low-latency updates and compresses storage by over 83%, reducing extraction costs by 3.2x for efficient multi-session operations.

VikingMem: A Unified Memory Base Management System for Stateful LLM-based Applications

Motivation and Core Paradigm

LLMs underpin an expanding array of interactive systems that demand robustness across multi-session, long-horizon engagements, including persistent agents, educational tutors, recommender systems, and collaborative assistants. Despite progress in context window scaling, relying exclusively on prompt-based persistence is impractical and semantically brittle for real-world, stateful applications. The challenge lies in maintaining a persistent, evolving memory that is both performant and adaptable to diverse downstream requirements.

VikingMem introduces a principled Memory Base paradigm, distinct from prior ad hoc caching or heuristic methods. Its design is grounded in three core principles:

• Selective Extraction: High-value, contextually salient information is programmatically extracted from raw, low information-density streams, filtering out noise through schema-driven event segmentation and LLM-guided saliency filtering.
• Inherent Statefulness and Evolution: The memory is not static. Entities (long-lived structured memories) are incrementally evolved via event-triggered updates, temporal prioritization, summarization, and explicit decay strategies.
• Generalizable Abstraction: A unified Event-Entity schema decouples business logic from data storage, enabling seamless adaptation across highly heterogeneous application domains.

These innovations aim to address persisting industry pain points: noisy memory ingestion, stale or contradictory context accumulation, and the proliferation of monolithic, inflexible memory architectures.

Event-Entity Schema and Operator Model

At the modeling core, VikingMem formalizes two primary abstractions:

• Event: An atomic, timestamped, schema-governed record representing a salient interaction or fact, distilled from raw session logs.
• Entity: A persistent construct (e.g., user profile, agent SOP) dynamically updated via aggregation expressions over events.

Events are ingested according to formally specified schemas that define the permitted structure, properties, and extraction logic. Entities, in turn, are defined with aggregation operators (\texttt{SUM}, \texttt{AVG}, \texttt{LLM_MERGE}, \texttt{TIME_COMPRESS}, etc.) specifying their update/maintenance logic in a declarative programmatic fashion.

Figure 1: Event and entity schema definition, and built-in operators in VikingMem.

This schema-driven mechanism separates extraction from business-specific rules, allowing VikingMem to act as a general substrate for variant memory requirements and to support efficient single-pass extraction regardless of complexity.

System Architecture and Pipeline

VikingMem is built atop the VikingDB vector engine, providing high-throughput, cloud-native vector storage and retrieval. The VikingMem pipeline consists of three core stages:

1. Memory Extraction: Utilizes a one-pass transformer inference pipeline driven by structured event/entity schemas. The extraction combines LLM-based saliency filtering (semantic pruning) with event-centric partitioning, efficiently identifying and segmenting relevant episodic units.

   Figure 2: VikingMem pipeline workflow from raw session ingestion to structured memory retrieval.

2. Memory Management: Handles deduplication, event-to-entity consolidation via novel operators, temporal compression (e.g., summarization, TTL-driven forgetting), and persistence. The Entity Update Algorithm (EUA) further enables efficient, LLM-free updates by generating and applying compact patch operations on existing entities.

   Figure 3: One-pass memory extraction paradigm supporting efficient high-fidelity memory updates.

   

   Figure 4: Entity update mechanism leveraging EUA for low-latency, LLM-free online updates.

3. Memory Retrieval: Employs multi-path recall—combining hybrid (dense/sparse) retrieval, time decay and business-importance reweighting, and an auxiliary keyword graph for robust factoid recovery. Multi-vector reranking (ColBERT-style late interaction with quantization and compression) yields highly effective and latency-bounded retrieval suitable for interactive workloads.

Empirical Evaluation

Comprehensive experiments demonstrate VikingMem's superiority across effectiveness, efficiency, and data retention. On LOCOMO and LongMemEval benchmarks:

• Effectiveness: Achieves up to 38% higher retrieval accuracy (LLM-Judge Score) compared to strong baselines such as Zep, Mem0, and Mirix. VikingMem consistently outperforms across both factual and reasoning-intensive queries, maintaining high scores even under increased session lengths.
• Efficiency: Delivers p95 retrieval latency as low as 0.39–0.89 sec, on par with lightweight baselines but vastly outperforming more complex memory systems (e.g., Mirix) that suffer 10x–25x longer latencies.
• Extraction/Storage: The one-pass extraction paradigm and EUA algorithm reduce extraction costs by 3.2x and latency by 1.7x for multi-memory-type workloads. Storage footprint is compressed by over 83% relative to naive raw-log retention, with no drop in answer quality.
• Ablation: Key modules such as intelligent segmentation and multi-vector rerank each provide ≥3–5% lift in judge score, confirming their impact.

Practical Applications and Extensibility

VikingMem supports deployment in a broad variety of industrial scenarios, including:

• Social chatbots (empathic companion memory),
• Cross-domain personalized search/recommendation,
• Meeting and collaboration tools (automated report/summary generation),
• Individualized education (progress, mastery tracking, adaptive recommendations),
• Autonomous Agent SOP consolidation and reuse.

A representative agent memory use case is illustrated below:

Figure 5: Transforming raw execution histories into structured, evolving agent tool memory using VikingMem.

The Event-Entity framework's programmability is confirmed by real-world operator-usage profiling, with education and agent memory preferring aggregation and semantic merging, while social domains rely more heavily on complex, temporal evolution.

Theoretical and Practical Implications

VikingMem systematizes memory management for LLM-based applications by abstracting away from prompt-centric engineering toward a schema-driven, composable algebra over events and entities. This approach enhances both modularity and scalability while supporting operational requirements for low-latency, high-throughput, and token-efficient workloads. Selective extraction with structured abstraction produces semantically dense memories, reducing both memory pollution and operational cost, and supporting trustworthiness in downstream applications.

On a theoretical front, VikingMem’s progressive event-to-entity model, with explicit operators and temporal weighting, aligns with cognitive science models of memory consolidation and decay—potentially inspiring further biologically/plausibility-driven research in AI memory substrates. The system’s open-source subset (OpenViking) further facilitates community adoption and experimental extension.

Future Directions

This memory base paradigm may underpin further advances in continual learning, persistent multi-agent systems, and privacy-aware state management for LLMs, while addressing challenges in automated auditability, long-term coherence, and adaptive context construction as LLM context windows grow. Ongoing research may refine granularity of event segmentation, context-aware prioritization policies, and evaluate memory base effects in settings with federated or privacy-sensitive state.

Conclusion

VikingMem establishes a unified, schema-driven, and operator-augmented architecture for stateful, long-term memory management in LLM-powered applications. Its event-entity abstraction, efficient pipeline, and robust empirical results substantiate the Memory Base model as an effective, scalable, and generalizable substrate for real-world, production-grade AI systems.

(2605.29640)