MIRIX Framework: Multi-Agent Memory System

• MIRIX is a modular multi-agent memory system that segments memory into six specialized components to enable context-rich retrieval for LLM-based AI agents.
• It employs a hierarchical architecture and active retrieval protocol to efficiently manage diverse multimodal data with robust privacy measures.
• Empirical benchmarks demonstrate up to 35% accuracy improvements and nearly 99.9% storage reduction compared to traditional memory frameworks.

MIRIX (Multi-Agent Memory System for LLM-Based Agents) is a modular, multi-agent framework designed to address the longstanding limitations in memory architectures for artificial intelligence agents, particularly those leveraging LLMs. By decomposing memory into six well-defined, specialized components and employing coordinated multi-agent management, MIRIX enables persistent, context-rich, and multimodal memory for user-facing AI systems. It offers robust handling of diverse data types, seamless active retrieval, efficient storage, and privacy-preserving mechanisms, setting new empirical benchmarks for memory-augmented AI agents (2507.07957).

1. Framework Architecture

At the core of MIRIX is a hierarchical, modular system comprising six specialized memory types: Core, Episodic, Semantic, Procedural, Resource Memory, and Knowledge Vault. Each memory component is managed by its dedicated Memory Manager, operating independently yet in concert with the overall architecture. A central Meta Memory Manager orchestrates task routing, memory updates, and retrieval strategy selection, functioning as the system's coordinator.

The architecture facilitates efficient, parallel operation among memory managers, supporting the following workflow:

• Incoming user input is analyzed by the Meta Memory Manager to determine relevant memory updates and retrievals.
• Specialized Memory Managers handle storage, updating, and retrieval within their respective domains.
• A Chat Agent interfaces with the entire memory system, executing an "Active Retrieval" protocol: it generates a retrieval topic from the query, searches all memory types, and tags each result by its memory source.

This structure supports both fine-grained memory organization and scalable, parallel memory operations, as depicted in the system's primary architectural diagram (Figure 1 in (2507.07957)), which illustrates the six modules and their interconnections.

2. Specialized Memory Components

Each MIRIX memory type fulfills a distinct role, ensuring comprehensive long-term storage and retrieval for LLM-based agents:

Memory Type	Core Functions	Representative Fields
Core Memory	Persistent, high-priority agent and user facts	Persona, Human (e.g., agent's tone, user's name)
Episodic Memory	Time-stamped user interaction and event logs	Event type, summary, details, actor, timestamp
Semantic Memory	Abstract, factual associations and knowledge	Concept name, summary, description, source
Procedural Memory	Structured workflows and task execution scripts	Step-by-step processes, structured formats
Resource Memory	Multimodal documents and detailed content	Title, summary, resource type, content/excerpt
Knowledge Vault	Secure, verbatim, sensitive user-specific data	Entry type, source, sensitivity, secret value

• Core Memory encodes persistent, readily accessible information such as the agent's persona and enduring user traits, segmented into "persona" and "human" blocks.
• Episodic Memory maintains a chronological log of events and user experiences with detailed context, facilitating temporal reasoning and continuity in interactions.
• Semantic Memory abstracts general knowledge and user-specific associations, supporting conceptual reasoning without event dependence.
• Procedural Memory contains structured instructions and task breakdowns (including JSON-encoded workflows), enabling efficient automation of routine or complex operations.
• Resource Memory manages both partial and full multimodal resources, such as screenshots and document excerpts, providing context-rich references beyond plain text.
• Knowledge Vault ensures secure storage and strict access control for sensitive or exact-match information; it implements access restrictions and encryption for high-sensitive entries.

3. Multi-Agent Coordination Mechanism

A defining feature of MIRIX is its multi-agent management model:

• Meta Memory Manager: Directs the flow of information, determining which memory modules require update or retrieval on new user events.
• Individual Memory Managers: Each manages storage, retrieval, and update for its respective memory type, operating in parallel to maximize throughput and separation of concerns.

During user interaction:

• The Chat Agent initiates a coarse retrieval sweep across all memory components.
• It refines retrieval through targeted queries, enabled by memory-type tagging and the "Active Retrieval" protocol.
• Redundancy is explicitly managed: near-duplicate entries (e.g., similar screenshots) are detected and discarded to optimize storage and operating efficiency.

This concurrent, role-differentiated scheme allows real-time responsiveness and minimizes both data duplication and retrieval latency.

4. Empirical Performance and Benchmarks

MIRIX has been evaluated on two advanced benchmarks, demonstrating superior performance over contemporary memory systems:

• ScreenshotVQA Benchmark: Requires answering queries based on sequences containing nearly 20,000 high-resolution computer screenshots.
  • MIRIX improved accuracy by approximately 35% over a Retrieval-Augmented Generation (RAG) baseline.
  • Achieved a storage reduction of roughly 99.9% relative to methods retaining raw images, with operational storage as low as 15–20 MB per sqlite database.
• LOCOMO Benchmark: Involves long-form conversational tasks (average dialogue length ∼26,000 tokens), with queries strictly answered from retrieved memory.
  • MIRIX attained a state-of-the-art LLM-as-a-Judge score of 85.38% using gpt-4.1-mini, outperforming baselines such as LangMem.
  • Particularly strong results were reported for multi-hop reasoning tasks, with improvements exceeding 24 percentage points over other models.

The framework’s effectiveness is concisely summarized by the provided equation:

$$\text{Accuracy}_{\text{improvement}} \approx 35\% \text{ (vs. RAG)} \ \text{Storage}_{\text{reduction}} \approx 99.9\% \text{ (vs. full-resolution image storage)}$$

5. Application and Deployment

MIRIX is deployed as a stand-alone, cross-platform personal assistant application with several operational features:

• Real-Time Screen Monitoring: A React-Electron frontend and Uvicorn backend capture screenshots at 1.5-second intervals. Redundant screenshots are filtered by visual similarity, and update flows are initiated upon collection of 20 unique images.
• Personalized Memory Base: Memory extraction and consolidation into the six memory components occur dynamically, using a streaming upload methodology (with the Gemini API) to reduce latency and maintain memory freshness.
• Privacy and Security: Knowledge Vault entries are end-to-end encrypted with granular permissions. All data is maintained locally via an sqlite backend to restrict exposure and preserve user privacy, especially for sensitive content.
• Visualization Interfaces: The assistant features both chat and memory-inspection interfaces, including tree visualizations for Semantic Memory and lists for Procedural Memory, enhancing transparency and user control over stored information.

A plausible implication is that this modular packaging could enable integration with personalized AI assistants and wearable computing platforms.

6. Comparative Innovations and Contributions

MIRIX introduces several innovations that differentiate it from prior and contemporary memory-augmented LLM frameworks:

• Compositional Memory Model: Explicit partitioning of memory into six functional types supports both finer retrieval granularity and storage optimization, in contrast with flat or monolithic memory stores.
• Multi-Modal and Multi-Agent System: Utilization of multimodal data—especially high-resolution screenshots and documents—extends beyond text-only approaches. The distributed agent architecture permits dynamic task routing and parallel execution.
• Active Retrieval Strategy: MIRIX’s retrieval protocol, which first generates context-specific topics and then integrates results tagged by memory type, contrasts with single-context or flat retrieval methodologies, enabling continually updated and relevant responses.
• Empirical Performance: Benchmarking reveals significant improvements over systems such as LangMem, Mem0, and traditional RAG approaches in both task accuracy and operational efficiency.
• Real-World Usability: The comprehensive application—including real-time interaction, visualization, and secure storage—demonstrates operational readiness and privacy-conscious design for end users.

7. Context and Outlook

MIRIX provides a scalable, robust blueprint for memory-augmented LLM systems, emphasizing structured memory, parallel management, and multimodal integration. Its architecture supports dynamic, real-time use cases and enables agents to reason contextually over long-term, personalized data at scale. This suggests potential applications in personalized AI, advanced digital assistants, and domains demanding stringent privacy controls. The engineering features of modularity, parallelism, and privacy set technical precedents for future research and system development in memory-driven AI agent architectures.