Omni-SimpleMem: Unified AI Memory Framework

• Omni-SimpleMem is a unified lifelong memory framework that formalizes memory as Multimodal Atomic Units (MAUs) to capture diverse data types and support efficient retrieval.
• It leverages selective ingestion, progressive retrieval, and knowledge graph augmentation to optimize the processing of multimodal signals and enhance inference efficiency.
• The AutoResearchClaw pipeline autonomously refines system design with iterative LLM-guided modifications, driving substantial benchmark improvements and operational robustness.

Omni-SimpleMem is a unified lifelong memory framework for AI agents that integrates a multimodal memory representation with efficient retrieval strategies, inspired by human cognitive architectures. Its design was autonomously discovered via the AutoResearchClaw pipeline, which iteratively executed architectural, data, and prompt modifications guided exclusively by LLM agents and benchmark feedback. Omni-SimpleMem formalizes memory as a set of "Multimodal Atomic Units" (MAUs), leverages hybrid retrieval strategies with dense-sparse and knowledge graph augmentation, and combines principles from both the O-Mem memory system and autoresearch-driven design. The system establishes new state-of-the-art results on challenging benchmarks while dramatically improving inference efficiency and demonstrates properties that make it particularly amenable to further autoresearch and autonomous system optimization (Liu et al., 1 Apr 2026, Wang et al., 17 Nov 2025).

1. Architectural Overview

Omni-SimpleMem's memory system is structured along three core principles—Selective Ingestion, Progressive Retrieval, and Structured Knowledge—instantiated in a pipeline with the following main stages:

• Selective Ingestion: Multimodal streams (text, image, audio, video) pass through novelty filters:
  • Vision: CLIP embeddings are computed, and frames are retained only if the cosine similarity with the last stored frame falls below threshold τ_high.
  • Audio: A VAD gate rejects silence.
  • Text: Jaccard overlap with recent MAU summaries discards high-duplication text above τ_dup.
  • Retained signals are summarized using modality-specific LLM prompts and embedded into a shared vector space.
• Multimodal Atomic Units (MAUs):

Each MAU is defined as $$M_i = \langle s_i, e_i, p_i, \tau_i, m_i, \ell_i \rangle$$, with: $s_i$: summary, $e_i$: normalized embedding, $p_i$: pointer to cold storage, $\tau_i$: timestamp, $m_i$: modality, $\ell_i$: graph links. Hot storage (summaries, embeddings, metadata) uses FAISS and JSON-Lines; cold storage maintains raw content (images, audio, text).

• Progressive Retrieval: Hybrid dense–sparse search retrieves MAUs:
  1. Hybrid Candidate Generation: Top-K dense (FAISS) and Top-L sparse (BM25 on summaries) are merged ($R(q) = D(q) \cup (K(q) \setminus D(q))$), preserving dense order.
  2. Pyramid Expansion: For $M_i$ in $R(q)$, progressively load s_i (summary), then full text/caption if score ≥ θ, and finally, the raw content in descending “score-per-token” order within budget B.

• Knowledge-Graph Augmentation: Entity–relational triples, extracted by LLM from summaries, are merged (using a hybrid similarity of α·cosine_embed + (1–α)·Jaro–Winkler above threshold τ_res) into a knowledge graph $s_i$0. At query time, entities in the query seed a neighborhood search (expanded to h hops, each node scored as $s_i$1). Linked MAUs for top graph nodes expand the hybrid candidate set.

This architecture is visualized in the primary pipeline figure as four interconnected blocks: novelty-filtering ingestion, MAU storage, hybrid+pyramid retrieval, and graph augmentation (Liu et al., 1 Apr 2026).

2. Mathematical Formulation

The system's central data structure, the MAU, is formalized by: $s_i$2 where $s_i$3 is computed as: $s_i$4 with Enc_m(·) an off-the-shelf encoder per modality.

Novelty Filtering (Example, Vision):

$s_i$5

Retrieval Scoring:

• Dense similarity: $s_i$6
• Sparse BM25: $s_i$7
• Hybrid candidate set: $s_i$8
• Knowledge graph score: $s_i$9

3. AutoResearchClaw Pipeline: Autonomous System Discovery

The AutoResearchClaw pipeline operationalizes a fully autonomous cycle for system improvement. The process executes 23 stages grouped into eight phases and is implemented as follows: $e_i$0 Key features include self-healing execution (automatic bugfixes for API/runtime errors), semantic failure diagnosis (targeted log analysis for low F1), and a multi-agent debate mechanism where LLMs propose and critique hypotheses before implementation (Liu et al., 1 Apr 2026).

4. Empirical Evaluation and Key Discoveries

Omni-SimpleMem achieves substantial gains across multiple benchmarks:

Benchmark	Baseline F1 / Accuracy	Final	Relative Improvement
LoCoMo	0.117	0.598	+411%
Mem-Gallery	0.254	0.797	+214%
PERSONAMEM	59.42% (A-Mem)	62.99%	+3.57 percentage points
Personalized Deep	36.43% (Mem0)	44.49%	+8.06 percentage points

Major contributions arise chiefly from:

• Bug fixes (+175% relative on LoCoMo iteration 1)
• Architectural changes (+44% relative on LoCoMo)
• Prompt engineering (up to +188% on specific Mem-Gallery categories)
• Hyperparameter tuning contributes <10% to total gains (Liu et al., 1 Apr 2026, Wang et al., 17 Nov 2025).

The system also achieves dramatic efficiency improvements over previous memory frameworks, reducing token cost by 94% and latency by 80% (LoCoMo Direct RAG: 2.6K tokens/query, 4.01s latency; Omni-SimpleMem: 1.5K tokens/query, 2.36s latency; GPU memory down from 33.16MB to 22.99MB) (Wang et al., 17 Nov 2025).

5. Relation to O-Mem and Personalization Mechanisms

Omni-SimpleMem is a distilled variant of O-Mem (Wang et al., 17 Nov 2025). It retains O-Mem’s conceptual division into working memory (topic-indexed), episodic memory (keyword/clue-indexed), and persona memory (fact- and attribute-based, updated on-the-fly). O-Mem’s mechanisms include:

• LLM-driven extraction of (topic, persona attribute, persona event) per interaction.
• Construction of clue-interaction and topic-interaction maps, and nearest-neighbor graphs for persona deduplication.
• Parallel, staged retrieval: working memory via topics, episodic memory via rarest clues, persona memory via fact/attribute matching, then a single LLM call for output generation.

Omni-SimpleMem simplifies some of these bookkeeping operations but follows the same principles—active user profiling, hierarchical retrieval, and efficient, structured updates.

6. Taxonomy of System Discoveries and Design Properties

Trajectory analysis of the AutoResearchClaw pipeline identifies six types of system discoveries:

1. Bug Fixes (e.g., response_format errors)
2. Architectural Changes (e.g., hybrid search, pyramid retrieval)
3. Prompt Engineering (constraint positioning, hallucination suppression)
4. Data Pipeline Repairs (tokenization, timestamp corrections)
5. Evaluation-Format Alignment (JSON enforcement)
6. Hyperparameter Tuning (top-k, thresholds, budgets)

Four properties particularly advantageous for autoresearch in this domain:

• Immediate scalar feedback (F1, accuracy) enabling tight feedback loops
• Modular code architecture (11 subpackages) facilitating fine-grained edits
• Fast experiment cycles (1–2 hours per round, enabling dozens of experiments in days)
• Robust version control and sandboxing for safe patch application/reverts

This suggests the suitability of multimodal memory domains for general-purpose autoresearch pipelines (Liu et al., 1 Apr 2026).

7. Limitations and Prospects

Omni-SimpleMem’s effectiveness depends on LLM extraction quality in persona profiling. Absence of explicit forgetting or decay can result in the accumulation of infrequently accessed, noisy memories. The current pipeline is primarily textual; extending to richer multimodal signals (audio, vision, sensors) may improve episodic recall. Privacy and compliance for persona abstraction and encryption remain open challenges. Future extensions could include vector-indexed sub-second search, adaptive per-user retrieval budgets, and reinforcement-learned policies for memory management (Wang et al., 17 Nov 2025).