O-Mem: Memory-Centric Architectures

• O-Mem is a family of memory-centric architectures and algorithms that unify optical, neuromorphic, and AI domains to enhance context-sensitive information retention and retrieval.
• Physical O-Mem devices leverage mem-emitters based on TMD monolayers to encode historical stimuli via non-Markovian dynamics, achieving sub-GHz switching and energy efficiency.
• Computational O-Mem frameworks employ hierarchical memory organization for LLM-powered agents and multimodal compression, significantly improving personalization, latency, and resource utilization.

O-Mem denotes a family of memory-centric architectures and algorithms spanning optical, neuromorphic, and artificial intelligence domains, unified by the principle of high-efficiency, context-sensitive information retention, retrieval, and manipulation. In recent literature, O-Mem has referred to both physical-memory devices based on mem-emitters and a computational framework for memory-efficient, long-horizon reasoning in LLM–powered agents and multimodal models. The following sections detail the diverse realization of O-Mem across these contexts, emphasizing design principles, underlying mathematical formalisms, system architectures, retrieval and update mechanisms, empirical results, and open research challenges.

1. Formal Definition and Taxonomy

O-Mem, in its broadest expression, subsumes two main realizations:

• Optical-Memory Devices (Physical O-Mem): Devices, such as mem-emitters, whose light-emission characteristics (intensity, wavelength) encode the time history of applied electrical or optical stimuli. These devices exhibit non-Markovian dynamics, with future emission states dependent on past input trajectories. Typical platforms rely on monolayer transition metal dichalcogenides (TMDs) atop dielectric substrates, where emission modulation arises via hysteresis in carrier population or radiative rates (Lopez-Richard et al., 2024).
• Omni-Memory Frameworks for AI Systems (Computational O-Mem): Machine learning memory modules designed for personalized, long-horizon, or audio-visual reasoning, characterized by dynamic profiling, hierarchical memory organization, and highly efficient, context-aware retrieval. Implementations target both agent personalization (Wang et al., 17 Nov 2025) and video-language compression/distillation (Wu et al., 26 May 2026, Zhao et al., 28 May 2026, Sun et al., 26 May 2026).

The unifying thread is a design emphasis on memory systems that adaptively compress, filter, and retrieve salient information, preserving the signal necessary for downstream inference or functional emission with minimal redundancy.

2. Physical O-Mem: Mem-Emitter Devices and Operating Principles

Mem-emitters provide an O-Mem physical instantiation whereby optical emission properties (intensity $Q$, energy $\hbar\omega$) are a function not just of instantaneous external fields but of the temporal trajectory of applied voltage, field, or illumination (Lopez-Richard et al., 2024). In canonical models:

$$Q = \frac{N}{\tau},\qquad \frac{1}{\tau} = K_{f,i} D(\Delta E_{f,i} - \hbar\omega)$$

where $N=N(\mathbf{F},\mathbf{x},t)$ is the excited-state population, $\tau$ the radiative lifetime, $K_{f,i}$ a transition matrix element, and $D(\cdot)$ an energy-conserving lineshape.

Two principal memory mechanisms are distinguished:

• Population-driven: Memory encoded in slow relaxation of carrier trapping/detrapping, yielding modulation of $N$ via rate equations:

$$\frac{d\,\delta n_i}{dt} = -\frac{\delta n_i}{\tau_i} + g_i^{(n)}(V(t)), \qquad Q \propto N_0 + \sum_i \delta n_i(V)$$

• Transition-rate-driven: Memory encoded in field- or substrate-induced modification of radiative rates or energy splittings, e.g., via Stark effect or wavefunction overlap:

$$\frac{1}{\tau} = K_{f,i}(F_{TMD})D[\Delta E_{f,i}(F_{TMD})-\hbar\omega]$$

Architectures typically entail atomic-thickness TMD monolayers ($\hbar\omega$0 ≈ 0.6–1 nm) on dielectrics ($\hbar\omega$1 ≈ 10–300 nm), with state variables $\hbar\omega$2 capturing both charge and dipole memory channels.

A hallmark is the emergence of stable hysteresis loops in $\hbar\omega$3 vs. external field, with loop area $\hbar\omega$4 quantifying memory capacity.

System-level advantages include sub-GHz–GHz switching, ms–s retention, voltage tunability (tens of meV per V/nm), and energy-efficient switching (femto- to attojoule/bit).

3. O-Mem in LLM-Powered Agents: Architecture and Memory Organization

In the context of LLM–based agents, O-Mem refers to an omni-memory system for long-horizon personalization and adaptive context retrieval (Wang et al., 17 Nov 2025). The architecture encompasses:

• Active User Profiling: An LLM component extracts user topics ($\hbar\omega$5), persona attributes ($\hbar\omega$6), and factual events ($\hbar\omega$7) from each interaction $\hbar\omega$8 via semantic parsing.
• Threefold Memory Store:
  • Persona Memory: Structured events ($\hbar\omega$9) and attribute clusters ($$Q = \frac{N}{\tau},\qquad \frac{1}{\tau} = K_{f,i} D(\Delta E_{f,i} - \hbar\omega)$$0) distilled via nearest-neighbor graph clustering in embedding space.
  • Working Memory: Topic-to-interaction dictionary $$Q = \frac{N}{\tau},\qquad \frac{1}{\tau} = K_{f,i} D(\Delta E_{f,i} - \hbar\omega)$$1
  • Episodic Memory: Token-to-interaction dictionary $$Q = \frac{N}{\tau},\qquad \frac{1}{\tau} = K_{f,i} D(\Delta E_{f,i} - \hbar\omega)$$2
• Hierarchical Parallel Retrieval: At query time, for user query $$Q = \frac{N}{\tau},\qquad \frac{1}{\tau} = K_{f,i} D(\Delta E_{f,i} - \hbar\omega)$$3:
  • Retrieve from working memory by topic similarity,
  • Retrieve from episodic memory (distinctive cue matching),
  • Retrieve from persona memory by profile similarity (cosine distance in attribute/event embedding).

No explicit semantic grouping is required prior to retrieval; parallel and on-the-fly retrieval minimizes retrieval noise and leverages associative as well as topical continuity. Update policies employ LLM-based Add/Ignore/Update operations with temporal clustering and distinctiveness filtering ($$Q = \frac{N}{\tau},\qquad \frac{1}{\tau} = K_{f,i} D(\Delta E_{f,i} - \hbar\omega)$$4).

This approach yields a statistically significant increase in both LoCoMo and PERSONAMEM benchmarks, outperforming earlier group-then-retrieve memory systems by 2.95–3.57 percentage points, while reducing average tokens per query ($$Q = \frac{N}{\tau},\qquad \frac{1}{\tau} = K_{f,i} D(\Delta E_{f,i} - \hbar\omega)$$51,500 vs. $$Q = \frac{N}{\tau},\qquad \frac{1}{\tau} = K_{f,i} D(\Delta E_{f,i} - \hbar\omega)$$645,000) and latency (2.36 s vs. 10.8 s) (Wang et al., 17 Nov 2025).

4. O-Mem for Multimodal Compression and Long-Context Reasoning

O-Mem underpins modern audio-video LLMs as a framework for selective, memory-centric context compression, balancing computational efficiency with long-range semantic retention (Wu et al., 26 May 2026, Sun et al., 26 May 2026, Zhao et al., 28 May 2026). The generalized pipeline consists of:

• Memory-Augmented Compression: The OMAC module implements a selective pipeline on raw video and audio tokens, constructing:
  • Coarse memory slots: query-guided frame summaries (visual) and audio anchors (acoustic).
  • Fine memory carriers: patch tokens with high contrast or distinctiveness within selected frames, and trimmed, temporally-merged audio tokens.

Mathematically, for video tokens $$Q = \frac{N}{\tau},\qquad \frac{1}{\tau} = K_{f,i} D(\Delta E_{f,i} - \hbar\omega)$$7 and audio tokens $$Q = \frac{N}{\tau},\qquad \frac{1}{\tau} = K_{f,i} D(\Delta E_{f,i} - \hbar\omega)$$8:

$$Q = \frac{N}{\tau},\qquad \frac{1}{\tau} = K_{f,i} D(\Delta E_{f,i} - \hbar\omega)$$9

Retain top-$N=N(\mathbf{F},\mathbf{x},t)$0 frames, extract patch set $N=N(\mathbf{F},\mathbf{x},t)$1 within each by contrast, and compute memory token $N=N(\mathbf{F},\mathbf{x},t)$2 via

$N=N(\mathbf{F},\mathbf{x},t)$3

Audio anchors are similarly scored and merged, with visual memory guiding the allocation of audio slots: $N=N(\mathbf{F},\mathbf{x},t)$4

• RL-based Compression-Aware Distillation: O-MARC shapes the reward signal to penalize degradation under compression, biasing the teacher-student RL objective toward behaviors invariant to token truncation:

$N=N(\mathbf{F},\mathbf{x},t)$5

• Empirical Results:
  • 30% token retention with O-Mem/OMAC achieves 1.53× latency speedup and a 34.7% memory reduction (Qwen2.5-Omni-3B: 15.8 GB vs. 24.2 GB), yet improves mean QA accuracy (45.8 vs. 44.1 for full-context, 41.0 for OmniZip) (Wu et al., 26 May 2026).
  • O-Mem compression remains robust under higher pruning ratios and scales across sequence lengths.

5. Memory Management for Long-Video Generation and Streaming AV-LLMs

For chunk-based autoregressive video generation and streaming audio-visual LLMs, O-Mem (also denoted as OmniMem) addresses the challenge of unbounded key-value cache growth by explicit, adaptive, and perturbation-aware memory selection (Zhao et al., 28 May 2026, Sun et al., 26 May 2026):

• Chunked KV Cache Management: Track $N=N(\mathbf{F},\mathbf{x},t)$6 latent chunks, each of $N=N(\mathbf{F},\mathbf{x},t)$7 tokens, maintaining historic $N=N(\mathbf{F},\mathbf{x},t)$8 and $N=N(\mathbf{F},\mathbf{x},t)$9 state with per-layer and per-head indexing.
• Adaptive Window Exclusion: Refines sparse retrieval by masking the local window when sufficient long-range history exists, thus countering recency bias in Top-K selection.
• Query-Shared KV Selection: Partitions the $\tau$0 tokens into $\tau$1 groups, allowing token-level Top-K selection to be amortized, reducing the union size from $\tau$2 to $\tau$3.
• Per-Head Scattered KV Access: Avoids union explosion by supporting per-head, noncontiguous access to selected blocks.

Performance on VBench-Long indicates that OmniMem increases the Dynamic Degree (temporal consistency/motion) by 52.3% over best existing baselines, with only marginal (1.7%) VRAM overhead (Zhao et al., 28 May 2026). Run-time is nearly linear in retained history length, not total context size.

For streaming audio-visual LLMs, modality-aware allocation and perturbation-based scoring are utilized:

$\tau$4

Budget-aware fine-tuning further improves retention, yielding +0.6–2.4% accuracy gains across VideoMME Long, LVBench, LVOmniBench (Sun et al., 26 May 2026).

6. Limitations, Challenges, and Future Directions

• Physical O-Mem devices are limited by the trade-off between rapid state-setting (low $\tau$5) and long retention ($\tau$6); device design must balance bias amplitude, pulse duration, and endurance.
• LLM-based O-Mem frameworks are contingent on effective user profiling, persona attribute clustering, and distinctiveness filtering—all of which may degrade under noisy or ambiguous interaction streams.
• Multimodal O-Mem faces the constraint of hyperparameter tuning for OMAC and model-dependent robustness to heavy compression; future improvements may include dynamic budget allocation, reinforcement-tuned selection, and extension to novel modalities.
• For streaming and long-context video models, the additional cost of hidden state retention and selection kernel computation remains a challenge; highly efficient similarity estimation and non-uniform chunk budgeting are plausible research directions (Sun et al., 26 May 2026).
• Dataset bottlenecks persist: benchmarks (e.g., UGC-AVQA, VBench-Long, LVBench) only partially cover the diversity and length of real-world scenarios.
• Extensions to explicit decay/forgetting, continual profile validation, or multi-modal integration (e.g., vision, speech, robotics) are anticipated.

O-Mem frameworks—whether as physical memory in optoelectronic devices or as computational modules within LLM-powered agents and multimodal models—provide a foundational approach to managing long-horizon information under strict resource constraints, with demonstrated advantages in personalization, context compression, and energy or compute efficiency (Lopez-Richard et al., 2024, Wang et al., 17 Nov 2025, Wu et al., 26 May 2026, Zhao et al., 28 May 2026, Sun et al., 26 May 2026).