FluxMem Adaptive Memory Framework

• FluxMem is a suite of adaptive memory frameworks that address long-horizon reasoning challenges by dynamically compressing, reorganizing, and evolving memory content.
• It employs hierarchical techniques—such as Temporal Adjacency Selection and Spatial Domain Consolidation—to optimize performance in multi-modal and multi-turn tasks.
• Empirical results demonstrate significant improvements in latency, GPU memory use, and overall task performance across streaming video and LLM agent benchmarks.

FluxMem is a family of adaptive memory frameworks designed to address the challenges of long-horizon reasoning, retrieval, and efficient representation in both LLM-based interactive agents and streaming video understanding systems. Across its major instantiations, FluxMem consistently replaces static, one-size-fits-all memory structures with mechanisms that either adaptively compress, reorganize, or evolve the memory content and connectivity according to the demands of the underlying environment, user queries, or the intrinsic dynamics of multi-modal streams (Xie et al., 2 Mar 2026, Fang et al., 27 May 2026, Lu et al., 15 Feb 2026).

1. General Principles and Motivation

FluxMem frameworks are motivated by two convergent needs: (1) the heterogeneity and scale of information in long-horizon tasks (such as multi-turn dialogues, web navigation, long videos), and (2) the brittleness or inefficiency of static, fixed-retrieval, or monolithic memory stores. Existing memory modules typically rely on simple lists, graphs, or buffers with hand-tuned thresholds and fixed interfaces, yielding brittleness under task variation and inefficiency under resource constraints (Lu et al., 15 Feb 2026, Fang et al., 27 May 2026). FluxMem differentiates itself by introducing context-sensitive structure selection, automated compression, or connectivity evolution—each targeting the deficiencies of prior schemes.

Three major paradigms instantiate the FluxMem approach:

• Hierarchical adaptive memory with automatic compression for streaming video (Xie et al., 2 Mar 2026).
• Multi-structure, context-adaptive episodic memory for LLM agents (Lu et al., 15 Feb 2026).
• Evolving heterogeneous memory graphs with feedback-driven refinement and consolidation for general agentic tasks (Fang et al., 27 May 2026).

2. Hierarchical Compression for Streaming Video Understanding

In the video domain, FluxMem implements a plug-and-play, training-free wrapper around any video-capable LLM, with a three-tier memory organization:

• Short-term memory ($\mathcal{M}^s$) holds the most recent frames in full fidelity for immediate grounding.
• Mid-term memory ($\mathcal{M}^m$) stores frames compressed with respect to temporal novelty.
• Long-term memory ($\mathcal{M}^l$) stores spatially consolidated anchors representing heavily compressed long-range information.

Memory promotion and compression are governed by two adaptive modules:

1. Temporal Adjacency Selection (TAS): Removes temporally redundant tokens by measuring token-wise novelty between adjacent frames using local cosine distances and selecting tokens via Otsu's self-adaptive thresholding (Xie et al., 2 Mar 2026).
2. Spatial Domain Consolidation (SDC): Further reduces tokens by identifying spatially repetitive regions using local graph connectivity and replacing connected components with anchor means, also using an Otsu-derived threshold.

Key parameters are dynamically set per frame; there is no manual threshold tuning except for a single proactive trigger parameter $\gamma$.

Streaming Update Loop

The sequential flow is as follows (summarized pseudocode):

$\mathcal{M}^m$0

Performance: On StreamingBench and OVO-Bench, FluxMem achieves state-of-the-art scores (76.4, 67.2) with substantial drops in latency (−69.9%) and peak GPU memory (−34.5%) compared to baselines, while discarding approximately 64% of visual tokens (Xie et al., 2 Mar 2026).

3. Multi-Structure Adaptive Memory for LLM Agents

For long-horizon textual agents, FluxMem (Lu et al., 15 Feb 2026) generalizes beyond flat memory buffers by maintaining a three-level memory hierarchy:

• Short-Term Interaction Memory (STIM): Fixed-size buffer for the most recent interactions.
• Mid-Term Episodic Memory (MTEM): Episodic units that can be linear chains, graphs, or hierarchical trees; structure is adaptively chosen by a learned selector.
• Long-Term Semantic Memory (LTSM): Abstracted, factual summaries consolidated from high-utility episodes.

Adaptive Structure Selection: At each step, an MLP-based selector uses a 12-dimensional feature vector (e.g., entity density, temporal density, topic diversity) to choose the optimal data structure for new episodes, supervised via response quality and memory utilization reward signals.

Probabilistic Memory Fusion: Instead of fixed similarity thresholds for merging new content, a Beta Mixture Model (BMM)-based probabilistic gate dynamically assigns pages to sessions, offering distribution-aware, robust fusion.

Lifecycle: The system supports automatic promotion and pruning within and across tiers, ensuring that memory composition evolves with access patterns, recency, and utility.

Empirical Results: On PERSONAMEM and LoCoMo, FluxMem offers average absolute performance improvements of 9.18% and 6.14% over strong memory baselines. Ablations demonstrate clear utility for each memory structure and for the BMM gate (Lu et al., 15 Feb 2026).

Dataset	Best Baseline	FluxMem	Δ (%)
PERSONAMEM	66.73%	72.43%	+9.18
LoCoMo	47.02 (F1)	49.95	+6.14

4. Evolving Memory Graphs: Connectivity and Consolidation

FluxMem generalizes further by treating memory as a heterogeneous graph $G = (V, E)$, with three distinct node types:

• Semantic nodes ($V_{sem}$): Factual/document chunks.
• Episodic nodes ($V_{epi}$): Trajectory records.
• Procedural nodes ($V_{proc}$): Distilled skills or reasoning circuits.

Connectivity evolves through three stages:

1. Initial Connection Formation: Retrieval and connection of semantic, episodic, and procedural nodes relevant to the current observation. Scoring integrates dense similarity, BM25, and LLM-based verification.
2. Feedback-Driven Refinement: On receipt of feedback, perform link expansion (repair missing connections), pruning (address misinformation/noise), or abstraction alignment (refine granularity).
3. Long-Term Consolidation: Cluster episodic nodes using k-means; distill procedural skills by LLM summarization; refine skills through Procedure Evolution Maturity Score (PEMS), which combines average trajectory success, skill succinctness, and stability in embedding space.

Metrics: Overall memory quality is measured by a convex combination of generalizability and maturity, $$M(G) = \alpha \cdot Gen(G) + (1-\alpha) \cdot Mat(G)$$, supporting trade-offs between broad coverage and robust long-term skill consolidation (Fang et al., 27 May 2026).

Empirical Results: On LoCoMo, Mind2Web, and GAIA, FluxMem consistently outperforms baselines (e.g., Full-Context, EverMemOS, AWM, Flash-Searcher) on LLM-Judge and task success metrics. Ablations show Stage II is most critical for factual recall, while Stage III drives advancements in reasoning-heavy, high-variance domains.

5. Core Algorithms and Compression/Selection Procedures

Distinct algorithmic elements underpin the various instantiations:

Pseudocode Examples

• TAS: Token retention determined by per-frame Otsu thresholding over local novelty scores; selection if novelty exceeds thresholds in either temporal direction.
• SDC: Spatial graph construction within frames, component clustering by local cosine similarity, with replacement by anchor mean vectors.
• Link Repair/Expansion: Add graph edges where feedback indicates under-exploitation, by top similarity/certainty.
• Abstraction Alignment: Use LLM prompt to alter the granularity of a memory node, updating its embedding.
• Procedural Circuit Distillation: Cluster episodic embeddings, induce a procedural skill via LLM, and add corresponding graph links; iterate until skill maturity stabilizes.

Automated Compression and Adaptivity

Adaptive thresholding (Otsu per-frame/per-component) is fundamental, distinguishing FluxMem from all methods with fixed or manually-tuned drop ratios (Xie et al., 2 Mar 2026, Lu et al., 15 Feb 2026). Probabilistic gating (BMM) ensures flexible, context-sensitive session merging (Lu et al., 15 Feb 2026), while graph evolution leverages LLM feedback and statistical metrics to guide refinement (Fang et al., 27 May 2026).

6. Benchmark Results and Ablation Findings

FluxMem variants have been evaluated on a spectrum of temporal reasoning, navigation, and long-context understanding tasks:

• Streaming video: Achieves 76.4 on StreamingBench and 67.2 on OVO-Bench in real-time, with ∼64% token reduction and significant decreases in GPU memory and latency (Xie et al., 2 Mar 2026).
• Long-horizon LLM agents: On PERSONAMEM and LoCoMo, FluxMem surpasses best baselines by 9.18% and 6.14% respectively (Lu et al., 15 Feb 2026).
• General agentic settings: Gains of 13–19 points over baselines on LoCoMo, Mind2Web, GAIA with strong statistical significance; ablations demonstrate criticality of staged graph refinement and procedural consolidation (Fang et al., 27 May 2026).

Domain	Baseline	FluxMem	Absolute Gain
StreamingBench	73.9	76.4	+2.5
OVO-Bench	63.3	67.2	+3.9
PERSONAMEM	66.73	72.43	+9.18
LoCoMo	47.02 (F1)	49.95	+6.14
Mind2Web (SR)	3.6–5.6%	8.1–9.6%	+3.6–4%

Ablations consistently show that either removing structural diversity, dropping adaptive gating in memory fusion, or limiting graph evolution all negatively impact performance, task robustness, or efficiency (Xie et al., 2 Mar 2026, Lu et al., 15 Feb 2026, Fang et al., 27 May 2026).

7. Limitations and Extensions

Identified limitations:

• Closed-loop refinement incurs latency and cost, especially where procedural distillation or feedback alignment requires multiple LLM inferences (Fang et al., 27 May 2026).
• Offline consolidation stages are batch-oriented; streaming, online variants are subjects for future work (Fang et al., 27 May 2026).
• Manual tuning for response triggers ($\gamma$) still exists for certain proactive features (Xie et al., 2 Mar 2026).
• Hyperparameter α in graph maturity/generalizability trade-off requires domain-aware calibration (Fang et al., 27 May 2026).

Plausible implications: Adaptive, multi-stage memory frameworks such as FluxMem may generalize to other sequential, non-stationary domains beyond language and video, provided context-sensitive memory structure selection and dynamic compression/adaptation can be instantiated for those modalities. A likely direction is developing online, low-latency procedural circuit distillation, or meta-learning structure selectors for broader task coverage.