MemTool: Memory Optimization Framework

• MemTool is a framework for dynamic memory optimization in LLM agents, offering modular mechanisms for tool acquisition, retention, and pruning.
• It supports three operational modes—Autonomous Agent, Workflow, and Hybrid—that balance adaptability and determinism in tool management.
• Empirical evaluations demonstrate that MemTool improves tool selection efficiency and task accuracy by effectively managing memory constraints in prolonged conversations.

MemTool is a framework and conceptual family of mechanisms for memory optimization and management in agents—primarily LLM agents—operating in dynamic tool-use, conversation, and knowledge-update regimes. The term spans both lightweight, short-term memory controllers for dynamic tool invocation in multi-turn dialogue and more architectural approaches for memory editability in machine reasoning systems. Applications encompass multi-turn LLM agents, memory-augmented retrieval systems, tool-use optimization, and explicit architectural memory designs. The defining principle is the modularization and explicit management of agent memory—acquisition, retention, pruning, and revision—around tools, models, or factual sequences, often supporting autonomy, deterministic workflow, and hybrid control.

1. Motivation: Memory Bottlenecks in Multi-Turn Tool-Using Agents

LLM agents engaging in multi-turn interactions with dynamic tool use confront two significant constraints: (i) context overflow, where the total number of active tools or external model contexts can exceed platform-imposed API limits (typically 128–512), and (ii) memory drift, where obsolete tools accumulate in the context window, degrading the agent’s performance and reasoning efficiency (Lumer et al., 29 Jul 2025). MemTool was created to provide a dynamic, agent-integrated memory manager. The core driver is to enable agents to dynamically search, introduce, and prune tools (or Model Context Protocol servers) over 100+ turns, maintaining both operational feasibility (API constraints) and high task-completion accuracy.

2. MemTool Framework Architectures and Agentic Modes

MemTool formalizes three operational regimes, each representing a continuum between agent autonomy and deterministic or scripted workflow:

• Autonomous Agent Mode: The LLM is instrumented with full agency, directly invoking Search_Tools(keywords) and Remove_Tools(tool_names) during both reasoning and answer generation. Search adds up to 5 tools via vector retrieval; removal prunes those deemed unnecessary. The agent decides when to trigger each step, yielding highly adaptive tool management but relying on the backbone model's reasoning capabilities (Lumer et al., 29 Jul 2025).
• Workflow Mode: Tool management is decoupled from answer generation. A fixed three-stage pipeline precedes every exchange: first, LLM_prune prunes tool memory; second, LLM_search suggests new search queries; third, the updated tool set is passed to the backbone LLM for answering. All steps are strictly deterministic, conferring robust memory control, but at the cost of responsiveness and adaptability.
• Hybrid Mode: Deterministic pruning is executed first, after which the LLM can autonomously search for tools but not remove them. Any overflow is resolved via re-invocation of the prune step. This partial agency offers a compromise—flexible tool incorporation with reliably bounded memory growth.

The framework itself sits interstitially between the dialogue history, a vector-embedded tool knowledge base (e.g., ScaleMCP), and the LLM, orchestrating dynamically available tool sets at each conversation turn.

3. Memory Management Mechanisms and Metrics

MemTool introduces quantitative metrics to measure and optimize short-term memory efficiency:

• Addition and Removal Counts: $A_t$ (tools added at turn $t$), $R_t$ (tools removed at turn $t$), $T_t$ (toolset size after $t$).
• Removal Ratio:

$$\text{RemovalRatio} = \frac{\sum_{t=1}^T R_t}{\sum_{t=1}^T A_t} \leq 1$$

• Rolling-Window Removal Ratio (3-turn average):

$$\text{AvgRemovalRatio}_{3T} = \frac{1}{T-2} \sum_{t=3}^T \frac{\sum_{i=t-2}^t R_i}{\sum_{i=t-2}^t A_i}$$

• Average Residual (post-bloat measure):

$$\text{AvgResidual}_{3T} = \frac{1}{T-2} \sum_{t=3}^T \left(\frac{1}{3}\sum_{i=t-2}^t T_i\right)$$

Memory operations are supported by embedding-based vector search (text-embedding-ada-002) for tool retrieval, ensuring that additions are semantically relevant to the user’s latest goal or query (Lumer et al., 29 Jul 2025).

4. Empirical Evaluation: Performance, Trade-offs, and Recommendations

Experiments span 13 commercially available LLMs (OpenAI GPT series, Google Gemini, Anthropic Claude, Meta LLaMA 3) on the ScaleMCP benchmark, with simulated 100-turn multi-tool interactions. Key findings (Lumer et al., 29 Jul 2025):

• Autonomous Agent Mode: Reasoning-optimized LLMs (GPT-o3, Claude Opus 4, Gemini 2.5 Pro/Flash) maintain high $\text{AvgRemovalRatio}_{3T}$ (0.90–0.94) and task completion scores (0.80–0.90), effectively pruning memory bloat. Medium-sized models fail to sustain tool removal (0–0.60) and degrade in both pruning and accuracy.
• Workflow Mode: All models attain efficient pruning ($t$0) but show a decline in downstream QA, especially for smaller models, due to the rigidity of the search pipeline.
• Hybrid Mode: Consistent, high pruning ($t$1) and strong task completion for sophisticated LLMs (e.g., Claude 3.7 Sonnet 0.88, GPT-o3 0.87).

Mode-specific recommendations:

• Deploy Autonomous Agent Mode for advanced models to maximize task accuracy and support on-the-fly tool discovery, but include explicit tool-count constraints.
• Use Workflow Mode as a memory controller for cost-sensitive or weaker models.
• Hybrid Mode balances adaptability and memory stability for mixed scenarios.

5. Connections to Long-Term, Versioned, and Architectural Memory

MemTool's focus is on short-term, dynamic tool management, but its principles extend to more general architectures for agent memory, knowledge versioning, and editability. Notably, the MeMo framework introduces multi-layer correlation matrix memories (CMMs) with atomic memo, forget, and retrieve operations. These are further enhanced by two auxiliary CMMs for version-aware indexing (V-CMM) and transaction storage (T-CMM). Transactions encode ordered memory edits (add, remove, replace, rollback) and ensure atomicity, traceability, and locality without model retraining (Li, 23 Jun 2026). This enables direct, history-preserving, or reversible memory updates—key for continual knowledge adaptation.

6. Limitations, Open Challenges, and Future Directions

Several limitations and unresolved issues persist:

• Agentic Pruning Reliability: Autonomous modes can fail catastrophically in small/medium models; prompts must explicitly encode memory constraints (Lumer et al., 29 Jul 2025).
• Workflow Rigidness: Inflexibility in deterministic modes prevents recovery from initial retrieval misses and complicates self-correction.
• Hybrid Looping: Agents in hybrid mode may trigger recursive pruning cycles if over-fetching occurs.
• Long-Term Personalization and Embedding Quality: Future work is needed on layering persistent, personalized memory and enhancing embedding fidelity to improve retrieval precision and adaptivity (Lumer et al., 29 Jul 2025).
• Adaptive Windowing: Current removal and residual metrics focus on a 3-turn window; extensions to adaptive windows for longer, more complex conversations are in progress.

The research agenda includes integrating multi-scale memory layers, supporting more complex transaction models (as in MeMo), and directly connecting tool-use memory with factual and ontological knowledge update frameworks (Li, 23 Jun 2026).

7. Broader Impact: Benchmarks and Practical Deployment

MemTool and related paradigms have directly catalyzed the creation of advanced benchmarks and empirical studies for memory-centric agent evaluation (e.g., Mem2ActBench for long-term memory utilization in tool selection and parameter grounding (Shen et al., 13 Jan 2026)). Analysis of failure cases—retrieval versus application—is now routine. Practical guidance includes pairing cheaper LLMs for deterministic pruning with higher-capacity models for answer generation and using hybrid architectures for real-world deployments.

Taken collectively, MemTool and its family of mechanisms represent a comprehensive, modular solution to the challenge of scalable, efficient, and accurate memory management for LLM-driven, tool-using agents operating in complex, persistent, multi-turn environments.