MemSearch-o1: Empowering Large Language Models with Reasoning-Aligned Memory Growth in Agentic Search

Abstract: Recent advances in LLMs have scaled the potential for reasoning and agentic search, wherein models autonomously plan, retrieve, and reason over external knowledge to answer complex queries. However, the iterative think-search loop accumulates long system memories, leading to memory dilution problem. In addition, existing memory management methods struggle to capture fine-grained semantic relations between queries and documents and often lose substantial information. Therefore, we propose MemSearch-o1, an agentic search framework built on reasoning-aligned memory growth and retracing. MemSearch-o1 dynamically grows fine-grained memory fragments from memory seed tokens from the queries, then retraces and deeply refines the memory via a contribution function, and finally reorganizes a globally connected memory path. This shifts memory management from stream-like concatenation to structured, token-level growth with path-based reasoning. Experiments on eight benchmark datasets show that MemSearch-o1 substantially mitigates memory dilution, and more effectively activates the reasoning potential of diverse LLMs, establishing a solid foundation for memory-aware agentic intelligence.

• The paper introduces a reasoning-aligned memory growth mechanism using part-of-speech-derived seeds to guide precise memory fragment extraction.
• The methodology dynamically grows and retraces memory fragments by aggregating seed-aligned evidence with cosine similarity and bridge potential for robust multi-hop reasoning.
• Empirical evaluations on diverse QA datasets show significant gains, including a 21.93% improvement on HotpotQA, while maintaining efficiency over extremely long contexts.

MemSearch-o1: Reasoning-Aligned Memory Growth for LLM Agentic Search

Motivation and Problem Formulation

Agentic search frameworks for LLM-based QA, especially deep search systems, require iterative planning, external corpus retrieval, and multi-hop reasoning. The accumulation of multi-round system memory leads to "memory dilution," where the signal-to-noise ratio rapidly degrades and the model’s attention overcontext is diluted. Existing memory management approaches, including various summarization, refinement, and pruning paradigms, either oversimplify semantic dependencies—incurring critical information loss—or become entangled with semantic drift across evolving search queries. To remedy these limitations, "MemSearch-o1: Empowering LLMs with Reasoning-Aligned Memory Growth in Agentic Search" (2604.17265) introduces a structured, reasoning-aligned memory growth and retracing mechanism that enables LLM agents to dynamically grow, filter, and reorganize memory at the fragment level anchored to part-of-speech-derived memory seeds.

Figure 1: Overview of MemSearch-o1, illustrating seed-token-driven memory growth and retrace for efficient, query-aligned reasoning.

Methodological Framework

MemSearch-o1 refactors the memory mechanism of agentic LLMs in deep search as follows:

• Memory Seed Extraction: At each reasoning step, MemSearch-o1 decomposes the current query into token groups based on linguistic parts of speech: subjects (nouns, pronouns), actions (verbs), degree modifiers (adjectives, adverbs), and temporal markers. These memory seeds provide fine-grained semantic anchors to guide evidence extraction and subsequent fragment growth.
• Memory Fragment Growth: For each memory seed, the agent retrieves relevant external documents. Leveraging explicit prompts and the identified seeds, the model extracts concise, seed-aligned fragments from the retrieved text, ensuring semantic tightness to the step-specific reasoning goal and avoiding entanglement with large, diluted contexts.
• Retracing and Memory Path Construction: All memory fragments from multiple rounds are aggregated, then retraced and filtered using a contribution function combining relevance (cosine similarity to the original query) and bridge potential (capacity to connect with other fragments). Greedy path construction forms a semantically smooth fragment path, supporting high-fidelity, multi-hop reasoning and answer generation.

This memory management paradigm shifts from monotonic, stream-like context concatenation to structured, fragment-level growth and dynamic retrace, which sharply mitigates context dilution and improves reasoning tractability.

Empirical Evaluation

Extensive benchmarking on eight QA datasets spanning Multi- and Single-Document QA (e.g., HotpotQA, 2WikiMQA, MuSiQue, DuReader, NarrativeQA, Qasper, MultiFieldQA-en/zh) demonstrates that MemSearch-o1 outperforms strong RAG, agentic search, and memory management baselines across both accuracy and efficiency metrics. Notably, with DeepSeek V3.1, MemSearch-o1 achieves a 21.93% absolute gain on HotpotQA and significant improvement on 2WikiMQA and MuSiQue.

• On single-document/localized evidence tasks (e.g., Qasper), where classical RAG methods are competitive, MemSearch-o1 maintains robust performance without oversearching—attributable to seed-based memory control.
• On extreme long-context tasks (up to 2M tokens in LongBookQA-zh), MemSearch-o1 demonstrates stable, strong reasoning, outperforming methods that suffer from oversearch or unreliable memory condensation.

Scalability and Ablation

Figure 2: QA performance of MemSearch-o1 across model sizes (Qwen2.5-0.5B to 72B) on 2WikiMQA.

MemSearch-o1 showcases consistent improvements with model scaling, activating deep search-based reasoning around 3B parameters and yielding monotonically better results with larger backbones. In contrast, Amber and earlier search/memory baselines achieve lower or less stable gains as size increases.

Ablation studies highlight essentiality:

• Removing memory fragment growth collapses performance to that of diluted context LLMs.
• Omitting retracing degrades performance due to the accumulation of off-target, redundant fragments.

Mechanistic Insights via Visualization

Figure 3: UMAP projection of memory paths—MemSearch-o1 extends retrieval and reasoning into broader, more goal-directed semantic regions compared to prior baselines.

Memory path visualization supports claims on semantic control and retrieval trajectory diversity. Even given identical retrieval regions, MemSearch-o1 provides expanded, coherent reasoning by guiding LLMs through deliberately constructed fragment chains, rather than compressing to an over-limited core.

Practical and Theoretical Implications

Practically, MemSearch-o1 enables multi-hop, agentic LLM systems to efficiently operate over extremely long external corpora while safeguarding essential evidence paths and maximizing the utilization of LLM latent reasoning. This is achieved without increasing generation cost or inference time compared to existing baselines. Cost-scale analysis reveals lower total token usage and time complexity $O(ND)$ (vs. $O(N^2D)$ for accumulation-based methods) across deep multi-round search.

Theoretically, this work contends that explicit reasoning-aligned memory growth and retracing at the token fragment level is both necessary and sufficient for robust long-context reasoning—contradicting the claim that generic memory summarization or pruning can suffice for stable multi-step agentic LLM search.

Future Directions

Improvements to the memory seed extraction pipeline (e.g., richer linguistic features or dynamic self-supervised selection), as well as adaptation to smaller LLMs or non-English/Multi-modal evidence domains, remain open research directions. The framework is immediately extensible to broader domains (e.g., scientific QA, narrative analysis) as demonstrated, but more sophisticated bridge/contribution scoring could further enhance long-range path coherence.

Conclusion

MemSearch-o1 establishes a robust framework for reasoning-aligned memory growth and retracing in agentic LLM search. By segmenting queries into linguistically motivated seed tokens and constructing memory paths via fragment growth and retrace, the approach substantially alleviates memory dilution, guarantees evidence completeness, and enhances multi-hop reasoning quality over long contexts. The method delivers state-of-the-art QA performance across diverse benchmarks and model scales, redefining memory management as a central axis for next-generation agentic AI systems.