DeepSearch-Evolve Overview

• DeepSearch-Evolve is a self-distillation framework that iteratively trains deep search agents using deterministic, verifiable intermediate progress signals.
• It leverages an offline Wikipedia-based environment, DeepSearch-World, with a large corpus to simulate web search and track entity-level progress.
• The iterative pipeline replaces sparse reward signals with process-level supervision, enhancing query reformulation, evidence gathering, and failure recovery.

Searching arXiv for the most relevant papers on DeepSearch-Evolve and closely related deep search agent training frameworks. arXiv search query: "DeepSearch-Evolve DeepSearch-World self-distillation deep search agents" DeepSearch-Evolve is a self-distillation framework for long-horizon deep search and web agents in which a model improves from its own tool-using trajectories inside a deterministic, verifiable environment. Introduced together with DeepSearch-World, the framework replaces dependence on stronger teacher models and sparse end-of-trajectory reinforcement signals with iterative supervised fine-tuning on self-generated, verified trajectories (Geng et al., 8 Jul 2026). Its central premise is that self-improvement in deep search becomes tractable when the environment can objectively verify intermediate progress, enabling process-level supervision over search, page reading, reflection, and recovery rather than only final-answer correctness (Geng et al., 8 Jul 2026).

1. Definition and problem setting

DeepSearch-Evolve addresses the problem of training tool-use agents for long-horizon search tasks, where supervised fine-tuning on fixed teacher-distilled trajectories tends to saturate and sparse-reward reinforcement learning provides weak supervision for extended interaction chains (Geng et al., 8 Jul 2026). The framework is presented as a form of self-evolving training for web agents, but its distinguishing feature is not merely iterative retraining; it is the coupling of that retraining to a deterministic and verifiable search environment that can supply objective signals about intermediate progress (Geng et al., 8 Jul 2026).

The paper identifies two common but inadequate self-improvement recipes for deep search. In one, supervised fine-tuning on self-collected positive trajectories is stable but constrained by the current backbone model, the diversity of collected trajectories, and the fact that it only imitates already-successful traces. In the other, sparse-reward reinforcement learning over full trajectories optimizes terminal answer correctness but does not disclose whether failure originated in query formulation, tool choice, evidence extraction, or final synthesis (Geng et al., 8 Jul 2026). DeepSearch-Evolve therefore reframes self-improvement as a self-distillation problem over verified process traces rather than as either fixed-demonstration imitation or end-only reward optimization.

A useful contrast is EvolveSearch, which also pursues self-evolution for search agents but does so by alternating rejection-sampling fine-tuning and reinforcement learning over open web search trajectories, with no human-written reasoning traces (Zhang et al., 28 May 2025). DeepSearch-Evolve shares the iterative reuse of the agent’s own successful rollouts, yet differs in making verifiability of intermediate search progress the primary mechanism for stabilizing self-improvement (Geng et al., 8 Jul 2026). This suggests that “DeepSearch-Evolve” names not only an iterative training loop but a specific design philosophy: self-distillation becomes reliable when the environment can ground and verify the agent’s internal progress.

2. DeepSearch-World as the enabling substrate

DeepSearch-Evolve is built on DeepSearch-World, an offline Wikipedia-based environment with reproducible search and reading tools (Geng et al., 8 Jul 2026). The environment constructs a local corpus from Wikipedia with roughly 10 million entries and exposes two deterministic tools: web_search_wiki(x), which returns ranked results with page title, caption or snippet, and a deterministic URL; and visit_wiki(url), which returns the full article content for that URL using a local offset index (Geng et al., 8 Jul 2026). These choices make the environment deterministic, offline, verifiable, and web-like (Geng et al., 8 Jul 2026).

The crucial mechanism is entity-level progress verification. For each question, the environment maintains a hidden target entity set

$\mathcal{T}_i=\{e_{i,1},\ldots,e_{i,H}\}.$

During rollout, it tracks a completed set $\mathcal{S}_t\subseteq\mathcal{T}_i$; when a tool response matches any unresolved entity in $\mathcal{T}_i\setminus\mathcal{S}_t$, that entity is added to the completed set (Geng et al., 8 Jul 2026). Progress verification is therefore order-free and objective after every tool call, and does not rely on LLM-based judging (Geng et al., 8 Jul 2026).

The environment also injects grounded reflection when a search fails. The first failure yields a generic revision signal; repeated failures produce stronger hints, including canonical entity names or fuzzy descriptions (Geng et al., 8 Jul 2026). This staged guidance explicitly supports a “search → fail → reflect → retry” interaction pattern. The resulting behaviors include query reformulation, failure recovery, and tracking of unresolved entities rather than mere repetition of search calls (Geng et al., 8 Jul 2026).

The importance of this environment design becomes clearer when compared with settings where intermediate verification is weaker or absent. EvolveSearch, for example, uses a hybrid reward over answer correctness and strict output format in a ReAct-like web-search setting, but relies on a judge model for answer evaluation and does not define environment-side entity-level progress verification (Zhang et al., 28 May 2025). DeepSearch-Evolve instead uses the environment itself to determine whether a tool call advanced the search process (Geng et al., 8 Jul 2026). A plausible implication is that the framework shifts supervision from heuristic or model-based post hoc judgment toward grounded process instrumentation.

3. Dataset construction and verifiability regime

DeepSearch-World contains 420K multi-hop QA tasks constructed from the Wikipedia hyperlink graph

$\mathcal{G}=(\mathcal{V},\mathcal{E})$

through an entity-centric random-walk and tree-based procedure (Geng et al., 8 Jul 2026). The construction begins from seed Wikipedia articles and performs BFS-like hyperlink exploration. At each step, an LLM, Gemini-3-Pro, selects the top $b$ relevant hyperlinks, with default $b=3$; exploration continues up to depth $d_{\max}=4$ or until $p_{\max}=6$ pages have been visited (Geng et al., 8 Jul 2026). Randomly sampled subtrees, specifically 2–4 subtrees, are then used to define sets of entities and relations from which a complex multi-hop question is generated (Geng et al., 8 Jul 2026).

To reduce memorization, the pipeline applies feature fuzzification: specific values are replaced with approximate descriptors, such as changing “2014” to “the 2010s” or “Beijing, China” to “a major Chinese city” (Geng et al., 8 Jul 2026). The authors also crawl the corresponding Wikipedia pages for all target entities to create a local corpus $\mathcal{C}$, ensuring that the evidence required by each question is retrievable in the offline environment (Geng et al., 8 Jul 2026).

In addition to the training pool, the work defines DeepSearch-Val, a validation set of 377 high-quality instances manually verified by a five-expert pool, with at least three experts per instance, and excluded from both training and trajectory generation (Geng et al., 8 Jul 2026). This separation matters because the framework’s central claim is not merely that it can generate self-training data, but that it can do so at scale while preserving an evaluation regime insulated from the self-generated training loop.

The emphasis on verifiable construction places DeepSearch-Evolve within a broader movement toward environments that measure actual search behavior rather than static answer production. In visual settings, VistaHop similarly argues that existing benchmarks overemphasize single-step understanding and under-measure iterative inspection, visual-anchor grounding, and long evidence chains (He et al., 2 Jun 2026). Although VistaHop is a benchmark rather than a training framework, both works converge on the view that deep search requires explicitly structured evidence paths and evaluation signals tied to those paths rather than only to final answers (He et al., 2 Jun 2026).

4. Iterative self-distillation pipeline

DeepSearch-Evolve follows an iterative self-evolving supervised fine-tuning loop (Geng et al., 8 Jul 2026). At round $R$, the current model $\mathcal{S}_t\subseteq\mathcal{T}_i$0 acts as teacher, generates trajectories in DeepSearch-World, and answer-correct trajectories are filtered and converted into ReAct-style supervision to train the next student $\mathcal{S}_t\subseteq\mathcal{T}_i$1 (Geng et al., 8 Jul 2026). The student then becomes the teacher for the next round (Geng et al., 8 Jul 2026).

The generation stage uses a three-phase scaffold:

Phase	Role	State or output
Plan	Initialize progress tracking	completed_list, todo_list, experience, information
Act	Execute tool-use steps	$\mathcal{S}_t\subseteq\mathcal{T}_i$2
End	Produce grounded final answer	concise answer from verified working memory

Within this scaffold, the teacher trajectory records tool calls, observations, progress-state updates, reflections, and the final answer (Geng et al., 8 Jul 2026). The state update is defined as

$\mathcal{S}_t\subseteq\mathcal{T}_i$3

where $\mathcal{S}_t\subseteq\mathcal{T}_i$4 is the grounded reflection signal (Geng et al., 8 Jul 2026).

Filtering occurs in two stages. First, only answer-correct trajectories are retained by exact verification. Second, Qwen3.5-9B applies a quality prompt to remove trajectories with redundant evidence, weak goal alignment, or inconsistent reasoning (Geng et al., 8 Jul 2026). The paper stresses that not all correct trajectories are suitable supervision: the retained set is intended to represent good agent behavior, not merely successful outcomes (Geng et al., 8 Jul 2026).

The scaffold is then converted into standard ReAct-form training examples. The thinking block at step $\mathcal{S}_t\subseteq\mathcal{T}_i$5 is rewritten as

$\mathcal{S}_t\subseteq\mathcal{T}_i$6

where $\mathcal{S}_t\subseteq\mathcal{T}_i$7 is a rewritten progress-state summary, $\mathcal{S}_t\subseteq\mathcal{T}_i$8 is rewritten reflection as self-correction, and $\mathcal{S}_t\subseteq\mathcal{T}_i$9 is the local rationale for the next action (Geng et al., 8 Jul 2026). Tool calls and observations remain unchanged (Geng et al., 8 Jul 2026). This conversion allows the deployed student to remain a standard ReAct policy while inheriting planning, memory tracking, and failure recovery from the scaffolded teacher traces (Geng et al., 8 Jul 2026).

Training is asynchronous. Generation continues while training runs in the background, and trajectories are mixed across rounds using importance sampling with decay factor

$\mathcal{T}_i\setminus\mathcal{S}_t$0

to prioritize recent verified data while retaining older data and reducing forgetting (Geng et al., 8 Jul 2026). The main configuration uses Qwen3.5-9B as backbone, 15 rounds in the appendix and 11 rounds in the main experiments, 10,000 questions per generation batch, 4,000 correct trajectories as the training trigger, a target mixed set of 4,000 per round, maximum trajectory length $\mathcal{T}_i\setminus\mathcal{S}_t$1, one epoch per round, learning rate $\mathcal{T}_i\setminus\mathcal{S}_t$2 with cosine schedule and 10% warmup, sequence length 32,768, and LlamaFactory with DeepSpeed ZeRO-2 (Geng et al., 8 Jul 2026). A later stage applies GRPO on 1,600 real-tool instances with Google SerpAPI and Jina to reduce the offline-to-real gap (Geng et al., 8 Jul 2026).

Compared with EvolveSearch, which alternates SFT and RL across $\mathcal{T}_i\setminus\mathcal{S}_t$3 data splits using filtered high-reward rollouts harvested from RL (Zhang et al., 28 May 2025), DeepSearch-Evolve is more purely a verified-trajectory self-distillation method. EvolveSearch uses reinforcement learning to generate exploration and rejection-sampling fine-tuning to consolidate that exploration (Zhang et al., 28 May 2025), whereas DeepSearch-Evolve emphasizes stable SFT over environment-verified trajectories (Geng et al., 8 Jul 2026). This suggests two distinct variants of self-evolution in search agents: one based on alternating RL and SFT, and another based on iterative self-distillation enabled by process verifiability.

5. Objectives, supervision signals, and behavioral induction

The core training objective in DeepSearch-Evolve is standard supervised fine-tuning over verified trajectories:

$\mathcal{T}_i\setminus\mathcal{S}_t$4

which the paper states is equivalent to token-level negative log-likelihood (Geng et al., 8 Jul 2026). This is contrasted with on-policy self-distillation:

$\mathcal{T}_i\setminus\mathcal{S}_t$5

where $\mathcal{T}_i\setminus\mathcal{S}_t$6 is the teacher distribution on student-induced prefixes (Geng et al., 8 Jul 2026). The paper argues that OPSD is riskier in long-horizon tool use because student prefixes can drift into low-quality states, making teacher soft labels noisy (Geng et al., 8 Jul 2026).

The framework’s behavioral effects arise from the environment and scaffold jointly. Because the environment tracks completed target entities, the agent learns to check whether each tool call advanced the goal, avoid premature final answers, and continue searching until evidence is sufficient (Geng et al., 8 Jul 2026). Grounded reflection promotes query reformulation and failure recovery, while the scaffold variables completed_list, todo_list, experience, and information encourage explicit subgoal decomposition, working-memory updates, and tracking of unresolved evidence (Geng et al., 8 Jul 2026). The paper also states that the model learns not to answer from snippets alone; it must visit pages and assemble corroborating evidence (Geng et al., 8 Jul 2026).

This process-level emphasis differs from sparse-reward RL for search agents. In EvolveSearch, the reward combines a format reward $\mathcal{T}_i\setminus\mathcal{S}_t$7 and answer reward $\mathcal{T}_i\setminus\mathcal{S}_t$8, with malformed outputs receiving zero and correctly formatted but incorrect answers receiving 0.5 (Zhang et al., 28 May 2025). DeepSearch-Evolve instead treats reliable intermediate verification as the key source of dense supervision (Geng et al., 8 Jul 2026). A plausible implication is that the framework substitutes explicit process measurement for reward shaping: rather than inferring search quality from end outcomes and formatting constraints, it encodes a notion of progress directly into the environment.

The paper’s ablations reinforce this interpretation. On SearchQA, rejection sampling by answer correctness and trajectory quality filtering together produce 58.2, exceeding 54.9 for rejection sampling only, 48.1 for quality filtering only, and 46.4 without both (Geng et al., 8 Jul 2026). On DeepSearch-Val, plain SFT gives 25.0, the full pipeline 31.9, removing state internalization 23.5, removing reflection rewriting 16.7, and removing both 14.8 (Geng et al., 8 Jul 2026). Reflection rewriting is therefore described as critical, while state internalization adds smaller but consistent gains (Geng et al., 8 Jul 2026).

6. Empirical performance and comparative position

DeepSearch-Evolve is evaluated on BrowseComp, BrowseComp-ZH, HLE, GAIA, xbench, HotpotQA, and also mentions Search-QA in the experimental setup (Geng et al., 8 Jul 2026). The principal comparison is between Qwen3.5-9B-Instruct and DeepSearch-World-9B. DeepSearch-World-9B achieves 31.2 on BrowseComp, 36.4 on BrowseComp-ZH, 25.7 on HLE, 61.5 on GAIA, 49.0 on xbench, and 93.4 on HotpotQA, compared with 7.4, 13.5, 16.7, 23.9, 20.0, and 45.3 respectively for the backbone (Geng et al., 8 Jul 2026). The corresponding gains are +23.8, +22.9, +9.0, +37.6, +29.0, and +48.1 (Geng et al., 8 Jul 2026).

The paper also compares with several open-source agents. WebSailor reports 6.7, 14.2, 12.8, 37.9, 34.3, and 92.8 on the same task sequence where available; Marco-DR reports 31.4, 47.1, --, 69.9, 42.0, --; MiroThinker-v1.0 reports 31.1, 40.2, 21.5, 66.4, 34.0, --; and PokeeResearch reports 6.2, --, 17.6, 49.2, --, 71.6 (Geng et al., 8 Jul 2026). Proprietary systems such as OpenAI Deep Research and OpenAI-o3 remain stronger on some tasks, with 51.5, 42.9, 26.6, 67.4 and 49.7, 58.1, 20.2, 70.5, 65.0 respectively where reported (Geng et al., 8 Jul 2026). DeepSearch-World-9B is thus positioned as much stronger than its backbone and competitive among open-source deep-search agents, especially on GAIA and HotpotQA (Geng et al., 8 Jul 2026).

Tool-use analysis further supports the claim that the framework changes behavior rather than merely answer calibration. Compared with Qwen3.5-9B-Instruct, DeepSearch-World-9B shows average interaction length 18.0 rounds versus 4.7, visit calls 5.4 versus 0.9, and advanced capability score 70% versus 19% (Geng et al., 8 Jul 2026). The paper interprets these differences as evidence of longer and more grounded search behavior (Geng et al., 8 Jul 2026).

The work’s comparative position is sharpened by related search-agent literature. EvolveSearch reports that its best variant, EvolveSearch-ite3, reaches 76.2 in-domain average and 53.7 out-of-domain average across seven multi-hop QA benchmarks, outperforming RLSearch-ite3 and DeepResearcher* in that setting (Zhang et al., 28 May 2025). Yet the benchmark regimes differ: EvolveSearch evaluates open web-search MHQA and emphasizes iterative RL–SFT bootstrapping (Zhang et al., 28 May 2025), whereas DeepSearch-Evolve emphasizes deterministic, verifiable long-horizon search in an offline environment (Geng et al., 8 Jul 2026). These results are therefore complementary rather than directly interchangeable.

7. Extensions, limitations, and broader significance

DeepSearch-Evolve’s main limitation is coverage. The environment is Wikipedia-only, which constrains domain diversity and real-world breadth, and the paper notes potential bias and coverage issues inherited from Wikipedia itself (Geng et al., 8 Jul 2026). It also states that the update rule still relies on evolving SFT, that RL-style updates or more advanced OPSD might improve generalization further, and that planning, error recovery, and tool-use strategies are currently injected through scaffolded SFT rather than learned directly by RL (Geng et al., 8 Jul 2026).

These limitations indicate where the framework may extend. One direction is toward multimodal or visual deep search. VistaHop defines Visual DeepSearch as an agentic, vision-centric setting that requires repeated image inspection, visual-anchor grounding, and long-chain evidence integration, and reports that the best evaluated model reaches only 24.31% Pass@1 under the strongest tool setting (He et al., 2 Jun 2026). Another direction is data evolution for multimodal agents. On-policy Data Evolution introduces a visual-native harness with persistent image-bank references and a closed-loop data generator that adapts to policy failures, improving Qwen3-VL-8B from 24.9% to 39.0% average across eight multimodal deep search benchmarks and Qwen3-VL-30B from 30.6% to 41.5% (Huang et al., 11 May 2026). These systems are not DeepSearch-Evolve itself, but they suggest that the underlying principle of policy-aware iterative improvement may generalize beyond text-only web search.

Within the search-agent literature, DeepSearch-Evolve’s most specific contribution is the claim that verifiable environments make self-evolution scalable because they provide reliable answer labels, intermediate process signals, selective filtering of good trajectories, and stable reuse of self-generated data (Geng et al., 8 Jul 2026). That claim distinguishes it from both fixed-demonstration SFT and sparse-reward RL, and also from broader notions of “deep search” that do not make intermediate progress measurable. A plausible implication is that future deep search systems may increasingly be organized around engineered environments whose main function is not only to supply tasks, but to expose a structured semantics of progress that self-improving agents can learn from directly.