LiteResearcher: Scalable Agentic RL Framework

• LiteResearcher is a scalable agentic reinforcement-learning framework that trains deep research agents using a deterministic, low-cost lite virtual world.
• It integrates a local search engine, browsing tools, and synthetic multi-hop QA generation with difficulty-aware on-policy GRPO training to drive performance.
• Empirical evaluations show that LiteResearcher-4B achieves 71.3% on GAIA-Text and 78.0% on XBench-DeepSearch, outperforming several state-of-the-art systems.

LiteResearcher is a scalable agentic reinforcement-learning training framework for a deep research agent that replaces live-web interaction during training with a local “lite virtual world” intended to mirror real-world search dynamics while remaining deterministic, low-latency, and low-cost (Li et al., 20 Apr 2026). The framework couples a large local corpus, local search and browse tools, synthetic and multi-hop question generation, and difficulty-aware on-policy GRPO training to produce a 4B-parameter research agent. In reported evaluations, LiteResearcher-4B achieves 71.3\% on GAIA-Text and 78.0\% on XBench-DeepSearch-2505, described as open-source state-of-the-art results, and is reported to outperform large-scale open-source and commercial systems including Tongyi DeepResearch and Claude-4.5 Sonnet on those benchmarks (Li et al., 20 Apr 2026).

1. Scope, problem setting, and design rationale

LiteResearcher addresses deep research as long-horizon, tool-using behavior in which an agent must iteratively search and browse, integrate evidence across many pages, perform cross-verification, enumeration, and aggregation, and sometimes run calculations or statistics (Li et al., 20 Apr 2026). This setting differs from paper-reading assistants and related-work drafting systems that operate on a single paper or a small retrieved set. Systems such as InsightGUIDE encode an expert reading methodology to produce a structured “map” of a paper for critical reading (Koloveas et al., 24 Sep 2025), while LitLLM uses retrieval, re-ranking, and generation to draft related-work sections from abstracts (Agarwal et al., 2024). LiteResearcher instead targets sustained, multi-step research behavior over a web-like environment (Li et al., 20 Apr 2026).

The framework is motivated by two coupled bottlenecks in agentic RL for deep research. First, hand-crafted synthetic data built on narrow corpora does not capture the “atomic search moves” of real web research, including direct fact lookup, aggregation over constraints, enumeration, cross-verification, and statistics (Li et al., 20 Apr 2026). Second, RL directly on the live web introduces variable latency, API failures, changing search results, and high monetary cost, making sustained on-policy training difficult (Li et al., 20 Apr 2026). LiteResearcher’s core claim is that agentic RL scales only if training is decoupled from the live internet while preserving web-like structure (Li et al., 20 Apr 2026).

This design situates LiteResearcher closer to RL-based deep-search systems than to static retrieval or summarization tools. A plausible implication is that the framework is best understood not primarily as an end-user interface, but as a training ecosystem for research agents.

2. Lite virtual world and local tool environment

The defining component of LiteResearcher is its lite virtual world, a local environment that preserves the structural properties of web search while eliminating live-web instability (Li et al., 20 Apr 2026). The environment is built around an enriched corpus of approximately 32 million pages spanning more than 1 million unique domains and 18 domain categories (Li et al., 20 Apr 2026). The corpus begins from a high-quality seed corpus including Wikipedia, BBC News, and other curated sources, totaling roughly 10 million pages, and is expanded through iterative web crawling driven by synthetic QA generation (Li et al., 20 Apr 2026).

The environment exposes two tools. The local search engine is implemented with Milvus and BGE-M3, using DiskANN hybrid retrieval, while the local browse tool is backed by PostgreSQL and serves page-level markdown content (Li et al., 20 Apr 2026). Search latency is reported as about 0.15 seconds and browse latency as about 0.17 seconds, compared with much higher latencies for online APIs (Li et al., 20 Apr 2026). The resulting environment is deterministic, low-variance, and incurs zero marginal cost after corpus construction (Li et al., 20 Apr 2026).

Synthetic task generation and corpus expansion are tightly linked. LiteResearcher first generates factual QA pairs from real webpages using an LLM prompt that asks for specific, concrete data points such as numbers, dates, names, locations, and percentages (Li et al., 20 Apr 2026). After a QA pair is created from a page, the source page is removed from the local corpus, forcing the agent to recover the answer through related pages rather than the original source (Li et al., 20 Apr 2026). Each QA pair is filtered through a seven-criterion rubric covering question independence, answer specificity and verifiability, unambiguity, clear answerability, non-open-endedness, non-triviality, and time specificity (Li et al., 20 Apr 2026). For each accepted QA, a commercial search API is used offline to fetch about 100 relevant real-world pages, which are crawled, deduplicated, filtered, and added back into the corpus (Li et al., 20 Apr 2026).

This environment differs from single-paper reading assistants, topic maps, or offline review suites. Lacuna, for example, precomputes a research map of summaries, concept elements, research directions, and proposals over machine learning papers (Weiss et al., 24 Jun 2026), whereas LiteResearcher constructs a tool-using training world for long-horizon action and evidence integration (Li et al., 20 Apr 2026).

3. Data engine, task construction, and curriculum

LiteResearcher’s data engine is organized around Corpus Extension & QA Synthesis and a Reinforcement Curriculum Learning engine (Li et al., 20 Apr 2026). The framework explicitly decomposes deep research into five atomic search capabilities: direct information, aggregation across multiple constraints, enumeration, cross-verification, and statistics or quantitative analysis (Li et al., 20 Apr 2026). The synthetic QA design is intentionally simple but scaled to a large corpus so that these capabilities are elicited naturally rather than through hand-authored logic templates (Li et al., 20 Apr 2026).

Training tasks come from several sources. The largest share is synthetic direct QA generated from the web-anchored corpus. A second source is multi-hop QA, generated by first building a web-grounded knowledge graph from seed entities and then sampling connected subgraphs for backward question generation (Li et al., 20 Apr 2026). Additional data include science-domain queries and QA distilled from open-source benchmarks and Tongyi DeepResearch trajectories, some used first in supervised fine-tuning and later incorporated into RL mixtures (Li et al., 20 Apr 2026).

LiteResearcher applies a difficulty-aware curriculum based on pass@8 of a reference model (Li et al., 20 Apr 2026). Only tasks of intermediate difficulty are retained for RL at a given stage; the implementation description states that tasks with pass@8 satisfying $1 \le c \le 7$ are used (Li et al., 20 Apr 2026). Stage 1 uses a mixture of approximately 7.6K synthetic direct QA queries and 2.8K multi-hop queries, while Stage 2 expands to about 11.1K synthetic direct QA queries, 3.3K multi-hop queries, and 1.8K science-domain queries (Li et al., 20 Apr 2026). The framework reports that Stage 1 alone saturates around GAIA $\sim 64.7\%$, whereas Stage 2, with harder tasks and longer context, pushes performance further (Li et al., 20 Apr 2026).

A plausible implication is that LiteResearcher treats curriculum design as part of the environment problem rather than as a secondary optimization detail. The data distribution is not fixed; instead, it is co-constructed with the corpus and adjusted stage by stage.

4. Agent architecture, supervised initialization, and GRPO training

The trained agent is LiteResearcher-4B, initialized from Qwen3-4B-Thinking-2507 (Li et al., 20 Apr 2026). The agent follows a ReAct-style loop that alternates between thoughts, actions, and observations until it returns a final answer in <answer>...</answer> format (Li et al., 20 Apr 2026). The action space consists of Search(q'), Browse(u, q'), and Finish(answer); tool outputs are inserted back into context for subsequent reasoning (Li et al., 20 Apr 2026).

Training proceeds in two phases. First, supervised fine-tuning uses 68K high-quality trajectories generated by Tongyi DeepResearch over synthetic QA and open-source QA sets, with trajectories cleaned to remove bad tool calls and repeated actions (Li et al., 20 Apr 2026). This SFT phase improves the base Qwen3-4B-Thinking model from 28.16\% to 55.58\% on GAIA and from 21.0\% to 64.25\% on XBench (Li et al., 20 Apr 2026). RL then starts from this checkpoint and uses only the RL objective, adding roughly 15–16 points on GAIA and XBench and surpassing the Tongyi teacher (Li et al., 20 Apr 2026).

The RL algorithm is Group Relative Policy Optimization. For each query $q$, $K=8$ rollouts are sampled from a rollout policy, and each rollout receives a binary reward based on an LLM judge that compares the predicted answer with the labeled answer (Li et al., 20 Apr 2026). The GRPO objective is given as

$$\mathcal{J}_{\text{GRPO}}(\theta) = \mathbb{E}_{q \sim P(Q), \{o_i\}_{i=1}^K \sim \pi_{\theta_{\text{old}}}} \left[ \frac{1}{K} \sum_{i=1}^K \min\!\Big( r_i(\theta) A_i,\; \text{clip}\big(r_i(\theta), 1-\epsilon_{\text{low}}, 1+\epsilon_{\text{high}}\big) A_i \Big) \right],$$

with

$$r_i(\theta)=\frac{\pi_\theta(o_i\mid q)}{\pi_{\theta_{\text{rollout}}}(o_i\mid q)}.$$

LiteResearcher removes KL and entropy terms and uses a strictly on-policy regime in which each batch is used for exactly one update and then discarded (Li et al., 20 Apr 2026). The paper reports that this choice is crucial for stability in long-horizon agentic tasks: an off-policy variant yields higher reward early but later declines, with lower final GAIA accuracy than the on-policy version (Li et al., 20 Apr 2026).

The RL setup uses a global batch size of 128 queries, $K=8$ rollouts per query, a learning rate of $1 \times 10^{-6}$, and response lengths up to 32K tokens in Stage 1 and 48K in Stage 2 (Li et al., 20 Apr 2026). The framework also applies Trajectory Importance Sampling correction to compensate for differences between the rollout and training engines (Li et al., 20 Apr 2026).

5. Empirical performance, efficiency, and training dynamics

LiteResearcher reports strong benchmark performance across deep research tasks. On GAIA-Text it reaches 71.3\%, and on XBench-DeepSearch-2505 it reaches 78.0\% (Li et al., 20 Apr 2026). Additional reported results include 83.1\% on Frames, 72.7\% on WebWalker, 22.0\% on HLE, 41.8\% on Seal-0, and 27.5\% and 32.5\% on BrowseComp EN and ZH respectively when using memory on evaluation (Li et al., 20 Apr 2026). The paper states that LiteResearcher-4B beats or matches systems including Claude-4.5-Sonnet, GLM-4.6, GPT-5-high on XBench, Tongyi DeepResearch 30B, WebSailor 30B, and AgentCPM-Explore-4B (Li et al., 20 Apr 2026).

The cost argument is central. During RL, LiteResearcher executes 45.8 million search calls and 27.4 million browse calls, for a total of 73.2 million tool calls (Li et al., 20 Apr 2026). If these calls were executed online, the estimated cost would range from roughly \$59K to \$243K depending on API providers (Li et al., 20 Apr 2026). By contrast, corpus construction requires about 220K Serper calls costing about \$220 once, after which training proceeds with zero marginal tool cost (Li et al., 20 Apr 2026). The environment is also reported as approximately 10–46 times faster per tool call than online alternatives (Li et al., 20 Apr 2026).

Training dynamics indicate that RL improves not only accuracy but also search behavior. Over training, mean reward rises from 0.42 to 0.70, mean response length drops from 18K to 12K tokens, mean turns decrease from 30 to 24, and context-overflow clip ratio falls from 0.28 to 0.02 (Li et al., 20 Apr 2026). The paper interprets this as RL removing repetitive, unproductive tool loops learned during SFT even without an explicit length penalty (Li et al., 20 Apr 2026). Stage 1 reduces tool calls and tokens by eliminating redundancy, while Stage 2 reintroduces some complexity as the agent tackles harder tasks (Li et al., 20 Apr 2026).

In the surrounding literature, LightSearcher addresses a related but distinct problem: the accuracy-efficiency trade-off in RL-based DeepSearch, reporting 39.6\% fewer tool calls and 48.6\% lower inference time than ReSearch while maintaining comparable accuracy on multi-hop QA (Lan et al., 7 Dec 2025). LiteResearcher’s emphasis is different: it makes large-scale training feasible by moving the environment offline, then uses strict on-policy RL and curriculum learning to improve long-horizon research behavior (Li et al., 20 Apr 2026).

6. Position in the literature, limitations, and subsequent extensions

LiteResearcher occupies a specific position among research-assistant systems. Reading-oriented assistants such as InsightGUIDE produce a structured map for a single paper and keep the source PDF central (Koloveas et al., 24 Sep 2025). Literature-review drafting systems such as LitLLM retrieve, rerank, and synthesize related-work sections from abstracts (Agarwal et al., 2024). Review-mapping systems such as Lacuna precompute summaries, concept elements, research directions, and proposals, then use those structures for deep report generation (Weiss et al., 24 Jun 2026). LiteResearcher differs in targeting agentic RL for deep research, with a local search-and-browse world specifically designed for large-scale policy optimization (Li et al., 20 Apr 2026).

The framework’s principal limitations are also explicit. The virtual world, though large, is still a frozen subset of the web and lacks changing trending content, personalization, and parts of the long tail (Li et al., 20 Apr 2026). Search fidelity is limited by page-level indexing, which may miss highly localized snippets (Li et al., 20 Apr 2026). Reward is binary and endpoint-based, with no dense or process-level supervision (Li et al., 20 Apr 2026). The 4B model remains small, and deeply nested browsing tasks such as BrowseComp remain challenging (Li et al., 20 Apr 2026). These constraints suggest that LiteResearcher solves the scalability problem of agentic RL more directly than the problems of dynamic knowledge freshness or fully robust evaluation.

The framework has already become the substrate for later proposals. MetaResearcher explicitly describes LiteResearcher as the infrastructure foundation for a “second generation” system that extends it with an evolving virtual world, discovery-oriented tasks such as hypothesis generation and contradiction resolution, a self-reflective meta-reward within GRPO, and a heterogeneous multi-agent swarm architecture (Yu et al., 18 Jun 2026). This suggests that LiteResearcher’s main historical significance lies not only in its benchmark scores, but in establishing a concrete recipe for training deep research agents: co-constructed data, a stable local tool environment, and difficulty-aware on-policy RL (Li et al., 20 Apr 2026).