MARS: Multi-Agent Deep Research Framework

• The paper introduces a dual-system architecture where System 1 extracts and compresses evidence while System 2 plans and synthesizes research answers.
• It employs bin-packing to manage context limitations and strategically utilizes tools such as Google Search, Scholar, and Python for dynamic research.
• Reinforcement learning jointly optimizes both systems, demonstrating enhanced performance on deep, knowledge-intensive QA tasks.

Searching arXiv for the MARS paper and closely related multi-agent research systems to ground the article. MARS, short for Multi-Agent System for Deep ReSearch, is a framework for dynamic, knowledge-intensive research that divides labor between two complementary cognitive roles inside a LLM: a fast, intuitive System 1 that reads, filters, and distills external information, and a slower, deliberate System 2 that plans, reasons, selects tools, and synthesizes answers (Chen et al., 6 Oct 2025). It is proposed to address two difficulties identified for large reasoning models: overuse of deliberate reasoning on easier tasks and dependence on static pretraining in environments that require current web or scholarly information. MARS combines this dual-system design with external tools such as Google Search, Google Scholar, and Python Interpreter, and trains the interaction end-to-end with a multi-agent reinforcement learning procedure derived from Group Relative Policy Optimization (GRPO) (Chen et al., 6 Oct 2025).

1. Definition and problem setting

MARS is defined as a framework intended to make LLMs better at dynamic, knowledge-intensive research by separating fast evidence distillation from slow deliberative reasoning (Chen et al., 6 Oct 2025). The motivating claim is that deep research tasks often require both current information retrieval and complex synthesis, yet prior approaches tend to fail in one of two ways: they either overload the reasoning model with excessive raw text or compress retrieved information so aggressively that important detail is lost (Chen et al., 6 Oct 2025).

The framework is explicitly motivated by two weaknesses of existing LLMs and large reasoning models. The first is overthinking on easy or medium tasks, where System 2-style deliberation consumes unnecessary tokens and time. The second is static knowledge in changing environments, where pretraining cutoff limits performance on tasks requiring recent web content, newly published research, or updated factual states (Chen et al., 6 Oct 2025). In that sense, MARS is not described merely as a retrieval-augmented model, but as a research architecture that reorganizes the interaction between retrieval, summarization, and reasoning.

The paper situates MARS against three classes of prior systems: direct LLM or LRM reasoning, RAG and iterative retrieval systems, and prior multi-agent systems whose agents remain cognitively homogeneous (Chen et al., 6 Oct 2025). Its central distinction is the explicit dual-process split: System 2 acts as the research planner and reasoning engine, while System 1 acts as the information extractor that summarizes large tool outputs into compact evidence. This suggests that the framework treats the bottleneck of deep research not only as a retrieval problem or a reasoning problem, but as a context-allocation problem under constrained inference budgets.

2. Dual-system architecture

The architecture is implemented as two prompts over the same base LLM, denoted $\pi_{\text{sys}_1}$ for System 1 and $\pi_{\text{sys}_2}$ for System 2 (Chen et al., 6 Oct 2025). System 2 maintains the evolving context $c_i$, containing the question and prior reasoning and tool summaries. At each turn it produces a reasoning step $s_i$, a tool request $t_i$, and a purpose $p_i$:

$s_i, (t_i, p_i) = \pi_{\text{sys}_2}(c_i)$

The role of the purpose variable $p_i$ is structurally important. It specifies what System 1 should extract from the tool result, so tool invocation is coupled to downstream summarization rather than treated as an unconstrained retrieval call (Chen et al., 6 Oct 2025). System 2 therefore performs not only reasoning and planning, but also supervisory control over the extraction objective.

System 1 is defined as the fast extraction module. It does not receive the full interaction history; instead, it sees the current tool outputs and the purpose $p_i$. Given tool outputs $\{o_{t_i}^{(1)}, \dots, o_{t_i}^{(n_{t_i})}\}$, it produces a distilled output $\pi_{\text{sys}_2}$0:

$\pi_{\text{sys}_2}$1

That distilled output is appended to the System 2 context:

$\pi_{\text{sys}_2}$2

The operational sequence is iterative: System 2 reasons from the current context, optionally invokes tools with a purpose, external tools return multiple outputs, System 1 distills those outputs in parallel, and System 2 resumes reasoning over the distilled evidence (Chen et al., 6 Oct 2025). The paper writes the overall answer distribution as a product over turns:

$\pi_{\text{sys}_2}$3

with the second term omitted when no tool is used (Chen et al., 6 Oct 2025). Architecturally, this formalization makes MARS a multi-turn tool-interaction system whose internal collaboration is asymmetric: System 2 controls search and reasoning depth, while System 1 controls evidence compression and context expansion.

3. Tool-mediated research workflow

MARS supports three external tools: Google Search, Google Scholar, and Python Interpreter (Chen et al., 6 Oct 2025). Google Search is used for broad and current web retrieval, Google Scholar for academic and research-specific sources, and Python Interpreter for structured computation and numeric reasoning. The paper states that Python outputs are usually passed directly back because they are short and structured, whereas Search and Scholar outputs are primarily processed by System 1 (Chen et al., 6 Oct 2025).

The framework’s account of deep research depends on a division between retrieval breadth and reasoning bandwidth. A single tool call may return multiple long web pages or papers, creating a compression problem before any deliberate synthesis can occur. MARS addresses this with First Fit Decreasing (FFD) bin-packing. The procedure counts tokens in each output, truncates any output that exceeds System 1’s maximum context length and isolates it, sorts the remaining outputs by length, and packs them into bins so each System 1 call can process a chunk efficiently (Chen et al., 6 Oct 2025). The paper states that it prefers FFD over Best Fit Decreasing for practical efficiency.

This bin-packing mechanism is not a peripheral engineering detail. In the MARS formulation, System 1 is useful precisely because it can process high-volume external information without overwhelming System 2. The use of bin-packing therefore operationalizes the framework’s core design principle: evidence should be compressed after retrieval and before deliberate reasoning, but in a way that respects context-window constraints and preserves sufficient structure for downstream synthesis.

Tool ablations on Humanity’s Last Exam further specify the functional division among the tools. Using all three tools is best overall; removing Google Search causes the largest overall performance drop, removing Python hurts math and physics most, and removing Google Scholar hurts CS/AI and “Other” categories most (Chen et al., 6 Oct 2025). This suggests that the framework treats tool choice as a domain-sensitive control problem rather than a uniform retrieval primitive.

4. Reinforcement learning formulation

MARS is trained with a multi-agent reinforcement learning framework that extends GRPO to jointly optimize both systems (Chen et al., 6 Oct 2025). For each question, the model generates multiple rollout trajectories. Each trajectory contains System 2 reasoning and tool-use decisions together with the corresponding System 1 extraction outputs for each tool call. The reward is assigned at the trajectory level, and all System 1 and System 2 samples within that trajectory share the same reward because the two systems are collaborative rather than competitive (Chen et al., 6 Oct 2025).

The reward is binary and is defined by an LLM evaluator following the official Humanity’s Last Exam protocol:

$\pi_{\text{sys}_2}$4

The final answer is extracted from the answer region in the response and checked against ground truth (Chen et al., 6 Oct 2025). For each question, the system samples $\pi_{\text{sys}_2}$5 trajectories. System 2 yields exactly $\pi_{\text{sys}_2}$6 samples, while System 1 yields a variable number because tool usage and bin-packing create a variable number of extraction instances. Advantages are computed by group-normalized rewards:

$\pi_{\text{sys}_2}$7

with $\pi_{\text{sys}_2}$8 (Chen et al., 6 Oct 2025). The total loss is

$\pi_{\text{sys}_2}$9

and each subsystem is optimized with a GRPO-style objective combining a clipped policy loss and KL regularization (Chen et al., 6 Oct 2025).

A practical difficulty is that System 2 contributes exactly $c_i$0 samples per question, while System 1 contributes a variable number. MARS therefore uses a sample balancing strategy: advantages are computed first for all samples, then System 1 samples are balanced to match the number of System 2 samples. If System 1 has more than $c_i$1 samples, it is down-sampled; if fewer, it is upsampled by duplication (Chen et al., 6 Oct 2025). The paper states that this preserves advantage estimation before balancing and prevents either subsystem from dominating optimization.

This training design implies that MARS does not learn only when to answer correctly. It also learns an interaction protocol: when to search, which tool to choose, what extraction purpose to specify, how to summarize retrieved evidence, and how to continue reasoning over distilled evidence. A plausible implication is that the framework treats deep research as a joint policy over reasoning and evidence reduction, rather than as a single long chain-of-thought augmented by tools.

5. Training data, implementation, and empirical performance

MARS is trained on a curated dataset assembled from public sources. The curation pipeline filters an initial pool of 5 million examples to a final 40K curated set through stages including academic-level filtering, deduplication, clarity filtering, graduate-level difficulty filtering, and best-of-16 verification with Google Search (Chen et al., 6 Oct 2025). The final reinforcement-learning mixture contains 5,050 sampled examples across the curated set, single-hop QA, multi-hop QA, and biomedical or clinical tasks (Chen et al., 6 Oct 2025).

The reported base models are Qwen2.5-7B-Instruct and Qwen3-8B. Training uses GRPO with learning rate $c_i$2, batch size 32, group size 16, temperature 1.0, and maximum interaction turns 10 (Chen et al., 6 Oct 2025). The prompt and response lengths are asymmetric by design: System 1 uses prompt length 23,552 and response length 8,192, whereas System 2 uses prompt length 3,072 and response length 28,672 (Chen et al., 6 Oct 2025). The asymmetry reflects the intended division of labor: large reading context for extraction and large generation budget for reasoning and synthesis.

The main benchmark is the text-only subset of Humanity’s Last Exam (HLE) with 2,154 questions, alongside seven knowledge-intensive QA tasks: Natural Questions, TriviaQA, PopQA, HotpotQA, 2WikiMultiHopQA, MuSiQue, and Bamboogle (Chen et al., 6 Oct 2025). On HLE, the paper reports MARS (Qwen2.5-7B): 7.38 average, a 3.86 percentage point gain over the base model Qwen2.5-7B-Instruct (Chen et al., 6 Oct 2025). Across the seven QA tasks, MARS reports 62.37 average with Qwen2.5-7B and 65.00 average with Qwen3-8B, with an 8.9% improvement over the previous SOTA C-3PO, and about 12.2% average improvement over C-3PO on the four multi-hop benchmarks (Chen et al., 6 Oct 2025).

The paper’s qualitative training analysis reports that HLE score improves steadily during RL, training reward stabilizes after an early rise, the number of tools per question increases over training, Google Search becomes the dominant tool choice, and both System 1 and System 2 response lengths increase (Chen et al., 6 Oct 2025). These observations are presented as evidence that the model learns more active tool use and richer extraction and reasoning behavior. Training reportedly ended after step 150 because length constraints were consistently exceeded (Chen et al., 6 Oct 2025).

6. Interpretation, related systems, and limitations

Within the deep-research literature represented here, MARS occupies a specific position: it is a dual-system research architecture whose principal concern is reasoning over large, changing, external information under context and token constraints (Chen et al., 6 Oct 2025). That focus distinguishes it from other contemporary multi-agent systems that use the same or similar acronym for different design problems.

The paper corpus shows that “MARS” is not a single research lineage but a reused acronym. “MARS: Reinforcing Multi-Agent Reasoning of LLMs through Self-Play in Strategic Games” studies self-play RL in cooperative and competitive games and reports transfer to reasoning benchmarks, including gains of 10.0% on AIME and 12.5% on GPQA-Diamond when integrated into multi-agent systems (Yuan et al., 17 Oct 2025). MarsRL addresses a Solver–Verifier–Corrector pipeline with agent-specific rewards and agentic pipeline parallelism for verifiable reasoning tasks, improving AIME2025 accuracy from 86.5% to 93.3% and BeyondAIME from 64.9% to 73.8% on Qwen3-30B-A3B-Thinking-2507 (Liu et al., 14 Nov 2025). A-MapReduce is explicitly positioned not as a classic deep-research system but as a framework for wide search, emphasizing parallel breadth-oriented retrieval, structured aggregation, and experiential memory rather than vertically structured deliberation (Chen et al., 1 Feb 2026). These neighboring systems indicate that multi-agent research architectures are diverging along at least three axes: strategic interaction, iterative verification, and retrieval topology.

The limitations described for MARS are mainly operational rather than theoretical. The framework depends on external tools, and its training and inference involve multi-turn tool use plus dual-system rollouts, making it more expensive than standard decoding (Chen et al., 6 Oct 2025). The balancing of two roles and multiple tools introduces additional coordination complexity. Evaluation on HLE and the QA benchmarks relies on LLM judges, which the paper notes as part of the evaluation pipeline rather than as a hard symbolic oracle (Chen et al., 6 Oct 2025). The reported dominance of Google Search in tool usage also suggests sensitivity of the learned tool policy to data distribution.

Taken together, the paper presents MARS as a response to a specific systems problem in LLM-based research: how to search widely enough to remain current, compress aggressively enough to fit within context limits, and still preserve sufficient evidence for slow, deliberate synthesis. Its substantive contribution is the claim that these goals can be pursued jointly by making evidence distillation and reasoning into separate but co-trained policies (Chen et al., 6 Oct 2025).