---
title: 'MARS: Multi-Agent Deep Research Framework'
url: https://www.emergentmind.com/topics/multi-agent-system-for-deep-research-mars
type: topic
---

# MARS: Multi-Agent Deep Research Framework

Searching arXiv for the MARS paper and closely related multi-agent research systems to ground the article.
MARS, short for **Multi-Agent System for Deep ReSearch**, is a framework for dynamic, knowledge-intensive research that divides labor between two complementary cognitive roles inside a large language model: a fast, intuitive **System 1** that reads, filters, and distills external information, and a slower, deliberate **System 2** that plans, reasons, selects tools, and synthesizes answers [2510.04935]. It is proposed to address two difficulties identified for large reasoning models: overuse of deliberate reasoning on easier tasks and dependence on static pretraining in environments that require current web or scholarly information. MARS combines this dual-system design with external tools such as Google Search, Google Scholar, and Python Interpreter, and trains the interaction end-to-end with a multi-agent reinforcement learning procedure derived from Group Relative Policy Optimization (GRPO) [2510.04935].

## 1. Definition and problem setting

MARS is defined as a framework intended to make large language models better at **dynamic, knowledge-intensive research** by separating fast evidence distillation from slow deliberative reasoning [2510.04935]. The motivating claim is that deep research tasks often require both current information retrieval and complex synthesis, yet prior approaches tend to fail in one of two ways: they either overload the reasoning model with excessive raw text or compress retrieved information so aggressively that important detail is lost [2510.04935].

The framework is explicitly motivated by two weaknesses of existing LLMs and large reasoning models. The first is **overthinking on easy or medium tasks**, where System 2-style deliberation consumes unnecessary tokens and time. The second is **static knowledge in changing environments**, where pretraining cutoff limits performance on tasks requiring recent web content, newly published research, or updated factual states [2510.04935]. In that sense, MARS is not described merely as a retrieval-augmented model, but as a research architecture that reorganizes the interaction between retrieval, summarization, and reasoning.

The paper situates MARS against three classes of prior systems: direct LLM or LRM reasoning, RAG and iterative retrieval systems, and prior multi-agent systems whose agents remain cognitively homogeneous [2510.04935]. Its central distinction is the explicit dual-process split: System 2 acts as the research planner and reasoning engine, while System 1 acts as the information extractor that summarizes large tool outputs into compact evidence. This suggests that the framework treats the bottleneck of deep research not only as a retrieval problem or a reasoning problem, but as a **context-allocation problem** under constrained inference budgets.

## 2. Dual-system architecture

The architecture is implemented as two prompts over the same base LLM, denoted $\pi_{\text{sys}_1}$ for System 1 and $\pi_{\text{sys}_2}$ for System 2 [2510.04935]. System 2 maintains the evolving context $c_i$, containing the question and prior reasoning and tool summaries. At each turn it produces a reasoning step $s_i$, a tool request $t_i$, and a purpose $p_i$:

$$
s_i, (t_i, p_i) = \pi_{\text{sys}_2}(c_i)
$$

The role of the purpose variable $p_i$ is structurally important. It specifies what System 1 should extract from the tool result, so tool invocation is coupled to downstream summarization rather than treated as an unconstrained retrieval call [2510.04935]. System 2 therefore performs not only reasoning and planning, but also supervisory control over the extraction objective.

System 1 is defined as the fast extraction module. It does not receive the full interaction history; instead, it sees the current tool outputs and the purpose $p_i$. Given tool outputs $\{o_{t_i}^{(1)}, \dots, o_{t_i}^{(n_{t_i})}\}$, it produces a distilled output $\tilde{o}_{t_i}$:

$$
\tilde{o}_{t_i} = \pi_{\text{sys}_1}\left(\text{Bin-Packing}\left(o_{t_i}^{(1)}, o_{t_i}^{(2)}, ..., o_{t_i}^{(n_{t_i})}\right), p_i\right)
$$

That distilled output is appended to the System 2 context:

$$
c_{i+1} = c_i \oplus \{s_i, t_i, p_i, \tilde{o}_{t_i}\}
$$

The operational sequence is iterative: System 2 reasons from the current context, optionally invokes tools with a purpose, external tools return multiple outputs, System 1 distills those outputs in parallel, and System 2 resumes reasoning over the distilled evidence [2510.04935]. The paper writes the overall answer distribution as a product over turns:

$$
\mathcal{P}(\text{answer}|q) = \prod_{i=1}^{N} \left[ \pi_{\text{sys}_2}(s_i, t_i, p_i|c_i)\cdot \pi_{\text{sys}_1}(\tilde{o}_{t_i}|\text{Bin-Packing}(o_{t_i}^{(1)}, o_{t_i}^{(2)}, ..., o_{t_i}^{(n_{t_i})}), p_i) \right]
$$

with the second term omitted when no tool is used [2510.04935]. Architecturally, this formalization makes MARS a multi-turn tool-interaction system whose internal collaboration is asymmetric: System 2 controls search and reasoning depth, while System 1 controls evidence compression and context expansion.

## 3. Tool-mediated research workflow

MARS supports three external tools: **Google Search**, **Google Scholar**, and **Python Interpreter** [2510.04935]. Google Search is used for broad and current web retrieval, Google Scholar for academic and research-specific sources, and Python Interpreter for structured computation and numeric reasoning. The paper states that Python outputs are usually passed directly back because they are short and structured, whereas Search and Scholar outputs are primarily processed by System 1 [2510.04935].

The framework’s account of deep research depends on a division between retrieval breadth and reasoning bandwidth. A single tool call may return multiple long web pages or papers, creating a compression problem before any deliberate synthesis can occur. MARS addresses this with **First Fit Decreasing (FFD)** bin-packing. The procedure counts tokens in each output, truncates any output that exceeds System 1’s maximum context length and isolates it, sorts the remaining outputs by length, and packs them into bins so each System 1 call can process a chunk efficiently [2510.04935]. The paper states that it prefers FFD over Best Fit Decreasing for practical efficiency.

This bin-packing mechanism is not a peripheral engineering detail. In the MARS formulation, System 1 is useful precisely because it can process high-volume external information without overwhelming System 2. The use of bin-packing therefore operationalizes the framework’s core design principle: evidence should be compressed **after retrieval and before deliberate reasoning**, but in a way that respects context-window constraints and preserves sufficient structure for downstream synthesis.

Tool ablations on Humanity’s Last Exam further specify the functional division among the tools. Using all three tools is best overall; removing **Google Search** causes the largest overall performance drop, removing **Python** hurts math and physics most, and removing **Google Scholar** hurts CS/AI and “Other” categories most [2510.04935]. This suggests that the framework treats tool choice as a domain-sensitive control problem rather than a uniform retrieval primitive.

## 4. Reinforcement learning formulation

MARS is trained with a multi-agent reinforcement learning framework that extends **GRPO** to jointly optimize both systems [2510.04935]. For each question, the model generates multiple rollout trajectories. Each trajectory contains System 2 reasoning and tool-use decisions together with the corresponding System 1 extraction outputs for each tool call. The reward is assigned at the **trajectory level**, and all System 1 and System 2 samples within that trajectory share the same reward because the two systems are collaborative rather than competitive [2510.04935].

The reward is binary and is defined by an LLM evaluator following the official Humanity’s Last Exam protocol:

$$
r(c_N, \text{ground truth}) = \begin{cases} 1, & \text{if Eval}_{\text{LLM}} = \text{Correct} \\ 0, & \text{otherwise} \end{cases}
$$

The final answer is extracted from the answer region in the response and checked against ground truth [2510.04935]. For each question, the system samples $G$ trajectories. System 2 yields exactly $G$ samples, while System 1 yields a variable number because tool usage and bin-packing create a variable number of extraction instances. Advantages are computed by group-normalized rewards:

$$
A_{\text{sys}_2}^k = \frac{r_{\text{sys}_2}^k - \text{mean}(\mathbf{r}_{\text{sys}_2})}{\text{std}(\mathbf{r}_{\text{sys}_2})}, \quad
A_{\text{sys}_1}^{k,j} = \frac{r_{\text{sys}_1}^{k,j} - \text{mean}(\mathbf{r}_{\text{sys}_1})}{\text{std}(\mathbf{r}_{\text{sys}_1})}
$$

with $r_{\text{sys}_1}^{k,j} = r_{\text{sys}_2}^k$ [2510.04935]. The total loss is

$$
\mathcal{L}_{\text{total}} = \mathcal{L}_{\text{sys}_2} + \mathcal{L}_{\text{sys}_1}
$$

and each subsystem is optimized with a GRPO-style objective combining a clipped policy loss and KL regularization [2510.04935].

A practical difficulty is that System 2 contributes exactly $G$ samples per question, while System 1 contributes a variable number. MARS therefore uses a **sample balancing strategy**: advantages are computed first for all samples, then System 1 samples are balanced to match the number of System 2 samples. If System 1 has more than $G$ samples, it is down-sampled; if fewer, it is upsampled by duplication [2510.04935]. The paper states that this preserves advantage estimation before balancing and prevents either subsystem from dominating optimization.

This training design implies that MARS does not learn only when to answer correctly. It also learns an interaction protocol: when to search, which tool to choose, what extraction purpose to specify, how to summarize retrieved evidence, and how to continue reasoning over distilled evidence. A plausible implication is that the framework treats deep research as a **joint policy over reasoning and evidence reduction**, rather than as a single long chain-of-thought augmented by tools.

## 5. Training data, implementation, and empirical performance

MARS is trained on a curated dataset assembled from public sources. The curation pipeline filters an initial pool of **5 million examples** to a final **40K curated set** through stages including academic-level filtering, deduplication, clarity filtering, graduate-level difficulty filtering, and best-of-16 verification with Google Search [2510.04935]. The final reinforcement-learning mixture contains **5,050 sampled examples** across the curated set, single-hop QA, multi-hop QA, and biomedical or clinical tasks [2510.04935].

The reported base models are **Qwen2.5-7B-Instruct** and **Qwen3-8B**. Training uses **GRPO** with learning rate $1 \times 10^{-6}$, batch size 32, group size 16, temperature 1.0, and maximum interaction turns 10 [2510.04935]. The prompt and response lengths are asymmetric by design: System 1 uses prompt length **23,552** and response length **8,192**, whereas System 2 uses prompt length **3,072** and response length **28,672** [2510.04935]. The asymmetry reflects the intended division of labor: large reading context for extraction and large generation budget for reasoning and synthesis.

The main benchmark is the text-only subset of **Humanity’s Last Exam (HLE)** with **2,154 questions**, alongside seven knowledge-intensive QA tasks: Natural Questions, TriviaQA, PopQA, HotpotQA, 2WikiMultiHopQA, MuSiQue, and Bamboogle [2510.04935]. On HLE, the paper reports **MARS (Qwen2.5-7B): 7.38 average**, a **3.86 percentage point gain** over the base model Qwen2.5-7B-Instruct [2510.04935]. Across the seven QA tasks, MARS reports **62.37 average** with Qwen2.5-7B and **65.00 average** with Qwen3-8B, with an **8.9% improvement over the previous SOTA C-3PO**, and about **12.2% average improvement** over C-3PO on the four multi-hop benchmarks [2510.04935].

The paper’s qualitative training analysis reports that HLE score improves steadily during RL, training reward stabilizes after an early rise, the number of tools per question increases over training, Google Search becomes the dominant tool choice, and both System 1 and System 2 response lengths increase [2510.04935]. These observations are presented as evidence that the model learns more active tool use and richer extraction and reasoning behavior. Training reportedly ended after step 150 because length constraints were consistently exceeded [2510.04935].

## 6. Interpretation, related systems, and limitations

Within the deep-research literature represented here, MARS occupies a specific position: it is a **dual-system research architecture** whose principal concern is reasoning over large, changing, external information under context and token constraints [2510.04935]. That focus distinguishes it from other contemporary multi-agent systems that use the same or similar acronym for different design problems.

The paper corpus shows that “MARS” is not a single research lineage but a reused acronym. **“MARS: Reinforcing Multi-Agent Reasoning of LLMs through Self-Play in Strategic Games”** studies self-play RL in cooperative and competitive games and reports transfer to reasoning benchmarks, including gains of **10.0% on AIME** and **12.5% on GPQA-Diamond** when integrated into multi-agent systems [2510.15414]. **MarsRL** addresses a Solver–Verifier–Corrector pipeline with agent-specific rewards and agentic pipeline parallelism for verifiable reasoning tasks, improving **AIME2025 accuracy from 86.5% to 93.3%** and **BeyondAIME from 64.9% to 73.8%** on Qwen3-30B-A3B-Thinking-2507 [2511.11373]. **A-MapReduce** is explicitly positioned not as a classic deep-research system but as a framework for **wide search**, emphasizing parallel breadth-oriented retrieval, structured aggregation, and experiential memory rather than vertically structured deliberation [2602.01331]. These neighboring systems indicate that multi-agent research architectures are diverging along at least three axes: strategic interaction, iterative verification, and retrieval topology.

The limitations described for MARS are mainly operational rather than theoretical. The framework depends on external tools, and its training and inference involve multi-turn tool use plus dual-system rollouts, making it more expensive than standard decoding [2510.04935]. The balancing of two roles and multiple tools introduces additional coordination complexity. Evaluation on HLE and the QA benchmarks relies on LLM judges, which the paper notes as part of the evaluation pipeline rather than as a hard symbolic oracle [2510.04935]. The reported dominance of Google Search in tool usage also suggests sensitivity of the learned tool policy to data distribution.

Taken together, the paper presents MARS as a response to a specific systems problem in LLM-based research: how to search widely enough to remain current, compress aggressively enough to fit within context limits, and still preserve sufficient evidence for slow, deliberate synthesis. Its substantive contribution is the claim that these goals can be pursued jointly by making evidence distillation and reasoning into separate but co-trained policies [2510.04935].

Source: https://www.emergentmind.com/topics/multi-agent-system-for-deep-research-mars