---
title: 'WideSeek-R1: Multi-Agent MARL Framework'
url: https://www.emergentmind.com/topics/wideseek-r1
type: topic
---

# WideSeek-R1: Multi-Agent MARL Framework

WideSeek-R1 is a multi-agent reinforcement learning (MARL) framework targeting broad information-seeking tasks where traditional single-agent “depth scaling” LLM paradigms reach organizational and context limitations. The system introduces a hierarchical architecture that leverages width scaling: a lead agent that decomposes complex, table-generative queries into parallel subtasks handled by subagents initialized from a shared LLM, enabling effective parallelism, context isolation, and scalable orchestration. WideSeek-R1, when trained on a curated dataset of diverse, broad information-seeking problems, approaches the performance of single-agent, ultra-large models with substantially fewer compute and parameter resources, thus redefining the frontier for information-centric multi-agent systems [2602.04634].

## 1. Motivation: Organizational Bottlenecks in Broad Information Seeking

Conventional LLM-based agents perform well on depth-oriented reasoning—the “deep research” paradigm—by extending chain-of-thought lengths or increasing context window sizes. However, for broad tasks that require extracting structured information about large numbers of entities simultaneously (e.g., generating a table summarizing attributes for 30–50 countries), two principal pathologies emerge:

- **Context pollution**: A monolithic agent tasked with assimilating hundreds of facts across unrelated entities rapidly exceeds its context capacity, degrading focus and accuracy.
- **Sequential inefficiency**: Single-agent protocols process each row or column in series, squandering wall-clock time and context utilization, and failing to capitalize on potential parallelism.

WideSeek-R1 addresses these issues with width scaling. The lead agent decomposes a broad input query $q$ (with $R$ entities and $C$ attributes) into $N$ independent subqueries, each handled by a subagent. Subagents execute information-seeking actions in isolated contexts using external tools, after which the lead agent aggregates their findings into a coherent, structured output.

This paradigm enables the system to efficiently produce complex tables of the form
$$
\{ x_{i,1}, \dots, x_{i,C} \}_{i=1}^R,
$$
where workload is distributed across subagents, preserving context clarity and maximizing wall-clock parallelism [2602.04634].

## 2. System Architecture: Lead-Subagent Hierarchy and Workflow

WideSeek-R1 instantiates both the lead agent and all subagents from a shared LLM backbone (e.g., Qwen3-4B), enforcing context isolation and granting access to specialized tool interfaces.

**Workflow**:
1. **Lead agent initialization**: Receives a system prompt and the user’s broad query $q$.
2. **Task decomposition** ($\texttt{create\_sub\_agents}$ tool): Lead agent identifies decomposition points and generates prompts for $k$ subagents.
3. **Parallel subagent rollout**: Each subagent receives a distinct subtask prompt:
   ```
   [SYSTEM: You are sub-agent; SUBTASK: prompt_a]
   ```
   Subagents iteratively employ external tools (`search`, `access`) and maintain private chain-of-thoughts until subtask completion.
4. **Aggregation and reflection**: The lead agent collects and integrates subagent outputs, periodically repeating decomposition and aggregation steps for up to $T$ turns.
5. **Finalization**: The system emits a unified Markdown table output.

Critical features include strict context isolation (subagents cannot access each other’s internal state) and strict parallelism (all subagents spawn and execute concurrently within system-imposed limits, e.g., $M = 10$ per turn). The architecture eliminates context collision and enables highly scalable orchestration [2602.04634].

## 3. Multi-Agent Reinforcement Learning Formulation

WideSeek-R1 employs an end-to-end multi-agent variant of Proximal Policy Optimization (PPO) for MARL across the lead agent and parallel subagents.

**Components**:

- **State space** for agent $a$ at rollout $i$, turn $t$:
  $$
  s_{i,a}^t = [p_a, q_a, o_{i,a}^1, tc_{i,a}^1, \dots, o_{i,a}^{t-1}, tc_{i,a}^{t-1}]
  $$
  where $p_a$ is the role prompt (lead or subagent), $q_a$ the assigned query, $o$ previous tokens, $tc$ tool call results.

- **Action space**: At each generation token, the policy $\pi_\theta$ selects either a language token or a structured tool call.

- **Reward function**:
  $$
  R_i = r_{\mathrm{ans}} + r_{\mathrm{format}} + r_{\mathrm{tool}} - r_{\mathrm{len}}
  $$
  - $r_{\mathrm{ans}}$ = item-level F1 between output and ground truth.
  - $r_{\mathrm{format}}$ = +0.1 for valid Markdown table.
  - $r_{\mathrm{tool}}$ = +0.05 for efficient tool usage.
  - $r_{\mathrm{len}}$ = penalty for overlong outputs.

- **Unified MARL loss**: All tokens of all agents in rollout $i$ share a group-normalized advantage:
  $$
  \hat A_i = \frac{R_i - \mu}{\sigma}
  $$
  PPO’s clipped loss is computed over all agent-token pairs, employing token- and agent-level reweighting to ensure that increased agent spawning improves $R_i$ rather than merely increasing capacity [2602.04634].

This protocol induces the lead agent to optimize decomposition strategies while ensuring subagents increase factual accuracy and efficiency via parallel execution.

## 4. Dataset Construction and Optimization Protocol

WideSeek-R1 is trained on a 20,000-instance corpus of broad information-seeking tasks generated via an automated pipeline:

- **Query synthesis**: HybridQA seeds and Gemini-3 prompts yield schema-constrained Markdown table queries, sampling row counts $R \in [10,50]$.
- **Answer verification**: Gemini-3 is prompted independently twice per query, accepting only those where corresponding cells match $\geq 90\%$.
- **Filtering**: Instances with poor inter-run consistency or $< 3$ rows are discarded; 73.3% are retained.
- **Statistics**: Median table $\approx$ 30 rows, 6 columns; dataset split equally with deep-QA data for hybrid training.

**Optimization details**:
- Backbone: Qwen3-4B.
- Batch size: 128 (split evenly across “wide” and “deep” batches).
- Learning rate: $1 \times 10^{-6}$, PPO clip $\epsilon=0.2$.
- Max context: 32K tokens.
- Training: 150 RL steps on 32 NVIDIA H100 GPUs, totaling $\sim 3,000$ GPU-hours [2602.04634].

## 5. Empirical Evaluation and Scaling Results

Evaluation is conducted on the WideSearch benchmark (Wong et al. 2025), comprising 200 queries balanced across English and Chinese.

**Metrics**:
- **Item F1**: cell-level F1 score
- **Row F1**: row-unit match
- **Success Rate**: perfect-table match

| Model                  | Item F1 Avg@4 |
|------------------------|--------------|
| SingleSeek-R1-4B       | 28.1%        |
| DeepSeek-R1-671B       | 41.3%        |
| WideSeek-R1-4B         | 40.0%        |

WideSeek-R1-4B, despite its moderate parameter count, approaches the performance of DeepSeek-R1-671B (a 167× larger model) and surpasses all multi-agent 4B/8B baselines. Scaling experiments reveal:

- **Depth scaling** (single agent, more turns) saturates at 30–35% Item F1.
- **Width scaling** (increased number of subagents per task) yields steady gains up to 40% Item F1 with 10 subagents.
- Naïve multi-agent methods (without MARL) incur interference and performance decay beyond 6 subagents; WideSeek-R1-4B maintains gains up to 10 subagents.

On seven standard QA datasets, WideSeek-R1-4B achieves Avg@4 $\sim$ 59.0%, outperforming both its single-agent version and multi-agent systems with substantially larger LLMs (e.g. OWL-8B, MiroFlow-8B) [2602.04634].

## 6. Limitations and Prospective Directions

### Identified Limitations
- **Coarse credit assignment**: The use of a single rollout score obscures errors attributable to decomposition versus local subagent execution.
- **Fixed two-level hierarchy**: Subagent recursion is disallowed; arbitrary-depth hierarchies may enable further organizational scaling.
- **Training cost**: Synchronous, collocated rollouts are latency-bound, impeding scalability.

### Future Work

- **Dynamic agent allocation**: Adaptive subagent spawning for task-dependent load balancing.
- **Recursive hierarchies**: Permitting multi-layer agent structures for ultra-broad or nested tasks.
- **Role-aware rewards**: Providing targeted feedback to specific agent roles (e.g., decomposition accuracy for the lead agent, local F1s for subagents).
- **Multi-domain generalization**: Extending width scaling to code synthesis, GUI automation, and real-time information domains.

This suggests a plausible trajectory toward universal information-seeking systems that maximize not only individual competence but also organizational intelligence via explicit parallelism and robust MARL optimization [2602.04634][2602.02636].

## 7. Position within Wide Research and Related Work

WideSeek-R1 addresses the same class of General Broad Information Seeking (GBIS) tasks as WideSeek and is evaluated on related or overlapping benchmarks (WideSearch, WideSeekBench). Both frameworks employ hierarchical planner-executor models, context isolation, and joint MARL. However, WideSeek-R1 demonstrates that end-to-end parallel orchestration with robust MARL leads to state-of-the-art item-level F1 without reliance on extra-large LLMs or hand-crafted workflow logic [2602.04634][2602.02636].

The shift from deep, sequential research architectures to dynamic, scalable “wide research” protocols marks a structural inflection point in information-seeking AI, opening a new axis for scaling by organizational capacity rather than raw model size.

Source: https://www.emergentmind.com/topics/wideseek-r1