MARAG-R1: Multi-Tool RAG Model

• MARAG-R1 is a reinforcement-learned multi-tool retrieval framework that replaces single-pass retrieval with iterative, dynamic evidence acquisition for corpus-level reasoning.
• It integrates four specialized tools—semantic search, keyword search, filtering, and aggregation—to iteratively refine evidence collection and improve factual synthesis.
• The training procedure combines supervised fine-tuning with reinforcement learning using multi-component rewards to achieve state-of-the-art performance on complex reasoning tasks.

Searching arXiv for MARAG-R1 and closely related agentic/multi-tool RAG work to ground the article in current literature. MARAG-R1 is a reinforcement-learned multi-tool retrieval-augmented generation framework designed to overcome the single-retriever, fixed top-$k$ bottleneck in conventional RAG. It equips a LLM with four retrieval tools—semantic search, keyword search, filtering, and aggregation—and trains the model to decide both how and when to use them through supervised fine-tuning followed by reinforcement learning. The framework is motivated by corpus-level reasoning tasks, where answering requires broader and more precise information access than a static retrieval pass can provide, and it is evaluated on GlobalQA, HotpotQA, and 2WikiMultiHopQA, where it substantially outperforms strong baselines and achieves new state-of-the-art results in corpus-level reasoning tasks (Luo et al., 31 Oct 2025).

1. Concept and scope

MARAG-R1 is introduced against the background that LLMs are limited by static pretraining knowledge, while standard RAG pipelines typically rely on a single retriever with fixed top-$k$ selection. The paper argues that this single-retriever paradigm restricts the model to a “narrow and static subset of the corpus,” which becomes the primary bottleneck for comprehensive external information acquisition, especially in tasks requiring corpus-level reasoning (Luo et al., 31 Oct 2025).

The framework addresses this bottleneck by replacing one-shot retrieval with multi-step, agentic retrieval. Rather than retrieving once and answering from a fixed evidence set, the model alternates between intermediate reasoning, tool invocation, and evidence accumulation. This design targets settings in which the system must gather sufficient evidence progressively, including Count, Sort, MinMax, TopK, and multi-hop question answering (Luo et al., 31 Oct 2025).

This placement distinguishes MARAG-R1 from conventional workflow RAG and from graph-based alternatives. The paper argues that graph-based RAG may improve global awareness but can lose document-level identifiers and fine-grained factual details during graph construction, whereas MARAG-R1 preserves direct access to original documents and their identifiers (Luo et al., 31 Oct 2025). A plausible implication is that the framework is intended not only to improve recall, but also to maintain provenance needed for set operations and exact aggregation.

2. Retrieval model and tool set

The defining feature of MARAG-R1 is its multi-tool retrieval environment. The model is equipped with four retrieval tools:

Tool	Paper notation	Role
Semantic Retriever	$F_{\mathrm{DR}}$	Dense semantic retrieval for broad contextual exploration
Keyword Retriever	$F_{\mathrm{KR}}$	Keyword-based matching for precise entity and attribute retrieval
Document Filter	$F_{\mathrm{DF}}$	Filtering documents by metadata attributes or logical constraints
Aggregation Tool	$F_{\mathrm{AG}}$	Statistical or structural synthesis such as counting, ranking, and set aggregation

The paper presents these tools as complementary. Semantic retrieval is used for broad contextual exploration, keyword retrieval for precise lexical matching, document filtering for reducing candidate sets under explicit constraints, and aggregation for operations such as counting, ranking, and set aggregation (Luo et al., 31 Oct 2025). The aggregation tool is particularly important for corpus-level reasoning tasks, because the answer may depend on a global operation over many documents rather than on a single supporting passage.

The framework’s broader design can be situated alongside recent training-free multi-agent RAG work, which also treats retrieval as a staged reasoning process rather than a one-shot module, although MARAG-R1 differs by learning tool coordination with reinforcement learning (Nguyen et al., 26 May 2025). It also contrasts with multimodal RAG systems that focus on reranking or evidence pruning after initial retrieval rather than expanding the retrieval action space itself (Hu et al., 29 May 2025).

3. Trajectory formulation and inference process

The paper formalizes training and inference around multi-step trajectories. For each query-answer pair $(Q, A^*)$, the model produces a trajectory

$$\mathcal{T} = \{S_1, S_2, \ldots, S_{|\mathcal{T}|}\},$$

where each intermediate step is

$S_t = (R_t, C_t, D_t), \quad t < |\mathcal{T}|.$

Here $R_t$ is intermediate reasoning, $k$0 is a tool invocation, and $k$1 is the resulting retrieved document output (Luo et al., 31 Oct 2025). The final step replaces the tool call with the predicted answer $k$2.

Operationally, the model begins from the user query, reasons about what information is missing, selects a tool, observes the retrieved output, and either continues retrieving or answers once sufficient information has been obtained (Luo et al., 31 Oct 2025). The paper does not provide a separate halting classifier or stopping threshold. Instead, the continue-versus-answer behavior is learned implicitly through sequence modeling and reinforced through the trajectory reward.

The framework therefore treats retrieval as a sequential decision problem over reasoning, actions, and observations. The paper does not explicitly formalize the setting as an MDP or POMDP, but the trajectory definition serves that role in practice. This suggests a retrieval-control view of RAG rather than a prompt-construction view.

4. Two-stage training procedure

MARAG-R1 is trained in two stages: supervised fine-tuning followed by reinforcement learning (Luo et al., 31 Oct 2025).

4.1 Supervised fine-tuning

The first stage constructs high-quality expert trajectories using GPT-4 with task instructions and examples, followed by rejection sampling to remove low-quality or redundant traces and ensure consistency and factual correctness. The resulting dataset is

$k$3

where each example contains a query, an expert trajectory, and a gold answer (Luo et al., 31 Oct 2025).

The supervised objective is standard autoregressive next-token prediction over trajectory steps: $k$4 This stage teaches the model to imitate expert multi-step retrieval behavior. The paper refers to the SFT-only model as MARAG-CS, for “cold start” (Luo et al., 31 Oct 2025).

4.2 Reinforcement learning

After imitation learning, the policy is further optimized by reinforcement learning in the retrieval environment. The goal is to improve tool coordination beyond what is present in demonstrations, especially because answer-only supervision is too sparse for effective learning (Luo et al., 31 Oct 2025).

The paper uses Reinforcement Learning with Leave-One-Out baseline (RLOO). For a batch of $k$5 sampled trajectories, the gradient estimator is

$k$6

The paper specifies that RL uses 5 sampled rollouts per query (Luo et al., 31 Oct 2025).

A central claim is that SFT initialization is necessary for stable exploration, while the leave-one-out baseline reduces variance without requiring a reference model or KL regularization (Luo et al., 31 Oct 2025). This places MARAG-R1 in a family of R1-style systems that use RL to refine multi-step reasoning behavior after supervised initialization, although its domain is retrieval control rather than pure reasoning (Yang et al., 29 May 2025, Kim et al., 29 May 2025).

5. Reward design

The RL stage optimizes a trajectory-level reward with three components: $k$7

5.1 Answer reward

The answer reward is token-level F1 between the predicted answer and the reference answer: $k$8 This provides partial credit instead of a binary signal (Luo et al., 31 Oct 2025).

5.2 Document coverage reward

The document coverage reward measures how well the retrieved document set matches the gold supporting documents: $k$9 with precision and recall over document identifiers. The paper gives the corresponding precision and recall definitions and computes an F1 score over predicted versus gold document sets (Luo et al., 31 Oct 2025). This component encourages both completeness and relevance of retrieval.

5.3 Tool exploration reward

The tool exploration reward encourages strategic but not excessive tool use. Let $F_{\mathrm{DR}}$0 be the number of tool calls in the sampled trajectory and $F_{\mathrm{DR}}$1 the number in the expert trajectory. The reward is defined piecewise: $F_{\mathrm{DR}}$2 This gives maximal reward when the model uses no more tool calls than the expert and penalizes overuse thereafter (Luo et al., 31 Oct 2025).

The three-part reward is one of the paper’s core design decisions. It makes retrieval quality a first-class training target rather than treating retrieval as latent infrastructure. This aligns MARAG-R1 with other recent systems that train evidence selection or retrieval control directly rather than relying on final-answer supervision alone (Wang et al., 27 Apr 2026, Luo et al., 31 Oct 2025).

6. Experimental setting

The main evaluation is conducted on GlobalQA, a corpus-level reasoning benchmark containing four task types: TopK, Count, Sort, and MinMax (Luo et al., 31 Oct 2025). These tasks are designed to stress comprehensive retrieval and explicit aggregation rather than local passage retrieval.

The paper also evaluates generalization on 2WikiMultiHopQA and HotpotQA (Luo et al., 31 Oct 2025). This choice connects MARAG-R1 to the broader multi-hop QA literature and shows whether a framework optimized for global retrieval transfers to more standard compositional QA tasks.

The experiments use Qwen2.5-Instruct models at 3B, 7B, and 14B scale as backbones. Retrieval uses BGE as the retriever, Qwen3-4B as the filtering model, a chunk-free retrieval strategy, and TopK $F_{\mathrm{DR}}$3 (Luo et al., 31 Oct 2025). Evaluation reports answer F1 and document F1@20, denoted D-F1@20, with the latter computed from precision and recall over retrieved versus gold evidence sets (Luo et al., 31 Oct 2025).

The baselines include StandardRAG, ITER-RETGEN, IRCoT, HyperGraphRAG, Search-R1, and ReCall (Luo et al., 31 Oct 2025). The ReCall comparison is especially important because it uses the same multi-tool retrieval environment but optimizes only with answer-level rewards, isolating the effect of MARAG-R1’s process-aware training design.

7. Empirical results

7.1 GlobalQA

On GlobalQA, MARAG-R1 substantially outperforms all reported baselines across model sizes (Luo et al., 31 Oct 2025).

For Qwen2.5-14B, MARAG-R1 achieves an average 31.22 F1 and 42.11 D-F1@20. In the same setting, ReCall achieves 14.25 F1 and 20.00 D-F1@20, MARAG-CS achieves 28.92 F1 and 39.83 D-F1@20, and StandardRAG achieves 1.51 F1 and 8.09 D-F1@20 (Luo et al., 31 Oct 2025).

For Qwen2.5-7B, MARAG-R1 reaches 28.60 F1 and 38.44 D-F1@20, compared with ReCall at 10.84 F1 and 17.61 D-F1@20 (Luo et al., 31 Oct 2025).

For Qwen2.5-3B, MARAG-R1 reaches 26.40 F1 and 37.05 D-F1@20, while ReCall reaches 6.41 F1 and 12.68 D-F1@20 (Luo et al., 31 Oct 2025).

The largest task-level strengths appear on Sort and MinMax, with Count remaining more difficult. For Qwen2.5-14B, the paper reports: TopK 32.65 / 38.81, Count 6.05 / 42.38, Sort 40.5 / 40.92, and MinMax 39.20 / 45.81 in F1 / D-F1@20 (Luo et al., 31 Oct 2025). This suggests that the framework is particularly effective when the problem requires structured aggregation over a sufficiently retrieved set, though counting remains comparatively challenging.

7.2 Generalization to multi-hop QA

On 2WikiMultiHopQA, MARAG-R1 achieves 22.00 EM / 26.93 F1, compared with StandardRAG at 12.00 / 18.24, ITER-RETGEN at 12.00 / 17.22, IRCoT at 5.00 / 18.50, and MARAG at 15.00 / 22.81 (Luo et al., 31 Oct 2025).

On HotpotQA, MARAG-R1 achieves 31.00 EM / 39.16 F1, compared with StandardRAG at 19.00 / 26.36, ITER-RETGEN at 17.00 / 25.56, IRCoT at 11.00 / 23.92, and MARAG at 22.00 / 34.05 (Luo et al., 31 Oct 2025).

These results indicate that the framework’s learned multi-tool retrieval strategy transfers beyond the corpus-level operations of GlobalQA to more conventional multi-hop QA tasks.

8. Ablations and interpretation

The ablation studies isolate the contributions of both training and tool design (Luo et al., 31 Oct 2025).

8.1 Training ablations

On Qwen2.5-14B GlobalQA, full MARAG-R1 achieves 31.22 / 42.11. Removing the tool-calls reward reduces this to 29.45 / 39.54. Removing the document reward reduces it to 28.87 / 39.53. Removing the answer reward causes a larger drop to 13.78 / 24.48. Removing SFT reduces performance to 14.25 / 20.00 (Luo et al., 31 Oct 2025).

This shows that SFT initialization and answer reward are indispensable, while document coverage and tool-calls reward provide consistent further gains. The pattern supports the paper’s interpretation that the three reward components are complementary.

8.2 Tool ablation

On Qwen2.5-7B, the full system achieves 27.35 / 38.19. Removing the document filter yields 22.57 / 32.14. Removing the semantic retriever yields 20.21 / 25.13. Removing the keyword retriever yields 13.03 / 24.08. Removing the aggregation function yields 10.48 / 19.54 (Luo et al., 31 Oct 2025).

Aggregation is therefore the most critical tool in the reported ablations, followed by keyword and semantic retrieval. This is consistent with the framework’s intended use for corpus-level reasoning, where retrieving documents is necessary but not sufficient unless the system can explicitly combine them.

8.3 Tool-usage analysis

The paper also compares answer quality, evidence coverage, and average number of tool calls. On Qwen2.5-14B, ReCall averages 4.53 calls, MARAG 2.81, MARAG-CS 6.27, and MARAG-R1 6.32 (Luo et al., 31 Oct 2025). The strongest systems make more tool calls, which the paper interprets as evidence that better performance depends on broader external information acquisition. Because MARAG-R1 and MARAG-CS make a similar number of calls but MARAG-R1 performs better, the improvement cannot be attributed to call volume alone; it reflects improved coordination.

8.4 Retrieval depth

Appendix analysis shows that performance rises quickly as the maximum retrieval steps increase from 2 to 5, and remains stable beyond 10 steps (Luo et al., 31 Oct 2025). This suggests that multi-step retrieval is essential, but that the RL-trained policy is robust to larger search depth.

9. Representative behavior

The paper’s most illustrative case study is an OR-query requiring union and min aggregation over document IDs (Luo et al., 31 Oct 2025). MARAG-R1 issues multiple searches for different subconditions, unions the resulting document sets, performs a keyword search for an additional clause, unions again to obtain 44 unique documents, and then applies a Min operation to identify the smallest document ID, yielding the correct answer 21 (Luo et al., 31 Oct 2025).

This example is important because it demonstrates behavior not reducible to “better retrieval.” The model is not merely finding better passages; it is executing a retrieval-and-aggregation program over document sets. This suggests that MARAG-R1 is best understood as a retrieval-control architecture with explicit evidence operations, not merely an enhanced retriever.

10. Relation to adjacent work

MARAG-R1 belongs to a broader trend of moving retrieval from a fixed preprocessing stage into the reasoning loop. Training-free multi-agent RAG systems similarly decompose retrieval and synthesis into multiple coordinated subtasks, but rely on prompting rather than reinforcement learning (Nguyen et al., 26 May 2025). Multimodal RAG work has likewise emphasized that evidence quality depends on reranking, grounding, and pruning, though those systems generally optimize evidence selection after initial retrieval rather than learning a richer retrieval action space (Hu et al., 29 May 2025, Wang et al., 27 Apr 2026).

This suggests that MARAG-R1 occupies a specific niche: it is a text-domain, corpus-level RAG framework that expands the model’s retrieval affordances and trains the policy over those affordances directly. A plausible implication is that its main contribution is architectural and procedural rather than representational.

11. Limitations and significance

The paper does not present a separate limitations section, but several constraints are evident from the design (Luo et al., 31 Oct 2025). MARAG-R1 incurs higher inference cost than one-shot RAG because it averages around 6 tool calls in strong settings. It depends on the quality of its semantic retriever, keyword retriever, filtering model, and aggregation implementation. It also requires GPT-4-generated expert trajectories, rejection sampling, and reinforcement learning in a retrieval environment, making training substantially more complex than conventional RAG.

Even so, the reported evidence supports the paper’s central claim: the single-retriever top-$F_{\mathrm{DR}}$4 paradigm is a major bottleneck for corpus-level reasoning, and MARAG-R1 overcomes that bottleneck by learning to coordinate multiple retrieval tools dynamically (Luo et al., 31 Oct 2025). Its strongest empirical value lies in tasks where answers depend on comprehensive evidence acquisition and explicit aggregation rather than on retrieving one relevant passage.

In that sense, MARAG-R1 can be understood as a transition point from retrieval-augmented generation to retrieval-governed reasoning: the model is not only grounded by retrieved evidence, but also trained to decide how evidence should be acquired, filtered, and combined to support final synthesis (Luo et al., 31 Oct 2025).