Multi-agent Retrieval-Augmented Framework

• Multi-agent retrieval-augmented frameworks are architectures that decompose retrieval, reasoning, and verification into specialized, interacting modules for improved evidence grounding.
• They employ adaptive and iterative mechanisms, including hybrid retrieval and reinforcement learning, to manage ambiguous queries and multi-hop dependencies.
• Applications span scientific QA, medical reasoning, counterspeech, and multimedia verification, demonstrating enhanced accuracy, citation integrity, and robust inference.

A multi-agent retrieval-augmented framework is a class of RAG systems in which retrieval, evidence selection, reasoning, answer generation, and verification are distributed across multiple specialized agents rather than collapsed into a single retrieve-and-generate pass. Recent formulations span scientific question answering, knowledge-graph QA, counterspeech generation, attributed QA, medical reasoning, clinical error detection, multimodal fact-checking, process monitoring, and heterogeneous enterprise data integration. Across these settings, the common objective is to improve grounding in external evidence while managing ambiguity, noisy retrieval, multi-hop dependencies, and trustworthiness requirements such as citations, traceability, or explicit verification (Besrour et al., 17 Oct 2025, Xu et al., 1 Sep 2025, Nguyen et al., 26 May 2025, Chang et al., 2024).

1. Definition, scope, and representative systems

The term denotes an architectural pattern rather than a single fixed pipeline. In SQuAI, the framework is a four-agent scientific QA system over the unarXive 2024 corpus of 2.3 million full-text arXiv papers, combining decomposition, hybrid retrieval, adaptive filtering, and citation-aware synthesis (Besrour et al., 17 Oct 2025). In AnchorRAG, it is a three-agent collaboration framework for open-world KG-based RAG without predefined anchor entities (Xu et al., 1 Sep 2025). In MA-RAG, it is a reasoning-driven pipeline with Planner, Step Definer, Extractor, and QA Agents, invoked on demand for ambiguous and multi-hop QA (Nguyen et al., 26 May 2025). In MAIN-RAG, it is a training-free filtering layer added post hoc to a traditional RAG pipeline through Predictor, Judge, and Final-Predictor agents (Chang et al., 2024). This suggests that the defining property is not the number of agents, but the explicit decomposition of retrieval-augmented inference into role-specialized, interacting modules.

Framework	Domain	Salient design
SQuAI	Scientific QA	Four agents, hybrid sparse-dense retrieval, adaptive filtering, in-line citations
AnchorRAG	Open-world KGQA	Predictor, parallel Retriever agents, Supervisor
MA-RAG	Complex QA	Planner, Step Definer, Extractor, QA Agents
MAIN-RAG	General QA	Training-free multi-agent document filtering
RAGentA	Attributed QA	Iterative filtering, cited answers, dynamic revision
SEMA-RAG	Medical QA	Interpreter, Explorer, Arbiter with sufficiency-driven retrieval

The scope of the pattern has expanded beyond text-only QA. MARF combines static and dynamic evidence for counterspeech against health misinformation (Anik et al., 9 Jul 2025). BLUEmed couples hybrid RAG with multi-agent debate for clinical terminology substitution detection (You et al., 12 Apr 2026). RAMA uses web retrieval plus an ensemble of multimodal judges for multimedia verification (Yang et al., 12 Jul 2025). HEP-CoPilot integrates publications, HEPData, and reconstructed plots through a directed graph of specialized agents for collider phenomenology (Cakir et al., 4 May 2026). A plausible implication is that multi-agent RAG has become a general strategy for externalized reasoning over heterogeneous evidence, not merely an optimization of the QA stack.

2. Role decomposition and coordination patterns

Most systems partition the pipeline into a small set of recurring functions: decomposition or planning, retrieval or traversal, evidence filtering or extraction, answer synthesis, and verification. SQuAI exemplifies this pattern directly: Agent-1 generates semantically distinct sub-questions, Agent-2 produces structured Q-A-E triplets for each sub-question–document pair, Agent-3 filters documents using a relevance confidence score, and Agent-4 synthesizes a long-form answer with in-line citations and citation context (Besrour et al., 17 Oct 2025). MA-RAG follows a similar but more explicitly sequential plan-execute structure in which a Planner first outputs a plan $P=\{s_1,\dots,s_n\}$, a Step Definer converts each step into an executable subquery, an Extractor selects only the relevant spans from retrieved passages, and a QA Agent answers each subquery before final assembly (Nguyen et al., 26 May 2025).

Other frameworks specialize coordination around the retrieval substrate. AnchorRAG begins with a Predictor that maps a natural-language question to top-$m$ candidate anchor entities through keyword extraction and relation-aware grounding, then launches $m$ Retriever agents in parallel for multi-hop traversal, while a Supervisor decides whether the aggregated evidence is sufficient to answer or whether more hops are needed (Xu et al., 1 Sep 2025). In SEMA-RAG, the Interpreter Agent first extracts a four-tuple clinical schema, the Explorer Agent performs iterative sufficiency-driven retrieval with evidence-gap analysis, and the Arbiter Agent produces a traceable adjudication report before answer selection (Huang et al., 16 May 2026). In MoodAngels, three Diagnosis Agents produce independent diagnostic opinions, a Judge Agent reconciles them, and two Debate Agents are invoked when disagreement persists (Xiao et al., 4 Jun 2025).

Coordination can also be centralized. In mRAG, a Coordinator Agent is the entry point for every question, maintaining history, documents, and draft responses while dispatching Planner, Searcher, Reasoner, Summarizer, Validator, and Generator/Reviser agents through structured prompts (Salemi et al., 12 Jun 2025). HEP-CoPilot similarly routes execution through a runtime DAG in which MissionControlAgent classifies intent, RouterAgent builds the graph, and specialized text, table, figure, equation, terminologist, reasoning, plotting, synthesis, and control agents execute when predecessor outputs are available (Cakir et al., 4 May 2026). This suggests two broad orchestration styles: pipeline-style staged collaboration and planner-mediated dynamic graphs.

3. Retrieval, evidence selection, and adaptive control

A central technical theme is that multi-agent RAG rarely treats retrieval as a single opaque step. Many frameworks adopt hybrid retrieval. SQuAI scores each document–subquery pair with

$\mathrm{score}(d,q)=\lambda\,\mathrm{BM25}(d,q)+(1-\lambda)\,\cosine(\mathbf{e}_d,\mathbf{e}_q),$

with $\lambda$ set to $0.35$ and dense embeddings from intfloat/e5-base-v2, while retrieval is performed over paper abstracts for scalability (Besrour et al., 17 Oct 2025). RAGentA also linearly interpolates BM25 and E5 embedding similarity, but with $\alpha=0.65$, and reports Recall@20 values of $0.5020$ for BM25, $0.4920$ for E5, and $0.5650$ for the hybrid retriever (Besrour et al., 20 Jun 2025). MARF applies the same general hybrid principle to a static medical knowledge base, using $m$0, while dynamic web evidence is collected separately and lightly fact-checked at a threshold of accuracy $m$1 before fusion (Anik et al., 9 Jul 2025).

A second recurring mechanism is adaptive filtering. MAIN-RAG defines a per-document log-odds relevance score

$m$2

then retains documents satisfying $m$3, where $m$4 is computed from the score distribution for the current query (Chang et al., 2024). SQuAI and RAGentA use closely related thresholds with $m$5 to discard weakly relevant documents while preserving recall for complex questions (Besrour et al., 17 Oct 2025, Besrour et al., 20 Jun 2025). In MA-RAG, the Extractor performs sentence- or span-level selection after dense retrieval, and ablations report 4–7 EM drops when the Extractor is removed, indicating that evidence filtering is not merely a compression step but a major determinant of downstream precision (Nguyen et al., 26 May 2025).

A third theme is iterative sufficiency control. SEMA-RAG formalizes retrieval as a self-evolving loop in which the Explorer Agent inspects the current evidence set $m$6, emits a binary sufficiency flag $m$7, describes the evidence gap $m$8, and proposes follow-up queries $m$9; performance peaks at 2–3 rounds rather than fixed-depth retrieval (Huang et al., 16 May 2026). SIRAG assigns this stopping decision to a Decision Maker agent and the filtering step to a Knowledge Selector, both trained with PPO under process-level rewards from an LLM judge (Wang et al., 17 Sep 2025). MAO-ARAG generalizes the idea further by letting a planner choose among Query Decomposition, Query Rewriter, Retrieval Agent, Document Selector, Answer Generator, and Answer Summarization modules over multiple turns, with a final reward that combines answer F1 with cost and format penalties (Chen et al., 1 Aug 2025). A common misconception is that multi-agent RAG simply means “more retrieval”; these systems instead make retrieval conditional, revisable, and termination-aware.

4. Training-free orchestration, reinforcement learning, and self-training

The family includes both training-free and learned variants. MAIN-RAG is explicitly training-free and uses prompted LLM agents without fine-tuning or extra data (Chang et al., 2024). MA-RAG is also training-free, relying on role prompts and on-demand invocation so that Planner runs once and the remaining agents run only per subtask (Nguyen et al., 26 May 2025). MASS-RAG presents a training-free synthesis design in which Summarizer, Extractor, and Reasoner agents produce complementary evidence views that are reconciled by a Synthesis Agent, all sharing the same frozen backbone and differing only by prompt templates (Xiao et al., 20 Apr 2026). BLUEmed likewise performs no fine-tuning on LLMs, operating entirely in zero-shot or few-shot prompting (You et al., 12 Apr 2026). One consequence is that multi-agent retrieval-augmented frameworks are not inherently coupled to parameter updates.

Other systems explicitly optimize inter-agent coordination. MMOA-RAG treats Query Rewriter, Selector, and Generator as cooperative agents in a Markov game with shared end-of-episode reward

$m$0

augmented by per-module penalties and a KL term relative to the SFT policy, and trains them with a MAPPO-style algorithm using a shared Llama-3-8B-Instruct backbone and centralized critic (Chen et al., 25 Jan 2025). SIRAG trains its Decision Maker and Knowledge Selector with PPO using a mixed reward $m$1, where the judge provides process-level supervision at intermediate steps (Wang et al., 17 Sep 2025). MAO-ARAG also uses PPO, but optimizes only the planner and explicitly trades off F1 against token cost, turn cost, and retrieval usage (Chen et al., 1 Aug 2025).

Self-training constitutes a third optimization regime. In mRAG, $m$2 full multi-agent trajectories are sampled per training question, scored by a composite reward weighting correctness and faithfulness at $m$3 and $m$4, and the best trajectory is turned into supervision for LoRA adapters shared across the trainable agents (Salemi et al., 12 Jun 2025). CoCoA takes a different route: CoCoA-zero first constructs multi-agent traces that separate internal and external knowledge induction, after which CoCoA distills the long chain $m$5 into a single model with SFT and DPO (Jiang et al., 3 Aug 2025). This suggests that “multi-agent” can describe either the inference architecture itself or a scaffold for generating supervision that is later compressed into a single forward pass.

5. Attribution, verification, and interpretability

Trustworthiness is a recurring design objective, but the mechanisms vary. SQuAI requires Agent-4 to place citations immediately after each factual sentence, merge citations when multiple documents support the same claim, and omit unsupported statements; it also surfaces the exact supporting sentence as citation context (Besrour et al., 17 Oct 2025). RAGentA adopts a similar claim-level citation scheme and formalizes faithfulness as the fraction of claims that are grounded, with correctness decomposed into coverage and relevance; a fourth Reviser agent decomposes the question into sub-questions, identifies uncovered gaps, performs follow-up retrieval, and synthesizes an updated answer (Besrour et al., 20 Jun 2025). In both cases, attribution is not a post-processing layer but part of the generation contract.

Verification often appears as a separate agent or protocol. In BLUEmed, two expert agents with source-partitioned evidence debate a classification, a judge performs cross-source verification through an additional retriever, and a cascading safety layer applies a Two-Term Rule, Expert Consensus Override, five domain-specific heuristics, and a confidence adjustment before the final label is emitted (You et al., 12 Apr 2026). In MoodAngels, Judge and Debate Agents are introduced precisely to surface contradictory evidence and to force both pro and con arguments before a final psychiatric diagnosis (Xiao et al., 4 Jun 2025). HEP-CoPilot assigns consistency checking to a ControlAgent after the reasoning and plotting stages (Cakir et al., 4 May 2026). SIRAG makes interpretability explicit by representing trajectories as trees in which each decision by the Decision Maker or Knowledge Selector can be scored by an LLM judge (Wang et al., 17 Sep 2025).

These designs also clarify a common misunderstanding: multi-agent decomposition alone does not guarantee factual reliability. BLUEmed reports that removing retrieval causes over-flagging and low precision, removing debate makes the system brittle under few-shot prompting and lowers recall, and removing the safety layer raises false positives by about 5–7% (You et al., 12 Apr 2026). MAIN-RAG explicitly lists reliance on LLM judgment accuracy as a limitation, since filtering can remove crucial context when the base LLM misjudges support (Chang et al., 2024). The stronger conclusion supported by the literature is narrower: multi-agent RAG improves trustworthiness when role separation is coupled to evidence control, explicit attribution, or structured verification.

6. Domains, empirical behavior, and limitations

Empirical gains are reported across many tasks, though under different metrics. SQuAI improves the combined metric of Answer Relevance, Contextual Relevance, and Faithfulness by up to $m$6 (approximately 12% relative gain) over a strong one-pass RAG baseline, with all Faithfulness scores exceeding $m$7, and releases a benchmark of 1,000 scientific question–answer–evidence triplets (Besrour et al., 17 Oct 2025). MAIN-RAG reports 2–11% improvement in answer accuracy over traditional RAG across four QA benchmarks and reduces irrelevant documents by roughly half relative to fixed top-$m$8 strategies, but at the cost of three LLM calls per retrieved document versus one in standard RAG (Chang et al., 2024). RAGentA improves correctness from $m$9 to $\mathrm{score}(d,q)=\lambda\,\mathrm{BM25}(d,q)+(1-\lambda)\,\cosine(\mathbf{e}_d,\mathbf{e}_q),$0 and faithfulness from $\mathrm{score}(d,q)=\lambda\,\mathrm{BM25}(d,q)+(1-\lambda)\,\cosine(\mathbf{e}_d,\mathbf{e}_q),$1 to $\mathrm{score}(d,q)=\lambda\,\mathrm{BM25}(d,q)+(1-\lambda)\,\cosine(\mathbf{e}_d,\mathbf{e}_q),$2 on 500 synthetic pairs, while hybrid retrieval raises Recall@20 by 12.5% over the best single retriever (Besrour et al., 20 Jun 2025).

In reasoning-intensive QA, MA-RAG reports Exact Match improvements such as NQ EM $\mathrm{score}(d,q)=\lambda\,\mathrm{BM25}(d,q)+(1-\lambda)\,\cosine(\mathbf{e}_d,\mathbf{e}_q),$3 for Llama3-8B and NQ EM $\mathrm{score}(d,q)=\lambda\,\mathrm{BM25}(d,q)+(1-\lambda)\,\cosine(\mathbf{e}_d,\mathbf{e}_q),$4, HotpotQA EM $\mathrm{score}(d,q)=\lambda\,\mathrm{BM25}(d,q)+(1-\lambda)\,\cosine(\mathbf{e}_d,\mathbf{e}_q),$5, and 2WikimQA EM $\mathrm{score}(d,q)=\lambda\,\mathrm{BM25}(d,q)+(1-\lambda)\,\cosine(\mathbf{e}_d,\mathbf{e}_q),$6 for GPT-4o-mini, with large drops when Planner or Extractor are removed (Nguyen et al., 26 May 2025). MMOA-RAG records Llama-3-8B results of HotpotQA Acc/EM/F1 $\mathrm{score}(d,q)=\lambda\,\mathrm{BM25}(d,q)+(1-\lambda)\,\cosine(\mathbf{e}_d,\mathbf{e}_q),$7, 2Wiki $\mathrm{score}(d,q)=\lambda\,\mathrm{BM25}(d,q)+(1-\lambda)\,\cosine(\mathbf{e}_d,\mathbf{e}_q),$8, and AmbigQA $\mathrm{score}(d,q)=\lambda\,\mathrm{BM25}(d,q)+(1-\lambda)\,\cosine(\mathbf{e}_d,\mathbf{e}_q),$9, outperforming its listed baselines (Chen et al., 25 Jan 2025). SIRAG raises average EM to $\lambda$0 across 2Wiki, HotpotQA, NQ, and PopQA, versus $\lambda$1 for SelfRAG and $\lambda$2 for standard RAG in the reported table (Wang et al., 17 Sep 2025). MAO-ARAG shows that adaptive planning can simultaneously increase Avg F1 from $\lambda$3 to $\lambda$4 and reduce token cost, calls, and turns relative to the best baseline in its comparison (Chen et al., 1 Aug 2025).

Specialized domains show the same pattern of evidence-grounded role separation. MARF achieves the best values in politeness, relevance, informativeness, and factual accuracy among its baselines, with the full model at $\lambda$5, $\lambda$6, $\lambda$7, and $\lambda$8 respectively (Anik et al., 9 Jul 2025). SEMA-RAG improves the strongest baseline by an average of $\lambda$9 accuracy points per backbone across five medical benchmarks and five LLM backbones (Huang et al., 16 May 2026). BLUEmed reaches accuracy $0.35$0, ROC-AUC $0.35$1, and PR-AUC $0.35$2 under few-shot prompting, outperforming both single-agent RAG and debate-only baselines (You et al., 12 Apr 2026). RAMA attains Accuracy $0.35$3, F1 $0.35$4, and MCC $0.35$5 on the multimedia verification benchmark, with total latency of about $0.35$6 s per sample (Yang et al., 12 Jul 2025). HEP-CoPilot is evaluated through case studies and receives mean LLM-as-judge scores of about $0.35$7 versus about $0.35$8 for PDF-only prompting, while reconstructing exclusion contours with less than 1–2% interpolation error (Cakir et al., 4 May 2026).

The principal limitations are equally consistent. Inference cost is repeatedly identified as a constraint: MAIN-RAG notes the overhead of multiple LLM calls, mRAG emphasizes higher inference cost from multi-agent workflows, and MAO-ARAG introduces an explicit cost-based penalty because a fixed pipeline struggles to balance performance and efficiency across queries (Chang et al., 2024, Salemi et al., 12 Jun 2025, Chen et al., 1 Aug 2025). Several papers also note dependence on agent quality or base-model capability: BLUEmed reports that the framework benefits most from models with sufficient instruction-following and clinical language understanding, while MA-RAG finds that reducing the capacity of QA, Planner, or Extractor causes the largest performance hits (You et al., 12 Apr 2026, Nguyen et al., 26 May 2025). The literature therefore supports a measured conclusion: multi-agent retrieval-augmented frameworks improve robustness and traceability by redistributing cognitive load across explicit roles, but they do so by adding orchestration complexity, judgment dependencies, and nontrivial latency or compute overhead.