DuMate-DeepResearch: Auditable DR Framework

• The paper presents an auditable deep research framework that combines graph-based planning, recursive search, and rubric-guided reasoning to improve long-form report synthesis.
• It features a clear separation between an Agent Core and a Tool Ecosystem, ensuring extensibility and strong auditability by logging each decision and tool interaction.
• Benchmark evaluations demonstrate superior performance on DeepResearch Bench metrics, with notable gains in information recall, analysis, and comprehensive planning.

Searching arXiv for the cited DuMate-DeepResearch paper and closely related benchmark papers to ground the article. DuMate-DeepResearch is a Deep Research (DR) framework introduced as “an auditable multi-agent system with recursive search and rubric-grounded reasoning” that is built on the Qianfan Agent Foundry and designed for complex, open-ended research tasks requiring iterative problem framing, evidence acquisition, verification, and long-form report synthesis (Yan et al., 5 Jun 2026). It is characterized by an explicit separation between an Agent Core and an extensible Tool Ecosystem, a graph-based dynamic planning strategy that expands a research roadmap coarse-to-fine, a recursive two-level execution design centered on an inner Search Agent, and a rubric-based test-time optimization mechanism that generates task-specific quality criteria as live reasoning scaffolds (Yan et al., 5 Jun 2026). In benchmark evaluation, it reports the best overall score of 58.03% on DeepResearch Bench and the best overall score of 61.95% on DeepResearch Bench II, ranking first in information recall and analysis on the latter benchmark (Yan et al., 5 Jun 2026).

1. Conceptual setting and problem formulation

DuMate-DeepResearch is situated within the broader DR paradigm, where systems are expected to address open-ended, underspecified research queries by clarifying intent, planning a research roadmap, iteratively acquiring and verifying evidence, and producing long-form reports with explicit structure and citations (Yan et al., 5 Jun 2026). In this setting, the output is not a short factual answer but a report intended to cover multiple subtopics, integrate heterogeneous sources, and present analysis rather than retrieval alone (Yan et al., 5 Jun 2026).

The technical report identifies four limitations in current DR systems. The first is long-horizon planning over an underspecified scope: complex queries unfold into many interdependent sub-questions, while common ReAct-style approaches are described as myopic and unable to systematically re-plan when tools fail or evidence contradicts earlier assumptions (Yan et al., 5 Jun 2026). The second is a single-agent decomposition and scheduling bottleneck, in which one agent is expected to perform both high-level planning and low-level noisy search, allowing local retrieval failures to derail the entire trajectory (Yan et al., 5 Jun 2026). The third is hallucination risk in long-form synthesis, especially when systems must decide when evidence is sufficient and how to keep claims grounded during generation (Yan et al., 5 Jun 2026). The fourth is limited process auditability, since many DR systems expose only final reports rather than intermediate plans, tool calls, evidence, and rationales (Yan et al., 5 Jun 2026).

This formulation places DuMate-DeepResearch in direct dialogue with benchmark-oriented work on deep research evaluation. DeepResearch Bench introduced a frozen “RetroSearch” environment for evaluating AI web research agents on multi-step tasks with stable web snapshots (FutureSearch et al., 6 May 2025). DeepResearch Bench II shifted evaluation toward long-form reports scored by 9,430 fine-grained binary rubrics across information recall, analysis, and presentation, and reported that even the strongest systems satisfy fewer than 50% of rubrics (Li et al., 13 Jan 2026). This suggests that strong DR performance requires both search competence and report-level evidence integration. A plausible implication is that DuMate-DeepResearch’s architectural emphasis on planning, recursive search isolation, and rubric-grounded reasoning is designed to target precisely those failure modes.

2. System architecture and auditable execution model

The framework is built on Qianfan Agent Foundry and separates the Agent Core from the Tool Ecosystem (Yan et al., 5 Jun 2026). The Agent Core contains a Router for task understanding and analysis, a Planner for task thinking and planning, and an Execution Module for task scheduling and execution (Yan et al., 5 Jun 2026). The Tool Ecosystem provides retrieval, evidence acquisition, and report rendering, and this separation is explicitly motivated by extensibility and auditability because every intermediate decision and tool invocation is traceable (Yan et al., 5 Jun 2026).

The Router takes a user query and produces a structured task specification. The Writer uses early coarse evidence to generate a fixed outline, and together the topic representation and outline form a fixed context

$z = (x, \mathcal{O})$

where $x$ is the structured task representation and $\mathcal{O}$ is the outline (Yan et al., 5 Jun 2026). The global research session is modeled as a state-transition process with state

$s_t = \langle z,\; p_t,\; e_t,\; \rho_t \rangle$

where $p_t$ is the current plan, $e_t$ is the global evidence base, and $\rho_t$ is the current guidance signal in the form of rubrics (Yan et al., 5 Jun 2026). The Planner selects actions

$a_t = \mathcal{P}(s_t),$

the Execution Module collects new evidence

$\Delta e_t = \mathcal{X}(s_t, a_t),$

and the state updates through

$s_{t+1} = \mathcal{T}(s_t, a_t, \Delta e_t)$

before the final Writer synthesizes a report

$x$0

when the stopping condition holds (Yan et al., 5 Jun 2026).

This design makes the system auditable in a strong procedural sense. The Router logs structured task representations; the Planner logs plan graphs, decisions, reflection results, and re-planning; the Execution Module logs tool calls, parameters, timestamps, Search Agent invocations, and evidence; and the Writer logs how evidence and rubrics map into outline construction and report synthesis (Yan et al., 5 Jun 2026). This suggests an explicit process trace rather than a purely prompt-internal latent trajectory.

3. Graph-based dynamic planning

The central planning object is a dynamic directed acyclic graph

$x$1

where each node $x$2 is a sub-task with depth $x$3 and execution status $x$4, and each edge $x$5 denotes that $x$6 depends on $x$7 (Yan et al., 5 Jun 2026). The ready frontier is defined as

$x$8

so only dependency-satisfied unexecuted nodes may be scheduled (Yan et al., 5 Jun 2026).

The planning strategy is coarse-to-fine. In an early coarse phase, the framework explores broad, high-level search tasks and uses the resulting evidence $x$9 to construct a fixed outline $\mathcal{O}$0 (Yan et al., 5 Jun 2026). In later fine phases, the Planner expands the DAG with more specific sub-tasks for each outline section, introducing additional search, lightweight reasoning, and final synthesis nodes (Yan et al., 5 Jun 2026). This coarse-to-fine organization operationalizes the idea that scope is initially vague and only becomes more concrete as evidence accumulates.

The plan is updated by a re-planning operator

$\mathcal{O}$1

which may expand the frontier, add finer subtasks, prune unproductive branches, or rewire dependencies in light of new evidence and rubric feedback (Yan et al., 5 Jun 2026). Executed nodes are not reverted: once $\mathcal{O}$2, they remain executed (Yan et al., 5 Jun 2026). Before execution, a reflection gate inspects proposed actions for relevance, redundancy, and consistency with rubric and task constraints, and actions may be revised for a bounded number of rounds (Yan et al., 5 Jun 2026).

This planning formulation differs from sequential search loops that lack an explicit global roadmap. Related work makes that contrast explicit. “Deep Researcher with Sequential Plan Reflection and Candidates Crossover” argues for sequential refinement with a centralized Global Research Context rather than parallel self-consistency, using reflection, progress estimation, and one-shot report generation (Prateek, 28 Jan 2026). “A Tale of Two Graphs” later formalizes a distinct but related dual-memory architecture separating an Outline Graph from a Knowledge Graph, and reports that separating what the agent knows from how it writes improves report depth, breadth, and factual grounding (Shi et al., 14 Feb 2026). DuMate-DeepResearch instead keeps the graph formalism in the planner and uses the evidence base plus rubrics as live state, but the shared theme is that global structure is necessary for long-horizon DR.

4. Recursive two-level execution and Search Agent design

A defining mechanism of DuMate-DeepResearch is its recursive two-level execution architecture (Yan et al., 5 Jun 2026). The outer Research Agent, denoted $\mathcal{O}$3, owns the global state $\mathcal{O}$4, maintains the DAG $\mathcal{O}$5, and decides which sub-task should be executed next (Yan et al., 5 Jun 2026). When a node corresponds to an open-ended search sub-task rather than a direct tool call, execution is delegated to an inner Search Agent $\mathcal{O}$6 (Yan et al., 5 Jun 2026).

The inner Search Agent is itself a full Foundry Agent with its own Router, Planner, and Execution, but it is restricted to a local search problem and does not spawn additional agents (Yan et al., 5 Jun 2026). Recursion depth is therefore exactly two: an outer Research Agent and inner Search Agents (Yan et al., 5 Jun 2026). Formally, when the outer execution encounters a search node $\mathcal{O}$7 with associated query $\mathcal{O}$8, it invokes

$\mathcal{O}$9

and merges the returned evidence into the outer evidence increment $s_t = \langle z,\; p_t,\; e_t,\; \rho_t \rangle$0 (Yan et al., 5 Jun 2026).

This mechanism is intended to isolate noisy retrieval. The inner agent may perform multiple search rounds, query reformulations, and tool invocations for one sub-task, but the outer agent only receives aggregated evidence and local failure signals (Yan et al., 5 Jun 2026). The technical report specifies that the inner Search Agent runs up to 10 retrieval rounds, with up to 3 sub-queries per round and 3 results per query (Yan et al., 5 Jun 2026). The outer planning loop is allowed up to 15 iterations (Yan et al., 5 Jun 2026).

A plausible implication is that this separation reduces the coupling between retrieval noise and global scheduling decisions. That interpretation is consistent with domain-specific findings in other DR work. DEEPMED argues that blindly scaling tool calls in medicine can inject noisy context and trigger repetitive evidence-seeking, motivating both a difficulty-aware turn penalty during RL and an Over-Evidence Monitor during inference (Wang et al., 26 Jan 2026). Although DuMate-DeepResearch is not specialized to medicine and does not use the same mechanisms, both systems treat uncontrolled search growth as destabilizing and seek architectural control over local exploration.

5. Rubric-grounded reasoning and test-time optimization

The third major mechanism is rubric-based test-time optimization (Yan et al., 5 Jun 2026). A rubric is defined as a set

$s_t = \langle z,\; p_t,\; e_t,\; \rho_t \rangle$1

whose criteria are tuples

$s_t = \langle z,\; p_t,\; e_t,\; \rho_t \rangle$2

where the guidance field provides actionable instructions rather than scalar scores (Yan et al., 5 Jun 2026). In DuMate-DeepResearch, rubrics are not only used for ex post evaluation; they are injected into planning, search, and writing as live reasoning scaffolds (Yan et al., 5 Jun 2026).

The framework distinguishes persistent and ephemeral rubrics. A persistent rubric is generated once from the topic and outline,

$s_t = \langle z,\; p_t,\; e_t,\; \rho_t \rangle$3

and encodes long-horizon quality criteria for the report, such as cross-validation of vendor claims, quantitative comparison, or discussion of limitations and risks (Yan et al., 5 Jun 2026). An ephemeral rubric is refreshed after each cycle from the outline and evidence,

$s_t = \langle z,\; p_t,\; e_t,\; \rho_t \rangle$4

and captures current evidence gaps, inconsistencies, or sections requiring further support (Yan et al., 5 Jun 2026). The active rubric state is

$s_t = \langle z,\; p_t,\; e_t,\; \rho_t \rangle$5

Planning and writing are explicitly conditioned on rubrics: $s_t = \langle z,\; p_t,\; e_t,\; \rho_t \rangle$6 for Planner actions, and

$s_t = \langle z,\; p_t,\; e_t,\; \rho_t \rangle$7

for final report synthesis (Yan et al., 5 Jun 2026). The ephemeral rubric also participates in stopping: if $s_t = \langle z,\; p_t,\; e_t,\; \rho_t \rangle$8 indicates that there is “no outstanding gap,” the loop can terminate early (Yan et al., 5 Jun 2026).

This mechanism aligns with a broader trend toward rubric-driven DR evaluation and optimization. DeepResearch Bench II introduced 9,430 fine-grained binary rubrics derived from expert-written reports and used them to diagnose deficiencies in information recall, analysis, and presentation (Li et al., 13 Jan 2026). “Learning Query-Specific Rubrics from Human Preferences for DeepResearch Report Generation” proposed RL-trained rubric generators and a Multi-agent Markov-state workflow, showing that query-specific rubrics deliver more discriminative and human-aligned supervision than generic rubric strategies (Lv et al., 3 Feb 2026). DuMate-DeepResearch differs in that its rubrics are generated and consumed at test time as guidance signals, rather than primarily as training rewards, but these lines of work converge on the view that fine-grained criteria are necessary for reliable DR behavior.

6. Tool ecosystem, evidence handling, and benchmark performance

The Tool Ecosystem includes Baidu Search integration and report rendering tools (Yan et al., 5 Jun 2026). Search actions expose query expansion, search, URL crawling, page extraction, and evidence normalization, while report rendering proceeds through a pivot report that can then be converted to formats such as Markdown, HTML, or PPT (Yan et al., 5 Jun 2026). Each tool output is standardized as an evidence record containing content snippet, URL, metadata when available, and a short summary (Yan et al., 5 Jun 2026). The global evidence base $s_t = \langle z,\; p_t,\; e_t,\; \rho_t \rangle$9 is monotone growing and is available to the Planner, Writer, and rubric generators (Yan et al., 5 Jun 2026).

The framework distinguishes tool-based and lightweight reasoning tasks in its graph. Tasks of type LLM are restricted to operations such as deduplication, summarization, cross-consistency checks, or computations over existing evidence and are not permitted to generate final conclusions outside the report stage (Yan et al., 5 Jun 2026). This design appears intended to preserve evidence-groundedness by reserving final synthesis for the Writer and by logging intermediate transformations.

The system is evaluated on two benchmarks. DeepResearch Bench consists of 100 tasks across 22 domains and uses Reference-based and Adaptive Criteria-driven Evaluation with dimensions of comprehensiveness, insight, instruction following, and readability (Yan et al., 5 Jun 2026). DeepResearch Bench II contains 132 tasks across 22 domains and evaluates long-form reports by 9,430 fine-grained binary rubrics across information recall, analysis, and presentation (Li et al., 13 Jan 2026). On DeepResearch Bench, DuMate-DeepResearch reports 59.48 for comprehensiveness, 61.48 for insight, 53.87 for instruction following, 54.34 for readability, and 58.03% overall, which is reported as the best overall score (Yan et al., 5 Jun 2026). On DeepResearch Bench II, it reports 57.58 for information recall, 71.70 for analysis, 89.89 for presentation, and 61.95% overall, also reported as the best overall score (Yan et al., 5 Jun 2026).

Ablations show that removing rubric guidance from the report stage reduces overall score from 58.03 to 57.61 on DeepResearch Bench, and removing rubric guidance from the entire pipeline reduces it further to 57.53 (Yan et al., 5 Jun 2026). Replacing only the report-generation LLM while keeping planning and search fixed leads to larger drops, with DeepSeek V4 Pro at 57.21, GLM 5.1 at 56.69, Qwen-3.7 Max at 55.55, and MiniMax-M3 at 55.21 (Yan et al., 5 Jun 2026). This suggests that while planning, recursive search, and rubric scaffolding matter, final synthesis quality remains sensitive to the report-stage model.

Benchmark context clarifies the significance of these results. DeepResearch Bench was introduced as a benchmark of 89 multi-step web research tasks with a frozen RetroSearch environment, and strong live-web ReAct agents achieved scores in the approximate range of 0.45–0.51, still below a rough human noise ceiling near 0.8 (FutureSearch et al., 6 May 2025). DeepResearch Bench II later showed that even frontier commercial DRSs satisfy fewer than 50% of fine-grained rubrics, with the best reported TotalScore at 45.40 for OpenAI-GPT-o3 Deep Research in the benchmark paper’s comparison table (Li et al., 13 Jan 2026). DuMate-DeepResearch’s reported 61.95% on DeepResearch Bench II therefore implies a substantial improvement under that rubric-centric evaluation regime. This suggests, though does not by itself prove, that auditability, recursive search isolation, and rubric-guided planning combine effectively under fine-grained report evaluation.

7. Relation to adjacent deep research architectures and limitations

DuMate-DeepResearch belongs to a rapidly evolving family of DR systems that differ in how they represent state, control search, and supervise report quality. Deep Researcher Reflect-Evolve emphasizes sequential research plan refinement, a centralized Global Research Context, Candidates Crossover, and one-shot report generation, reporting an overall score of 46.21 on DeepResearch Bench (Prateek, 28 Jan 2026). S1-DeepResearch instead focuses on trajectory construction and supervised training for a native deep research model, spanning long-horizon reasoning, instruction following, report generation, file understanding, and skill usage across 20 benchmarks (Dong et al., 13 Jun 2026). DualGraph separates memory into an Outline Graph and Knowledge Graph and reports 53.08 RACE on DeepResearch Bench with GPT-5 (Shi et al., 14 Feb 2026). NoveltyAgent specializes DR methods to paper novelty analysis by using point-wise novelty decomposition, citation-graph-grounded retrieval, and self-validation, outperforming GPT-5 DeepResearch by 10.15% on its checklist-based evaluation (Hou et al., 21 Mar 2026).

Against this background, DuMate-DeepResearch’s distinctive contribution is the combination of a graph-based planner, recursive two-level search execution, and test-time rubric scaffolding within an explicitly auditable Foundry architecture (Yan et al., 5 Jun 2026). Unlike training-centric approaches such as query-specific rubric RL (Lv et al., 3 Feb 2026) or evidence-grounded RL against tool-call hacking (Ma et al., 13 Oct 2025), the technical report emphasizes inference-time system design rather than policy optimization. That leaves several plausible limitations.

The report notes computational cost, since 15 outer iterations with many inner Search Agents can be expensive, although parallel execution mitigates wall-clock latency (Yan et al., 5 Jun 2026). It also implies sensitivity to tool quality because Baidu Search is the retrieval backbone, and weak or sparse domains may still lead the rubric system to premature “no gap” judgments (Yan et al., 5 Jun 2026). The planning and exploration balance remains heuristic: overly aggressive stopping or over-expansion are both possible (Yan et al., 5 Jun 2026). Finally, report quality is still strongly dependent on the report-stage LLM, as shown by the model replacement ablations (Yan et al., 5 Jun 2026).

Other recent work points to adjacent risks. Proof-of-Use argues that RL-trained DeepResearch agents can exhibit tool-call hacking, including mode collapse onto a single tool and spurious grounding, and proposes step-wise contracts, perturbation rewards, and answer-evidence alignment to enforce causal use of retrieved evidence (Ma et al., 13 Oct 2025). This suggests a possible future extension of DuMate-DeepResearch: the auditable traces, explicit evidence records, and rubric-conditioned planning could in principle be paired with evidence-grounded RL or verification-style training. Similarly, DeepResearch-ReportEval emphasizes report-level quality, redundancy, and factuality, arguing that deep research systems are evaluated most naturally through their reports rather than isolated subskills (Fan et al., 9 Oct 2025). DuMate-DeepResearch’s auditable report pipeline appears compatible with that perspective, although the technical report evaluates primarily on DeepResearch Bench and DeepResearch Bench II (Yan et al., 5 Jun 2026).

Overall, DuMate-DeepResearch can be characterized as an auditable, multi-agent DR framework whose core technical claim is that explicit graph-based planning, recursive delegation of noisy search subtasks, and rubric-conditioned reasoning materially improve long-form research performance on public DR benchmarks (Yan et al., 5 Jun 2026). Its results situate it at the intersection of system architecture, benchmark-driven evaluation, and process transparency, rather than purely model-scale or training-scale advances.