Vision-DeepResearch Paradigm

• The Vision-DeepResearch paradigm is a multimodal framework characterized by iterative, cross-modal reasoning that integrates visual and textual evidence for high-reliability decision making.
• It employs multi-round visual localization combined with textual retrieval, leveraging synthetic supervision and reinforcement learning to optimize complex reasoning tasks.
• Empirical results demonstrate significant performance improvements over static models, underscoring its effectiveness in handling noisy, real-world environments.

The Vision-DeepResearch paradigm defines a new multimodal agentic framework integrating LLMs, advanced visual perception, tool use, and iterative search to achieve high-reliability, evidence-grounded reasoning in complex real-world scenarios requiring coupled vision and language understanding. Agentic MLLMs under this paradigm perform dozens of interleaved reasoning and tool-call steps—alternating multi-entity, multi-scale visual queries with textual search—and are trained end-to-end via a combination of synthetic supervision and reinforcement learning to internalize the entire search–ground–reason–synthesize process. This paradigm breaks from static dataset or pipeline models, emphasizing long-horizon, noisy-environment research that robustly localizes, retrieves, aggregates, and verifies visual and textual evidence across non-trivial search spaces (Huang et al., 29 Jan 2026, Zeng et al., 2 Feb 2026, Geng et al., 7 Aug 2025, Zheng et al., 20 May 2025).

1. Formal Definition and Conceptual Foundations

A Vision-DeepResearch agent formally operates on inputs $(I, q)$, where $I$ is an image and $q$ a textual query, and outputs an answer $a$ through iterative, interleaved visual and textual search and reasoning. Its trajectory is structured as:

$$C_{\mathrm{multimodal}} = \{I, q,\ \langle R_t^v, A_t^v, O_t^v\rangle_{t=1}^{T_v},\ \langle R_u^t, A_u^t, O_u^t\rangle_{u=1}^{T_t},\ a_{\text{output}}\}$$

where $R$ are model-generated reasoning states, $A$ are tool-call actions (either vision- or text-search), and $O$ are observations. The agent:

• Alternates (multi-turn) between visual grounding (entity proposal, bounding-box crops, multi-scale image querying) and text-based evidence gathering.
• Aggregates retrieved evidence into memory states $V_t^v$, $V_t^t$ at each round.
• Terminates visual search adaptively using a learned judge model, then switches to pure textual search when entity-level visual information is exhausted (Huang et al., 29 Jan 2026, Zeng et al., 2 Feb 2026).

The paradigm is fundamentally iterative and cross-modal, rejecting single-pass retrieval or one-shot QA in favor of active, multi-stage evidence collection and grounding.

2. Pipeline Architecture and Core Algorithmic Workflow

The canonical Vision-DeepResearch workflow comprises:

1. Planning and Subgoal Decomposition: MLLM-based planner decomposes $q$ into subgoals $s_i$ (e.g., “identify object in image,” “locate logo,” “connect to knowledge graph”).
2. Multi-round Visual Localization: For each subgoal, propose entity bounding boxes $B_t = \{b_1,\dots,b_n\}$; perform multi-scale cropping for each entity and run vision-search tool calls ($A_t^v$).
3. Textual Retrieval and Reasoning: After visual evidence $V_t^v$ is deemed sufficient (by judge model $h_t^v$), generate textual queries to external search engines, iteratively refine based on accumulated memory $V_t^t$.
4. Evidence Aggregation: At each round, update multimodal memory $M_r = M_{r-1} \cup \{E_r, R_r\}$ with new entities $E_r$ and retrieved data $R_r$; combine scores $\alpha S_{\text{vis}} + \beta S_{\text{text}}$ to guide subsequent tool calls.
5. LLM-based Synthesis: Final answer $a_{\text{output}}$ generated based on the entire memory of visual and textual context accumulated (Huang et al., 29 Jan 2026, Zeng et al., 2 Feb 2026).

Algorithmic details emphasize asynchronous multi-threaded rollouts of tool calls, adaptive cropping and search, and integration of both intermediate observations and auxiliary (e.g., summarization, code interpreter) LLM tools (Huang et al., 29 Jan 2026, Geng et al., 7 Aug 2025).

3. Benchmarking, Dataset Construction, and Empirical Results

The Vision-DeepResearch paradigm is benchmarked using question sets specifically designed to prevent shortcut exploitation and demand real visual grounding, multi-round search, and cross-modal evidence fusion. The “Vision-DeepResearch Benchmark” (VDR-Bench (Zeng et al., 2 Feb 2026)) exemplifies this, with:

• 2,000 questions spanning 10 domains; entity-level cropping and human VLM-verified ground truths.
• Multi-hop complexity enforced by random-walk-based knowledge graph expansions.
• Three protocol settings: Direct Answer (no search), Cropped-Image Search + Text Search (CIS+TS), and CIS+TS with forced multi-round visual reasoning (MVF).

Empirical findings:

• Baseline models (Gemini-2.5-Pro, GPT-5, Claude-4-Sonnet) achieve only 5–10% “direct” accuracy, but >2x improvement with CIS+TS and additional 13–15 percentage point boost using MVF.
• SOTA open Vision-DeepResearch-30B model attains 56.9% average accuracy, outperforming closed and open baselines under these demanding conditions (Huang et al., 29 Jan 2026).

4. Optimization: Training Paradigms and Reward Schemes

Internalization of deep-research capabilities in Vision-DeepResearch agents proceeds via:

• Large-scale synthetic supervision (SFT): 30,000 curated multimodal trajectories plus auxiliary VQA-driven and synthetic fuzzy QA data.
• Reinforcement learning (RL): Fine-tuning using Group Relative Policy Optimization (GRPO), where the episode-level reward is binary correct/incorrect graded by an LLM-as-judge, and trajectory-level metrics (format compliance, tool usage) provide inductive incentive for high-efficiency, low-redundancy tool calling (Huang et al., 29 Jan 2026, Zheng et al., 20 May 2025).
• Reward design: Conditional rewards—granting tool-use bonus only for correct answers—lead to emergent exploration → efficient exploitation behavior, including “plan first, zoom only when needed” policies and significant improvements in grounding accuracy and hallucination mitigation (Zheng et al., 20 May 2025).

5. Advancements, Limitations, and Representative Model Behaviors

Vision-DeepResearch agents demonstrate:

• Highly distributed, long-horizon reasoning (dozens of agent steps, $T_{\text{max}}\approx 50$; $>100$ tool calls per episode).
• Multi-entity, multi-scale visual search, with entity recall tightly correlated with final answer accuracy.
• Coordinated vision–language planning, memory updating, and learning of when to stop search or switch modalities.
• Broad, compositional generalization: extracting, comparing, and verifying cross-modal evidence chains over complex search environments (Huang et al., 29 Jan 2026, Geng et al., 7 Aug 2025, Zheng et al., 20 May 2025).

Limitations include sparse environment rewards in harder settings, RL instability, fixed toolset constraints (most implementations do not yet support e.g., dynamic web interactions, segmentation tools), and grounding drift under difficult visual perturbations. Open challenges include adaptive cropping, richer reward shaping, integration of dynamic web tools, and video/temporal extension (Zeng et al., 2 Feb 2026, Geng et al., 7 Aug 2025).

6. Comparison with Related Paradigms and Broader Research Context

Vision-DeepResearch extends the DeepResearch framework from text to vision and multimodal domains (Zhang et al., 18 Aug 2025, Xu et al., 14 Jun 2025), enforcing evidence-grounded, structured reasoning:

• Contrasts with pipeline models (fixed pre/post-perception) and static visual QA, enabling perception and reasoning to be interleaved, RL-optimized, and memory-integrated.
• Benchmarks such as BrowseComp-VL, VDR-Bench, and HLE-VL assess not just accuracy but also tool-use patterns, entity recall, and reliability under “Google-proof” and noisy conditions (Geng et al., 7 Aug 2025, Zeng et al., 2 Feb 2026).
• Models like DeepEyes further illustrate the paradigm of end-to-end RL with interleaved multimodal CoT and reward-driven emergence of “thinking with images” (Zheng et al., 20 May 2025).

The paradigm also serves as a testbed for broader advances in capability-attributed data curation (Li et al., 27 Sep 2025), information-theoretic and spatiotemporal learning (Betti et al., 2022), and biologically plausible system-level integration (Wei et al., 2021).

7. Design Guidelines, Metrics, and Future Trajectories

Practical design best practices for Vision-DeepResearch include:

• Enforcing entity-level, multi-round visual localization rather than relying on whole-image or single-pass retrieval.
• Balancing visual and textual cues in evidence aggregation and using knowledge-graph expansions to generate non-trivial multi-hop queries.
• Training and evaluating agents with both semantic accuracy and entity recall to preclude priors or shortcut exploitation.
• RL-based reward shaping and adaptive control to promote both accuracy and efficiency.

Current metrics used include answer accuracy, entity recall, answer-explanation faithfulness, tool-use entropy, and LLM-based semantic grading (Zeng et al., 2 Feb 2026, Huang et al., 29 Jan 2026).

Open research directions highlight RL-learned cropping policies, end-to-end optimization of perception–reasoning–retrieval loops, entity-level memory and matching beyond LLM-judged recall, extension to video and temporal multimodal sequences, and richer tool and evidence source suites (e.g., 3D models, AR scene graphs).

The Vision-DeepResearch paradigm crystallizes as a tightly coupled, cross-modal agentic system that unifies perception, search, memory, reasoning, and controlled synthesis under a single, iteratively learned architecture—enabling progress on a spectrum of high-complexity, real-world tasks that require both depth of reasoning and robust vision-language integration (Huang et al., 29 Jan 2026, Zeng et al., 2 Feb 2026, Geng et al., 7 Aug 2025, Zheng et al., 20 May 2025).