SearchEyes: Towards Frontier Multimodal Deep Search Intelligence via Search World Simulation

Abstract: Training multimodal search agents to perform multi-hop reasoning remains challenging due to a fundamental structural disconnect: existing pipelines construct training data, search environments, and reward signals independently, causing synthesized structural metadata to be discarded, environments to rely on irreproducible external engines, and RL rewards to remain sparse at the trajectory level. We present \textbf{SearchEyes}, which uses a typed knowledge graph as the backbone of a \emph{simulated search world} that unifies all three components. We propose \textbf{Perception-Knowledge Chains (PKC)} to sample constrained multi-hop paths over the visual-knowledge intersection of Wikidata5M, retaining hop-level entity metadata that simultaneously defines a self-contained search world and step-level reward anchors. We further propose \textbf{Hop-Anchored Policy Optimization (HaPO)}, which reuses these anchors for step-level credit assignment without a separately trained process reward model. Experiments on six multimodal knowledge-intensive benchmarks show that SearchEyes achieves state-of-the-art performance among open-source multimodal search agents, with SearchEyes-27B improving over the strongest open-source baseline by 6.2 points on average.%

• The paper introduces SearchEyes, which unifies data synthesis, environment simulation, and RL credit assignment via typed knowledge graphs to enable robust multi-hop reasoning.
• It employs a Perception-Knowledge Chain synthesis method and Hop-Anchored Policy Optimization, achieving significant accuracy gains and efficient parameter scaling.
• Empirical results on six benchmarks demonstrate state-of-the-art performance, with competitive metrics against larger models using far fewer parameters.

SearchEyes: Unifying Multimodal Multi-Hop Search Agents via Knowledge Graph Simulation and Step-Level RL Credit

Motivation and Problem Formulation

The construction of multimodal search agents that perform robust multi-hop reasoning tasks is fundamentally impeded by a disconnect between data generation, environment simulation, and reward signal construction. Previous pipelines synthesize training data, design environments—typically dependent on external search APIs—and construct RL rewards independently. This fragmentation yields irreproducible, brittle environments and sparse, high-variance RL signals, thereby limiting policy learning efficacy for long-horizon compositional tasks.

SearchEyes addresses this disconnect by treating a typed knowledge graph as a unified simulation environment, data generator, and step-level reward anchor. This co-design paradigm leverages Wikidata5M entities—cross-joined with Wikipedia textual and visual content—to enable end-to-end agentic RL on frontier, open-domain, multimodal question-answering and reasoning objectives.

Figure 1: SearchEyes system overview, showing the knowledge graph as a unified backbone for question synthesis, environment interaction, and step-level rewards.

Methodology

SearchEyes advances a two-pronged solution: (1) Perception-Knowledge Chain (PKC) synthesis for training data and environment construction, and (2) Hop-Anchored Policy Optimization (HaPO) for RL-driven step-level policy optimization.

Knowledge Graph Construction & Self-Contained Environment

A typed entity-relation knowledge graph is built from intersected subsets of Wikidata5M, Wiki6M (entity text), and Wikipedia images, yielding 1.2M entities (340K with high-quality images) and 5.8M relation triples with entities typed by four semantic domains. This supports context-rich, deterministic retrieval: the agent’s tools (text_search, lookup, visual_search) operate over this closed-world environment, ensuring full reproducibility and tracking of ground-truth entity access.

Perception-Knowledge Chain Synthesis

PKC samples multi-hop paths in the knowledge graph constrained by (i) strict alternation between perception (visual grounding) and knowledge (text-based retrieval) hops; (ii) disambiguation constraints to enforce multi-branch reasoning (treewidth $\leq 2$); (iii) anti-shortcut and hub-avoidance filters; (iv) domain diversity requirements for robust compositional coverage.

This process outputs detailed structural metadata: each composed instance includes the full gold chain of entities to be traversed, ensuring intermediate states are annotated and disallowing training/evaluation shortcuts via leakage.

Figure 3: PKC path-sampling and data generation pipeline; all entities maintain path-aligned metadata, supporting environment construction and step-level credit.

RL and Policy Optimization

The agent undergoes a multi-stage training regime:

• Supervised Fine-Tuning (SFT): Trajectories are distilled from privileged rollouts (with retrieval boost and denoised observations), then the agent is trained via causal language modeling masking out environmental observations.
• Hop-Anchored Policy Optimization (HaPO): During RL, for each question and sampled batch of trajectories, the full chain of ground-truth entity “anchors” is used for credit assignment. Step-level advantages are constructed by grouping all trajectory steps that retrieve the same anchor entity, then computing group-relative outcomes. The resulting per-token advantage interpolates between trajectory-level (full episode reward) and hop-level (anchor-aligned) signals. Additional algorithmic improvements include fatal-aware masking of degenerate suffixes, one-sided clamping for positive advantage retention, and smooth asymmetric gating to control gradient flow and avoid large destructive updates.

  Figure 2: Complete post-training pipeline with SFT and HaPO stages. Step-level anchors enable low-variance, multi-hop credit assignment.

Empirical Results

State-of-the-Art Performance

SearchEyes is evaluated on six established multimodal knowledge-intensive QA/web-search benchmarks (SimpleVQA, VDR, MMSearch, LiveVQA, BrowseComp-VL, FVQA) and the bespoke, held-out VisSearch Bench (guaranteed multi-hop visual structure). Key results:

• SearchEyes-27B achieves 82.4 on MMSearch and 79.1 on FVQA, competitive with Gemini-3.1-Pro using 6× fewer parameters, and surpasses all open-source baselines by 6.2 points.
• SearchEyes-9B matches or exceeds systems with $\sim$3-4× larger backbones (e.g., OpenSearch-VL-30B at 59.8%).

  Figure 4: Model scale/accuracy trade-off; SearchEyes sets an open-source state-of-the-art frontier in both overall and parameter efficiency.

Notably, PKC/HAPO ablations reveal that P–K alternation and anti-shortcut filtering in synthesis, and hop-anchored credit in training, yield the largest accuracy gains (up to +7.7/-4.2 versus removing constraints, and +4.0 over standard GRPO). Baselines using unconstrained web/corpus sampling or trajectory-only reward lag far behind.

Transfer and Generalization

On VisSearch Bench, which specifically evaluates multi-hop and modality alternation, proprietary models (including GPT-5, Kimi-K2.5) score below 10%, confirming that SearchEyes’ explicit structural constraints and anchor-based training transfer to genuinely hard, compositional reasoning objectives.

Theoretical and Practical Implications

SearchEyes establishes that knowledge graph simulation enables joint optimization of data quality, environment fidelity, and RL credit assignment. The design removes dependencies on irreproducible web APIs, yielding stable, scalable environments for step-level policy improvement and resource-efficient model scaling. The use of hop-anchored credit eliminates the need for bespoke process reward models, yielding a practical blueprint for RL on long-horizon compositional tasks. The joint P–K alternation and fine-grained filtering ensure the learned policy generalizes beyond shallow tool invocation to true multi-modal reasoning, as supported by both quantitative and qualitative results.

Future Directions

The approach sets a new benchmark for environment and data co-design in agentic multimodal RL. Future research directions include expanding knowledge graph coverage, incorporating richer ontological structures (ontology-based constraints, dynamic relation typing), scaling to massively parallel simulated worlds for self-evolving curriculum learning, and transferring these techniques to reinforcement learning from human feedback (RLHF) pipelines for robust real-world compositional generalization.

Conclusion

SearchEyes introduces a unified, structurally grounded framework for training multimodal multi-hop search agents by tightly coupling data synthesis, environment simulation, and step-level RL signals through typed knowledge graphs. Empirical results show substantial gains over prior open-source and proprietary methods, indicating the critical role of structural metadata preservation and credit assignment for future agentic AI development.

For comprehensive evaluation and ablation records, see "SearchEyes: Towards Frontier Multimodal Deep Search Intelligence via Search World Simulation" (2607.05943).