Open Deep Search: Democratizing Search with Open-source Reasoning Agents (2503.20201v1)

Abstract: We introduce Open Deep Search (ODS) to close the increasing gap between the proprietary search AI solutions, such as Perplexity's Sonar Reasoning Pro and OpenAI's GPT-4o Search Preview, and their open-source counterparts. The main innovation introduced in ODS is to augment the reasoning capabilities of the latest open-source LLMs with reasoning agents that can judiciously use web search tools to answer queries. Concretely, ODS consists of two components that work with a base LLM chosen by the user: Open Search Tool and Open Reasoning Agent. Open Reasoning Agent interprets the given task and completes it by orchestrating a sequence of actions that includes calling tools, one of which is the Open Search Tool. Open Search Tool is a novel web search tool that outperforms proprietary counterparts. Together with powerful open-source reasoning LLMs, such as DeepSeek-R1, ODS nearly matches and sometimes surpasses the existing state-of-the-art baselines on two benchmarks: SimpleQA and FRAMES. For example, on the FRAMES evaluation benchmark, ODS improves the best existing baseline of the recently released GPT-4o Search Preview by 9.7% in accuracy. ODS is a general framework for seamlessly augmenting any LLMs -- for example, DeepSeek-R1 that achieves 82.4% on SimpleQA and 30.1% on FRAMES -- with search and reasoning capabilities to achieve state-of-the-art performance: 88.3% on SimpleQA and 75.3% on FRAMES.

Open Deep Search (ODS) is presented as a framework designed to enhance open-source LLMs with sophisticated search and reasoning capabilities, aiming to reduce the performance disparity compared to proprietary systems like Perplexity Sonar Reasoning Pro and OpenAI's GPT-4o Search Preview (Alzubi et al., 26 Mar 2025 ). The core contribution lies in augmenting a user-selected base LLM with two specialized components: an Open Reasoning Agent (ORA) and an Open Search Tool (OST). This modular design allows for integration with various open-source LLMs, thereby "democratizing" access to advanced search functionalities previously dominated by closed systems.

ODS Architecture and Components

The ODS framework operates by integrating an Open Reasoning Agent (ORA) and an Open Search Tool (OST) with a base LLM chosen by the user. The architecture facilitates a dynamic interplay where the LLM's generative and understanding capabilities are coupled with structured reasoning and targeted information retrieval.

1. Base LLM: Any capable open-source LLM can serve as the foundation (e.g., DeepSeek-R1 is mentioned). Its role is to understand the initial query, process information retrieved by the tools, and synthesize the final response.
2. Open Reasoning Agent (ORA): This component acts as the orchestrator. Upon receiving a task or query, the ORA interprets it and formulates a plan, which involves a sequence of actions. These actions can include invoking various tools, critically including the OST. The ORA manages the flow of information between the LLM and the tools, deciding when to search, what to search for, and how to integrate the search results back into the reasoning process.
3. Open Search Tool (OST): This is described as a novel web search tool designed to outperform existing proprietary search APIs or tools used by other systems. It handles the interaction with web search infrastructure, likely involving query formulation/rewriting, result fetching, snippet extraction, and potentially re-ranking or filtering based on relevance to the ORA's specific information need at that point in the reasoning chain.

The interaction flow can be conceptualized as follows: User Query -> Base LLM -> ORA (interprets task, plans actions) -> ORA decides to search -> ORA calls OST with specific query -> OST executes search, retrieves/processes results -> OST returns results to ORA -> ORA provides results/context to Base LLM -> Base LLM processes/synthesizes -> ORA potentially initiates further actions or generates final response via Base LLM.

graph LR A[User Query] --> B(Base LLM); B --> C{Open Reasoning Agent (ORA)}; C -- Interprets Task & Plans --> C; C -- Decides to Search --> D[Open Search Tool (OST)]; D -- Executes Web Search --> E((Web/Search Index)); E -- Returns Raw Results --> D; D -- Processes Results --> D; D -- Returns Formatted Snippets --> C; C -- Provides Context/Results --> B; B -- Synthesizes Information --> B; C -- Further Actions? --> C; C -- Generate Final Response --> F[Final Answer]; B --> F;

Open Search Tool (OST) Functionality

While the paper abstract (Alzubi et al., 26 Mar 2025 ) doesn't detail the internal mechanics of the OST, it explicitly claims superior performance compared to proprietary counterparts. This suggests the OST likely incorporates advanced techniques beyond simple API calls to standard search engines. Potential functionalities could include:

• Adaptive Query Formulation: Dynamically generating or refining search queries based on the ORA's reasoning state and the information needed.
• Multi-Source Retrieval: Aggregating results from multiple search backends or indices.
• Intelligent Snippet Extraction: Identifying and extracting the most relevant passages from retrieved web pages, possibly using auxiliary models.
• Fact Verification/Cross-Referencing: Comparing information across multiple sources to improve reliability.
• Result Re-ranking: Ordering search results based on relevance determined by the context of the ORA's current sub-task, not just keyword matching.

Implementing the OST would require access to a web search index (e.g., via APIs like Bing, Google, or potentially open-source indices) and significant engineering to build the surrounding logic for query processing, result filtering, and relevance assessment.

Open Reasoning Agent (ORA) Implementation

The ORA is central to ODS's capability. It functions as a meta-controller coordinating the LLM and tools. Implementing the ORA likely involves techniques from agent-based LLM systems, such as:

• Prompt Engineering: Designing sophisticated prompts that instruct the base LLM to act as the reasoning agent, decompose problems, select appropriate tools (like the OST), and format tool calls.
• ReAct Framework (or similar): Employing paradigms like "Reasoning and Acting" where the agent explicitly verbalizes its reasoning steps, plans actions, executes them (calls tools), observes the results, and adjusts its plan accordingly within a loop.
• Tool Definition and Integration: Defining the available tools (OST and potentially others like calculators, code interpreters) in a format the LLM can understand and invoke, often involving parsing structured outputs (e.g., JSON) from the tools.
• State Management: Maintaining the context and history of the reasoning process, including previous actions, observations, and intermediate thoughts, to guide subsequent steps.

An illustrative pseudocode for the ORA loop might look like:

def ora_process(query, base_LLM, available_tools): """ Conceptual implementation of the Open Reasoning Agent loop. """ state = {"history": [], "plan": None, "observations": []} prompt = construct_initial_prompt(query, available_tools) while not task_is_complete(state): # 1. Reason step (using the base LLM) reasoning_prompt = update_prompt_with_state(prompt, state) LLM_output = base_LLM.generate(reasoning_prompt) thought, action_request = parse_LLM_output(LLM_output) # e.g., parse thought process and tool call state["history"].append({"thought": thought, "action": action_request}) if action_request: # 2. Action step tool_name = action_request["tool_name"] tool_input = action_request["input"] if tool_name in available_tools: tool = available_tools[tool_name] # e.g., tool is the OST instance observation = tool.execute(tool_input) state["observations"].append(observation) else: state["observations"].append(f"Error: Tool '{tool_name}' not found.") # Potentially add error handling / replanning logic else: # No action requested, assume LLM is providing final answer or needs to refine plan pass # Logic to determine if task is complete or requires more reasoning # Update state and check termination condition update_task_completion_status(state) final_answer_prompt = construct_final_answer_prompt(state) final_answer = base_LLM.generate(final_answer_prompt) return final_answer

This loop structure allows the agent to iteratively refine its understanding and gather necessary information via the OST before synthesizing the final output using the base LLM.

Evaluation and Performance Claims

ODS was evaluated on two question-answering benchmarks: SimpleQA and FRAMES. The key performance claims are:

• Baseline Improvement: ODS significantly improves upon existing open-source LLM baselines when augmented with its search and reasoning capabilities. For instance, DeepSeek-R1 achieves 82.4% accuracy on SimpleQA and 30.1% on FRAMES alone. With ODS integration, its performance jumps to 88.3% on SimpleQA and 75.3% on FRAMES.
• Competitive with SOTA: The results position ODS as competitive with, and sometimes superior to, state-of-the-art proprietary systems. Notably, on the FRAMES benchmark, ODS is reported to outperform the GPT-4o Search Preview baseline by 9.7% in accuracy (achieving 75.3% vs. a presumed baseline around 65.6%, although the exact baseline number isn't provided in the abstract).
• Generalizability: The framework is presented as general, capable of enhancing any LLM, suggesting its architecture is not tied to a specific model.

Table: Reported Performance on Benchmarks

System	Benchmark	Metric	Score
DeepSeek-R1 (Base)	SimpleQA	Accuracy	82.4%
DeepSeek-R1 + ODS	SimpleQA	Accuracy	88.3%
DeepSeek-R1 (Base)	FRAMES	Accuracy	30.1%
DeepSeek-R1 + ODS	FRAMES	Accuracy	75.3%
GPT-4o Search Preview (Baseline)	FRAMES	Accuracy	~65.6% (inferred)

These results suggest that the combination of the ORA's structured reasoning and the OST's effective information retrieval provides substantial gains, particularly on complex tasks represented by the FRAMES benchmark, which likely require multi-step reasoning and integration of information from multiple sources.

Practical Implications and Deployment

The primary implication of ODS is enabling the development of high-performance search and question-answering systems using entirely open-source components. This lowers the barrier to entry for organizations wanting advanced AI search capabilities without relying on proprietary APIs.

Implementation Considerations:

• LLM Choice: Performance will depend heavily on the chosen base LLM's reasoning and instruction-following capabilities. Models like DeepSeek-R1, Llama 3, or Mistral Large could be suitable candidates.
• OST Backend: The OST needs a backend search capability. This could involve commercial search APIs (Bing, Google), potentially incurring costs and rate limits, or integrating with open search indices (like Common Crawl) which requires significant infrastructure.
• Computational Resources: Running the base LLM and potentially the ORA logic (if it involves separate models or complex prompting) requires substantial compute resources (GPU memory and processing power), similar to hosting any large LLM. The OST might add further overhead depending on its implementation.
• Latency: The multi-step nature of the ORA (Reason -> Act -> Observe -> Reason...) can introduce latency compared to a single LLM call. Optimizing the interaction loop and the speed of the OST is crucial for real-time applications.
• Robustness: Agent-based systems can sometimes fail due to planning errors, incorrect tool usage, or infinite loops. Robust error handling and potentially mechanisms for plan correction are needed for reliable deployment.

ODS could be applied in various domains, including enterprise search, research assistants, customer support bots, and automated fact-checking systems, providing deeper insights and more accurate answers than standard keyword search or basic LLM question-answering.

Conclusion

Open Deep Search (ODS) presents a modular framework for augmenting open-source LLMs with advanced reasoning and web search capabilities (Alzubi et al., 26 Mar 2025 ). By combining an Open Reasoning Agent (ORA) for orchestration and an Open Search Tool (OST) for effective information retrieval, ODS demonstrates significant performance improvements on question-answering benchmarks, achieving results competitive with leading proprietary systems. Its open-source nature promises to make sophisticated AI search more accessible, though practical implementation requires careful consideration of LLM choice, search backend integration, computational resources, and system robustness.