Nomad: Autonomous Exploration and Discovery

Abstract: We introduce Nomad, a system for autonomous data exploration and insight discovery. Given a corpus of documents, databases, or other data sources, users rarely know the full set of questions, hypotheses, or connections that could be explored. As a result, query-driven question answering and prompt-driven deep-research systems remain limited by human framing and often fail to cover the broader insight space. Nomad addresses this problem with an exploration-first architecture. It constructs an explicit Exploration Map over the domain and systematically traverses it to balance breadth and depth. It generates and selects hypotheses and investigates them with an explorer agent that can use document search, web search, and database tools. Candidate insights are then checked by an independent verifier before entering a reporting pipeline that produces cited reports and higher-level meta-reports. We also present a comprehensive evaluation framework for autonomous discovery systems that measures trustworthiness, report quality, and diversity. Using a corpus of selected UN and WHO reports, we show that Nomad produces more trustworthy and higher-quality reports than baselines, while also producing more diverse insights over several runs. Nomad is a step toward autonomous systems that not only answer user questions or conduct directed research, but also discover which questions, research directions, and insights are worth surfacing in the first place.

• The paper demonstrates that Nomad’s autonomous exploration and explicit map construction enable discovery beyond traditional user-specified queries.
• It employs both bottom-up and top-down strategies to build a hierarchical Topic Tree, integrated with an explorer–verifier loop for evidence-backed insight validation.
• Evaluations reveal that Nomad outperforms baseline LLM agents with superior numeric grounding, factuality, and inter-report diversity.

Nomad: Autonomous Data Exploration and Discovery

Motivation and Problem Statement

"Nomad: Autonomous Exploration and Discovery" (2603.29353) targets a core limitation in current LLM-based research agents: the inability of query- or prompt-driven systems to autonomously expand beyond user-specified directions, thereby missing non-obvious, high-value insights present in a corpus. Traditional systems confine exploration to the information space demanded by the user’s initial query, resulting in narrow coverage and high redundancy. Nomad is explicitly architected to autonomously traverse and interrogate the full expanse of a document or data corpus by constructing and systematically traversing an explicit Exploration Map, aiming for maximal coverage, diversity, and validated insight generation.

Figure 1: Query-oriented research approaches (left) examine only the hypothesized portions of a corpus; Nomad’s exploration-first model (right) autonomously constructs a domain-wide map, traverses it, and surfaces diverse, evidence-backed insights.

System Architecture and Core Components

At its core, Nomad implements a pipeline grounded on an explicit Exploration Map, a structured, dynamically constructed, heterogeneous graph over topics, concepts, hypotheses, insights, and underlying documents.

Figure 2: Nomad pipeline—explicit map construction, breadth-prioritized topic selection, hypothesis generation, explorer–verifier loop, and evidence-audited reporting.

Exploration Map Construction

The Exploration Map is realized via both bottom-up and top-down mechanisms:

• Bottom-up: When a corpus is available, Nomad first extracts and disambiguates concept nodes across all documents (Figure 3). It then clusters and organizes these concepts into a hierarchical Topic Tree (Figure 4), using hybrid embedding-based clustering and LLM-guided tree construction for scalable handling of large concept sets.

  Figure 3: Post-concept layer construction—documents are linked to unified, disambiguated concepts; hypotheses are generated per concept node based on insight potential.

  

  Figure 4: Full exploration map—concepts are nested within the Topic Tree; the map is traversed breadth-first for maximal topical diversity.

• Top-down: When only a goal is specified, Nomad prompts LLMs—informed by web search or global knowledge—to construct a hierarchical Topic Tree reflecting the goal’s diverse subtopics.
• Both approaches can be dynamically expanded as new data arrive or as topical drift is detected; web search-based expansion is orchestrated using seed-guided topic insertion (Figure 5).

  Figure 6: Real WHO exploration map instance showcasing non-uniform branching and depth, driven by insight potential evaluation and shaped by both document-derived and goal-directed topics.

  

  Figure 7: Comparative construction: top-down yields globally broad but coarse maps, while bottom-up is corpus-specific, granular, and shaped by emergent insight potential.

Topic Selection and Hypothesis Generation

Nomad employs a breadth-first traversal strategy for the Topic Tree, prioritizing heterogeneity, and tracks both "insight potential" (available unexplored hypothesis count) and exploration depth per topic node. Upon reaching topical frontiers with low unexplored hypotheses, new hypotheses are batch-generated by LLMs, graded for relevance, impact, and diversity via composite, metric-weighted scoring.

Explorer–Verifier Loop

Insight generation and validation are rigorously decoupled. The explorer agent operates in an explicit observe–reason–act loop, leveraging tool subagents (document search, SQL, web search), each exposing a uniform interface but encapsulating complex retrieval and composition logic internally. Upon surfacing a candidate insight, the independent verifier decomposes it into atomic sub-claims and independently validates each via tool calls, awarding a per-claim faithfulness score.

Figure 8: Explorer–verifier loop: LLM explorer iteratively collects evidence; verifier decomposes claims and cross-examines independently, feeding actionable feedback for further refinement or proceeding to report generation upon successful validation.

Figure 9: AI-generated summary of explorer–verifier turn sequence for a real-world insight, demonstrating iteration and refinement enforced by externalized reasoning and validation cycles.

Reporting, Meta-Reporting, and Auditing

Nomad’s reporting phase assembles finalized insights and their traceable evidence into multi-section, rigorously structured, and citation-audited documents. Report composition begins with outline generation, followed by targeted section filling, presentation of data in diverse forms (texts, tables, charts), and a downstream citation-auditing pass to enforce evidentiary rigor.

Figure 10: Report generation workflow—from structured outline and insight database, content is generated, audited, and synthesized into compact, consumable artifacts (e.g., executive posters).

Meta-reports aggregate pools of insights using entity-based, thematically clustered, and multi-stage refinement synthesis for higher-level thematic overviews.

Figure 11: Meta-report pipeline—highlights are distilled per audience, grouped into entity-aligned clusters, and synthesized in a progressive structure–depth–focus sequence.

Figure 12: Example synthesized meta-report for the WHO analyst instance, merging diverse reports into an executive-level thematic overview.

Evaluation Framework and Comparative Results

Nomad introduces a comprehensive evaluation suite quantifying trustworthiness (numeric grounding, factuality), quality (analytical depth, coverage, originality, actionability, presentation), intra-report distinctness (low redundancy), and inter-report diversity (embedding-based coverage of unique insights across runs).

Key results:

• Numeric Grounding: Nomad achieves 70.4% (bottom-up) and 66.3% (top-down), compared to 17.3–46.2% for GPT Researcher and 34.9–64.1% for o3-deep-research, reflecting substantial gains in evidence-backed quantification.
• Factuality: 65.2–65.9% versus 28.5–52.2% (GPT Researcher) and 40.2–73.3% (o3-deep-research), with Nomad especially superior in settings where full-corpus exploration is critical.
• Quality: Aggregated scores (61.0–62.6%) exhibit strong gains in originality (+20–40 points) and actionability (+10–15 points) relative to baselines, at only a modest cost to within-report distinctness.
• Diversity: Nomad’s inter-report diversity is 0.43–0.47, compared to 0.08–0.20 for baselines, indicating far broader coverage of the insight space across independent runs.
• Contradictory claim: The paper asserts that current LLM agent frameworks systematically produce redundant, low-diversity research outputs and that Nomad circumvents this with explicit, data-driven exploration control mechanisms.

  Figure 13: Radar profiles—Nomad dominates in trustworthiness, overall quality, and diversity across both corpus- and goal-driven settings; baselines show narrower profiles, further supporting robustness of Nomad’s output diversity and reliability.

  

  Figure 14: Nomad achieves high both quantity and numeric grounding of claims per report, with grounding rate 75–81%, compared to 20–32% for o3-deep-research and 4–20% for GPT Researcher.

Theoretical and Practical Implications

Nomad’s explicit coverage and validation structure addresses foundational weaknesses in stochastic LLM agent rollouts: over-concentration on user-specified or high-probability paths, low serendipity, and risk of hallucinated or ungrounded claims. By treating research as autonomous exploration rather than task execution, Nomad reshapes the control state of LLM-driven discovery, supporting practical deployments where breadth, novelty, and trustworthiness are critical—e.g., policy review, enterprise intelligence, and strategic forecasting.

These architectural insights generalize: any long-horizon LLM reasoning workflow that must cover open-ended or massively multidimensional corpora should benefit from explicit, map-based exploration, iterative cross-verification, and modular, citation-preserving reporting pipelines. Adoption of these patterns is likely necessary for next-generation, fully autonomous research and analysis agents, especially in settings where high stakes or lack of human-in-the-loop supervision preclude failure modes seen in current generative agent frameworks.

Future Directions

Research avenues highlighted include:

• Richer, subgraph-leveraged insight generation within the exploration map
• Dynamic, adaptive control strategies (including MCTS-style or exploration–exploitation traversal schemes)
• Multimodal and cross-corpus exploration map amalgamation
• End-to-end ablation and efficiency analysis on system-level choices (e.g., explorer/verifier model choice, map update policies)
• Extension to non-text modalities (structured data, images, video, audio) and improvement in transparent reasoning explainability

Conclusion

Nomad (2603.29353) systematically advances the state of autonomous discovery by combining explicit, breadth-prioritizing exploration, rigorous claim verification, and structured, evidence-audited reporting. The empirical results demonstrate clear superiority on diversity, originality, and trustworthiness in both closed- and open-domain research settings. The underlying paradigm—exploration-first, control-state-driven, and verifier-separated—will likely form a core building block for future scalable, robust, and insight-generating AI research agents.