DataSTORM: Dual-System Research Framework

• DataSTORM is a dual-system framework that combines LLM-agentic thesis-driven analysis with simulation ensemble management to support rigorous decision-making.
• It utilizes a three-stage pipeline integrating web and database querying, multi-agent exploration, and iterative thesis refinement for evidence-grounded narrative generation.
• Empirical evaluations show significant improvements in insight recall and summary quality, while its simulation module efficiently explores alternative timelines in complex dynamical domains.

DataSTORM encompasses two distinct systems at the intersection of reasoning and exploration over large, multifaceted data assets: (1) an LLM-agentic framework for deep thesis-driven analysis over structured databases and web corpora (Liu et al., 7 Apr 2026); and (2) a platform for simulation ensemble management and alternative timeline exploration in complex dynamical domains (Azad et al., 2024). Both employ advanced methodologies to drive insight discovery, decision support, and narrative synthesis in the presence of massive, heterogeneous information sources.

1. LLM-Agentic Deep Research: System Overview

DataSTORM (Liu et al., 7 Apr 2026) is an LLM-based agentic system structured as a three-stage pipeline targeting both structured databases and internet sources, culminating in the automated generation of evidence-grounded analytical narratives. The architecture tightly integrates multi-agent exploration, exploratory data analysis (EDA), and iterative thesis refinement, reframing deep research as a thesis-driven process.

The pipeline operates as follows:

• Warm-Start Module (Co-STORM): Ingests an internet corpus $\mathcal I$ and user query $q$, initiating a lightweight report $r_0$ and seed insight bank $B_0$ through multi-agent discourse over web evidence.
• Multi-Agent Exploration Module: Consists of a Planner agent that generates exploration questions tagged by destination (database or internet); an Executor agent which handles SQL synthesis and execution (implementing a ReAct-style reasoning loop and outputting both SQL $s_{i,j}$ and answer $a_{i,j}$); a Query Consistency Module ensuring predicate alignment and context-aware rectification; Inductive Surfacing via automatic computation of summary statistics after SQL execution; and a Thesis Generation & Refinement agent, invoked every $p$ layers.
• Final Report Generation: Comprised of staged narrative assembly with outline generation (Stage A), evidence-grounded section drafting (Stage B), citation grounding and revision (Stages C & D), and linguistic polishing (Stage E).

A unified insight bank $B_i$ maintains evidence from both database (quantitative) and web (qualitative) sources, annotated with provenance and merged into the evolving narrative arc via the system’s dynamic thesis $t_i$.

2. Exploratory Data Analysis and Data Storytelling Integration

The system encodes EDA principles both in its control flow and content surfacing:

• The Planner acts deductively by posing targeted “what-if” or “why” questions; the Executor acts inductively by surfacing bottom-up patterns and summary statistics after each SQL execution (including $\text{distinct\_pct}(C)$, $q$0, and for numeric columns, $q$1).
• Query Consistency ensures alignment of filter predicates across questions and enforces normalization by issuing follow-ups in the face of detected semantic drift.
• The system's Data Storytelling module generates candidate theses $q$2 from $q$3 (the insight bank at layer $q$4), attaches research strategies for each, and invokes periodic refinement to maintain a coherent “through-line.”
• The report planning module (OutlineGen) programmatically links thesis, evidence, and narrative section structure.

3. Formal Thesis-Driven Analytical Process

The thesis-driven analytical workflow consists of:

• Candidate Thesis Discovery: Given $q$5, the thesis generation LLM $q$6 generates up to three candidate theses $q$7. Each $q$8 is scored for coherence with $q$9 using a function $r_0$0, yielding $r_0$1.
• Iterative Validation via Cross-Source Hypotheses: For each $r_0$2, corresponding evidence sets $r_0$3 (quantitative/database) and $r_0$4 (qualitative/web) are extracted, and a consistency score $r_0$5 is computed via embedding similarity.
• Convergence Criteria: Stopping is based on narrative and evidence bank stability: if the symmetric difference $r_0$6 and $r_0$7, or a fixed number of $r_0$8 layers is reached.

This process ensures research output converges to a supported, focused analytical narrative.

4. Cross-Source Querying and Evidence Integration

In each layer, the Planner emits up to $r_0$9 questions $B_0$0, dispatched to either SQL execution or web search. For database destinations, the Executor uses a ReAct reasoning loop:

• Thought: Choose relevant tables/columns
• Actions: $B_0$1
• Loop continues until a stopping criterion is satisfied

All answers $B_0$2, enriched with summary statistics and destination metadata, are merged into $B_0$3. The InsightFilter retains the top $B_0$4 insights per iteration by relevance to the current thesis ($B_0$5), applying LLM-based re-ranking. This produces a parallel evidence structure where both qualitative and quantitative results are treated equivalently in downstream thesis and report generation.

5. Empirical Evaluation and Benchmarking

5.1 Quantitative Metrics on InsightBench

Performance is assessed using:

• Insight-level recall: $B_0$6, with match scored by LLM similarity.
• Summary-level score: $B_0$7, judged by LLM.

Results:

System	Judge	Insight Recall	Summary Score
AgentPoirot	Qwen-3-30B	49.9%	51.5%
DataSTORM	Qwen-3-30B	69.3%	58.7%
AgentPoirot	GPT-4o	47.1%	46.6%
DataSTORM	GPT-4o	61.9%	52.5%

DataSTORM demonstrated a +19.4% (Qwen) and +14.8% (GPT-4o) absolute improvement in insight-level recall and substantial gains in summary score (Liu et al., 7 Apr 2026).

5.2 ACLED Benchmark and Human Evaluation

Key metrics: reference-induced matching, RACE framework (Comprehensiveness, Depth, Instruction-following, Readability, 50 = parity), and database use ratio.

System	Ref-Match	RACE	DB Use Ratio
OpenAI DR (CSV/MCP)	51.2/48.5	46.8/46.1	23.3/30.4%
DataSTORM	61.8	52.6	66.4%

Human expert assessment over 20 topics found DataSTORM outperforming the baseline on 6/7 rubric dimensions, with significant improvement in Originality (+0.84 pp, $B_0$8), and winning 57.5% of pairwise preferences versus the baseline.

This suggests that the thesis-driven and EDA-anchored design enables DataSTORM to surface both more relevant and original insights, with a higher proportion of claims explicitly grounded in structured data compared to proprietary LLM research systems. A plausible implication is that this architecture can generalize to other settings requiring rigorous multi-source synthesis and reasoning.

6. Algorithmic Structure and Reproducibility

The system’s operational core is summarized as follows:

$s_{i,j}$1

Key subroutines include LLM-driven question generation, ReAct SQL execution, LLM-based filter and thesis refinement, and convergence/stopping checks. This formal structure supports reproduction and extension in novel data-centric research contexts (Liu et al., 7 Apr 2026).

7. Relation to Simulation Ensembles: DataStorm-EM

DataStorm also labels a platform for managing and analyzing large ensembles of simulation instances—DataStorm-EM (Azad et al., 2024)—designed for high-complexity domains requiring the exploration of alternative system trajectories under uncertainty.

DataStorm-EM is architectured over four modules:

• Ensemble Generation: Directed acyclic model graphs (DS-Actors), sampling of parameter vectors, and distributed execution orchestration.
• Parameter Management & Optimization: Solves a 0–1 quadratic program to optimize the selection of simulation instances for diversity and informativeness under budget $B_0$9.
• Timeline Extraction & Analysis: Hierarchical agglomerative clustering of timelines based on weighted time-series distances, selection of medoid representatives, and optional 2D projection for exploratory analysis.
• Visualization & Exploration Tools: Rich web-based interfaces (React + D3) for timeline plotting, cluster mapping, parallel coordinates of parameter vectors, interactive drill-down, and scenario summaries.

In practice, DataStorm-EM achieves efficient workflow orchestration, scalable clustering, and interactive exploration for ensembles up to 5,000 simulation instances on cloud/HPC clusters, with visualization overhead maintained below 5 minutes for $s_{i,j}$0 timelines. Prototypes in pandemic response and urban sustainability further demonstrate its domain-general applicability. The system leverages a modern stack (Python, Ansible, Docker, Apache Kafka, Neo4j, Parquet, SciPy, scikit-learn) and supports extension via new model wrappers and analytic plug-ins (Azad et al., 2024).