EXPERIGEN: AI-Driven Scientific Discovery Framework

• EXPERIGEN is an agentic framework defined by a two-phase, Bayesian-inspired search that integrates LLM-based hypothesis generation with rigorous empirical testing.
• Its Generator and Experimenter agents collaboratively propose and refine natural-language hypotheses over unstructured, multimodal datasets, enhancing both novelty and predictive accuracy.
• Empirical evaluations show 2–4x more statistically significant hypotheses, 7–17% predictive gains, and impactful real-world A/B test results across diverse research domains.

Searching arXiv for the specified paper and closely related context papers. EXPERIGEN is an agentic framework for end-to-end scientific discovery in social and behavioral research, introduced to unify automatic hypothesis generation and empirical validation over raw, unstructured, and potentially multimodal or relational data. It is presented as a response to the slow manual loop of observing phenomena, formulating hypotheses, and testing them experimentally, and is explicitly organized around a Bayesian optimization inspired two-phase search in which a Generator proposes candidate hypotheses and an Experimenter evaluates them empirically (Gupta et al., 8 Feb 2026). In the reported evaluation, the framework is described as discovering more statistically significant hypotheses than prior approaches, producing hypotheses that are more predictive, and extending naturally to multimodal and relational datasets; the same work also reports expert assessment of novelty and research-worthiness and a real-world A/B deployment of an LLM-generated hypothesis (Gupta et al., 8 Feb 2026).

1. Conceptual scope and motivation

EXPERIGEN is situated in the setting of data-driven social science research, where discovery is framed as an iterative cycle of observation, hypothesis generation, and experimental validation. The motivating claim is that manual cycles can take months, while existing data-driven methods either generate hypotheses without rigorous validation or validate only pre-specified hypotheses. EXPERIGEN is introduced specifically to close this gap by operationalizing end-to-end discovery rather than supporting only isolated substeps (Gupta et al., 8 Feb 2026).

The framework is defined by a division of labor between two agents. A Generator agent, implemented as an LLM, proposes testable natural-language hypotheses. An Experimenter agent, described as a ReAct-style LLM augmented with a sandboxed Python kernel and an LLM feature extractor, operationalizes and evaluates those hypotheses through aligned statistical procedures. The architecture therefore treats hypothesis generation and hypothesis testing as coupled computational processes rather than as separate pipelines (Gupta et al., 8 Feb 2026).

A central design feature is the explicit attempt to balance breadth and rigor. The outer loop searches for novel candidates, while the inner loop refines candidate hypotheses through repeated empirical checks. This suggests an effort to preserve exploratory search over a combinatorial hypothesis space while constraining acceptance through statistical evidence. In the paper’s framing, this is the mechanism by which the system bridges automatic ideation and empirical falsification (Gupta et al., 8 Feb 2026).

2. Two-phase search over the hypothesis space

The framework formalizes the dataset as $D = \{(x_i, y_i)\}_{i=1}^N$, where $x_i$ may include text, image, or metadata and $y_i$ is the outcome. The hypothesis space is denoted by $\mathcal{H}$ and consists of natural-language, testable hypotheses. The stated objective is to identify a bank $\mathcal{H}^\ast \subset \mathcal{H}$ of supported hypotheses $H$ for which the empirical evaluation satisfies $E_H(D)=\text{“supported”}$ (Gupta et al., 8 Feb 2026).

The first stage is the outer acquisition-guided seed generation loop. At outer iteration $i=1 \ldots I$, the Generator induces an implicit proposal distribution

$q_i(H)\approx q(H \mid D, \mathcal{H}_{i-1}),$

conditioned on a dataset summary and the previously accepted bank $\mathcal{H}_{i-1}$. Seed hypotheses are elicited to implicitly maximize the acquisition objective

$x_i$0

where $x_i$1 is plausibility arising from the Generator’s prior belief and $x_i$2 is a novelty bonus defined as semantic distance from previously accepted hypotheses, for example in embedding space (Gupta et al., 8 Feb 2026).

The second stage is the inner local refinement loop. Each seed $x_i$3 enters up to $x_i$4 refinement steps. At step $x_i$5, the Generator proposes a refinement $x_i$6 conditioned on short-term memory

$x_i$7

The Experimenter constructs features $x_i$8 and runs aligned statistical tests on $x_i$9 to produce evidence

$y_i$0

where $y_i$1 is a p-value and $y_i$2 is an effect size such as an odds ratio or correlation coefficient. If any $y_i$3 satisfies Bonferroni-corrected significance,

$y_i$4

it becomes a candidate for acceptance into $y_i$5; otherwise the Generator uses the returned guidance to refine the hypothesis further (Gupta et al., 8 Feb 2026).

This search strategy is described as Bayesian-optimization-inspired rather than as a literal implementation of classical Bayesian optimization. A plausible implication is that the framework borrows the explore-exploit logic of acquisition functions without assuming a conventional continuous surrogate model over the discrete space of natural-language hypotheses.

3. Generator and Experimenter agents

The Generator agent receives three classes of inputs: a compact dataset summary including schema, statistics, and samples; either the accepted hypothesis bank $y_i$6 or the session memory $y_i$7; and a description of the Experimenter’s capabilities, including code-based and LLM-based feature extraction. Its output is a JSON object with three fields: "hypothesis", "request", and "test" (Gupta et al., 8 Feb 2026).

The combined outer-plus-inner pseudocode is given in the source as:

$\mathcal{H}^\ast \subset \mathcal{H}$4

Within this process, the seed-generation prompt emphasizes novelty and plausibility, particularly by discouraging duplication of $y_i$8. During refinement, the prompting objective changes: the Generator is instructed to make local adjustments in response to Experimenter feedback, for example by controlling for confounds, adding qualifiers, or narrowing feature definitions (Gupta et al., 8 Feb 2026).

The Experimenter is defined as a composite agent. It first plans through ReAct, including statements of the form “I need feature $y_i$9; I will apply test $\mathcal{H}$0; I must control covariates $\mathcal{H}$1.” It then executes this plan in a sandboxed Python kernel that persists across refinement steps. The implementation supports programmatic features such as readability, regex-derived indicators, grouping operations, and temporal ranks, together with statistical tests including t-test, $\mathcal{H}$2, Mann–Whitney U, and logistic regression. When a hypothesis requires semantic constructs, such as sentiment or rhetorical devices, the Experimenter invokes an LLM feature extractor. The return object is a structured report containing features, test choice, $\mathcal{H}$3, $\mathcal{H}$4, confidence intervals, and guidance for subsequent refinement (Gupta et al., 8 Feb 2026).

An explicit example of a tool call is a $\mathcal{H}$5 test implemented as:

$\mathcal{H}^\ast \subset \mathcal{H}$5

This operational design makes the hypothesis-testing stage both symbolic and statistical: natural-language requests are translated into feature engineering and inferential procedures, then fed back into the next iteration of natural-language refinement.

4. Data modalities, feature realization, and statistical criteria

A stated property of the framework is that the Generator operates only over a compact schema summary and is therefore agnostic to modality. The burden of modality-specific realization falls on the Experimenter, which uses two tool classes: code-based relational transformations and LLM/VLM extraction (Gupta et al., 8 Feb 2026).

For relational and metadata-dependent hypotheses, code-based feature construction can transform the whole dataset. The examples given include within-thread position, represented as the rank of a comment among its siblings; temporal lags, such as response time relative to post time; and grouped aggregates such as user-level means. The paper explicitly notes that no per-sample embedding is needed for such features and that pandas/groupby code can suffice (Gupta et al., 8 Feb 2026).

For semantic or visual features, the framework uses closed-set prompting. The text template is of the form: “You are a judge. Text: ‘…’ Feature: X. Allowed labels: {yes, no}.” The image template is of the form: “You are a VLM. Image: <path>. Feature: Y. Allowed labels: {A,B,C}.” The stated interpretation is that extension to audio, video, or graphs only requires replacing the extractor, while Generator prompts remain unchanged (Gupta et al., 8 Feb 2026). This suggests a modular decomposition between hypothesis proposal and feature realization.

The statistical criteria are explicitly defined. Predictive accuracy is measured using in-domain and out-of-distribution classification accuracy, either through two-step LLM inference or logistic regression on hypothesis-derived features. A hypothesis is considered statistically valid if its feature coefficient in a multivariate regression has $\mathcal{H}$6 after Bonferroni correction, equivalently $\mathcal{H}$7. False Discovery Rate is defined as

$\mathcal{H}$8

Reported effect-size measures include the Odds Ratio for binary outcomes,

$\mathcal{H}$9

the correlation coefficient

$\mathcal{H}^\ast \subset \mathcal{H}$0

and Cohen’s $\mathcal{H}^\ast \subset \mathcal{H}$1,

$\mathcal{H}^\ast \subset \mathcal{H}$2

For expert validation, the reported Likert-style constructs are novelty, research-worthiness, quality, and perceived researcher expertise on a 1–5 scale or closely related ordinal labels (Gupta et al., 8 Feb 2026).

5. Empirical performance relative to baselines

Across 10 tasks, consisting of 5 from HypoBench and 5 described as a “hard set” of multimodal or metadata tasks, EXPERIGEN is reported to achieve predictive gains of 7–17 percentage points over state-of-the-art hypothesis-generation baselines and to dominate on 9/10 in-domain or out-of-distribution splits (Gupta et al., 8 Feb 2026). The same evaluation reports 2–4 times more Bonferroni-corrected significant hypotheses and an FDR below 5%, compared with 20–25% for prior methods (Gupta et al., 8 Feb 2026).

Representative out-of-distribution results are summarized below.

Task	Baseline OOD	ExperiGen OOD
Deception	63.2%	85.2%
Persuasion	64.2%	88.8%
CMV	60.1%	79.4%

These reported gains are not limited to a single modality. The paper explicitly states that the framework naturally extends to multimodal and relational datasets, and the benchmark construction includes tasks requiring multimodal or metadata-based reasoning (Gupta et al., 8 Feb 2026). A plausible implication is that the claimed performance advantage derives not only from better textual hypothesis generation but from the ability to instantiate hypotheses through richer feature pipelines.

The paper’s abstract further summarizes the results at a coarser level: EXPERIGEN “consistently discovers 2-4x more statistically significant hypotheses that are 7-17 percent more predictive than prior approaches” (Gupta et al., 8 Feb 2026). Because the details section phrases the predictive result in “percentage-point improvement” terms while the abstract uses “percent more predictive,” the safest reading is to preserve both formulations as reported rather than normalize them.

6. Expert review and real-world validation

To evaluate qualities beyond statistical support, the study recruited 5 senior faculty with 5–20 years of experience and top-journal publications to assess 25 EXPERIGEN-generated hypotheses. The evaluation dimensions were novelty, research-worthiness, clarity and testability quality, and the level of research expertise reflected in the hypothesis design (Gupta et al., 8 Feb 2026).

The reported outcomes were that 88% of the reviewed hypotheses were rated moderately or strongly novel, 70% were recommended for pursuit as serious research, the mean quality rating was 4.24/5, the mean confidence rating was 4.32/5, and 68% of the designs were judged at senior graduate-student level or above (Gupta et al., 8 Feb 2026). The abstract presents a closely related summary: among 25 reviewed hypotheses, 88% were rated moderately or strongly novel, 70% were deemed impactful and worth pursuing, and most demonstrated rigor comparable to senior graduate-level research (Gupta et al., 8 Feb 2026).

These evaluations are presented as a response to a potential misconception that statistical significance alone establishes scientific usefulness. The paper explicitly states that hypotheses must also be novel, empirically grounded, and actionable to drive real scientific progress (Gupta et al., 8 Feb 2026). The expert review is therefore intended to assess qualities not captured by predictive accuracy or corrected p-values.

The strongest validation claim concerns deployment. In collaboration with a Fortune 500 brand, the framework generated the hypothesis: “Lead-gen forms that are horizontally centered and have a soft-shadow effect achieve higher sign-up rates.” A 50/50 A/B test yielded 15,100 sign-ups for the challenger and 3,400 for the control, corresponding to a reported +344% uplift, with a two-proportion z-test yielding $\mathcal{H}^\ast \subset \mathcal{H}$3 and an estimated $29,570 in paid-traffic savings over one week (Gupta et al., 8 Feb 2026). The paper characterizes this as the first reported A/B deployment of automatically generated LLM-hypotheses with statistically significant, large real-world impact (Gupta et al., 8 Feb 2026).

7. Limitations, extensions, and research significance

The reported limitations are threefold. First, the framework depends on LLM quality: feature-extractor errors matter, and the paper states that 20% label noise reduces out-of-distribution accuracy by 4–5 percentage points. Second, compute cost remains substantial for large datasets, even though it is described as sublinear with data size. Third, unobserved confounders remain a risk because the statistical procedures still rely on assumptions that may not fully eliminate hidden structure (Gupta et al., 8 Feb 2026).

The future directions listed in the paper are correspondingly targeted. These include integrating causal structure learning in order to propose interventions rather than only associations; extending the system to richer modalities such as audio, video, time series, and network data through specialized extractors; reducing cost by distilling hypotheses or using smaller LLMs in early iterations; and incorporating active learning by selectively crowdsourcing ambiguous features (Gupta et al., 8 Feb 2026). These proposals indicate that the present system is framed primarily as an associative discovery engine rather than a complete causal inference framework.

The application domains named in the paper include marketing, public policy, healthcare, education, and computational social science. The examples are specific: A/B testing creative variants and call-to-action strategies in marketing, identifying determinants of voter persuasion in public policy, discovering textual or visual interventions in healthcare and education, and automated insight generation at scale in computational social science (Gupta et al., 8 Feb 2026).

Taken together, the reported evidence positions EXPERIGEN as a framework for linking natural-language hypothesis search with empirical testing in heterogeneous data environments. The paper’s concluding interpretation is that it accelerates hypothesis generation and validation while producing research-quality, intervention-ready discoveries across text, image, and relational datasets (Gupta et al., 8 Feb 2026). A plausible implication is that its main contribution lies not in any single modeling component, but in the orchestration of proposal, operationalization, statistical correction, and iterative refinement within a unified agentic loop.