ActiveSciBench: Adaptive Discovery Benchmark

• ActiveSciBench is a benchmark suite that evaluates closed-loop scientific discovery through adaptive experimental design in enzyme kinetics and gene regulatory networks.
• It integrates variable selection in enzyme kinetics and perturbation-driven causal graph inference in gene regulatory networks to recover underlying mechanisms.
• The benchmark emphasizes hypothesis-driven experimentation by requiring agents to iteratively design experiments under fixed budgets to disambiguate competing mechanisms.

ActiveSciBench is a benchmark suite for evaluating closed-loop scientific discovery with active experimentation in two scientifically grounded settings: enzyme kinetics and gene regulatory networks. Introduced alongside LLM-AutoSciLab, it is designed to move beyond static, fixed-dataset inference by requiring an agent to select informative experiments under a fixed budget, identify relevant variables or perturbations, and recover the true governing mechanism—either an equation in ActiveSciBench-Chem or a signed causal graph in ActiveSciBench-GRN (Kabra et al., 21 May 2026). Its central premise is that scientific discovery is a budget-constrained process in which hypotheses guide data acquisition and observations refine the hypothesis space, rather than a purely supervised mapping from fixed observations to models.

1. Motivation and discovery problem

ActiveSciBench is motivated by an identifiability bottleneck that arises when multiple mechanisms fit the same limited observations. In that setting, static supervised learning on fixed datasets cannot force competing hypotheses to diverge, so it can yield locally plausible but incorrect models that fail to generalize. The benchmark therefore recasts discovery as adaptive data acquisition: the agent must design experiments that maximize contrast among candidate mechanisms and thereby resolve uncertainty efficiently.

Each task is an oracle-based experimental system with hidden mechanism and parameters. The learner has a fixed query budget $B$ and must generate candidate hypotheses, select experiments adaptively to disambiguate those hypotheses, refine mechanisms with new evidence, and recover the ground truth at budget exhaustion. In ActiveSciBench-Chem, part of the problem is variable selection: the interface exposes seven biochemical variables, but only a subset is mechanistically relevant in any given task. In ActiveSciBench-GRN, the learner must choose perturbations such as knock-up or knock-down interventions and infer the signed, directed causal graph.

This framing makes mechanism discrimination integral to evaluation. The benchmark targets recovery of the true underlying law or graph, not merely predictive fit on a passively observed dataset.

2. Composition and domain coverage

ActiveSciBench comprises two datasets with complementary scientific structure and inference targets (Kabra et al., 21 May 2026).

Suite	Composition	Recovery target
ActiveSciBench-Chem	57 curated enzyme-kinetics tasks in Easy, Medium, and Hard tiers	ground-truth rate law and relevant variables
ActiveSciBench-GRN	45 tasks per seed from five canonical regulatory motif families, with three topological variants and three difficulty regimes	exact signed directed causal graph

ActiveSciBench-Chem organizes 57 curated enzyme-kinetics tasks across three tiers. Easy tasks cover standard families such as Michaelis–Menten and competitive inhibition. Medium tasks introduce structured compositions such as mixed or substrate inhibition. Hard tasks include extended mechanisms such as cooperative or allosteric regulation. The tasks are simulator-driven and mechanistically grounded.

ActiveSciBench-GRN contains 45 tasks per seed generated from five canonical regulatory motif families: activation chain, coherent feedforward loop, incoherent feedforward loop, negative feedback, and toggle or bistable circuits. Each motif family is provided in three topological variants and three difficulty regimes. Difficulty reflects increasingly nonlinear dynamics, including saturation, bistability, and switching, which make full graph recovery progressively harder.

Taken together, the two suites broaden discovery beyond symbolic regression. Chem covers compositional biochemical kinetics with environmental modifiers and inhibitors, whereas GRN covers perturbation-driven causal graph inference with nonlinear regulatory dynamics.

3. Experimental interfaces and mechanistic targets

In ActiveSciBench-Chem, an experiment is specified by an assay condition vector in a shared seven-dimensional interface,

$x = (C_A, C_I, C_B, C_P, Enz, T, pH),$

corresponding to substrate, inhibitor, second substrate, product concentration, enzyme loading, temperature, and pH. The oracle returns the initial rate $r_0$ and auxiliary mass-balance observables, while the true rate law and parameters remain hidden.

The chem tasks draw from representative mechanistic families including Michaelis–Menten,

$v = \frac{V_{\max}[S]}{K_M + [S]},$

competitive inhibition,

$$v = \frac{V_{\max}[S]}{K_M\!\left(1 + \frac{[I]}{K_I}\right) + [S]},$$

Hill cooperativity,

$v = V_{\max} \frac{[S]^n}{K^n + [S]^n},$

substrate inhibition, ping–pong bisubstrate mechanisms, Arrhenius temperature dependence, product feedback or inhibition, and extended allosteric or cooperative variants. The benchmark also includes structured compositions of these families and extended or nonstandard mechanisms such as ordered sequential bisubstrate, allosteric activation, anti-cooperative Hill behavior, mixed or cooperative inhibition, monotonic pH dependence, metal-ion activation, autocatalytic feedback, and dual inhibition.

In ActiveSciBench-GRN, an experiment is a discrete intervention on selected nodes, for example knock-up or knock-down. The oracle returns downstream expression changes for all nodes. The hidden mechanism is a signed directed graph together with nonlinear regulatory kinetics. The underlying dynamics are modeled by

$\frac{d x_i}{dt} = f_i(\mathbf{x}, \theta),$

where $x_i$ is expression, $\theta$ are kinetic parameters, and $f_i$ encodes regulation from parent nodes. The causal structure is

$x = (C_A, C_I, C_B, C_P, Enz, T, pH),$0

with sign $x = (C_A, C_I, C_B, C_P, Enz, T, pH),$1, where direction encodes influence and sign encodes activation or repression.

Both suites are noiseless by default, with controlled Gaussian noise used for robustness analysis. In both cases, ground-truth laws or graphs remain inaccessible during interaction and are used only for evaluation.

4. Closed-loop protocol and acquisition logic

ActiveSciBench evaluates a budget-constrained iterative protocol rather than one-shot model fitting (Kabra et al., 21 May 2026). At iteration $x = (C_A, C_I, C_B, C_P, Enz, T, pH),$2, the agent maintains

$x = (C_A, C_I, C_B, C_P, Enz, T, pH),$3

with accumulated data $x = (C_A, C_I, C_B, C_P, Enz, T, pH),$4, structured memory $x = (C_A, C_I, C_B, C_P, Enz, T, pH),$5, and current hypothesis set $x = (C_A, C_I, C_B, C_P, Enz, T, pH),$6. The loop consists of four operations: proposing candidate mechanisms and search regions via GenHyp, choosing an acquisition mode based on bootstrap confidence $x = (C_A, C_I, C_B, C_P, Enz, T, pH),$7, selecting the next experiment via Acquire, and refining hypotheses after observing the oracle response via RefineHyp with confidence gating ConfGate.

The benchmark distinguishes two acquisition modes. If $x = (C_A, C_I, C_B, C_P, Enz, T, pH),$8, the mode is Disambiguate: experiments are chosen to maximize disagreement among candidate mechanisms. If $x = (C_A, C_I, C_B, C_P, Enz, T, pH),$9, the mode is Refine: experiments focus on reducing residual uncertainty within the current family, for example through local parameter estimation.

For Chem, disagreement at a candidate point $r_0$0 is defined by

$r_0$1

where $r_0$2 are predictions from candidate mechanisms. For GRN, if an intervention $r_0$3 yields predicted response vectors $r_0$4, the disagreement objective is

$r_0$5

The benchmark enforces a fixed query budget $r_0$6 and does not assign heterogeneous per-experiment costs. Search regions are bounds returned by the agent, and within those bounds a small diverse batch of high-disagreement points is sampled for oracle queries. This setup isolates the value of hypothesis-conditioned acquisition under explicit budget pressure.

5. Evaluation criteria and empirical behavior

For Chem, evaluation combines predictive and mechanistic criteria (Kabra et al., 21 May 2026). Predictive fidelity is measured by RMSLE,

$r_0$7

Numerical exact accuracy is

$r_0$8

Symbolic accuracy evaluates whether the recovered law is symbolically equivalent to the ground truth up to algebraic rewriting and fitted constants. The reported procedure uses an LLM judge prompted to decide whether there exist constants that make the candidate expression exactly equivalent to the ground-truth expression; a “Yes” is counted as correct.

For GRN, the benchmark reports edge-level precision, recall, and F1 over adjacency, sign accuracy for activation versus repression, exact graph accuracy as exact signed adjacency matrix match, and motif accuracy when used. Sample efficiency is measured as the multiplicative number of oracle queries a comparison method needs to match the fixed-budget performance of the proposed method.

On ActiveSciBench-Chem, the closed-loop agent achieves 35.09% symbolic accuracy and 50.88% exact accuracy overall, with RMSLE $r_0$9. Reported baselines include Bayesian Experimental Design at 31.58% symbolic accuracy and 39.47% exact accuracy overall, and PySR on fixed datasets at 7.89% symbolic accuracy and 15.79% exact accuracy. The method also remains robust on Hard tasks, with 42.86% symbolic accuracy and 52.38% exact accuracy.

On ActiveSciBench-GRN, the reported performance is 72.49% edge F1, 31.11% exact graph recovery, and 98.15% sign accuracy overall. Baselines cited in the benchmark include GIES with F1 $v = \frac{V_{\max}[S]}{K_M + [S]},$0 and exact accuracy $v = \frac{V_{\max}[S]}{K_M + [S]},$1, and uncertainty sampling with F1 $v = \frac{V_{\max}[S]}{K_M + [S]},$2 and exact accuracy $v = \frac{V_{\max}[S]}{K_M + [S]},$3. The results indicate that sign prediction is relatively easy compared with exact topology recovery, whereas exact topology requires targeted perturbations.

Across NewtonBench, Chem, and GRN, hypothesis-guided experimentation is reported as 2–5× more sample-efficient than the strongest baselines. Representative fixed-budget comparisons show that baselines need 2.33–4.60× more queries to match the agent’s recovery, while LLM-only variants often need 5–14× more. Ablation studies identify removal of hypothesis-conditioned acquisition as the largest source of degradation across suites, and they further indicate that memory and hypothesis diversity matter more in settings requiring relevant-variable identification or accumulation of perturbation evidence.

6. Position relative to adjacent benchmarks and practical use

ActiveSciBench is explicitly positioned against both interactive law-discovery benchmarks and static inference benchmarks (Kabra et al., 21 May 2026). NewtonBench evaluates interactive symbolic law recovery when relevant variables are already known and remains equation-only, whereas ActiveSciBench adds hidden relevant-variable selection in Chem, perturbation-driven causal graph recovery in GRN, and budget-limited oracle access across two domains. Traditional symbolic regression systems such as AI Feynman and PySR, and GRN inference benchmarks such as GeneNetWeaver and GENIE3, assume fixed datasets and do not evaluate closed-loop experiment design.

This makes ActiveSciBench distinct in what it rewards. It emphasizes adaptive discovery, requiring agents to choose experiments that separate mechanisms rather than merely reduce predictive uncertainty. A plausible implication is that performance on static regression or graph-inference corpora does not directly transfer to this benchmark unless the method can also control data acquisition.

The benchmark is released with code and data at the LLM-AutoSciLab repository. The repository provides scripts to instantiate oracles, configure budgets and seeds, and run closed-loop agents or baselines. Default fixed-budget comparisons in the reported experiments use $v = \frac{V_{\max}[S]}{K_M + [S]},$4 for Chem and $v = \frac{V_{\max}[S]}{K_M + [S]},$5 for GRN. Chem uses the seven-dimensional assay interface and GRN uses a discrete perturbation interface; each run builds $v = \frac{V_{\max}[S]}{K_M + [S]},$6 online, while ground-truth mechanisms and graphs are stored internally for evaluation. The reported implementation uses PySR for symbolic refinement and signed-graph fitting with BFGS for graph refinement. Any method that can iteratively propose experiments and return mechanisms can, in principle, be evaluated in the same framework.

7. Limitations, assumptions, and extension paths

ActiveSciBench uses simulator-based oracles, and therefore does not capture full laboratory complexity, including failures, batch effects, protocol constraints, or cost heterogeneity (Kabra et al., 21 May 2026). Its performance also depends on the quality and diversity of LLM-generated hypotheses and on the coverage of the refinement toolchain. These assumptions matter because the benchmark isolates the epistemic value of adaptive experimentation, but it does so in a controlled environment.

Robustness and scalability remain open issues. Although the reported agent shows improved robustness to noise in ablations, exact topology recovery in GRN remains noise-sensitive. Scaling to larger GRNs, richer biochemistry, or multi-modal measurements is identified as requiring stronger priors and more efficient acquisition. Domain coverage is also curated rather than exhaustive: Chem includes textbook and extended mechanisms but not the full space of biochemical kinetics, while GRN motifs cover common primitives but not the full diversity of cellular regulation.

The paper identifies several directions for extension, including multi-omics readouts, richer environmental controls, continuous cost models, and real lab deployments with operational constraints. Real-world validation would require integration with physical instrumentation APIs, safety gates, and domain-specific acquisition cost models. In that sense, ActiveSciBench functions as a clean, budget-controlled testbed for active mechanism recovery, while leaving open the substantial engineering and methodological work required for deployment in self-driving laboratories.