ActiveSciBench-GRN: Benchmark for Gene Networks

• ActiveSciBench-GRN is a benchmark for closed-loop discovery in gene regulatory networks that evaluates adaptive experiment design and hypothesis refinement.
• It employs small synthetic GRN models with predefined motifs to measure performance through edge, sign, and exact graph accuracy metrics.
• The framework integrates dynamic experiment selection and posterior updating to resolve both topological ambiguity and kinetic uncertainty.

Searching arXiv for the benchmark paper and closely related work to ground the encyclopedia entry. arXiv query: "LLM-AutoSciLab ActiveSciBench-GRN" ActiveSciBench-GRN is a benchmark for closed-loop scientific discovery in gene regulatory networks (GRNs), introduced as one of the two datasets in ActiveSciBench within "LLM-AutoSciLab: Closed-Loop Scientific Discovery via Active Experimentation with LLMs" (Kabra et al., 21 May 2026). It models discovery as a budget-constrained process requiring adaptive experiment design, variable selection, and recovery of true mechanisms, rather than static inference from a fixed dataset. In this setting, hypotheses guide data acquisition and the resulting observations refine the hypothesis space, so the benchmark is explicitly designed around active data acquisition, signed causal-graph recovery, and iterative mechanism refinement under limited interventions (Kabra et al., 21 May 2026).

1. Benchmark scope and task construction

ActiveSciBench-GRN comprises 45 gene-regulatory-network tasks. The task catalog is built from five canonical regulatory motifs: activation chain, coherent feed-forward, incoherent feed-forward, negative feedback, and toggle switch. Each motif is instanced in three topological variants and three difficulty levels—Easy, Medium, and Hard—yielding $5 \times 3 \times 3 = 45$ tasks per seed (Kabra et al., 21 May 2026).

Each task uses a small synthetic network with $n=5$ nodes: one signal node, three regulators, and one distractor $R$. The graph is directed, with adjacency entries $A_{ij} \in \{0,1\}$, and a sign matrix $S_{ij} \in \{-1,+1\}$ when $A_{ij}=1$, otherwise $S_{ij}=0$. The sparsity cap is at most six edges, and variants uniformly sample wiring patterns consistent with the relevant motif family (Kabra et al., 21 May 2026).

The benchmark therefore fixes a tightly controlled combinatorial regime in which topology, sign, and intervention strategy all matter. A plausible implication is that the small-network design isolates the effects of active experiment selection and mechanism disambiguation, rather than conflating them with large-scale estimation error.

2. Dynamical-system formulation and oracle model

Each node $i$ evolves according to an ordinary differential equation of the form

$$\frac{dx_i}{dt} = -\gamma_i x_i + \sum_{j=1}^{n} A_{ji} \cdot f_{ji}(x_j; \theta_{ji}).$$

This assigns each task a mechanistic dynamics model rather than a purely graph-theoretic one (Kabra et al., 21 May 2026). The benchmark specifies typical regulatory terms for activation and repression. For activation,

$$f_{ji}(x_j; \theta) = \lambda_{ji} \cdot \frac{x_j^{h_{ji}}}{K_{ji} + x_j^{h_{ji}}},$$

and for repression,

$n=5$0

The task-specific parameters are $n=5$1, drawn per task. By default, there is no observation noise, expressed as $n=5$2, although Gaussian noise $n=5$3 can be injected on measured $n=5$4 (Kabra et al., 21 May 2026).

The ground-truth mechanism is represented by both an adjacency matrix $n=5$5, encoding edge presence or absence, and a sign matrix $n=5$6. This means that recovery is not limited to edge existence; it also requires correct regulatory polarity (Kabra et al., 21 May 2026).

3. Hypothesis space and posterior representation

In the benchmark’s closed-loop formulation, a candidate GRN hypothesis is encoded as $n=5$7, where the graph $n=5$8 is paired with fitted kinetic parameters $n=5$9. The framework maintains a weighted set of hypotheses $R$0 (Kabra et al., 21 May 2026).

The graph prior is sparsity-favoring:

$R$1

The parameter prior $R$2 is taken as, for example, independent log-Normal or Gamma distributions on $R$3. Under a hypothesis $R$4, the likelihood is

$R$5

The posterior over graphs is then written as

$R$6

These definitions make explicit that ActiveSciBench-GRN is not merely an intervention benchmark; it is a benchmark over a structured hypothesis space in which discrete graph uncertainty and continuous parameter uncertainty coexist (Kabra et al., 21 May 2026). This suggests that experimental design must resolve both topological ambiguity and kinetic ambiguity, even in a small network.

4. Active experiment selection

The benchmark is designed around choosing interventions that most effectively disambiguate the current hypothesis set. The next experiment $R$7, such as “knock-down node $R$8,” is ideally selected to maximize expected information gain:

$R$9

In practice, this is approximated by inter-hypothesis disagreement:

$A_{ij} \in \{0,1\}$0

The operational procedure is: for each intervention in the candidate set, predict the response under each current hypothesis, compute the variance across those predictions, and select the intervention with maximal disagreement. The selected intervention is then submitted to the oracle, yielding the next observation (Kabra et al., 21 May 2026).

This criterion is central to the benchmark’s purpose. Hypothesis-conditioned selection focuses perturbations where competing signed graphs predict diverging responses, rapidly eliminating spurious indirect edges (Kabra et al., 21 May 2026). In that sense, ActiveSciBench-GRN evaluates whether a discovery system can use mechanistic disagreement to drive data collection, rather than merely fit observations after the fact.

5. Closed-loop refinement protocol

After observing $A_{ij} \in \{0,1\}$1, the benchmarked framework updates each hypothesis weight according to its predictive adequacy:

$A_{ij} \in \{0,1\}$2

An equivalent unnormalized posterior update is

$A_{ij} \in \{0,1\}$3

For each surviving graph $A_{ij} \in \{0,1\}$4, the kinetic parameters are then refit via

$A_{ij} \in \{0,1\}$5

with BFGS on the negative log-posterior given as an example implementation (Kabra et al., 21 May 2026).

The benchmark description also specifies a confidence-gated mode switch. A bootstrap-based confidence score $A_{ij} \in \{0,1\}$6 is computed. If $A_{ij} \in \{0,1\}$7, the procedure remains in a “Disambiguate” regime using $A_{ij} \in \{0,1\}$8; once $A_{ij} \in \{0,1\}$9, it switches to a “Refine” mode in which experiments focus on reducing residual parameter uncertainty within the current high-confidence $S_{ij} \in \{-1,+1\}$0 (Kabra et al., 21 May 2026).

This separation between disambiguation and refinement is structurally important. A plausible implication is that the benchmark probes whether a method can detect when graph uncertainty is still dominant and when it is instead efficient to spend interventions on parameter refinement inside a mostly resolved topology.

6. Evaluation criteria and reported performance

ActiveSciBench-GRN evaluates graph recovery using edge precision, recall, and $S_{ij} \in \{-1,+1\}$1, as well as exact graph accuracy and sign accuracy. The metrics are defined as follows:

• $S_{ij} \in \{-1,+1\}$2
• $S_{ij} \in \{-1,+1\}$3
• $S_{ij} \in \{-1,+1\}$4
• Exact graph accuracy: $S_{ij} \in \{-1,+1\}$5
• Sign accuracy: fraction of edges with correct sign (Kabra et al., 21 May 2026)

Under a budget $S_{ij} \in \{-1,+1\}$6 interventions, the main reported results are:

Method	Edge $S_{ij} \in \{-1,+1\}$7 (%)	Exact (%)
Random sampling	35.56	2.22
Uncertainty sampling	50.10	4.44
LLM-only	50.41	0.00
Code-assisted LLM	54.67	0.00
LLM-AutoSciLab	72.49	31.11

For the same setting, the corresponding sign accuracies are 84.44 for random sampling, 88.15 for uncertainty sampling, 79.44 for LLM-only, 85.98 for code-assisted LLM, and 98.15 for LLM-AutoSciLab (Kabra et al., 21 May 2026).

The benchmark summary further states that, to match $S_{ij} \in \{-1,+1\}$8 $S_{ij} \in \{-1,+1\}$9, the strongest active baselines require 3–4 times more queries: code-assisted LLM requires approximately $A_{ij}=1$0, uncertainty sampling approximately $A_{ij}=1$1, and classical methods more than $A_{ij}=1$2. More broadly, the enclosing paper reports 31.1% exact graph recovery on ActiveSciBench-GRN and states that hypothesis-guided experimentation is 2–5 times more sample-efficient than the strongest competing baselines (Kabra et al., 21 May 2026).

These results indicate that the benchmark discriminates sharply between passive or weakly adaptive strategies and explicitly hypothesis-conditioned active selection. Near-perfect sign accuracy above 98% is reported as evidence that the framework reliably distinguishes activation versus repression once topology is correct (Kabra et al., 21 May 2026).

7. Interpretation, limitations, and position within closed-loop discovery

ActiveSciBench-GRN is presented as a controlled, closed-loop benchmark for signed causal-graph discovery under an experimental budget (Kabra et al., 21 May 2026). Its defining feature is that discovery proceeds by iterating over hypothesis generation, intervention choice, evidence acquisition, and mechanism refinement. The benchmark is therefore aligned with the claim that scientific discovery is a closed-loop process in which hypotheses guide data acquisition and observations refine the hypothesis space (Kabra et al., 21 May 2026).

The benchmark’s limitations are stated explicitly. It uses small synthetic networks with $A_{ij}=1$3 and idealized ODE oracles; real single-cell data include off-target effects, heterogeneous noise, and batch confounders. Recovery quality also depends on LLM hypothesis quality and coverage, so prompt engineering remains consequential, and unseen motif families may challenge recovery. In addition, scalability to larger GRNs with $A_{ij}=1$4 is currently limited by the combinatorial explosion of candidate graphs and experiment design space (Kabra et al., 21 May 2026).

A common misconception would be to treat ActiveSciBench-GRN as a standard graph-reconstruction dataset. The benchmark is more specific than that: it evaluates adaptive intervention policies, weighted hypothesis maintenance, posterior updating, and budgeted mechanism recovery in a signed dynamical system. Conversely, it should not be taken as a direct surrogate for real biological discovery. The synthetic, low-noise setting is deliberately controlled, and the paper itself identifies real-data complications absent from the benchmark (Kabra et al., 21 May 2026).

Within the LLM-AutoSciLab framework, ActiveSciBench-GRN functions as a testbed for whether active experimentation can resolve ambiguity that static supervised learning on fixed observations cannot. The reported gap between baseline methods and hypothesis-driven selection suggests that, in this benchmark regime, targeting structural ambiguity is more effective than passively fitting edge weights or relying on unguided sampling (Kabra et al., 21 May 2026).