Causal Components and Generalization of the Explore-Then-Commit Protocol

Determine which components of the explore-then-commit protocol—structured acquisition, automatic diagnostics, restricted measurement batches, interpreter access, and fresh synthesis—cause its observed improvements, and establish whether those improvements generalize beyond noiseless direct-equation scientific law-discovery tasks.

Background

The paper evaluates a protocol in which a LLM proposes candidate laws, a programmatic planner selects measurements, and a fresh synthesis prompt produces a final law from fixed observations. The complete configuration achieves lower numerical error with fewer measurements than the implemented baselines, but several interventions vary simultaneously, including measurement-batch constraints, automatic diagnostics, hypothesis prompting, interpreter access, and fresh synthesis. Consequently, the reported results do not isolate which components produce the improvements.

The experiments use noiseless, direct-output benchmark tasks from NewtonBench. The authors therefore leave unresolved whether the observed measurement efficiency and numerical-fit advantages extend to harder scientific laws, alternative benchmark settings, noisy observations, or indirect dynamical systems.