EmpiricalBench: Equation Recovery Benchmark

• EmpiricalBench is a benchmark that assesses symbolic regression algorithms by measuring their ability to recover human-interpretable empirical formulas from both original and synthetic datasets.
• It employs evaluation metrics such as exact match rate and tree edit distance to compare candidate equations against known ground truths.
• The benchmark guides algorithm tuning by emphasizing interpretability, model selection, and the rediscovery of established scientific laws.

EmpiricalBench is a benchmark introduced in the context of symbolic regression for science, specifically within the software libraries PySR and SymbolicRegression.jl. It quantifies the capacity of symbolic regression algorithms to recover historical empirical equations known in the scientific literature, from both original and synthetic datasets. The benchmark is designed to evaluate interpretable machine learning models in their ability to rediscover human-interpretable symbolic forms that underlie scientific phenomena.

1. Definition and Core Purpose

EmpiricalBench serves as a standardized testing suite to assess symbolic regression algorithms by measuring their success in reconstructing empirical formulas from given data. Unlike purely predictive benchmarks or black-box accuracy metrics, EmpiricalBench is oriented toward equation recovery, reflecting the scientific process where interpretable closed-form expressions are sought. The benchmark covers tasks in which the goal is not merely to fit the data but to extract the original symbolic equation that generated the observations, thereby providing an interpretable explanatory model. Recovery is assessed on both historical equations and synthetic functions with known structure.

2. Benchmark Structure and Evaluation Methodology

EmpiricalBench is comprised of a curated set of benchmark tasks, each associated with a historical empirical equation (e.g., the van der Waals equation, Planck's law, or the Michaelis-Menten model) or a synthetically generated equation with known structure. For each task, the benchmark presents a dataset, either derived from actual measurements or sampled from the empirical law, and the symbolic regression algorithm must return candidate expressions.

The evaluation methodology compares the recovered expressions to the ground truth using symbolic equivalence criteria. Typical measures include string match, tree edit distance, and equivalence over a grid of input values (numerical identity). The benchmark may record metrics such as exact recovery rate, mean normalized edit distance, and the fraction of variables and operators matched.

A representative table structure:

Benchmark Task	Recovery Criterion	Score Metric
Michaelis-Menten	Symbolic Equivalence	Exact Match Rate
van der Waals	Tree Edit Distance	Mean Edit Distance
Synthetic Polynomial	Operator Presence	Variable Recall

3. Application in Symbolic Regression Algorithms

EmpiricalBench is integral to the development and assessment of symbolic regression packages, such as PySR and SymbolicRegression.jl. In this context, the benchmark is used to evaluate the underlying evolutionary search algorithms responsible for proposing candidate symbolic models. These algorithms typically proceed through an "evolve–simplify–optimize" loop, iteratively generating and refining symbolic expressions and optimizing unknown scalar constants.

Performance on EmpiricalBench provides concrete feedback on aspects such as:

• Search strategy effectiveness: How well does the evolutionary loop recover correct formulas?
• Numeric optimization robustness: Are coefficients discovered to high accuracy?
• Simplification and generalization: Are outputs minimal and interpretable?

Algorithm tuning—including operator selection, population management, and regularization—can be guided by EmpiricalBench results.

4. Impact on Scientific Interpretability and Model Selection

By focusing on empirical equation recovery, EmpiricalBench directly advances the interpretability of machine learning in scientific domains. The benchmark operationalizes the goal of rediscovering physical laws rather than only fitting data, thus favoring models that provide both predictive performance and human-understandable rationale. Its adoption allows researchers to rigorously compare symbolic regression strategies, calibrate complexity-vs.-accuracy tradeoffs, and select models that are likely to generalize or be adopted in scientific workflows.

A plausible implication is that EmpiricalBench could facilitate broader acceptance of symbolic regression as a standard scientific modeling tool, given its emphasis on equation discovery.

5. Historical Context and Benchmark Scope

EmpiricalBench represents an evolution in the benchmarking of machine learning for science, moving beyond conventional tabular or predictive benchmarks toward the recovery of interpretable, human-recognizable mathematical forms. The benchmark includes both canonical problems known for historical equation fitting and synthetic problems to stress-test regression in high-noise or multi-modality settings.

The scope covers equations from physics, biology, chemistry, and engineered systems; each benchmark instance is clearly documented with its origin, physical meaning, and typical input/output domains. This breadth is designed to reflect the diversity of modeling challenges encountered in empirical sciences.

6. Integration, Accessibility, and Extensions

EmpiricalBench is integrated into PySR and SymbolicRegression.jl libraries. It is accessible to users via public source code, documentation, and standardized usage interfaces. The benchmark is intended to be extensible, allowing practitioners to add new tasks, equations, or datasets to reflect emerging modeling needs. Results can be reported, compared, and reproduced across research groups, facilitating cumulative progress and transparency in the assessment of scientific machine learning algorithms.

A plausible implication is that, by establishing open protocols and sharing empirical benchmark results, the community can accelerate the development of interpretable machine learning methods specifically tuned for scientific equation discovery.