Reproducibility of complete seeded calibration searches

Establish whether a full seeded numerical calibration search can be reproduced from the archived knowledge package, experiment definition, optimizer settings, and evaluation records, rather than reproducing only individual saved evaluations.

Background

The numerical replay experiment reproduced eleven individual saved parameter evaluations, including several byte-identical metric files, but it did not replay complete seeded searches or regenerate agent-authored formulations.

The authors therefore leave unresolved whether the archive is sufficient to reproduce an entire seeded search, including its sequence of evaluations and final result.

References

The same apparatus can test the questions these conditions raise: whether a frozen knowledge package and experiment definition hold at a new site, with a different observation combination or under a different task, and whether a full seeded search can be reproduced from the archive.

— Delegating Earth science model calibration to knowledge-guided LLM agents  (2610.18378 - Zhu et al., 13 Oct 2026) in Section 4, Discussion and conclusion