Robustness-conditional peer-review protocols for cross-pipeline review

Establish peer-review protocols conditioned on robustness that empirically map how cross-pipeline review degrades from interpolative to extrapolative regimes, specify a minimum diversity requirement across participating LLMs, and define demarcation rules limiting extrapolative checks to provenance, schema, and data-card compliance.

Background

Mature instrumentation pipelines may act as reviewers of sibling pipelines, but reliability depends on whether cases lie inside or outside validation envelopes. Cross-domain or extrapolative review risks confounding by shared LLM artifacts.

The authors call for formal protocols that respect robustness regimes, enforce diversity among LLMs, and restrict extrapolative endorsements to non-mechanistic checks.

References

Nine open questions will determine whether instrumented data matures into a recognised substrate for scientific machine learning. Robustness-conditional peer-review protocols. The community needs an empirical map of how cross-pipeline review degrades from interpolative to extrapolative regimes, a minimum-diversity requirement on participating LLMs, and a demarcation rule restricting extrapolative checks to provenance, schema, and data-card compliance.

— Instrumented data for causal scientific machine learning  (2606.07865 - Wilke, 5 Jun 2026) in Section 7, Methodological questions for the community, Item 6