Calibration requirements for promoting agent-reported bands to calibrated intervals

Determine the amount and type of physical ground-truth measurement required to convert agent-reported uncertainty bands into calibrated intervals for a given problem class in verification-and-validation instrumented pipelines, potentially using conformal prediction over extraction–measurement residuals.

Background

Instrumented data carries uncertainty bands that originate from perception-layer extraction. These bands are initially self-reported and must be empirically calibrated to be trustworthy. The authors call for formal calibration protocols and suggest conformal prediction as a candidate approach.

Establishing the calibration burden is necessary before the instrumented data can be reliably consumed as ground truth for training, validation, or surrogate modeling.

References

Nine open questions will determine whether instrumented data matures into a recognised substrate for scientific machine learning. Calibration protocols. How much physical ground truth is needed before agent-reported bands can be promoted to calibrated intervals on a class? Conformal prediction over extraction--measurement residuals is a natural candidate.

— Instrumented data for causal scientific machine learning  (2606.07865 - Wilke, 5 Jun 2026) in Section 7, Methodological questions for the community, Item 1