Principled integration of heterogeneous risk-model evidence

Develop principled methods for selecting and aggregating experimental or simulated evaluations, historical or observational data, and expert-elicited parameter estimates as inputs to AI risk models.

Background

AI risk models may draw on three broad evidence sources: experimental or simulated evaluations and benchmarks, historical or observational data and incident reports, and expert-elicited parameter estimates or structured expert judgments. Each source has important limitations, and the paper argues that robust future models will likely need to combine them. The unresolved issue is how to choose among these sources and aggregate them in a principled way so that the resulting risk estimates remain meaningful and appropriately calibrated.

References

All three types have both advantages and drawbacks and, in the future, we believe it will be desirable to combine them. However, principled ways of selecting and aggregating these sources of information remain an open problem.

— Open Problems in AI Risk Modeling: Insights from a Workshop on the Technical Foundations of AI Risk Modeling  (2609.03178 - Jackson et al., 2 Sep 2026) in Section 4.1, “Data Types and Challenges”