Minimum detection-rate threshold for AI-assisted PRA model review

Determine the minimum error detection rate that a specifically trained generative AI must achieve when reviewing Probabilistic Risk Assessment (PRA) models, such as Event Trees, Fault Trees, or dynamic statechart-based models, to justify leveraging the technology in practice.

Background

The paper explores the use of generative AI as a support tool for reviewing PRA models, noting that AI can potentially achieve high error and inconsistency detection rates. However, the authors emphasize that adopting such a tool requires clarity on performance thresholds to ensure the benefits outweigh the risks.

Within this context, the authors explicitly raise unresolved questions about what detection performance is sufficient to warrant the use of AI in PRA review workflows. Establishing a minimum acceptable detection rate is necessary to guide deployment, evaluation, and regulatory acceptance of AI-assisted reviews.