Required rigor for high-stakes AI evaluations

Ascertain how to determine the appropriate level of rigor for evaluations of decision-making AI systems in high-stakes settings, relative to other applications, so that confidence levels match the potential consequences of deployment.

Background

Different deployment contexts demand different levels of evaluation confidence. High-stakes uses (e.g., health, finance, critical infrastructure) arguably require stronger assurance than consumer applications, but current practice lacks principled guidance on tailoring rigor.

Clarifying how to set and justify rigor thresholds by use case would support risk-proportional governance and compliance efforts.