Scope of Risks Effectively Detectable via AI Red-Teaming

Determine which categories of undesirable behaviors, model limitations, and misuse risks in generative AI systems can or should be effectively detected and mitigated through AI red-teaming exercises.

Background

The paper notes that policy definitions of AI red-teaming, including those in the U.S. Executive Order, are vague about what kinds of harms or vulnerabilities such exercises can realistically surface and mitigate. To ground future standards, the authors highlight that the basic scope of what red-teaming can effectively catch remains unresolved.

Clarifying the classes of risks that AI red-teaming can reliably expose is essential for scoping evaluations, allocating resources, and setting expectations for regulators and stakeholders.