Reliability of identifying misuse in closed foundation model monitoring for vulnerability detection

Establish whether monitoring and moderation of closed foundation model services can reliably identify illicit use of such models for automated vulnerability detection given the dual-use nature of security testing.

Background

The authors compare open and closed release paradigms and note that even with better monitoring capabilities in closed systems, distinguishing legitimate security testing from malicious automated vulnerability detection is nontrivial.

This reliability question affects the comparative assessment of marginal risk between open and closed model releases and highlights the need for evidence about the effectiveness of monitoring systems deployed by closed model providers.