Scale thorough mechanistic analysis of model computations

Develop scalable methodologies for thorough mechanistic analysis of the computations inside modern neural networks so that auditors can obtain strong, general assurances about model behavior across broad classes of inputs, extending beyond small or simple tasks.

Background

The authors argue that mechanistic understanding of internal computations can provide stronger assurances than black-box testing alone, because it enables predictive reasoning over classes of inputs rather than individual examples. They note prior work achieving detailed analyses on simple tasks but emphasize that scaling this level of understanding remains unresolved.

This scaling challenge is central to using interpretability for audits: without scalable, thorough analysis, auditors cannot reliably translate internal insights into broad behavioral guarantees for large, complex models.