Information requirements for dynamic model registries

Determine what specific information should be stored and how it should be verified in registries that track AI models over time, including versioning and post-deployment modifications, to enable oversight of dynamic systems.

Background

Modern AI systems frequently change after deployment through updates, filters, and components, complicating oversight. Registries could track these changes, but their design requires consensus on what data to record and how to verify it.

Clarifying registry information requirements would aid reproducibility, accountability, and alignment with regulatory needs.

References

One approach could be to have registries that track models over time, however, it's not clear what information should be stored in such a registry, nor how the information could be verified.

— Open Problems in Technical AI Governance (2407.14981 - Reuel et al., 2024) in Section 5.3.2 Verification of Dynamic Systems