Dice Question Streamline Icon: https://streamlinehq.com

Defining verifiable model registry requirements for dynamic systems

Specify the canonical information that should be stored in registries tracking frequently updated, multi‑component AI systems and develop mechanisms to verify the accuracy and integrity of registry entries over time.

Information Square Streamline Icon: https://streamlinehq.com

Background

Modern AI systems are dynamic, comprising multiple evolving components, which creates oversight challenges.

Model registries could support tracking and verification, but it is unclear what information is necessary and how it should be validated to provide reliable accountability.

References

However, it's not clear what information should be stored in such a registry, nor how the information could be verified.

Open Problems in Technical AI Governance (2407.14981 - Reuel et al., 20 Jul 2024) in Section 5.3.2 “Verification of Dynamic Systems”