Dice Question Streamline Icon: https://streamlinehq.com

Standardizing training compute measurement with quantization and dropout

Determine how training compute metrics used for governance should account for techniques such as quantization and dropout, and propose a standardized definition and measurement methodology.

Information Square Streamline Icon: https://streamlinehq.com

Background

Training compute is widely used as a risk proxy, but modern training techniques complicate consistent measurement.

Standardizing how compute metrics incorporate quantization, dropout, and other methods is necessary for fair and effective regulation.

References

It's also unclear how measures of training compute should take into account techniques such as quantization and drop-out.

Open Problems in Technical AI Governance (2407.14981 - Reuel et al., 20 Jul 2024) in Section 7.1 “Translation of Governance Goals into Policies and Requirements”