Dice Question Streamline Icon: https://streamlinehq.com

Establish documentation requirements for dataset provenance and ADMS design

Establish a comprehensive standard specifying what should be included when documenting the origin of datasets and the design of automated decision-making systems (ADMS) to support effective ethics-based auditing.

Information Square Streamline Icon: https://streamlinehq.com

Background

The authors emphasize the importance of documentation (e.g., model cards and reporting formats) for traceability and accountability in EBA. However, they note it remains unresolved what precise information about dataset provenance and ADMS design should be mandated to enable meaningful audits.

References

To mention a few: was the right ADMS built, and was it build right? (Dobbe et al., 2019); what are the appropriate ethical assessment criteria for ADMS? (D'Agostino & Durante, 2018); what should be included when documenting the origin of a dataset or the design of an ADMS? (Raji et al., 2020); how to account for the power asymmetries between system owners, regulating bodies and decision-subjects when designing EBA procedures? (Crawford et al., 2019); and, who within STS is responsible for distributed moral action? (Floridi, 2016b). These questions are left for future research.

— Ethics-Based Auditing of Automated Decision-Making Systems: Intervention Points and Policy Implications (2111.04380 - Mokander et al., 2021) in Section 5 (Discussion: limitations and risks associated with EBA), paragraph beginning “It should also be noted that when analysing STS...”