Dice Question Streamline Icon: https://streamlinehq.com

Determine whether an ADMS is the right system and whether it is built correctly

Determine, for a given decision-making task, whether the selected automated decision-making system (ADMS) appropriately addresses the task ('built the right system') and verify whether the ADMS has been constructed correctly ('built the system right').

Information Square Streamline Icon: https://streamlinehq.com

Background

The paper frames automated decision-making systems (ADMS) as components of larger sociotechnical systems and argues that ethics-based auditing (EBA) must consider both processes and impacts. Within this framing, the authors note that a systems approach leaves several detailed questions unanswered.

A core unresolved issue they list is how to assess both problem selection (whether the ADMS is appropriate for the task) and implementation correctness (whether it is built correctly), which is foundational for any EBA process to be meaningful.

References

To mention a few: was the right ADMS built, and was it build right? (Dobbe et al., 2019); what are the appropriate ethical assessment criteria for ADMS? (D'Agostino & Durante, 2018); what should be included when documenting the origin of a dataset or the design of an ADMS? (Raji et al., 2020); how to account for the power asymmetries between system owners, regulating bodies and decision-subjects when designing EBA procedures? (Crawford et al., 2019); and, who within STS is responsible for distributed moral action? (Floridi, 2016b). These questions are left for future research.

— Ethics-Based Auditing of Automated Decision-Making Systems: Intervention Points and Policy Implications (2111.04380 - Mokander et al., 2021) in Section 5 (Discussion: limitations and risks associated with EBA), paragraph beginning “It should also be noted that when analysing STS...”