The Fall of an Algorithm: Characterizing the Dynamics Toward Abandonment (2404.13802v2)

Abstract: As more algorithmic systems have come under scrutiny for their potential to inflict societal harms, an increasing number of organizations that hold power over harmful algorithms have chosen (or were required under the law) to abandon them. While social movements and calls to abandon harmful algorithms have emerged across application domains, little academic attention has been paid to studying abandonment as a means to mitigate algorithmic harms. In this paper, we take a first step towards conceptualizing "algorithm abandonment" as an organization's decision to stop designing, developing, or using an algorithmic system due to its (potential) harms. We conduct a thematic analysis of real-world cases of algorithm abandonment to characterize the dynamics leading to this outcome. Our analysis of 40 cases reveals that campaigns to abandon an algorithm follow a common process of six iterative phases: discovery, diagnosis, dissemination, dialogue, decision, and death, which we term the "6 D's of abandonment". In addition, we highlight key factors that facilitate (or prohibit) abandonment, which include characteristics of both the technical and social systems that the algorithm is embedded within. We discuss implications for several stakeholders, including proprietors and technologists who have the power to influence an algorithm's (dis)continued use, FAccT researchers, and policymakers.

• The paper introduces a comprehensive 6-phase model that captures the lifecycle of algorithm abandonment from initial discovery to final decommissioning.
• It employs thematic analysis of 40 real-world cases to pinpoint socio-technical factors influencing abandonment decisions.
• The study highlights implications for technologists and policymakers, emphasizing the need for contestable, transparent algorithm design.

Analyzing Algorithm Abandonment: A Thematic Exploration

The paper "The Fall of an Algorithm: Characterizing the Dynamics Toward Abandonment" offers a comprehensive examination of the seldom-discussed but increasingly significant phenomenon of algorithm abandonment. This paper presents a thematic analysis of algorithm abandonment, a decision by organizations to discontinue the development, deployment, or usage of an algorithmic system due to its potential or actual harms. Through an investigation of 40 cases of algorithmic systems that were either successfully abandoned or faced calls for abandonment, the authors outline a structured insight into the dynamics of this process.

Key Insights and Themes

The authors identify a six-phase model they term the "6 D's of abandonment," which they argue encapsulates the dynamics of algorithm abandonment: discovery, diagnosis, dissemination, dialogue, decision, and death. These phases outline a generally observed lifecycle in the abandonment of algorithmic systems.

• Discovery: This initial phase concerns the public awareness of an algorithm's existence and the emergence of its potential harms. The paper notes that this could either be sudden, with a catalytic event, or gradual, with growing recognition of an algorithm's problematic aspects. In many instances, discovery was characterized by realizing the algorithm's existence or its adverse effects, leading to wider critique.
• Diagnosis: This phase involves an in-depth examination to ascertain the extent of the harms caused by the algorithm. This is often manifested through algorithm audits and investigations into design decisions. The paper notes how external audits, sometimes involving affected communities, played critical roles in diagnosing systemic issues within algorithms.
• Dissemination: Having diagnosed the harms, efforts are made to raise awareness and put pressure on algorithm owners through public discourse, often leveraging media coverage to amplify calls for action. This dissemination plays a crucial role in getting the attention of policymakers and larger public entities that can influence the fate of the algorithm.
• Dialogue: The increased visibility often leads to dialogue between algorithm owners and critics. The paper examines how these dialogues, which may occur in various forms—ranging from formal legislative debates to informal social media discussions—can either obstruct or facilitate the abandonment decision, depending on the receptivity and responses from the algorithm owners.
• Decision: At this critical juncture, a decision is made either to repair, ignore, or abandon the algorithm. The paper highlights the complexity of this decision-making process and its dependence on numerous factors, including social pressures and regulatory landscapes.
• Death: Following a decision to abandon, the final phase involves the steps taken to halt the algorithm's operations and address its lasting impacts. This phase also contemplates the extent to which reparations are made, both to individuals harmed and in larger systemic contexts.

Factors Influencing Abandonment

The authors identify several socio-technical factors that influence whether an algorithm is abandoned. These include perspectives like the public awareness of the algorithm, its auditability, and the broader regulatory environment. Importantly, the paper emphasizes the complexities of the ecosystems surrounding algorithms, where algorithmic systems are rarely isolated and often deeply embedded in larger socio-technical infrastructures.

Implications for Stakeholders

The analysis in this paper has substantial implications for several stakeholders, including technologists, policymakers, and FAccT researchers. The authors suggest that technologists should aim to design systems that are easily contestable and abandonable, especially at different lifecycle stages. For policymakers, the findings underscore the importance of creating explicit legal frameworks that facilitate the ban or restriction of harmful algorithms. Meanwhile, researchers are encouraged to build tools and infrastructures that help in auditing and making algorithmic systems more transparent and accountable.

Concluding Thoughts

This paper is an essential contribution to the discourse on algorithmic fairness and accountability, spotlighting the critical yet underexplored area of algorithm abandonment. It prompts further exploration into how organizations can create more ethical and less harmful technological systems by considering abandonment as a legitimate response to irreparable algorithmic harms. As AI and algorithms increasingly permeate various facets of everyday life, understanding the dynamics of their abandonment becomes ever more crucial.