Prediction-Based Decisions and Fairness: A Catalogue of Choices, Assumptions, and Definitions

Abstract: A recent flurry of research activity has attempted to quantitatively define "fairness" for decisions based on statistical and ML predictions. The rapid growth of this new field has led to wildly inconsistent terminology and notation, presenting a serious challenge for cataloguing and comparing definitions. This paper attempts to bring much-needed order. First, we explicate the various choices and assumptions made---often implicitly---to justify the use of prediction-based decisions. Next, we show how such choices and assumptions can raise concerns about fairness and we present a notationally consistent catalogue of fairness definitions from the ML literature. In doing so, we offer a concise reference for thinking through the choices, assumptions, and fairness considerations of prediction-based decision systems.

• The paper's main contribution is a comprehensive catalogue of fairness definitions and assumptions in prediction-based decisions, setting a foundation for equitable ML applications.
• It discusses methodological choices in data selection, model design, and sensitive variable handling that shape fairness outcomes in machine learning contexts.
• The study quantifies trade-offs among various fairness metrics and proposes causal frameworks to improve the assessment of equitable decision-making.

Prediction-Based Decisions and Fairness: Addressing Choices, Assumptions, and Definitions

The paper "Prediction-Based Decisions and Fairness: A Catalogue of Choices, Assumptions, and Definitions" by Mitchell et al. tackles the pressing issue of fairness in prediction-based decision systems, particularly those relying on ML and statistical predictions. Given the widespread adoption of such systems in a variety of sectors—including lending, hiring, law enforcement, and public health—concerns over fair and equitable decision-making have become paramount. This paper attempts to systematize the definitions of fairness, delineate assumptions underlying decision-making, and provide frameworks for assessing fairness across different domains.

Key Themes and Contributions

1. Overview of Fairness in ML

The paper acknowledges the chaotic growth of the fairness discourse in ML, resulting in diverse terminology and definitions. By proposing a comprehensive catalogue of fairness definitions, the authors aim to create a consistent framework to address fairness concerns. They elaborate on the implicit assumptions inherent in prediction-based decisions and how these assumptions may result in biased outcomes across different groups, particularly along lines of race, gender, and socioeconomic status.

2. Methodological Assumptions and Fairness

Mitchell et al. discuss the various assumptions and choices made in the development and implementation of predictive models that impact fairness:

• Policy-related Choices: Decisions about deploying models, the populations they affect, and the decision spaces available are all critical in determining the fairness of a predictive system. Notably, the broader social objective behind a model influences its fairness.
• Data-Driven Decisions: The quality and selection of training data, model selection, and evaluation metrics play a crucial role in shaping model fairness. Problems such as sampling bias and measurement error are identified as significant challenges to equitable predictions.
• Sensitivity to Group Membership: The paper addresses how sensitive variables, often legally designated as protected characteristics, are managed in predictive modeling. There's a discussion about the importance of intersectionality in analyzing fairness across combined attributes such as race and gender.

3. Mathematical Formulations of Fairness

The authors outline various mathematical notions of fairness that have been proposed in academic literature, grounded in both predictive and decision-based frameworks. These include:

• Accuracy and Error Rate Balances: Metrics such as equal false positive rates, true negative rates, and balanced accuracy across groups.
• Predictive Parity and Calibration: Ensuring similar predictive performance metrics for different demographic groups.
• Demographic Parity: A focus on equal decision rates irrespective of the actual outcome.

Despite providing an expansive view of these definitions, the paper highlights fundamental incompatibilities between them, asserting that simultaneous satisfaction of multiple fairness constraints is not always achievable.

4. Causal Frameworks for Fairness

The authors explore causal reasoning to better understand and address fairness issues. By assessing causal pathways and potential outcomes under counterfactual settings (hypothetical changes in sensitive attributes), it becomes possible to evaluate the fairness of predictions on a more granular level. However, they note the complexity of these methods and their reliance on robust assumptions.

5. Implications for Future Research and Policy

The roadmap detailed in this paper outlines a clear pathway for practitioners and researchers to navigate the fairness landscape in ML. The authors call for incorporating insights from both social contexts and quantitative metrics for a more holistic evaluation of fairness. These considerations emphasize that fairness is not just a theoretical challenge but a deeply practical one that impacts real-world outcomes.

Conclusion

The paper serves as a cornerstone in the field of fair machine learning by cataloguing the vast array of fairness definitions and highlighting their applications and limits. By dissecting the methodological choices and laying out potential paths forward, the paper invites deeper engagement with fairness as an iterative, evaluative, and empathetic process. Such work underpins the ongoing effort to reconcile data-driven decision-making with the equitable treatment of all individuals affected by these systems. Future research aims must include broader multidisciplinary collaboration to tackle the intricate social dynamics embedded in technological systems.