The Measure and Mismeasure of Fairness (1808.00023v3)

Abstract: The field of fair machine learning aims to ensure that decisions guided by algorithms are equitable. Over the last decade, several formal, mathematical definitions of fairness have gained prominence. Here we first assemble and categorize these definitions into two broad families: (1) those that constrain the effects of decisions on disparities; and (2) those that constrain the effects of legally protected characteristics, like race and gender, on decisions. We then show, analytically and empirically, that both families of definitions typically result in strongly Pareto dominated decision policies. For example, in the case of college admissions, adhering to popular formal conceptions of fairness would simultaneously result in lower student-body diversity and a less academically prepared class, relative to what one could achieve by explicitly tailoring admissions policies to achieve desired outcomes. In this sense, requiring that these fairness definitions hold can, perversely, harm the very groups they were designed to protect. In contrast to axiomatic notions of fairness, we argue that the equitable design of algorithms requires grappling with their context-specific consequences, akin to the equitable design of policy. We conclude by listing several open challenges in fair machine learning and offering strategies to ensure algorithms are better aligned with policy goals.

An Analysis of "The Measure and Mismeasure of Fairness"

The paper "The Measure and Mismeasure of Fairness" by Corbett-Davies et al. provides a comprehensive evaluation of existing fairness definitions in the field of machine learning, particularly focusing on how these definitions interact with decision policies in various contexts. The authors categorize these definitions into two broad families: those constraining effects on disparities (e.g., error rate parity) and those limiting effects of protected attributes (e.g., anti-classification). Through rigorous analysis, the paper argues that these formal definitions often result in Pareto-dominated decision policies—where no improvement is possible on one objective without worsening another.

Critical Evaluation of Fairness Definitions

The paper begins by categorizing fairness definitions into two groups: those that address the effects of decisions on disparity measures, and those that limit how protected attributes, like race, influence decisions. The authors rigorously analyze popular fairness measures such as demographic parity and equalized odds, revealing these measures tend to be at odds with decision policies that would maximize utility. Their argument hinges on the statistical concept of inframarginality, which effectively demonstrates why in many cases, achieving a given fairness constraint leads to less utility for individuals—especially in contexts where utility should reflect individual best interests, like medical diagnoses.

Implications on Decision-Making Policies

One of the significant insights throughout the paper is that adhering to strict fairness constraints can paradoxically harm those they are meant to protect. For instance, the paper discusses how responding to counterfactual fairness would require decisions that ignore differences in risk distribution between groups, consequently lowering aggregate utility. Such decisions would not align with many institutions’ objectives, such as increasing student diversity while maintaining academic standards or maximizing patient health outcomes. Specifically, they showed that in cases with multiple objectives, fairness constraints can force one to choose policies not lying on the Pareto frontier, meaning there are other feasible policies that can improve outcomes on all objectives simultaneously.

Theoretical Framework and Statistical Analysis

The authors introduce a sound framework for understanding the limitations of fairness definitions. Through mathematical proofs, they make the case for the non-existence of feasible policies satisfying fairness constraints like equalized odds or demographic parity without incurring utility losses. Their statistical proofs employ the notation of randomness across possible decisions to analytically show why focusing solely on error metrics in fairness is problematic due to the inherent trade-offs unaddressed by such metrics.

Consequentialist Perspective and Recommendations

The paper leans towards a consequentialist perspective—suggesting fairness should grapple with the real-world consequences rather than solely adhering to rigid mathematical definitions. This shift in perspective aligns more closely with views in policy design rather than purely algorithmic measures. As a way forward, the authors suggest recognizing the difficulty in balancing decision priorities and that fairness in algorithm design should be contextually grounded, customized to the specific needs and goals of the given application area. They advocate for embracing the complexity inherent in fairness and suggest algorithms are integral to policy-making choices and not detached, technical entities.

Open Challenges and Conclusion

Concluding the research, several challenges and strategies are proposed for the advancement of fair machine learning. Crucially, improving data collection practices, comprehensively understanding calibration across different groups, and identifying robust frameworks for algorithmic assessment are seen as pivotal aspects moving forward. The paper underscores how ongoing legislative and ethical debate shape emerging definitions of fairness in contexts like lending, medicine, and criminal justice. This work pushes the envelope by making a solid case for fair ML research, making clear the pitfalls of rigidly applied fairness definitions without due consideration of real-world applicability and context.

Overall, this paper enriches the dialogue on fair AI systems, calling for an informed and careful approach to incorporate fairness truly aligned with broader policy goals. The analytical depth provided by the authors is commendable and sets a precedent for future studies that seek to bridge the gap between fairness in theory and practice.