Inherent Trade-Offs in the Fair Determination of Risk Scores (1609.05807v2)

Abstract: Recent discussion in the public sphere about algorithmic classification has involved tension between competing notions of what it means for a probabilistic classification to be fair to different groups. We formalize three fairness conditions that lie at the heart of these debates, and we prove that except in highly constrained special cases, there is no method that can satisfy these three conditions simultaneously. Moreover, even satisfying all three conditions approximately requires that the data lie in an approximate version of one of the constrained special cases identified by our theorem. These results suggest some of the ways in which key notions of fairness are incompatible with each other, and hence provide a framework for thinking about the trade-offs between them.

• The paper proves that no algorithm can simultaneously satisfy calibration within groups, balance for the positive class, and balance for the negative class except under perfect prediction or equal base rates.
• It employs rigorous mathematical proofs to establish that inherent trade-offs exist among fairness conditions in probabilistic classification.
• The study guides algorithm design and policy-making by highlighting the practical limitations and computational challenges of fair risk score assignments.

Inherent Trade-Offs in the Fair Determination of Risk Scores

The paper "Inherent Trade-Offs in the Fair Determination of Risk Scores" by Kleinberg, Mullainathan, and Raghavan addresses critical challenges in algorithmic fairness, particularly in the context of probabilistic classification. The authors propose and formalize three fairness conditions central to debates on algorithmic bias and discrimination: Calibration within groups, Balance for the positive class, and Balance for the negative class. The core contribution of the paper is their proof that, except under highly constrained conditions, no method can satisfy these three fairness conditions simultaneously. This research reveals inherent trade-offs between key notions of fairness, providing a theoretical framework to understand these limitations.

Core Fairness Conditions

The three fairness conditions formalized in this paper are as follows:

1. Calibration within Groups: For each subgroup, if the algorithm assigns a probability $z$ to a person, then $z$ should represent the actual fraction of people with probability $z$ who are positive instances.
2. Balance for the Positive Class: The average predicted score for positive instances should be the same across different groups.
3. Balance for the Negative Class: The average predicted score for negative instances should be the same across different groups.

These conditions stem from various domains such as criminal justice, internet advertising, and medical diagnosis, where algorithmic decisions significantly impact outcomes.

Theoretical Findings

The authors' primary theorem establishes that these fairness conditions cannot be satisfied simultaneously, except in two specific cases:

• Perfect Prediction: Every prediction made by the algorithm is entirely accurate, i.e., it predicts the positive and negative classes with absolute certainty.
• Equal Base Rates: The two groups have an identical proportion of positive instances.

The rigorous proof involves examining the calibration condition and deriving that, for the fairness conditions to hold, the datasets either must allow perfect prediction or must have equal base rates. The authors provide both a formal mathematical argument and an intuitive explanation, underscoring the practical implications of their theoretical results.

Approximate Fairness

The paper further explores approximate versions of the fairness conditions and demonstrates that even achieving these approximately necessitates the data to be within an approximate version of the identified special cases. The authors introduce a continuous function to quantify this approximation, reinforcing their initial impossibility results.

Practical and Theoretical Implications

The implications of this research are profound both in theory and practice.

• Algorithm Design: Designers of machine learning algorithms must acknowledge that ensuring fairness across multiple dimensions is impossible in most practical scenarios. Trade-offs will be necessary, depending on the specific fairness condition that is prioritized.
• Policy and Regulation: Policymakers need to understand that regulatory frameworks mandating simultaneous satisfaction of diverse fairness conditions are inherently unfeasible. The paper can guide the development of more realistic guidelines that recognize these trade-offs.
• Future Research: This work opens new avenues for research, particularly in developing algorithms that can accompany fairness through tailored trade-offs or optimizing fairness under specific constrained setups.

Computational Complexity

The authors provide insights into the computational complexity of achieving these fairness conditions, especially in non-trivial cases. For example, they show that determining whether a non-trivial, integral fair risk assignment exists is NP-complete. This result emphasizes the practical limitations and computational challenges in implementing fair decision-making algorithms.

Conclusion and Future Directions

"Inherent Trade-Offs in the Fair Determination of Risk Scores" elucidates fundamental constraints in achieving simultaneous fairness across multiple dimensions. This acknowledgment shifts the narrative from attempting to achieve a utopian fairness to making informed choices about which fairness aspects to prioritize. Future research might explore dynamic fairness criteria that adapt based on contextual requirements or develop more nuanced fairness definitions tailored to specific applications.

This paper provides a critical foundation to understand the limitations and possibilities of fair algorithms, urging both researchers and practitioners to carefully consider the trade-offs inherent in algorithmic decision-making.