Fairness Under Unawareness: Assessing Disparity When Protected Class Is Unobserved (1811.11154v1)

Abstract: Assessing the fairness of a decision making system with respect to a protected class, such as gender or race, is challenging when class membership labels are unavailable. Probabilistic models for predicting the protected class based on observable proxies, such as surname and geolocation for race, are sometimes used to impute these missing labels for compliance assessments. Empirically, these methods are observed to exaggerate disparities, but the reason why is unknown. In this paper, we decompose the biases in estimating outcome disparity via threshold-based imputation into multiple interpretable bias sources, allowing us to explain when over- or underestimation occurs. We also propose an alternative weighted estimator that uses soft classification, and show that its bias arises simply from the conditional covariance of the outcome with the true class membership. Finally, we illustrate our results with numerical simulations and a public dataset of mortgage applications, using geolocation as a proxy for race. We confirm that the bias of threshold-based imputation is generally upward, but its magnitude varies strongly with the threshold chosen. Our new weighted estimator tends to have a negative bias that is much simpler to analyze and reason about.

• The paper decomposes bias in threshold-based imputation, pinpointing inter- and intra-geolocation outcome variations.
• It introduces a weighted estimator leveraging soft classification to reduce bias compared to traditional proxy models.
• Simulations on HMDA data validate the approach, highlighting improved interpretability and potential for regulatory fairness assessments.

Fairness Under Unawareness: Evaluating Bias and Disparity Using Unobserved Protected Class Estimations

The challenge of ensuring fairness in automated decision-making when protected class memberships, such as race or gender, are not observable is a prominent issue in compliance assessments and algorithmic accountability. This paper addresses this complexity by evaluating the biases intrinsically linked to the use of proxy models that estimate protected class membership for members of unobserved demographics. A common approach in regulatory contexts is to employ the Bayesian Improved Surname Geocoding (BISG) method, which uses proxy variables such as surname and geolocation to estimate race or ethnicity. The main focus of this paper is to analyze biases associated with this method and to propose an alternative estimation technique that may offer more precise assessments.

Key Contributions and Findings

1. Bias Decomposition in Threshold-Based Imputation: The authors break down the biases involved in estimating demographic disparities using threshold-based imputation methods and highlight two main sources of bias: inter-geolocation outcome variation and intra-geolocation outcome variation. This decomposition is crucial as it allows practitioners to better understand and predict when such estimation methods might lead to over- or underestimation of disparities. It is demonstrated that the degree of bias is influenced heavily by the chosen threshold, further complicating the utilization of such methods without sufficient ground-truth knowledge.
2. Proposal of a Weighted Estimator: The paper introduces a weighted estimator that leverages soft classification rather than hard imputation. This new estimator replaces binary label assignments with a probabilistic assessment of class membership, reducing the bias to a function of the conditional covariance between outcomes and true class membership. This approach significantly simplifies the analysis and understanding of bias compared to threshold-based methods.
3. Simulations and Case Studies: Through simulations and analysis of the Home Mortgage Disclosure Act (HMDA) dataset, the paper validates its theoretical framework. The numerical results confirm the presence of an upward bias in threshold-based estimates, influenced heavily by the socio-economic variation tied to geolocation. The proposed weighted estimator generally shows a negative bias but provides a more transparent and interpretable source of bias, making it potentially favorable for regulatory assessments.

Implications and Future Directions

Practically, the paper underscores the need for careful consideration in choosing methods for assessing fairness when protected class data are missing. The introduced weighted estimator presents a promising alternative for such evaluations. Theoretically, this work enhances the understanding of bias dynamics in proxy-based fairness assessments.

Future work could address several dimensions unexamined in this paper. First, determining optimal proxy model construction and parameterization could enhance the robustness of estimations across diverse domains and geographies. Second, exploring the generalization of these methods to more complex, multifaceted models of decision-making beyond binary classification could broaden applicability. Finally, integrating privacy-preserving technologies and secure computation methods might alleviate data sharing concerns, enabling more extensive real-world validation.

In summary, while the problem of fair assessments under unawareness remains complex, this paper provides valuable insights and methodological advancements that could shape regulatory approaches and improve algorithmic fairness in high stakes environments.