Post-processing for Individual Fairness (2110.13796v1)

Abstract: Post-processing in algorithmic fairness is a versatile approach for correcting bias in ML systems that are already used in production. The main appeal of post-processing is that it avoids expensive retraining. In this work, we propose general post-processing algorithms for individual fairness (IF). We consider a setting where the learner only has access to the predictions of the original model and a similarity graph between individuals, guiding the desired fairness constraints. We cast the IF post-processing problem as a graph smoothing problem corresponding to graph Laplacian regularization that preserves the desired "treat similar individuals similarly" interpretation. Our theoretical results demonstrate the connection of the new objective function to a local relaxation of the original individual fairness. Empirically, our post-processing algorithms correct individual biases in large-scale NLP models such as BERT, while preserving accuracy.

• The paper introduces a novel graph smoothing framework that enforces individual fairness without retraining, using Laplacian regularization.
• The authors provide theoretical insights and empirical evidence that localized fairness constraints maintain model accuracy in NLP tasks.
• The paper demonstrates that the method effectively reduces biases, such as gender-based inconsistencies, in deployed ML models.

Post-processing for Individual Fairness: An Analytical Perspective

In addressing the pervasive issue of algorithmic bias in ML models, "Post-processing for Individual Fairness" by Petersen et al. provides an innovative approach, focused on post-processing algorithms to enforce Individual Fairness (IF). The paper explores a critical area of algorithmic fairness—ensuring similar treatments for similar individuals—through a computationally efficient method that circumvents the need for retraining costly models.

Key Contributions and Methodology

The authors propose a novel framework that treats the problem of enforcing IF as a graph smoothing problem, effectively leveraging graph Laplacian regularization. This enables the integration of fairness without compromising predictive accuracy, especially when dealing with large-scale NLP models like BERT. The approach assumes access to model predictions and a similarity graph rather than original model parameters or retraining capabilities, creating a practical solution appealing to many ML practitioners.

Significant contributions of the work include:

1. Graph Smoothing Formulation: The transformation of the IF post-processing task into a graph smoothing problem using graph Laplacian regularization encapsulates the principle of "treat similar individuals similarly", driving the fairness constraints. This framework accommodates large datasets by proposing a coordinate descent algorithm that efficiently handles the computational load.
2. Theoretical Insights: The paper provides a theoretical basis for how graph Laplacian regularization enforces local IF, distinguishing it from global IF, which tends to negatively impact model performance due to its extensive constraints.
3. Empirical Validation: Experiments conducted with BERT models demonstrate the method's efficacy in correcting biases while maintaining prediction accuracy. Comparative analysis with direct IF constraint methods highlights the advantage of the localized approach in balancing fairness and accuracy.
4. Consistency Across Frameworks: The work extends beyond binary classification to accommodate multi-dimensional outputs and proposes generalization for various types of output space discrepancy measures, such as KL divergence and other Bregman divergences.

Numerical Results and Discussion

The empirical evaluation presents substantial improvement in enforcing individual fairness over the traditional IF constraint methods. The authors used sentiment prediction tasks and text classification in identity-sensitive datasets to benchmark their method against existing techniques such as SenSeI.

A notable result from their experiments is a marked improvement in gender-based prediction consistency in a biography classification task with minimal accuracy loss. Similarly, in toxicity detection tasks, the method significantly reduces the bias in predictions tied to identity words.

Implications and Future Directions

The implications of this research are profound for the deployment of fair ML systems, especially in high-stakes domains involving text-based decision-making systems. By facilitating post-processing fairness without environmental and computational costs associated with retraining, the paper provides a pathway for ethical adoption of pre-trained models.

Future work could explore the scalability of graph-based approaches for fairness in more complex model architectures and diverse datasets. Additionally, investigating the trade-offs in localized versus global regularization to find an optimal balance that can be tailored to varied contexts and fairness definitions remains an open question. Overall, this paper expands the toolkit available for enhancing fairness in deployed ML models, emphasizing environmental and computational sustainability.