Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraints

Abstract: Language is increasingly being used to define rich visual recognition problems with supporting image collections sourced from the web. Structured prediction models are used in these tasks to take advantage of correlations between co-occurring labels and visual input but risk inadvertently encoding social biases found in web corpora. In this work, we study data and models associated with multilabel object classification and visual semantic role labeling. We find that (a) datasets for these tasks contain significant gender bias and (b) models trained on these datasets further amplify existing bias. For example, the activity cooking is over 33% more likely to involve females than males in a training set, and a trained model further amplifies the disparity to 68% at test time. We propose to inject corpus-level constraints for calibrating existing structured prediction models and design an algorithm based on Lagrangian relaxation for collective inference. Our method results in almost no performance loss for the underlying recognition task but decreases the magnitude of bias amplification by 47.5% and 40.5% for multilabel classification and visual semantic role labeling, respectively.

• The paper introduces the RBA framework that uses corpus-level constraints and Lagrangian relaxation to significantly reduce gender bias amplification.
• Empirical results reveal that RBA reduces bias amplification by 40.5% for vSRL and 47.5% for MLC without degrading model performance.
• The methodology rigorously quantifies bias through bias scores, underscoring how ML models can amplify societal stereotypes in training datasets.

Reducing Gender Bias Amplification using Corpus-level Constraints

Introduction

The paper "Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraints" by Zhao et al. addresses the significant and well-documented issue of gender bias in machine learning models, particularly those used in visual recognition tasks. The paper empirically demonstrates that datasets for tasks like multilabel object classification (MLC) and visual semantic role labeling (vSRL) are imbued with substantial gender biases, which are exacerbated by the models trained on these datasets. To mitigate this, the authors propose an innovative calibration approach that incorporates corpus-level constraints through an algorithm based on Lagrangian relaxation.

Empirical Demonstration of Bias

The paper highlights two primary tasks—vSRL using the imSitu framework and MLC using the MS-COCO dataset. The authors provide compelling empirical evidence that both datasets are laden with substantial gender biases. For instance, activities such as cooking are depicted far more frequently with female agents in the training datasets. Astonishingly, the training dataset reflects a bias of 33% more likelihood of associating cooking with females, which the model amplifies to 68% at test time. This disturbing amplification underscores the critical need for mechanisms to quantify and mitigate such biases.

Measurement and Quantification of Bias

The authors develop a robust framework to rigorously measure and visualize biases in the datasets and subsequently in the models. They employ bias scores to represent the correlation between gender and certain activities or objects. These scores are validated against the distribution in an evaluation set to measure bias amplification. For instance, with the verb cooking, the bias towards females in the imSitu training dataset is 0.66, which balloons to 0.84 in the model predictions on the evaluation set.

RBA: Reducing Bias Amplification

To address bias amplification, the paper introduces the novel RBA (Reducing Bias Amplification) framework. RBA functions by imposing corpus-level constraints designed to match the distribution of gender indicators in the training dataset with those in the evaluation set. This effectively curbs the tendency of the model to disproportionately align certain activities or objects with a specific gender.

The RBA algorithm is iterative, employing Lagrangian relaxation to solve the complex optimization problem imposed by these constraints. The iterative nature of the algorithm allows the decomposition of the joint inference, making it feasible to integrate with existing inference methods. On benchmark datasets like imSitu and MS-COCO, the application of RBA results in a substantial reduction in the mean bias amplification—by 40.5% for vSRL and by 47.5% for MLC—without compromising the performance of the underlying recognition task.

Implications and Future Directions

The implications of this research are manifold. Practically, this approach can be readily adopted in real-world applications, attenuating the reinforcement of societal stereotypes by AI systems. Theoretically, this work paves the way for more nuanced and comprehensive treatments of bias reduction in machine learning models.

Future research could build upon this by exploring various structured predictors to assess if and how they amplify bias differently. Additionally, expanding the scope beyond gender to include other dimensions of bias such as race and ethnicity would be critical. Investigating the effectiveness of RBA on other structured tasks, like pronoun reference resolution, opens new avenues for ensuring fairness in AI systems across different domains.

Conclusion

The paper by Zhao et al. provides a compelling and methodologically sound approach to tackling gender bias amplification in visual recognition tasks. Employing corpus-level constraints and leveraging Lagrangian relaxation, the proposed RBA framework significantly mitigates bias without adversely affecting model performance. This research constitutes a vital step towards the development of AI systems that are not only technically robust but also socially responsible.