Rethinking Bias Mitigation: Fairer Architectures Make for Fairer Face Recognition (2210.09943v3)

Abstract: Face recognition systems are widely deployed in safety-critical applications, including law enforcement, yet they exhibit bias across a range of socio-demographic dimensions, such as gender and race. Conventional wisdom dictates that model biases arise from biased training data. As a consequence, previous works on bias mitigation largely focused on pre-processing the training data, adding penalties to prevent bias from effecting the model during training, or post-processing predictions to debias them, yet these approaches have shown limited success on hard problems such as face recognition. In our work, we discover that biases are actually inherent to neural network architectures themselves. Following this reframing, we conduct the first neural architecture search for fairness, jointly with a search for hyperparameters. Our search outputs a suite of models which Pareto-dominate all other high-performance architectures and existing bias mitigation methods in terms of accuracy and fairness, often by large margins, on the two most widely used datasets for face identification, CelebA and VGGFace2. Furthermore, these models generalize to other datasets and sensitive attributes. We release our code, models and raw data files at https://github.com/dooleys/FR-NAS.

Rethinking Bias Mitigation in Face Recognition through Architecture Design

The paper "Rethinking Bias Mitigation: Fairer Architectures Make for Fairer Face Recognition" addresses a critical challenge in the deployment of face recognition systems: the presence of biases based on socio-demographic attributes. Widely used in sensitive applications, these systems often exhibit performance disparities, leading to unjust outcomes. Traditional methods have attempted to mitigate these biases primarily through data preprocessing, in-processing adjustments, or post-processing corrections. However, these strategies have been inadequate for domains like face recognition. The authors propose an innovative shift in focus from these traditional methods to the intrinsic designs of neural network architectures, positing that biases may be inherent within the architectures themselves.

The research conducted introduces the concept of leveraging Neural Architecture Search (NAS) alongside Hyperparameter Optimization (HPO) to explore the space of network architectures for fairness. This approach diverges from conventional techniques that typically fix architectures while exploring debiasing strategies. By jointly optimizing architectures and hyperparameters specifically for fairness alongside accuracy, the authors report significant improvements in both these dimensions.

The authors conducted NAS+HPO experiments to identify architectures that achieve a balance between accuracy and fairness. They utilized a multi-objective approach that evaluates the trade-offs between these objectives, employing datasets such as CelebA and VGGFace2, prominent benchmarks in face recognition tasks. The paper's results indicate that the newly derived architectures Pareto-dominate existing high-performance models and bias mitigation techniques across multiple metrics, including on datasets not seen during training, thereby evidencing generalization across data distributions and sensitive attributes.

An intriguing insight from this paper is the reported independence of fairness from model size, suggesting that factors other than the number of parameters contribute to fairness. This finding disrupts the prevalent notion that larger models inherently improve fairness through increased capacity and expressiveness. Additionally, the paper highlights that architectures identified through NAS demonstrated lower linear separability of protected attributes, hinting at the complex interactions between network design and bias.

In practice, these findings imply the potential to deploy more equitable face recognition systems that do not compromise on overall performance. Theoretically, this paper opens avenues for future work exploring architecture design principles intrinsically aligned with fairness. The authors advocate for further exploration of these principles in diverse contexts and datasets to verify the broader applicability of their conclusions.

From an academic standpoint, this research enriches the discourse on fairness in machine learning by introducing architectural considerations to the forefront of bias mitigation strategies. Traditional perspectives often focus on data-driven solutions; however, this paper underscores the significance of architectural biases as an indispensable component of this dialogue.

In conclusion, the implications of this work extend beyond face recognition, suggesting a paradigm shift in approaching fairness across AI systems. It stands as an invitation to researchers to reevaluate how fairness objectives can inform all stages of model development, from architecture search to deployment. This work ultimately contributes a nuanced perspective to the design of machine learning systems that are not only performant but also principled in terms of equity.