Characterising Bias in Compressed Models (2010.03058v2)

Abstract: The popularity and widespread use of pruning and quantization is driven by the severe resource constraints of deploying deep neural networks to environments with strict latency, memory and energy requirements. These techniques achieve high levels of compression with negligible impact on top-line metrics (top-1 and top-5 accuracy). However, overall accuracy hides disproportionately high errors on a small subset of examples; we call this subset Compression Identified Exemplars (CIE). We further establish that for CIE examples, compression amplifies existing algorithmic bias. Pruning disproportionately impacts performance on underrepresented features, which often coincides with considerations of fairness. Given that CIE is a relatively small subset but a great contributor of error in the model, we propose its use as a human-in-the-loop auditing tool to surface a tractable subset of the dataset for further inspection or annotation by a domain expert. We provide qualitative and quantitative support that CIE surfaces the most challenging examples in the data distribution for human-in-the-loop auditing.

• The paper introduces the concept of Compression Identified Exemplars (CIE) to pinpoint examples with elevated post-compression error rates.
• The paper shows that pruning and quantization amplify bias by disproportionately deteriorating performance on underrepresented and sensitive attributes.
• The paper advocates human-in-the-loop auditing using CIEs to efficiently target and mitigate fairness issues in compressed models.

Characterizing Bias in Compressed Models

The research presented in the paper "Characterising Bias in Compressed Models" addresses the effects of common model compression techniques, such as pruning and quantization, on deep neural networks. The authors conduct a thorough analysis of how these techniques impact model bias, with particular focus on underrepresented and sensitive features within datasets.

Main Contributions

• Compression Identified Exemplars (CIE): The paper introduces the concept of Compression Identified Exemplars (CIE), which are examples in a dataset that experience disproportionately high errors post-compression. Such exemplars highlight the significant variation in model performance on this subset, suggesting that compression can amplify existing biases.
• Algorithmic Bias Amplification: The authors establish that pruning and quantization techniques not only preserve the top-level accuracy metrics but also exacerbate errors in subsets associated with underrepresented or sensitive attributes, such as gender or age.
• Human-in-the-Loop Auditing: A key proposal of the paper is the utility of CIE as a tool for human-in-the-loop auditing. This enables practitioners to focus on a tractable subset of data for further inspection, making it an efficient method for identifying and correcting model errors concerning protected attributes.

Methodology and Findings

The research examines the implications of model compression using CelebA, a dataset containing celebrity images annotated with gender and binary attribute labels. The paper particularly investigates disparities across gender and age sub-groups by comparing model performances pre- and post-compression.

1. Accuracy Discrepancies: Although the top-line accuracy metrics, such as top-1 accuracy, remain largely unchanged with compression, the model's performance deteriorates significantly for a minority of examples, indicating potential unfairness for certain sub-groups.
2. Disparate Impact: The results reveal that the compression-induced errors disproportionately affect underrepresented and sensitive attributes. For instance, the false positive rates for minority demographics increase considerably, suggesting compression may compromise model fairness.
3. Auditing via CIE: CIEs serve as an effective means to identify the most challenging examples. The paper demonstrates that focusing testing efforts on CIEs can provide clearer insights into the skewed error distributions caused by compression, offering a foundation for targeted auditing and bias mitigation strategies.

Implications and Future Directions

• Theoretical Implications: The findings highlight that compression techniques, despite their efficiency benefits, can inadvertently increase model bias, necessitating a more nuanced understanding of compression trade-offs beyond accuracy metrics.
• Practical Implications: For practitioners, the tool of CIEs presents an opportunity to improve model robustness and fairness, especially in applications involving protected attributes, where fairness is crucial, such as hiring, healthcare, and surveillance.
• Future Work: Further research may explore the development of compression methods that minimize bias amplification. Moreover, extending this analysis to more diverse datasets and model architectures could validate the generalizability of the presented findings.

In summary, while model compression is an essential technique for deploying deep learning models under resource constraints, it is crucial to consider its impact on model fairness. The introduction of CIEs and the subsequent analyses provide valuable insights and tools for addressing such biases, paving the way for more equitable AI systems.