Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
38 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
41 tokens/sec
o3 Pro
7 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

A Unified Framework and Dataset for Assessing Societal Bias in Vision-Language Models (2402.13636v2)

Published 21 Feb 2024 in cs.CV, cs.CL, and cs.CY

Abstract: Vision-LLMs (VLMs) have gained widespread adoption in both industry and academia. In this study, we propose a unified framework for systematically evaluating gender, race, and age biases in VLMs with respect to professions. Our evaluation encompasses all supported inference modes of the recent VLMs, including image-to-text, text-to-text, text-to-image, and image-to-image. Additionally, we propose an automated pipeline to generate high-quality synthetic datasets that intentionally conceal gender, race, and age information across different professional domains, both in generated text and images. The dataset includes action-based descriptions of each profession and serves as a benchmark for evaluating societal biases in vision-LLMs (VLMs). In our comparative analysis of widely used VLMs, we have identified that varying input-output modalities lead to discernible differences in bias magnitudes and directions. Additionally, we find that VLM models exhibit distinct biases across different bias attributes we investigated. We hope our work will help guide future progress in improving VLMs to learn socially unbiased representations. We will release our data and code.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize File Document Download Save Streamline Icon: https://streamlinehq.com PDF File Document Download Save Streamline Icon: https://streamlinehq.com Markdown Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (43)
  1. 2023. Gpt-4v(ision) system card.
  2. Shikha Bordia and Samuel R. Bowman. 2019. Identifying and reducing gender bias in word-level language models. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Student Research Workshop, pages 7–15, Minneapolis, Minnesota. Association for Computational Linguistics.
  3. Joy Buolamwini and Timnit Gebru. 2018. Gender shades: Intersectional accuracy disparities in commercial gender classification. In Conference on Fairness, Accountability and Transparency, FAT 2018, 23-24 February 2018, New York, NY, USA, volume 81 of Proceedings of Machine Learning Research, pages 77–91. PMLR.
  4. Dall-eval: Probing the reasoning skills and social biases of text-to-image generation models. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 3043–3054.
  5. Holistic analysis of hallucination in gpt-4v(ision): Bias and interference challenges.
  6. Does object recognition work for everyone? In IEEE Conference on Computer Vision and Pattern Recognition Workshops, CVPR Workshops 2019, Long Beach, CA, USA, June 16-20, 2019, pages 52–59. Computer Vision Foundation / IEEE.
  7. A friendly face: Do text-to-image systems rely on stereotypes when the input is under-specified? ArXiv, abs/2302.07159.
  8. Sourojit Ghosh and Aylin Caliskan. 2023. ’person’ == light-skinned, western man, and sexualization of women of color: Stereotypes in stable diffusion. In Conference on Empirical Methods in Natural Language Processing.
  9. Visogender: A dataset for benchmarking gender bias in image-text pronoun resolution. ArXiv, abs/2306.12424.
  10. Sepehr Janghorbani and Gerard de Melo. 2023. Multi-modal bias: Introducing a framework for stereotypical bias assessment beyond gender and race in vision–language models. ArXiv, abs/2303.12734.
  11. Mistral 7b.
  12. mindall-e on conceptual captions. https://github.com/kakaobrain/minDALL-E.
  13. Parameter-efficient modularised bias mitigation via AdapterFusion. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 2738–2751, Dubrovnik, Croatia. Association for Computational Linguistics.
  14. Sustainable modular debiasing of language models. In Findings of the Association for Computational Linguistics: EMNLP 2021, pages 4782–4797, Punta Cana, Dominican Republic. Association for Computational Linguistics.
  15. Survey of social bias in vision-language models. arXiv preprint arXiv:2309.14381.
  16. BLIP-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pages 19730–19742. PMLR.
  17. Towards debiasing sentence representations. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5502–5515, Online. Association for Computational Linguistics.
  18. Improved baselines with visual instruction tuning.
  19. Visual instruction tuning. In NeurIPS.
  20. StereoSet: Measuring stereotypical bias in pretrained language models. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 5356–5371, Online. Association for Computational Linguistics.
  21. CrowS-pairs: A challenge dataset for measuring social biases in masked language models. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1953–1967, Online. Association for Computational Linguistics.
  22. Gpt-4 technical report.
  23. Sdxl: Improving latent diffusion models for high-resolution image synthesis.
  24. Hierarchical text-conditional image generation with CLIP latents. CoRR, abs/2204.06125.
  25. Null it out: Guarding protected attributes by iterative nullspace projection. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7237–7256, Online. Association for Computational Linguistics.
  26. Lauren A. Rhue. 2018. Racial influence on automated perceptions of emotions. CJRN: Race & Ethnicity (Topic).
  27. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695.
  28. High-resolution image synthesis with latent diffusion models. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022, New Orleans, LA, USA, June 18-24, 2022, pages 10674–10685. IEEE.
  29. Photorealistic text-to-image diffusion models with deep language understanding. In Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurIPS 2022, New Orleans, LA, USA, November 28 - December 9, 2022.
  30. No classification without representation: Assessing geodiversity issues in open data sets for the developing world. arXiv: Machine Learning.
  31. "i’m sorry to hear that": Finding new biases in language models with a holistic descriptor dataset. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, EMNLP 2022, Abu Dhabi, United Arab Emirates, December 7-11, 2022, pages 9180–9211. Association for Computational Linguistics.
  32. Tejas Srinivasan and Yonatan Bisk. 2021. Worst of both worlds: Biases compound in pre-trained vision-and-language models. CoRR, abs/2104.08666.
  33. Ryan Steed and Aylin Caliskan. 2021. Image representations learned with unsupervised pre-training contain human-like biases. In FAccT ’21: 2021 ACM Conference on Fairness, Accountability, and Transparency, Virtual Event / Toronto, Canada, March 3-10, 2021, pages 701–713. ACM.
  34. Harini Suresh and John V. Guttag. 2021. A framework for understanding sources of harm throughout the machine learning life cycle. In EAAMO 2021: ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization, Virtual Event, USA, October 5 - 9, 2021, pages 17:1–17:9. ACM.
  35. Any-to-any generation via composable diffusion. arXiv preprint arXiv:2305.11846.
  36. Gemini: A family of highly capable multimodal models.
  37. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805.
  38. Llama 2: Open foundation and fine-tuned chat models.
  39. Bleaching text: Abstract features for cross-lingual gender prediction. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, ACL 2018, Melbourne, Australia, July 15-20, 2018, Volume 2: Short Papers, pages 383–389. Association for Computational Linguistics.
  40. Measuring and reducing gendered correlations in pre-trained models. Technical report.
  41. Predictive inequity in object detection. CoRR, abs/1902.11097.
  42. Vlstereoset: A study of stereotypical bias in pre-trained vision-language models. In Proceedings of the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing, AACL/IJCNLP 2022 - Volume 1: Long Papers, Online Only, November 20-23, 2022, pages 527–538. Association for Computational Linguistics.
  43. Bias in generative ai, (work in progress). https://www.andrew.cmu.edu/user/ales/cib/bias_in_gen_ai.pdf.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (3)
  1. Ashutosh Sathe (9 papers)
  2. Prachi Jain (12 papers)
  3. Sunayana Sitaram (54 papers)