Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
41 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
41 tokens/sec
o3 Pro
7 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

OpenBias: Open-set Bias Detection in Text-to-Image Generative Models (2404.07990v2)

Published 11 Apr 2024 in cs.CV and cs.AI

Abstract: Text-to-image generative models are becoming increasingly popular and accessible to the general public. As these models see large-scale deployments, it is necessary to deeply investigate their safety and fairness to not disseminate and perpetuate any kind of biases. However, existing works focus on detecting closed sets of biases defined a priori, limiting the studies to well-known concepts. In this paper, we tackle the challenge of open-set bias detection in text-to-image generative models presenting OpenBias, a new pipeline that identifies and quantifies the severity of biases agnostically, without access to any precompiled set. OpenBias has three stages. In the first phase, we leverage a LLM to propose biases given a set of captions. Secondly, the target generative model produces images using the same set of captions. Lastly, a Vision Question Answering model recognizes the presence and extent of the previously proposed biases. We study the behavior of Stable Diffusion 1.5, 2, and XL emphasizing new biases, never investigated before. Via quantitative experiments, we demonstrate that OpenBias agrees with current closed-set bias detection methods and human judgement.

PDF HTML Abstract

OpenBias: Open-set Bias Detection in Text-to-Image Generative Models

The paper "OpenBias: Open-set Bias Detection in Text-to-Image Generative Models" addresses the significant challenge of identifying biases in text-to-image (T2I) generative models without relying on predefined sets of bias categories. This research is particularly relevant as T2I models see increasing deployment and more widespread use, necessitating an investigation into potential biases these systems might inadvertently amplify or propagate.

The authors introduce a novel framework, OpenBias, which operates in an open-set scenario, aiming to detect and quantify biases in T2I models such as Stable Diffusion, including its versions 1.5, 2, and XL. Unlike prior efforts that often focus on predefined sets of biases, OpenBias seeks to uncover biases potentially overlooked in prior studies.

Methodology Overview

OpenBias comprises a three-stage pipeline leveraging LLMs and Vision Question Answering (VQA) models. The key stages are:

  1. Bias Proposal: Utilizing a LLM, OpenBias first proposes potential biases using a set of real-world captions. By generating bias proposals without any precompiled list, the framework taps into vast uncharted domains of biases beyond typical categories such as gender or race.
  2. Image Generation: For each proposed bias, the associated generative model processes the same set of captions to produce images. This stage serves as a placeholder for generating visual representations that might harbor the proposed biases.
  3. Bias Assessment: A VQA model evaluates the generated images to recognize and quantify the presence of the candidate biases. This modeling of bias assessment enables OpenBias to effectively capture the subtleties in bias propagation in a contextual and context-free manner.

Key Results

The paper contains robust numerical evaluations that substantiate the framework's claims. Quantitative experiments exhibit remarkable agreement between OpenBias and conventional closed-set bias detection methodologies as well as with human judgment, validating the framework's reliability. The research also highlights differences in bias prevalence across different configurations of the Stable Diffusion model, demonstrating the nuanced discoveries that an open-set approach enables.

Theoretical and Practical Implications

Theoretically, this research extends our understanding of bias detection in machine learning models by challenging the notion that biases must be pre-specified. It proposes that there are often naïve assumptions embedded in datasets which may lead to unchecked bias propagation. Practically, OpenBias opens the door for AI developers and researchers to identify bias in a more comprehensive fashion, potentially improving the fairness of model outputs in real-world applications.

Future Speculations

Moving forward, the framework's adaptability presents various opportunities for further exploration. Several next steps can include deploying OpenBias to critically assess other forms of generative AI, such as language generation models, and extending it to include audio-visual and multimodal systems. Additionally, integrating OpenBias into bias mitigation protocols could further enhance the fairness of T2I models.

In summary, OpenBias provides an invaluable tool for broadening the scope of bias detection in T2I models, advocating for a shift towards a more dynamic and context-sensitive understanding of bias. This research represents a crucial advancement in the ongoing efforts to foster more equitable machine learning systems.

File Document Download Save Streamline Icon: https://streamlinehq.com PDF File Document Download Save Streamline Icon: https://streamlinehq.com Markdown Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (76)
  1. Does data repair lead to fair models? curating contextually fair data to reduce model bias. In WACV, 2022.
  2. Spatext: Spatio-textual representation for controllable image generation. In CVPR, 2023.
  3. Easily accessible text-to-image generation amplifies demographic stereotypes at large scale. In Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency, 2023.
  4. Man is to computer programmer as woman is to homemaker? debiasing word embeddings. In NeurIPS, 2016.
  5. On the opportunities and risks of foundation models. arXiv preprint, 2021.
  6. Sega: Instructing text-to-image models using semantic guidance. In NeurIPS, 2023a.
  7. Mitigating inappropriateness in image generation: Can there be value in reflecting the world’s ugliness? arXiv preprint, 2023b.
  8. Instructpix2pix: Learning to follow image editing instructions. In CVPR, 2023.
  9. Language models are few-shot learners. In NeurIPS, 2020.
  10. Emerging properties in self-supervised vision transformers. In ICCV, 2021.
  11. Video chatcaptioner: Towards the enriched spatiotemporal descriptions. arXiv preprint, 2023.
  12. Reproducible scaling laws for contrastive language-image learning. In CVPR, 2023.
  13. Dall-eval: Probing the reasoning skills and social biases of text-to-image generation models. In ICCV, 2023.
  14. Improving fairness using vision-language driven image augmentation. In WACV, 2024.
  15. An image is worth 16x16 words: Transformers for image recognition at scale. In ICLR, 2021.
  16. Diffusion self-guidance for controllable image generation. In NeurIPS, 2023.
  17. Fair diffusion: Instructing text-to-image generation models on fairness. arXiv preprint, 2023.
  18. An image is worth one word: Personalizing text-to-image generation using textual inversion. arXiv preprint, 2022.
  19. Bias and fairness in large language models: A survey. arXiv preprint, 2023.
  20. Unified concept editing in diffusion models. In WACV, 2024.
  21. Pair-diffusion: A comprehensive multimodal object-level image editor. arXiv preprint, 2023.
  22. Visual programming: Compositional visual reasoning without training. In CVPR, 2023.
  23. Women also snowboard: Overcoming bias in captioning models. In ECCV, 2018.
  24. Prompt-to-prompt image editing with cross attention control. In ICLR, 2022.
  25. Classifier-free diffusion guidance. In NeurIPS 2021 Workshop on Deep Generative Models and Downstream Applications, 2021.
  26. Promptcap: Prompt-guided image captioning for vqa with gpt-3. In ICCV, 2023a.
  27. Tifa: Accurate and interpretable text-to-image faithfulness evaluation with question answering. In ICCV, 2023b.
  28. Composer: Creative and controllable image synthesis with composable conditions. In ICML, 2023.
  29. Learning fair classifiers with partially annotated group labels. In CVPR, 2022.
  30. Fairface: Face attribute dataset for balanced race, gender, and age for bias measurement and mitigation. In WACV, 2021.
  31. A style-based generator architecture for generative adversarial networks. In CVPR, 2019.
  32. Alias-free generative adversarial networks. In NeurIPS, 2021.
  33. Repfair-gan: Mitigating representation bias in gans using gradient clipping. arXiv preprint, 2022.
  34. Vilt: Vision-and-language transformer without convolution or region supervision. In ICML, 2021.
  35. Bias-to-text: Debiasing unknown visual biases through language interpretation. arXiv preprint, 2023.
  36. The india face set: International and cultural boundaries impact face impressions and perceptions of category membership. Frontiers in Psychology, 2021.
  37. mPLUG: Effective and efficient vision-language learning by cross-modal skip-connections. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, 2022a.
  38. Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation. In ICML, 2022b.
  39. BLIP-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In Proceedings of the 40th International Conference on Machine Learning, 2023.
  40. Microsoft COCO: common objects in context. In ECCV, 2014.
  41. Improved baselines with visual instruction tuning. In NeurIPS 2023 Workshop on Instruction Tuning and Instruction Following, 2023a.
  42. Visual instruction tuning. In NeurIPS, 2023b.
  43. The chicago face database: A free stimulus set of faces and norming data. 2015.
  44. Chicago face database: Multiracial expansion. Behavior Research Methods, 2020.
  45. Social biases through the text-to-image generation lens. In Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society, 2023.
  46. Learning from failure: De-biasing classifier from biased classifier. NeurIPS, 2020.
  47. Biases in large language models: Origins, inventory, and discussion. ACM Journal of Data and Information Quality, 2023.
  48. Glide: Towards photorealistic image generation and editing with text-guided diffusion models. In International Conference on Machine Learning, 2022.
  49. Dinov2: Learning robust visual features without supervision. In Transactions on Machine Learning Research, 2023.
  50. SDXL: Improving latent diffusion models for high-resolution image synthesis. In ICLR, 2024.
  51. Learning transferable visual models from natural language supervision. In ICML, 2021.
  52. Hierarchical text-conditional image generation with clip latents. arXiv preprint, 2022.
  53. High-resolution image synthesis with latent diffusion models. In CVPR, 2022.
  54. Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation. In CVPR, 2023.
  55. Photorealistic text-to-image diffusion models with deep language understanding. NeurIPS, 2022.
  56. Intra-processing methods for debiasing neural networks. In NeurIPS, 2020.
  57. Conceptnet 5.5: An open multilingual graph of general knowledge. In AAAI, 2017.
  58. Selective annotation makes language models better few-shot learners. In ICLR, 2023.
  59. Unbiased image synthesis via manifold-driven sampling in diffusion models. arXiv preprint, 2023.
  60. Reclip: A strong zero-shot baseline for referring expression comprehension. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics, 2022.
  61. Vipergpt: Visual inference via python execution for reasoning. In ICCV, 2023.
  62. Improving the fairness of deep generative models without retraining. arXiv preprint, 2021.
  63. Llama: Open and efficient foundation language models. arXiv preprint, 2023.
  64. Fairness definitions explained. In Proceedings of the international workshop on software fairness, 2018.
  65. Git: A generative image-to-text transformer for vision and language. In Transactions on Machine Learning Research, 2022a.
  66. Ofa: Unifying architectures, tasks, and modalities through a simple sequence-to-sequence learning framework. In ICML, 2022b.
  67. Towards fairness in visual recognition: Effective strategies for bias mitigation. In CVPR, 2020.
  68. Chain-of-thought prompting elicits reasoning in large language models. In NeurIPS, 2022.
  69. Allen R Wilcox. Indices of qualitative variation. Technical report, Oak Ridge National Lab., Tenn., 1967.
  70. Fairgan: Fairness-aware generative adversarial networks. In 2018 IEEE International Conference on Big Data (Big Data). IEEE, 2018.
  71. From image descriptions to visual denotations: New similarity metrics for semantic inference over event descriptions. Transactions of the Association for Computational Linguistics, 2014.
  72. Iti-gen: Inclusive text-to-image generation. In ICCV, 2023a.
  73. Adding conditional control to text-to-image diffusion models. In ICCV, 2023b.
  74. Men also like shopping: Reducing gender bias amplification using corpus-level constraints. In EMNLP, 2017.
  75. Gender bias in coreference resolution: Evaluation and debiasing methods. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2018.
  76. Chatgpt asks, blip-2 answers: Automatic questioning towards enriched visual descriptions. In Transactions on Machine Learning Research, 2023.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (9)
  1. Moreno D'Incà (7 papers)
  2. Elia Peruzzo (9 papers)
  3. Massimiliano Mancini (66 papers)
  4. Dejia Xu (37 papers)
  5. Vidit Goel (13 papers)
  6. Xingqian Xu (23 papers)
  7. Zhangyang Wang (374 papers)
  8. Humphrey Shi (97 papers)
  9. Nicu Sebe (270 papers)
Citations (9)
View on Semantic Scholar
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets

Youtube Logo Streamline Icon: https://streamlinehq.com

YouTube