Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
38 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
41 tokens/sec
o3 Pro
7 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Quantifying Bias in Text-to-Image Generative Models (2312.13053v1)

Published 20 Dec 2023 in cs.CV and cs.CR
Quantifying Bias in Text-to-Image Generative Models

Abstract: Bias in text-to-image (T2I) models can propagate unfair social representations and may be used to aggressively market ideas or push controversial agendas. Existing T2I model bias evaluation methods only focus on social biases. We look beyond that and instead propose an evaluation methodology to quantify general biases in T2I generative models, without any preconceived notions. We assess four state-of-the-art T2I models and compare their baseline bias characteristics to their respective variants (two for each), where certain biases have been intentionally induced. We propose three evaluation metrics to assess model biases including: (i) Distribution bias, (ii) Jaccard hallucination and (iii) Generative miss-rate. We conduct two evaluation studies, modelling biases under general, and task-oriented conditions, using a marketing scenario as the domain for the latter. We also quantify social biases to compare our findings to related works. Finally, our methodology is transferred to evaluate captioned-image datasets and measure their bias. Our approach is objective, domain-agnostic and consistently measures different forms of T2I model biases. We have developed a web application and practical implementation of what has been proposed in this work, which is at https://huggingface.co/spaces/JVice/try-before-you-bias. A video series with demonstrations is available at https://www.youtube.com/channel/UCk-0xyUyT0MSd_hkp4jQt1Q

PDF HTML Abstract

Understanding Bias in Text-to-Image Generation

The Rise of Text-to-Image Models

In the field of artificial intelligence, the capability to convert written descriptions into vivid images has been a crucial advancement. The technology behind text-to-image (T2I) generative models has reached impressive levels of sophistication, allowing for a broad range of applications, from creative art generation to marketing.

Evaluating Model Biases

However, with power comes responsibility. As these models are trained on vast amounts of internet-sourced data, they run the risk of inheriting the biases present in their training material. Such biases, whether they pertain to gender, ethnicity, or other factors, can lead to unfair or harmful representations.

Recognizing this, a new paper proposes a methodology to quantify biases in T2I generative models beyond just social biases. The researchers introduce three metrics: distribution bias, which examines the frequency of generated objects; Jaccard hallucination, accounting for inaccuracies like omitted or additional objects; and generative miss-rate, assessing the alignment of generated images with input.

Controlled Experiments and Backdoor Injections

To assess biases, the researchers compared four advanced T2I models under normal conditions against variants with deliberately induced biases via backdoor injections—an increasingly concerning issue in neural networks. Backdoor attacks can subtly alter a model's output upon recognizing a specific input 'trigger'. By controlling these triggers, the paper reveals the potential to manipulate and measure biases within the models.

Findings and Insights

The paper's controlled experiments uncovered a distinct gender bias and an underrepresentation of women in the outputs of T2I models. Furthermore, the paper showed that task-oriented conditions, such as marketing scenarios, were highly susceptible to manipulated output spaces, frequently skewing the results towards specific brands when a bias is introduced.

Broader Implications

Beyond the specifics of the T2I model bias, this paper's implications are significant for the AI community. It establishes a benchmark for evaluating and recognizing bias in generative models and lays the groundwork for the creation of more responsible AI technologies.

Conclusions

As AI continues to integrate deeply into society, identifying, quantifying, and mitigating biases within AI systems is vital. By understanding and addressing these issues proactively, we can ensure that the benefits of AI are enjoyed equitably across society. This paper represents a significant step towards achieving fairness and reliability in the rapidly evolving field of generative AI models.

File Document Download Save Streamline Icon: https://streamlinehq.com PDF File Document Download Save Streamline Icon: https://streamlinehq.com Markdown Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (54)
  1. N. Mehrabi, F. Morstatter, N. Saxena, K. Lerman, and A. Galstyan, “A survey on bias and fairness in machine learning,” ACM Computing Surveys, vol. 54, no. 6, pp. 1–35, 2021.
  2. C. Bird, E. Ungless, and A. Kasirzadeh, “Typology of risks of generative text-to-image models,” in Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society, 2023, pp. 396–410.
  3. J. Cho, A. Zala, and M. Bansal, “Dall-eval: Probing the reasoning skills and social biases of text-to-image generation models,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, October 2023, pp. 3043–3054.
  4. S. Luccioni, C. Akiki, M. Mitchell, and Y. Jernite, “Stable bias: Evaluating societal representations in diffusion models,” in Thirty-seventh Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2023, pp. 1–14. [Online]. Available: https://openreview.net/forum?id=qVXYU3F017
  5. R. Naik and B. Nushi, “Social biases through the text-to-image generation lens,” arXiv preprint arXiv:2304.06034, 2023.
  6. P. Seshadri, S. Singh, and Y. Elazar, “The bias amplification paradox in text-to-image generation,” arXiv preprint arXiv:2308.00755, 2023.
  7. C. T. Teo, M. Abdollahzadeh, and N.-M. Cheung, “Fair generative models via transfer learning,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 37, no. 2, 2023, pp. 2429–2437.
  8. R. Gozalo-Brizuela and E. C. Garrido-Merchan, “Chatgpt is not all you need. a state of the art review of large generative ai models,” arXiv preprint arXiv:2301.04655, 2023.
  9. N. Akhtar, A. Mian, N. Kardan, and M. Shah, “Advances in adversarial attacks and defenses in computer vision: A survey,” IEEE Access, vol. 9, pp. 155 161–155 196, 2021.
  10. X. Huang, D. Kroening, W. Ruan, J. Sharp, Y. Sun, E. Thamo, M. Wu, and X. Yi, “A survey of safety and trustworthiness of deep neural networks: Verification, testing, adversarial attack and defence, and interpretability,” Computer Science Review, vol. 37, pp. 1–35, 2020.
  11. S. Kaviani and I. Sohn, “Defense against neural trojan attacks: A survey,” Neurocomputing, vol. 423, pp. 651–667, 2021.
  12. J. Vice, N. Akhtar, R. Hartley, and A. Mian, “Bagm: A backdoor attack for manipulating text-to-image generative models,” arXiv preprint arXiv:2307.16489, 2023.
  13. S.-Y. Chou, P.-Y. Chen, and T.-Y. Ho, “How to backdoor diffusion models?” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, June 2023, pp. 4015–4024.
  14. W. Chen, D. Song, and B. Li, “Trojdiff: Trojan attacks on diffusion models with diverse targets,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, June 2023, pp. 4035–4044.
  15. A. B. Arrieta, N. Díaz-Rodríguez, J. Del Ser, A. Bennetot, S. Tabik, A. Barbado, S. García, S. Gil-López, D. Molina, R. Benjamins et al., “Explainable artificial intelligence (xai): Concepts, taxonomies, opportunities and challenges toward responsible ai,” Information fusion, vol. 58, pp. 82–115, 2020.
  16. D. Pessach and E. Shmueli, “A review on fairness in machine learning,” ACM Computing Surveys, vol. 55, no. 3, pp. 1–44, 2022.
  17. R. Wolfe and A. Caliskan, “Markedness in visual semantic ai,” in Proceedings of the ACM Conference on Fairness, Accountability, and Transparency, ser. FAccT ’22, 2022, p. 1269–1279. [Online]. Available: https://doi.org/10.1145/3531146.3533183
  18. E. Ferrara, “Should chatgpt be biased? challenges and risks of bias in large language models,” arXiv preprint arXiv:2304.03738, 2023.
  19. P. P. Liang, C. Wu, L.-P. Morency, and R. Salakhutdinov, “Towards understanding and mitigating social biases in language models,” in Proceedings of the 38th International Conference on Machine Learning, ser. Proceedings of Machine Learning Research, M. Meila and T. Zhang, Eds., vol. 139.   PMLR, 18–24 Jul 2021, pp. 6565–6576.
  20. A. Abid, M. Farooqi, and J. Zou, “Persistent anti-muslim bias in large language models,” in Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society, ser. AIES ’21, 2021, pp. 298–306. [Online]. Available: https://doi.org/10.1145/3461702.3462624
  21. R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer, “High-resolution image synthesis with latent diffusion models,” arXiv preprint arXiv:2112.10752, 2021.
  22. A. Shakhmatov, A. Razzhigaev, A. Nikolich et al., “Kandinsky 2.1,” https://huggingface.co/kandinsky-community/kandinsky-2-2-decoder, 2023.
  23. S. AI, “Deepfloyd-if,” https://huggingface.co/DeepFloyd/IF-I-M-v1.0, 2023.
  24. M. Qraitem, K. Saenko, and B. A. Plummer, “Bias mimicking: A simple sampling approach for bias mitigation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, June 2023, pp. 20 311–20 320.
  25. N. Garcia, Y. Hirota, Y. Wu, and Y. Nakashima, “Uncurated image-text datasets: Shedding light on demographic bias,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, June 2023, pp. 6957–6966.
  26. L. Gustafson, C. Rolland, N. Ravi, Q. Duval, A. Adcock, C.-Y. Fu, M. Hall, and C. Ross, “Facet: Fairness in computer vision evaluation benchmark,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, October 2023, pp. 20 370–20 382.
  27. C. Schumann, S. Ricco, U. Prabhu, V. Ferrari, and C. Pantofaru, “A step toward more inclusive people annotations for fairness,” in Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society, ser. AIES ’21, 2021, pp. 916–925. [Online]. Available: https://doi.org/10.1145/3461702.3462594
  28. A. Wang, A. Liu, R. Zhang, A. Kleiman, L. Kim, D. Zhao, I. Shirai, A. Narayanan, and O. Russakovsky, “Revise: A tool for measuring and mitigating bias in visual datasets,” International Journal of Computer Vision, vol. 130, no. 7, pp. 1790–1810, 2022.
  29. K. Karkkainen and J. Joo, “Fairface: Face attribute dataset for balanced race, gender, and age for bias measurement and mitigation,” in Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 2021, pp. 1548–1558.
  30. Y. Li, Y. Jiang, Z. Li, and S.-T. Xia, “Backdoor learning: A survey,” IEEE Transactions on Neural Networks and Learning Systems, pp. 1–18, 2022.
  31. B. Wu, H. Chen, M. Zhang, Z. Zhu, S. Wei, D. Yuan, and C. Shen, “Backdoorbench: A comprehensive benchmark of backdoor learning,” Advances in Neural Information Processing Systems, vol. 35, pp. 10 546–10 559, 2022.
  32. S. Zhai, Y. Dong, Q. Shen, S. Pu, Y. Fang, and H. Su, “Text-to-image diffusion models can be easily backdoored through multimodal data poisoning,” arXiv preprint arXiv:2305.04175, 2023.
  33. M. Zheng, Q. Lou, and L. Jiang, “Trojvit: Trojan insertion in vision transformers,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, June 2023, pp. 4025–4034.
  34. J. W. Cho, D.-J. Kim, H. Ryu, and I. S. Kweon, “Generative bias for robust visual question answering,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, June 2023, pp. 11 681–11 690.
  35. J. Lim, Y. Kim, B. Kim, C. Ahn, J. Shin, E. Yang, and S. Han, “Biasadv: Bias-adversarial augmentation for model debiasing,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, June 2023, pp. 3832–3841.
  36. J. Vice, N. Akhtar, R. Hartley, and A. Mian, “Marketable foods (mf) dataset,” https://ieee-dataport.org/documents/marketable-foods-mf-dataset, 2023.
  37. A. P. Bradley, “The use of the area under the roc curve in the evaluation of machine learning algorithms,” Pattern recognition, vol. 30, no. 7, pp. 1145–1159, 1997.
  38. Q. Xiao, G. Li, and Q. Chen, “Image outpainting: Hallucinating beyond the image,” IEEE Access, vol. 8, pp. 173 576–173 583, 2020.
  39. Y. Li, R. Panda, Y. Kim, C.-F. R. Chen, R. S. Feris, D. Cox, and N. Vasconcelos, “Valhalla: Visual hallucination for machine translation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 5216–5226.
  40. A. Gunjal, J. Yin, and E. Bas, “Detecting and preventing hallucinations in large vision language models,” arXiv preprint arXiv:2308.06394, 2023.
  41. Z. Ji, N. Lee, R. Frieske, T. Yu, D. Su, Y. Xu, E. Ishii, Y. J. Bang, A. Madotto, and P. Fung, “Survey of hallucination in natural language generation,” ACM Computing Surveys, vol. 55, no. 12, pp. 1–38, mar 2023. [Online]. Available: https://doi.org/10.1145/3571730
  42. G. A. Miller, “Wordnet: A lexical database for english,” Communications of the ACM, vol. 38, no. 11, p. 39–41, nov 1995. [Online]. Available: https://doi.org/10.1145/219717.219748
  43. T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár, and C. L. Zitnick, “Microsoft coco: Common objects in context,” in Proceedings of the European Conference on Computer Vision, D. Fleet, T. Pajdla, B. Schiele, and T. Tuytelaars, Eds., 2014, pp. 740–755.
  44. J. Li, D. Li, C. Xiong, and S. Hoi, “BLIP: Bootstrapping language-image pre-training for unified vision-language understanding and generation,” in Proceedings of the 39th International Conference on Machine Learning, ser. Proceedings of Machine Learning Research, K. Chaudhuri, S. Jegelka, L. Song, C. Szepesvari, G. Niu, and S. Sabato, Eds., vol. 162.   PMLR, 17–23 Jul 2022, pp. 12 888–12 900.
  45. A. Ramesh, P. Dhariwal, A. Nichol, C. Chu, and M. Chen, “Hierarchical text-conditional image generation with clip latents,” arXiv preprint arXiv:2204.06125, 2022.
  46. C. Saharia, W. Chan, S. Saxena et al., “Photorealistic text-to-image diffusion models with deep language understanding,” in Advances in Neural Information Processing Systems, S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, Eds., vol. 35, 2022, pp. 36 479–36 494.
  47. C. Raffel, N. Shazeer, A. Roberts et al., “Exploring the limits of transfer learning with a unified text-to-text transformer,” Journal of Machine Learning Research, vol. 21, no. 1, pp. 1–67, jan 2020.
  48. P. Young, A. Lai, M. Hodosh, and J. Hockenmaier, “From image descriptions to visual denotations: New similarity metrics for semantic inference over event descriptions,” Transactions of the Association for Computational Linguistics, vol. 2, pp. 67–78, 2014.
  49. A. Krizhevsky, G. Hinton et al., “Learning multiple layers of features from tiny images,” Technical Report, University of Toronto, 2009.
  50. V. Kinakh, “Stable imagenet-1k dataset,” https://www.kaggle.com/datasets/vitaliykinakh/stable-imagenet1k, 2022.
  51. P. Sharma, N. Ding, S. Goodman, and R. Soricut, “Conceptual captions: A cleaned, hypernymed, image alt-text dataset for automatic image captioning,” in Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, 2018, pp. 2556–2565.
  52. O. Russakovsky, J. Deng, H. Su et al., “Imagenet large scale visual recognition challenge,” International journal of computer vision, vol. 115, pp. 211–252, 2015.
  53. J. Nam, H. Cha, S. Ahn, J. Lee, and J. Shin, “Learning from failure: De-biasing classifier from biased classifier,” Advances in Neural Information Processing Systems, vol. 33, pp. 20 673–20 684, 2020.
  54. H. Bahng, S. Chun, S. Yun, J. Choo, and S. J. Oh, “Learning de-biased representations with biased representations,” in International Conference on Machine Learning.   PMLR, 2020, pp. 528–539.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (4)
  1. Jordan Vice (10 papers)
  2. Naveed Akhtar (77 papers)
  3. Richard Hartley (73 papers)
  4. Ajmal Mian (136 papers)
Citations (5)
View on Semantic Scholar
Youtube Logo Streamline Icon: https://streamlinehq.com

YouTube