Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
38 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
41 tokens/sec
o3 Pro
7 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Gender Bias in Transformer Models: A comprehensive survey (2306.10530v1)

Published 18 Jun 2023 in cs.CL and cs.AI

Abstract: Gender bias in AI has emerged as a pressing concern with profound implications for individuals' lives. This paper presents a comprehensive survey that explores gender bias in Transformer models from a linguistic perspective. While the existence of gender bias in LLMs has been acknowledged in previous studies, there remains a lack of consensus on how to effectively measure and evaluate this bias. Our survey critically examines the existing literature on gender bias in Transformers, shedding light on the diverse methodologies and metrics employed to assess bias. Several limitations in current approaches to measuring gender bias in Transformers are identified, encompassing the utilization of incomplete or flawed metrics, inadequate dataset sizes, and a dearth of standardization in evaluation methods. Furthermore, our survey delves into the potential ramifications of gender bias in Transformers for downstream applications, including dialogue systems and machine translation. We underscore the importance of fostering equity and fairness in these systems by emphasizing the need for heightened awareness and accountability in developing and deploying language technologies. This paper serves as a comprehensive overview of gender bias in Transformer models, providing novel insights and offering valuable directions for future research in this critical domain.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize File Document Download Save Streamline Icon: https://streamlinehq.com PDF File Document Download Save Streamline Icon: https://streamlinehq.com Markdown Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (66)
  1. A. Nadeem, B. Abedin, and O. Marjanovic, “Gender bias in ai: A review of contributing factors and mitigating strategies,” 2020.
  2. A. Nadeem, O. Marjanovic, B. Abedin et al., “Gender bias in ai-based decision-making systems: a systematic literature review,” Australasian Journal of Information Systems, vol. 26, 2022.
  3. R. Schwartz, A. Vassilev, K. Greene, L. Perine, A. Burt, P. Hall et al., “Towards a standard for identifying and managing bias in artificial intelligence,” NIST Special Publication, vol. 1270, pp. 1–77, 2022.
  4. B. Baldwin, J. Reynar, M. Collins, J. Eisner, A. Ratnaparkhi, J. Rosenzweig, A. Sarkar, and S. Bangalore, “University of pennsylvania: description of the university of pennsylvania system used for muc-6,” in Sixth Message Understanding Conference (MUC-6): Proceedings of a Conference Held in Columbia, Maryland, November 6-8, 1995, 1995.
  5. Y. Wang and D. Redmiles, “Implicit gender biases in professional software development: An empirical study,” in 2019 IEEE/ACM 41st International Conference on Software Engineering: Software Engineering in Society (ICSE-SEIS).   IEEE, 2019, pp. 1–10.
  6. A.-M. Simundic, “Bias in research,” Biochemia medica, vol. 23, no. 1, pp. 12–15, 2013.
  7. P. Turney, “Bias and the quantification of stability,” Machine Learning, vol. 20, pp. 23–33, 1995.
  8. N. Mehrabi, F. Morstatter, N. Saxena, K. Lerman, and A. Galstyan, “A survey on bias and fairness in machine learning,” ACM Computing Surveys (CSUR), vol. 54, no. 6, pp. 1–35, 2021.
  9. M.-E. Brunet, C. Alkalay-Houlihan, A. Anderson, and R. Zemel, “Understanding the origins of bias in word embeddings,” in International conference on machine learning.   PMLR, 2019, pp. 803–811.
  10. O. Papakyriakopoulos, S. Hegelich, J. C. M. Serrano, and F. Marco, “Bias in word embeddings,” in Proceedings of the 2020 conference on fairness, accountability, and transparency, 2020, pp. 446–457.
  11. J. Zhao, T. Wang, M. Yatskar, R. Cotterell, V. Ordonez, and K.-W. Chang, “Gender bias in contextualized word embeddings,” in Proceedings of NAACL-HLT, 2019, pp. 629–634.
  12. “Amazon scraps secret ai recruiting tool that showed bias against women,” https://www.reuters.com/article/us-amazon-com-jobs-automation-insight-idUSKCN1MK08G, accessed: 2023-01-03.
  13. G. Stanovsky, N. A. Smith, and L. Zettlemoyer, “Evaluating gender bias in machine translation,” in Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019, pp. 1679–1684.
  14. M. O. Prates, P. H. Avelar, and L. C. Lamb, “Assessing gender bias in machine translation: a case study with google translate,” Neural Computing and Applications, vol. 32, pp. 6363–6381, 2020.
  15. B. Savoldi, M. Gaido, L. Bentivogli, M. Negri, and M. Turchi, “Gender Bias in Machine Translation,” Transactions of the Association for Computational Linguistics, vol. 9, pp. 845–874, 08 2021. [Online]. Available: https://doi.org/10.1162/tacl_a_00401
  16. R. Tang, M. Du, Y. Li, and X. Hu, “Mitigating gender bias in captioning systems,” 06 2020.
  17. L. A. Hendricks, K. Burns, K. Saenko, T. Darrell, and A. Rohrbach, “Women also snowboard: Overcoming bias in captioning models,” in Proceedings of the European Conference on Computer Vision (ECCV), 2018, pp. 771–787.
  18. Y. Hirota, Y. Nakashima, and N. Garcia, “Quantifying societal bias amplification in image captioning,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 13 450–13 459.
  19. M. Thelwall, “Gender bias in sentiment analysis,” Online Information Review, 2018.
  20. M. H. Asyrofi, Z. Yang, I. N. B. Yusuf, H. J. Kang, F. Thung, and D. Lo, “Biasfinder: Metamorphic test generation to uncover bias for sentiment analysis systems,” IEEE Transactions on Software Engineering, vol. 48, no. 12, pp. 5087–5101, 2022.
  21. S. Kiritchenko and S. Mohammad, “Examining gender and race bias in two hundred sentiment analysis systems,” in Proceedings of the Seventh Joint Conference on Lexical and Computational Semantics, 2018, pp. 43–53.
  22. A. Caliskan, P. P. Ajay, T. Charlesworth, R. Wolfe, and M. R. Banaji, “Gender bias in word embeddings: a comprehensive analysis of frequency, syntax, and semantics,” in Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society, 2022, pp. 156–170.
  23. J. H. Park, J. Shin, and P. Fung, “Reducing gender bias in abusive language detection,” in Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, 2018, pp. 2799–2804.
  24. K. Lu, P. Mardziel, F. Wu, P. Amancharla, and A. Datta, “Gender bias in neural natural language processing,” Logic, Language, and Security: Essays Dedicated to Andre Scedrov on the Occasion of His 65th Birthday, pp. 189–202, 2020.
  25. S. Bordia and S. Bowman, “Identifying and reducing gender bias in word-level language models,” in Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Student Research Workshop, 2019, pp. 7–15.
  26. D. Hovy and S. Prabhumoye, “Five sources of bias in natural language processing,” Language and Linguistics Compass, vol. 15, no. 8, p. e12432, 2021.
  27. A. Garimella, C. Banea, D. Hovy, and R. Mihalcea, “Women’s syntactic resilience and men’s grammatical luck: Gender-bias in part-of-speech tagging and dependency parsing,” in Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics.   Florence, Italy: Association for Computational Linguistics, Jul. 2019, pp. 3493–3498. [Online]. Available: https://aclanthology.org/P19-1339
  28. T. Lingren, L. Deleger, K. Molnar, H. Zhai, J. Meinzen-Derr, M. Kaiser, L. Stoutenborough, Q. Li, and I. Solti, “Evaluating the impact of pre-annotation on annotation speed and potential bias: natural language processing gold standard development for clinical named entity recognition in clinical trial announcements,” Journal of the American Medical Informatics Association, vol. 21, no. 3, pp. 406–413, 2014.
  29. S. Kiritchenko and S. Mohammad, “Examining gender and race bias in two hundred sentiment analysis systems,” in Proceedings of the Seventh Joint Conference on Lexical and Computational Semantics.   New Orleans, Louisiana: Association for Computational Linguistics, Jun. 2018, pp. 43–53. [Online]. Available: https://aclanthology.org/S18-2005
  30. D. Hovy, F. Bianchi, and T. Fornaciari, ““you sound just like your father” commercial machine translation systems include stylistic biases,” in Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 2020, pp. 1686–1690.
  31. A. Peng, B. Nushi, E. Kıcıman, K. Inkpen, S. Suri, and E. Kamar, “What you see is what you get? the impact of representation criteria on human bias in hiring,” in Proceedings of the AAAI Conference on Human Computation and Crowdsourcing, vol. 7, 2019, pp. 125–134.
  32. T. Wolf, L. Debut, V. Sanh, J. Chaumond, C. Delangue, A. Moi, P. Cistac, T. Rault, R. Louf, M. Funtowicz et al., “Huggingface’s transformers: State-of-the-art natural language processing,” arXiv preprint arXiv:1910.03771, 2019.
  33. A. Gillioz, J. Casas, E. Mugellini, and O. Abou Khaled, “Overview of the transformer-based models for nlp tasks,” in 2020 15th Conference on Computer Science and Information Systems (FedCSIS).   IEEE, 2020, pp. 179–183.
  34. A. M. Braşoveanu and R. Andonie, “Visualizing transformers for nlp: a brief survey,” in 2020 24th International Conference Information Visualisation (IV).   IEEE, 2020, pp. 270–279.
  35. A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” Advances in neural information processing systems, vol. 30, 2017.
  36. P. Budzianowski and I. Vulić, “Hello, it’s gpt-2-how can i help you? towards the use of pretrained language models for task-oriented dialogue systems,” in Proceedings of the 3rd Workshop on Neural Generation and Translation, 2019, pp. 15–22.
  37. L. Floridi and M. Chiriatti, “Gpt-3: Its nature, scope, limits, and consequences,” Minds and Machines, vol. 30, pp. 681–694, 2020.
  38. Y. Liu, M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen, O. Levy, M. Lewis, L. Zettlemoyer, and V. Stoyanov, “Roberta: A robustly optimized bert pretraining approach,” arXiv preprint arXiv:1907.11692, 2019.
  39. P. He, X. Liu, J. Gao, and W. Chen, “Deberta: Decoding-enhanced bert with disentangled attention,” arXiv preprint arXiv:2006.03654, 2020.
  40. T. Bolukbasi, K.-W. Chang, J. Y. Zou, V. Saligrama, and A. T. Kalai, “Man is to computer programmer as woman is to homemaker? debiasing word embeddings,” Advances in neural information processing systems, vol. 29, 2016.
  41. T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell et al., “Language models are few-shot learners,” Advances in neural information processing systems, vol. 33, pp. 1877–1901, 2020.
  42. B. D. Lund and T. Wang, “Chatting about chatgpt: how may ai and gpt impact academia and libraries?” Library Hi Tech News, 2023.
  43. A. Borji, “A categorical archive of chatgpt failures,” arXiv preprint arXiv:2302.03494, 2023.
  44. M. Ortega-Martín, Ó. García-Sierra, A. Ardoiz, J. Álvarez, J. C. Armenteros, and A. Alonso, “Linguistic ambiguity analysis in chatgpt,” arXiv preprint arXiv:2302.06426, 2023.
  45. S. Khan, M. Naseer, M. Hayat, S. W. Zamir, F. S. Khan, and M. Shah, “Transformers in vision: A survey,” ACM computing surveys (CSUR), vol. 54, no. 10s, pp. 1–41, 2022.
  46. R. Jangir, N. Hansen, S. Ghosal, M. Jain, and X. Wang, “Look closer: Bridging egocentric and third-person views with transformers for robotic manipulation,” IEEE Robotics and Automation Letters, vol. 7, no. 2, pp. 3046–3053, 2022.
  47. X. Han, Z. Zhang, N. Ding, Y. Gu, X. Liu, Y. Huo, J. Qiu, Y. Yao, A. Zhang, L. Zhang et al., “Pre-trained models: Past, present and future,” AI Open, vol. 2, pp. 225–250, 2021.
  48. A. Silva, P. Tambwekar, and M. Gombolay, “Towards a comprehensive understanding and accurate evaluation of societal biases in pre-trained transformers,” in Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2021, pp. 2383–2389.
  49. P. Awasthi, M. Kleindessner, and J. Morgenstern, “Equalized odds postprocessing under imperfect group information,” in Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics, ser. Proceedings of Machine Learning Research, S. Chiappa and R. Calandra, Eds., vol. 108.   PMLR, 26–28 Aug 2020, pp. 1770–1780. [Online]. Available: https://proceedings.mlr.press/v108/awasthi20a.html
  50. P. Garg, J. Villasenor, and V. Foggo, “Fairness metrics: A comparative analysis,” in 2020 IEEE International Conference on Big Data (Big Data).   IEEE, 2020, pp. 3662–3666.
  51. R. Robinson, “Assessing gender bias in medical and scientific masked language models with stereoset,” arXiv preprint arXiv:2111.08088, 2021.
  52. I. Beltagy, K. Lo, and A. Cohan, “Scibert: A pretrained language model for scientific text,” in Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), 2019, pp. 3615–3620.
  53. B. Li, H. Peng, R. Sainju, J. Yang, L. Yang, Y. Liang, W. Jiang, B. Wang, H. Liu, and C. Ding, “Detecting gender bias in transformer-based models: a case study on bert,” arXiv preprint arXiv:2110.15733, 2021.
  54. P. Joniak and A. Aizawa, “Gender biases and where to find them: Exploring gender bias in pre-trained transformer-based language models using movement pruning,” arXiv preprint arXiv:2207.02463, 2022.
  55. X. Bao and Q. Qiao, “Transfer learning from pre-trained bert for pronoun resolution,” in Proceedings of the first workshop on gender bias in natural language processing, 2019, pp. 82–88.
  56. S. Beamer, K. Asanović, and D. Patterson, “The gap benchmark suite,” arXiv preprint arXiv:1508.03619, 2015.
  57. R. Wolfe and A. Caliskan, “Low frequency names exhibit bias and overfitting in contextualizing language models,” in Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, 2021, pp. 518–532.
  58. J. Vig, S. Gehrmann, Y. Belinkov, S. Qian, D. Nevo, Y. Singer, and S. Shieber, “Investigating gender bias in language models using causal mediation analysis,” Advances in Neural Information Processing Systems, vol. 33, pp. 12 388–12 401, 2020.
  59. R. Bhardwaj, N. Majumder, and S. Poria, “Investigating gender bias in bert,” Cognitive Computation, vol. 13, no. 4, pp. 1008–1018, 2021.
  60. N. Reimers and I. Gurevych, “Sentence-bert: Sentence embeddings using siamese bert-networks,” in Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), 2019, pp. 3982–3992.
  61. C. R. S. Basta, M. Ruiz Costa-Jussà, and J. A. Rodríguez Fonollosa, “Towards mitigating gender bias in a decoder-based neural machine translation model by adding contextual information,” in Proceedings of the The Fourth Widening Natural Language Processing Workshop.   Association for Computational Linguistics, 2020, pp. 99–102.
  62. W. Luo, “Encoder-decoder based neural machine translation,” Ph.D. dissertation, 2019.
  63. M. Bartl, M. Nissim, and A. Gatt, “Unmasking contextual stereotypes: Measuring and mitigating bert’s gender bias,” arXiv preprint arXiv:2010.14534, 2020.
  64. M. Kaneko, A. Imankulova, D. Bollegala, and N. Okazaki, “Gender bias in masked language models for multiple languages,” arXiv preprint arXiv:2205.00551, 2022.
  65. D. de Vassimon Manela, D. Errington, T. Fisher, B. van Breugel, and P. Minervini, “Stereotype and skew: Quantifying gender bias in pre-trained and fine-tuned language models,” in EACL 2021-16th Conference of the European Chapter of the Association for Computational Linguistics, Proceedings of the Conference.   Association for Computational Linguistics, 2021, pp. 2232–2242.
  66. A. Lindqvist, M. G. Sendén, and E. A. Renström, “What is gender, anyway: a review of the options for operationalising gender,” Psychology & sexuality, vol. 12, no. 4, pp. 332–344, 2021.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (4)
  1. Praneeth Nemani (6 papers)
  2. Yericherla Deepak Joel (1 paper)
  3. Palla Vijay (1 paper)
  4. Farhana Ferdousi Liza (5 papers)
Citations (2)
View on Semantic Scholar