Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
119 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

GeniL: A Multilingual Dataset on Generalizing Language (2404.05866v2)

Published 8 Apr 2024 in cs.CL

Abstract: Generative LLMs are transforming our digital ecosystem, but they often inherit societal biases, for instance stereotypes associating certain attributes with specific identity groups. While whether and how these biases are mitigated may depend on the specific use cases, being able to effectively detect instances of stereotype perpetuation is a crucial first step. Current methods to assess presence of stereotypes in generated language rely on simple template or co-occurrence based measures, without accounting for the variety of sentential contexts they manifest in. We argue that understanding the sentential context is crucial for detecting instances of generalization. We distinguish two types of generalizations: (1) language that merely mentions the presence of a generalization ("people think the French are very rude"), and (2) language that reinforces such a generalization ("as French they must be rude"), from non-generalizing context ("My French friends think I am rude"). For meaningful stereotype evaluations, we need to reliably distinguish such instances of generalizations. We introduce the new task of detecting generalization in language, and build GeniL, a multilingual dataset of over 50K sentences from 9 languages (English, Arabic, Bengali, Spanish, French, Hindi, Indonesian, Malay, and Portuguese) annotated for instances of generalizations. We demonstrate that the likelihood of a co-occurrence being an instance of generalization is usually low, and varies across different languages, identity groups, and attributes. We build classifiers to detect generalization in language with an overall PR-AUC of 58.7, with varying degrees of performance across languages. Our research provides data and tools to enable a nuanced understanding of stereotype perpetuation, a crucial step towards more inclusive and responsible language technologies.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (57)
  1. Andrea E Abele and Bogdan Wojciszke. 2014. Communal and agentic content in social cognition: A dual perspective model. In Advances in experimental social psychology, volume 50, pages 195–255. Elsevier.
  2. Casteism in India, but not racism - a study of bias in word embeddings of Indian languages. In Proceedings of the First Workshop on Language Technology and Resources for a Fair, Inclusive, and Safe Society within the 13th Language Resources and Evaluation Conference, pages 1–7, Marseille, France. European Language Resources Association.
  3. Evaluating the underlying gender bias in contextualized word embeddings. GeBNLP 2019, page 33.
  4. Camiel J Beukeboom and Christian Burgers. 2017. Linguistic bias. In Oxford research encyclopedia of communication.
  5. Re-contextualizing fairness in NLP: The case of India. In Proceedings of the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 727–740, Online only. Association for Computational Linguistics.
  6. Seegull multilingual: a dataset of geo-culturally situated stereotypes. arXiv preprint arXiv:2403.05696.
  7. Stereotyping norwegian salmon: An inventory of pitfalls in fairness benchmark datasets. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 1004–1015.
  8. Man is to computer programmer as woman is to homemaker? debiasing word embeddings. Advances in neural information processing systems, 29.
  9. Detecting racial stereotypes: An italian social media corpus where psychology meets nlp. Information Processing & Management, 60(1):103118.
  10. A multilingual dataset of racial stereotypes in social media conversational threads. In Findings of the Association for Computational Linguistics: EACL 2023, pages 686–696.
  11. Semantics derived automatically from language corpora contain human-like biases. Science, 356(6334):183–186.
  12. Tessa ES Charlesworth and Mahzarin R Banaji. 2022. Word embeddings reveal social group attitudes and stereotypes in large language corpora. Handbook of language analysis in psychology, pages 494–508.
  13. Gender stereotypes in natural language: Word embeddings show robust consistency across child and adult language corpora of more than 65 million words. Psychological Science, 32(2):218–240.
  14. Marked personas: Using natural language prompts to measure stereotypes in language models. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1504–1532, Toronto, Canada. Association for Computational Linguistics.
  15. “be nice to your wife! the restaurants are closed”: Can gender stereotype detection improve sexism classification? In Findings of the Association for Computational Linguistics: EMNLP 2021, pages 2833–2844.
  16. Stepmothers are mean and academics are pretentious: What do pretrained language models learn about you? In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 1477–1491, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
  17. Detecting gender stereotypes: Lexicon vs. supervised learning methods. In Proceedings of the 2020 CHI conference on human factors in computing systems, pages 1–11.
  18. Hate speech classifiers learn normative social stereotypes. Transactions of the Association for Computational Linguistics, 11:300–319.
  19. Stereotype and skew: Quantifying gender bias in pre-trained and fine-tuned language models. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 2232–2242.
  20. Building socio-culturally inclusive stereotype resources with community engagement. In Thirty-seventh Conference on Neural Information Processing Systems Datasets and Benchmarks Track.
  21. Building stereotype repositories with complementary approaches for scale and depth. In Proceedings of the First Workshop on Cross-Cultural Considerations in NLP (C3NLP), pages 84–90, Dubrovnik, Croatia. Association for Computational Linguistics.
  22. On measuring and mitigating biased inferences of word embeddings. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 7659–7666.
  23. Sunipa Dev and Jeff Phillips. 2019. Attenuating bias in word vectors. In The 22nd International Conference on Artificial Intelligence and Statistics, pages 879–887. PMLR.
  24. On measures of biases and harms in NLP. In Findings of the Association for Computational Linguistics: AACL-IJCNLP 2022, pages 246–267, Online only. Association for Computational Linguistics.
  25. Sound framework: Analyzing (so) cial representation in (un) structured (d) ata. arXiv preprint arXiv:2311.17259.
  26. WinoQueer: A community-in-the-loop benchmark for anti-LGBTQ+ bias in large language models. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 9126–9140, Toronto, Canada. Association for Computational Linguistics.
  27. A survey of race, racism, and anti-racism in nlp. arXiv preprint arXiv:2106.11410.
  28. A model of (often mixed) stereotype content: Competence and warmth respectively follow from perceived status and competition. In Social cognition, pages 162–214. Routledge.
  29. A little bird told me your gender: Gender inferences in social media. Information Processing & Management, 58(3):102541.
  30. Word embeddings quantify 100 years of gender and ethnic stereotypes. Proceedings of the National Academy of Sciences, 115(16):E3635–E3644.
  31. Michael Alexander Kirkwood Halliday. 1973. Explorations in the functions of language.
  32. Social biases in NLP models as barriers for persons with disabilities. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5491–5501, Online. Association for Computational Linguistics.
  33. Jacqui Hutchison and Douglas Martin. 2015. The evolution of stereotypes. Evolutionary perspectives on social psychology, pages 291–301.
  34. StereoMap: Quantifying the awareness of human-like stereotypes in large language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 12236–12256, Singapore. Association for Computational Linguistics.
  35. SeeGULL: A stereotype benchmark with broad geo-cultural coverage leveraging generative models. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 9851–9870, Toronto, Canada. Association for Computational Linguistics.
  36. Casteist but not racist? quantifying disparities in large language model bias between india and the west. arXiv preprint arXiv:2309.08573.
  37. Bias out-of-the-box: An empirical analysis of intersectional occupational biases in popular generative language models. Advances in neural information processing systems, 34:2611–2624.
  38. The abc of stereotypes about groups: Agency/socioeconomic success, conservative–progressive beliefs, and communion. Journal of personality and social psychology, 110(5):675.
  39. Measuring bias in contextualized word representations. In Proceedings of the First Workshop on Gender Bias in Natural Language Processing, pages 166–172, Florence, Italy. Association for Computational Linguistics.
  40. Quantifying social biases in contextual word representations. In 1st ACL Workshop on Gender Bias for Natural Language Processing.
  41. Intersectional stereotypes in large language models: Dataset and analysis. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 8589–8597, Singapore. Association for Computational Linguistics.
  42. Deciphering stereotypes in pre-trained language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 11328–11345.
  43. StereoSet: Measuring stereotypical bias in pretrained language models. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 5356–5371, Online. Association for Computational Linguistics.
  44. CrowS-pairs: A challenge dataset for measuring social biases in masked language models. pages 1953–1967.
  45. The measurement of meaning. 47. University of Illinois press.
  46. BBQ: A hand-built bias benchmark for question answering. In Findings of the Association for Computational Linguistics: ACL 2022, pages 2086–2105, Dublin, Ireland. Association for Computational Linguistics.
  47. On releasing annotator-level labels and information in datasets. In Proceedings of the Joint 15th Linguistic Annotation Workshop (LAW) and 3rd Designing Meaning Representations (DMR) Workshop, pages 133–138, Punta Cana, Dominican Republic. Association for Computational Linguistics.
  48. Stereotyping and impression formation: How categorical thinking shapes person perception. 2007) The Sage Handbook of Social Psychology: Concise Student Edition. London: Sage Publications Ltd, pages 68–92.
  49. Characteristics of harmful text: Towards rigorous benchmarking of language models. Advances in Neural Information Processing Systems, 35:24720–24739.
  50. Gender bias in coreference resolution. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers), pages 8–14, New Orleans, Louisiana. Association for Computational Linguistics.
  51. Rachel M Schmitz and Emily Kazyak. 2016. Masculinities in cyberspace: An analysis of portrayals of manhood in men’s rights activist websites. Social Sciences, 5(2):18.
  52. The woman worked as a babysitter: On biases in language generation. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3407–3412, Hong Kong, China. Association for Computational Linguistics.
  53. An information-theoretic approach and dataset for probing gender stereotypes in multilingual masked language models. In Findings of the Association for Computational Linguistics: NAACL 2022, pages 921–932.
  54. Yi Chern Tan and L Elisa Celis. 2019. Assessing social and intersectional biases in contextualized word representations. Advances in neural information processing systems, 32.
  55. Zeerak Waseem. 2016. Are you a racist or am i seeing things? annotator influence on hate speech detection on twitter. In Proceedings of the first workshop on NLP and computational social science, pages 138–142.
  56. Gender bias in coreference resolution: Evaluation and debiasing methods. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers), pages 15–20, New Orleans, Louisiana. Association for Computational Linguistics.
  57. Learning gender-neutral word embeddings. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 4847–4853, Brussels, Belgium. Association for Computational Linguistics.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (5)
  1. Aida Mostafazadeh Davani (13 papers)
  2. Sagar Gubbi (4 papers)
  3. Sunipa Dev (28 papers)
  4. Shachi Dave (12 papers)
  5. Vinodkumar Prabhakaran (48 papers)

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now