Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
91 tokens/sec
GPT-4o
12 tokens/sec
Gemini 2.5 Pro Pro
o3 Pro
5 tokens/sec
GPT-4.1 Pro
15 tokens/sec
DeepSeek R1 via Azure Pro
33 tokens/sec
Gemini 2.5 Flash Deprecated
12 tokens/sec
2000 character limit reached

I Am Not Them: Fluid Identities and Persistent Out-group Bias in Large Language Models (2402.10436v1)

Published 16 Feb 2024 in cs.CL

Abstract: We explored cultural biases-individualism vs. collectivism-in ChatGPT across three Western languages (i.e., English, German, and French) and three Eastern languages (i.e., Chinese, Japanese, and Korean). When ChatGPT adopted an individualistic persona in Western languages, its collectivism scores (i.e., out-group values) exhibited a more negative trend, surpassing their positive orientation towards individualism (i.e., in-group values). Conversely, when a collectivistic persona was assigned to ChatGPT in Eastern languages, a similar pattern emerged with more negative responses toward individualism (i.e., out-group values) as compared to collectivism (i.e., in-group values). The results indicate that when imbued with a particular social identity, ChatGPT discerns in-group and out-group, embracing in-group values while eschewing out-group values. Notably, the negativity towards the out-group, from which prejudices and discrimination arise, exceeded the positivity towards the in-group. The experiment was replicated in the political domain, and the results remained consistent. Furthermore, this replication unveiled an intrinsic Democratic bias in LLMs, aligning with earlier findings and providing integral insights into mitigating such bias through prompt engineering. Extensive robustness checks were performed using varying hyperparameter and persona setup methods, with or without social identity labels, across other popular LLMs.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (112)
  1. Chatgpt in healthcare: Exploring ai chatbot for spontaneous word retrieval in aphasia. In Companion Publication of the 2023 Conference on Computer Supported Cooperative Work and Social Computing, pages 1–5, 2023.
  2. On the opportunities and risks of foundation models. arXiv preprint arXiv:2108.07258, 2021.
  3. Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901, 2020.
  4. “kelly is a warm person, joseph is a role model”: Gender biases in llm-generated reference letters. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 3730–3748, 2023.
  5. Towards mitigating llm hallucination via self reflection. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 1827–1843, 2023.
  6. Whose opinions do language models reflect? In Proceedings of the 40th International Conference on Machine Learning, ICML’23, 2023.
  7. Chatgpt is more likely to be perceived as male than female. arXiv preprint arXiv:2305.12564, 2023.
  8. “fifty shades of bias”: Normative ratings of gender bias in GPT generated English text. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 1862–1876. Association for Computational Linguistics, December 2023.
  9. From pretraining data to language models to downstream tasks: Tracking the trails of political biases leading to unfair NLP models. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 11737–11762. Association for Computational Linguistics, 2023.
  10. Di Zhou and Yinxian Zhang. Red ai? inconsistent responses from gpt3. 5 models on political issues in the us and china. arXiv preprint arXiv:2312.09917, 2023.
  11. Large language models propagate race-based medicine. NPJ Digital Medicine, 6(1):195, 2023.
  12. Chatgpt exhibits gender and racial biases in acute coronary syndrome management. medRxiv, pages 2023–11, 2023.
  13. Llms–the good, the bad or the indispensable?: A use case on legal statute prediction and legal judgment prediction on indian court cases. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 12451–12474, 2023.
  14. Human heuristics for ai-generated language are flawed. Proceedings of the National Academy of Sciences, 120(11):e2208839120, 2023.
  15. An integrative theory of intergroup conflict. Organizational identity: A reader, 56(65):9780203505984–16, 1979.
  16. Social identifications: A social psychology of intergroup relations and group processes. Routledge, 2006.
  17. Out-group homogeneity: Judgments of variability at the individual and group levels. Journal of personality and social psychology, 54(5):778, 1988.
  18. Social categorization and intergroup behaviour. European journal of social psychology, 1(2):149–178, 1971.
  19. Out-group animosity drives engagement on social media. Proceedings of the National Academy of Sciences, 118(26):e2024292118, 2021.
  20. In conversation with artificial intelligence: aligning language models with human values. Philosophy & Technology, 36(2):1–24, 2023.
  21. Geert Hofstede. Culture and organizations. International studies of management & organization, 10(4):15–41, 1980.
  22. Alan S Waterman. The psychology of individualism. New York: Praeger, 1984.
  23. Shalom H Schwartz. Individualism-collectivism: Critique and proposed refinements. Journal of cross-cultural psychology, 21(2):139–157, 1990.
  24. Daphna Oyserman. The lens of personhood: Viewing the self and others in a multicultural society. Journal of personality and social psychology, 65(5):993, 1993.
  25. Harry C Triandis. A theoretical framework for the study of diversity. 1995.
  26. Cultural incongruencies in artificial intelligence. arXiv preprint arXiv:2211.13069, 2022.
  27. Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems, 35:27730–27744, 2022.
  28. Using large language models to simulate multiple humans and replicate human subject studies. In International Conference on Machine Learning, pages 337–371. PMLR, 2023.
  29. Do llms exhibit human-like response biases? a case study in survey design. arXiv preprint arXiv:2311.04076, 2023.
  30. Evaluating large language models in generating synthetic hci research data: a case study. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, pages 1–19, 2023.
  31. Out of one, many: Using language models to simulate human samples. Political Analysis, 31(3):337–351, 2023.
  32. Chatgpt outperforms crowd workers for text-annotation tasks. Proceedings of the National Academy of Sciences, 120(30):e2305016120, 2023.
  33. Generative language models exhibit social identity biases. arXiv preprint arXiv:2310.15819, 2023.
  34. Large language models show human-like content biases in transmission chain experiments. Proceedings of the National Academy of Sciences, 120(44):e2313790120, 2023.
  35. Multilingual language models are not multicultural: A case study in emotion. In Proceedings of the 13th Workshop on Computational Approaches to Subjectivity, Sentiment, & Social Media Analysis, pages 202–214, 2023.
  36. Cultural alignment in large language models: An explanatory analysis based on hofstede’s cultural dimensions. arXiv preprint arXiv:2309.12342, 2023.
  37. William D Crano. Milestones in the psychological analysis of social influence. Group Dynamics: Theory, Research, and Practice, 4(1):68, 2000.
  38. Henri Tajfel. Experiments in intergroup discrimination. Scientific american, 223(5):96–103, 1970.
  39. Robert M Sapolsky. Behave: The biology of humans at our best and worst. Penguin, 2017.
  40. Let me count the ways in which i respect thee: Does competence compensate or compromise lack of liking from the group? European Journal of Social Psychology, 35(2):263–279, 2005.
  41. Social identity theory: Constructive and critical advances. Springer-Verlag Publishing, 1990.
  42. Michael A Hogg. Social identity theory. Springer, 2016.
  43. Stereotypes and social cognition. Sage Publications, Inc, 1994.
  44. The perception of variability within in-groups and out-groups: Implications for the law of small numbers. Journal of personality and social psychology, 38(1):141, 1980.
  45. Social identity and perceived group homogeneity: Evidence for the ingroup homogeneity effect. European Journal of Social Psychology, 20(4):269–286, 1990.
  46. An information sampling explanation for the in-group heterogeneity effect. Psychological Review, 127(1):47, 2020.
  47. John C Turner. Social comparison and social identity: Some prospects for intergroup behaviour. European journal of social psychology, 5(1):1–34, 1975.
  48. Marilynn B Brewer. The psychology of prejudice: Ingroup love or outgroup hate?, 55 j, 1999.
  49. “in-group love” and “out-group hate” as motives for individual participation in intergroup conflict: A new game paradigm. Psychological science, 19(4):405–411, 2008.
  50. “in-group love” and “out-group hate” in repeated interaction between groups. Journal of Behavioral Decision Making, 25(2):188–195, 2012.
  51. Collective narcissism and its social consequences. Journal of personality and social psychology, 97(6):1074, 2009.
  52. The paradox of in-group love: Differentiating collective narcissism advances understanding of the relationship between in-group and out-group attitudes. Journal of Personality, 81(1):16–28, 2013.
  53. Social discrimination and tolerance in intergroup relations: Reactions to intergroup difference. Personality and social psychology review, 3(2):158–174, 1999.
  54. Avner Greif. Cultural beliefs and the organization of society: A historical and theoretical reflection on collectivist and individualist societies. Journal of political economy, 102(5):912–950, 1994.
  55. Cultural variation in the self-concept. In The self: Interdisciplinary approaches, pages 18–48. Springer, 1991.
  56. Theodore M Singelis. The measurement of independent and interdependent self-construals. Personality and social psychology bulletin, 20(5):580–591, 1994.
  57. Irina Cozma. How are individualism and collectivism measured. Romanian Journal of Applied Psychology, 13(1):11–17, 2011.
  58. Partisans without constraint: Political polarization and trends in american public opinion. American Journal of Sociology, 114(2):408–446, 2008.
  59. Fear among the extremes: How political ideology predicts negative emotions and outgroup derogation. Personality and social psychology bulletin, 41(4):485–497, 2015.
  60. Polarization and conflict: Theoretical and empirical issues, 2008.
  61. Political polarization on twitter. In Proceedings of the international aaai conference on web and social media, volume 5, pages 89–96, 2011.
  62. Social media, political polarization, and political disinformation: A review of the scientific literature. Political polarization, and political disinformation: a review of the scientific literature (March 19, 2018), 2018.
  63. Robert W McGee. Is chat gpt biased against conservatives? an empirical study. An Empirical Study (February 15, 2023), 2023.
  64. David Rozado. The political biases of chatgpt. Social Sciences, 12(3):148, 2023.
  65. Detecting cross-geographic biases in toxicity modeling on social media. In Proceedings of the Seventh Workshop on Noisy User-generated Text (W-NUT 2021), pages 313–328, 2021.
  66. Picking on the same person: Does algorithmic monoculture lead to outcome homogenization? Advances in Neural Information Processing Systems, 35:3663–3678, 2022.
  67. Co-writing with opinionated language models affects users’ views. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, pages 1–15, 2023.
  68. Taxonomy of risks posed by language models. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency, pages 214–229, 2022.
  69. Emilio Ferrara. Social bot detection in the age of chatgpt: Challenges and opportunities. First Monday, 2023.
  70. The spread of low-credibility content by social bots. Nature communications, 9(1):1–9, 2018.
  71. Rethinking individualism and collectivism: evaluation of theoretical assumptions and meta-analyses. Psychological bulletin, 128(1):3, 2002.
  72. Horizontal and vertical dimensions of individualism and collectivism: A theoretical and measurement refinement. Cross-cultural research, 29(3):240–275, 1995.
  73. Probing pre-trained language models for cross-cultural differences in values. In Proceedings of the First Workshop on Cross-Cultural Considerations in NLP (C3NLP), pages 114–130. Association for Computational Linguistics, May 2023.
  74. R OpenAI. Gpt-4 technical report. arXiv, pages 2303–08774, 2023.
  75. Towards making the most of chatgpt for machine translation. In Findings of EMNLP 2023, 2023.
  76. Surfacing racial stereotypes through identity portrayal. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency, pages 1604–1615, 2022.
  77. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805, 2023.
  78. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288, 2023.
  79. David S. Yeager Christopher J. Bryan Margarett Clapper Susannah Chandhok Johannes C. Eichstaedt Cameron Hecht Jeremy Jamieson Meghann Johnson Michaela Jones Danielle Krettek-Cobb Leslie Lai Nirel JonesMitchell Desmond C. Ong Carol S. Dweck James J. Gross & James W. Pennebaker Dorottya Demszky, Diyi Yang. Using large language models in psychology. Nature Reviews Psychology, 2023.
  80. Large language models are zero-shot reasoners. Advances in neural information processing systems, 35:22199–22213, 2022.
  81. How can we know what language models know? Transactions of the Association for Computational Linguistics, 8:423–438, 2020.
  82. Making pre-trained language models better few-shot learners. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 3816–3830, 2021.
  83. Large language models meet nl2code: A survey. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 7443–7464, 2023.
  84. From primed concepts to action: A meta-analysis of the behavioral effects of incidentally presented words. Psychological bulletin, 142(5):472, 2016.
  85. Scott Clifford. Compassionate democrats and tough republicans: How ideology shapes partisan stereotypes. Political Behavior, 42(4):1269–1293, 2020.
  86. Mitigating political bias in language models through reinforced calibration. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 14857–14866, 2021.
  87. Merriam-Webster. What is the difference between a democrat and a republican?, 2023.
  88. Negativity bias, negativity dominance, and contagion. Personality and social psychology review, 5(4):296–320, 2001.
  89. Negative information weighs more heavily on the brain: the negativity bias in evaluative categorizations. Journal of personality and social psychology, 75(4):887, 1998.
  90. Eva Cetinic. The myth of culturally agnostic ai models. arXiv preprint arXiv:2211.15271, 2022.
  91. Jun Li Jeung and Janet Yi-Ching Huang. Correct me if i am wrong: Exploring how ai outputs affect user perception and trust. In Companion Publication of the 2023 Conference on Computer Supported Cooperative Work and Social Computing, pages 323–327, 2023.
  92. Automation accuracy is good, but high controllability may be better. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, CHI ’19, page 1–8, New York, NY, USA, 2019. Association for Computing Machinery.
  93. Partha Pratim Ray. Chatgpt: A comprehensive review on background, applications, key challenges, bias, ethics, limitations and future scope. Internet of Things and Cyber-Physical Systems, 2023.
  94. Artificial intelligence can persuade humans on political issues. 2023.
  95. Catalina L Toma. Perceptions of trustworthiness online: the role of visual and textual information. In Proceedings of the 2010 ACM conference on Computer supported cooperative work, pages 13–22, 2010.
  96. Judging truth. Annual review of psychology, 71:499–515, 2020.
  97. The nature and origins of misperceptions: Understanding false and unsupported beliefs about politics. Political Psychology, 38:127–150, 2017.
  98. S3superscript𝑆3S^{3}italic_S start_POSTSUPERSCRIPT 3 end_POSTSUPERSCRIPT: Social-network simulation system with large language model-empowered agents. arXiv preprint arXiv:2307.14984, 2023.
  99. Ai model gpt-3 (dis)informs us better than humans. Science Advances, 9(26):eadh1850, 2023.
  100. Lost in transformation: Rediscovering llm-generated campaigns in social media. In Multidisciplinary International Symposium on Disinformation in Open Online Media, pages 72–87. Springer, 2023.
  101. Quantifying search bias: Investigating sources of bias for political searches in social media. In Proceedings of the 2017 ACM conference on computer supported cooperative work and social computing, pages 417–432, 2017.
  102. The rise of social bots. Communications of the ACM, 59(7):96–104, 2016.
  103. Why johnny can’t prompt: how non-ai experts try (and fail) to design llm prompts. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, pages 1–21, 2023.
  104. Toward value scenario generation through large language models. In Companion Publication of the 2023 Conference on Computer Supported Cooperative Work and Social Computing, pages 212–220, 2023.
  105. Factuality challenges in the era of large language models. arXiv preprint arXiv:2310.05189, 2023.
  106. Towards interpretable mental health analysis with large language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 6056–6077, 2023.
  107. Evaluating the performance of chatgpt in ophthalmology: An analysis of its successes and shortcomings. Ophthalmology Science, page 100324, 2023.
  108. Shaping the emerging norms of using large language models in social computing research. In Companion Publication of the 2023 Conference on Computer Supported Cooperative Work and Social Computing, pages 569–571, 2023.
  109. Shiona McCallum. Chatgpt banned in italy over privacy concerns. BBC News, 2023.
  110. Emilio Ferrara. Should chatgpt be biased? challenges and risks of bias in large language models. arXiv preprint arXiv:2304.03738, 2023.
  111. Platform (in) justice: A call for a global research agenda. In Companion Publication of the 2023 Conference on Computer Supported Cooperative Work and Social Computing, pages 411–414, 2023.
  112. Dynamic remodeling of in-group bias during the 2008 presidential election. Proceedings of the National Academy of Sciences, 106(15):6187–6191, 2009.
Citations (2)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets