Angry Men, Sad Women: Large Language Models Reflect Gendered Stereotypes in Emotion Attribution (2403.03121v3)

Abstract: LLMs reflect societal norms and biases, especially about gender. While societal biases and stereotypes have been extensively researched in various NLP applications, there is a surprising gap for emotion analysis. However, emotion and gender are closely linked in societal discourse. E.g., women are often thought of as more empathetic, while men's anger is more socially accepted. To fill this gap, we present the first comprehensive study of gendered emotion attribution in five state-of-the-art LLMs (open- and closed-source). We investigate whether emotions are gendered, and whether these variations are based on societal stereotypes. We prompt the models to adopt a gendered persona and attribute emotions to an event like 'When I had a serious argument with a dear person'. We then analyze the emotions generated by the models in relation to the gender-event pairs. We find that all models consistently exhibit gendered emotions, influenced by gender stereotypes. These findings are in line with established research in psychology and gender studies. Our study sheds light on the complex societal interplay between language, gender, and emotion. The reproduction of emotion stereotypes in LLMs allows us to use those models to study the topic in detail, but raises questions about the predictive use of those same LLMs for emotion applications.

• The paper demonstrates that LLMs replicate gender stereotypes by attributing anger to men and sadness to women using over 200K gender-event pairs.
• The authors employ persona-based prompts with models like GPT-4 and LLaMA to reveal statistically significant biases in emotion attribution.
• The study highlights ethical concerns in emotion analysis, urging interdisciplinary solutions to mitigate gender bias in NLP applications.

Overview of Gendered Stereotypes in Emotion Attribution in LLMs

The paper entitled "Angry Men, Sad Women: LLMs Reflect Gendered Stereotypes in Emotion Attribution" provides a thorough investigation into how LLMs replicate gender-based stereotypes in emotion attribution. This research addresses a notable gap in current NLP literature regarding the intersection of gender bias and emotion analysis. While societal norms and biases, particularly regarding gender, have been extensively documented in various applications, this paper focuses specifically on emotion attribution, providing the first comprehensive paper of its kind on state-of-the-art LLMs.

The research scrutinizes five prominent LLMs, examining how these models attribute emotions when prompted with scenarios connected to gendered personas. The authors present compelling evidence that these models predict emotions consistent with stereotypical narratives—women are more frequently associated with sadness, and men with anger—aligning with established psychological and gender studies. Using persona-based prompting techniques, the paper explores 200K+ gender-event pairs to corroborate these misalignments.

Key Findings and Methodology

The authors utilize the International Survey On Emotion Antecedents And Reactions (ISEAR) dataset alongside modern LLMs—among them, LLaMA and GPT-4—to assess whether LLMs inherently reflect societal gender stereotypes. Their methodology involves using persona-based prompts to ask models what emotion a gendered persona might experience during a given scenario. The reevaluations of these contexts reveal statistically significant differences in the way emotions are attributed across genders.

Quantitative results clearly show that stereotypes such as "angry men" and "sad women" persist within model outputs. Such findings underline that LLMs tend to amplify disparities rather than merely mirror real-world lived experiences. The disparity underlines the models' failure to objectively replicate the true socio-emotional experiences presented in datasets—a potential pitfall in using LLMs for emotion-driven applications.

Implications and Future Perspectives

This paper's revelations highlight significant implications for any NLP applications involving emotion recognition or sentiment analysis, particularly in areas demanding nuanced understandings, such as mental health diagnostics and human-computer interaction. The propagation of such stereotypes poses ethical and practical concerns, as these models may unwittingly perpetuate harmful biases and provide skewed emotional intelligence assessments.

The authors advocate for a reconsideration of how such bias-laden models are used in emotion-related applications. They suggest that societal norms embedded into these models could lead to disproportionate dangers, including impacting judgments and opportunities that involve human emotions. Emphasizing interdisciplinary approaches, the authors call for integrating insights from psychology and social sciences to craft more inclusive and equitable NLP systems.

Conclusion

This paper is significant in advancing the discourse on gender stereotyping in AI, urging for greater scrutiny in developing and deploying LLMs, particularly for emotion attribution tasks. It prompts critical discussions about how to responsibly utilize AI in emotionally sensitive domains and the importance of advancing fairness and inclusiveness within these evolving technologies. By shedding light on these recurring issues, the paper encourages broader investigation into ensuring ethical and unbiased AI systems capable of fostering both technical excellence and societal well-being.