Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
38 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
41 tokens/sec
o3 Pro
7 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

AffirmativeAI: Towards LGBTQ+ Friendly Audit Frameworks for Large Language Models (2405.04652v1)

Published 7 May 2024 in cs.HC

Abstract: LGBTQ+ community face disproportionate mental health challenges, including higher rates of depression, anxiety, and suicidal ideation. Research has shown that LGBTQ+ people have been using LLM-based chatbots, such as ChatGPT, for their mental health needs. Despite the potential for immediate support and anonymity these chatbots offer, concerns regarding their capacity to provide empathetic, accurate, and affirming responses remain. In response to these challenges, we propose a framework for evaluating the affirmativeness of LLMs based on principles of affirmative therapy, emphasizing the need for attitudes, knowledge, and actions that support and validate LGBTQ+ experiences. We propose a combination of qualitative and quantitative analyses, hoping to establish benchmarks for "Affirmative AI," ensuring that LLM-based chatbots can provide safe, supportive, and effective mental health support to LGBTQ+ individuals. We benchmark LLM affirmativeness not as a mental health solution for LGBTQ+ individuals or to claim it resolves their mental health issues, as we highlight the need to consider complex discrimination in the LGBTQ+ community when designing technological aids. Our goal is to evaluate LLMs for LGBTQ+ mental health support since many in the community already use them, aiming to identify potential harms of using general-purpose LLMs in this context.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize File Document Download Save Streamline Icon: https://streamlinehq.com PDF File Document Download Save Streamline Icon: https://streamlinehq.com Markdown Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (42)
  1. Ali Alkhatib and Michael Bernstein. 2019. Street-Level Algorithms: A Theory at the Gaps Between Policy and Decisions. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. ACM, Glasgow Scotland Uk, 1–13. https://doi.org/10.1145/3290605.3300760
  2. Mental health, social adversity, and health-related outcomes in sexual minority adolescents: a contemporary national cohort study. The Lancet Child & Adolescent Health 4, 1 (Jan. 2020), 36–45. https://doi.org/10.1016/S2352-4642(19)30339-6
  3. On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?. In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency (FAccT ’21). Association for Computing Machinery, New York, NY, USA, 610–623. https://doi.org/10.1145/3442188.3445922
  4. Markus Bidell. 2005. The Sexual Orientation Counselor Competency Scale: Assessing Attitudes, Skills, and Knowledge of Counselors Working With Lesbian, Gay, and Bisexual Clients. Counselor Education and Supervision 44 (06 2005). https://doi.org/10.1002/j.1556-6978.2005.tb01755.x
  5. Pew Research Center. 2016. Reddit News Users More Likely to Be Male, Young, and Digital in Their News Preferences. https://www.pewresearch.org/journalism/2016/02/25/reddit-news-users-more-likely-to-be-male-young-and-digital-in-their-news-preferences/
  6. Adolescents’ Sexual Orientation and Behavioral and Neural Reactivity to Peer Acceptance and Rejection: The Moderating Role of Family Support. Clinical Psychological Science (2023), 21677026231158574.
  7. Stress-Related Growth, Coming Out, and Internalized Homonegativity in Lesbian, Gay, and Bisexual Youth. An Examination of Stress-Related Growth Within the Minority Stress Model. Journal of Homosexuality 58, 1 (Dec. 2010), 117–137. https://doi.org/10.1080/00918369.2011.533631
  8. A Survey on In-context Learning. arXiv:2301.00234 [cs.CL]
  9. LGBTQ-AI? Exploring Expressions of Gender and Sexual Orientation in Chatbots. In Proceedings of the 3rd Conference on Conversational User Interfaces (Bilbao (online), Spain) (CUI ’21). Association for Computing Machinery, New York, NY, USA, Article 2, 4 pages. https://doi.org/10.1145/3469595.3469597
  10. Delivering Cognitive Behavior Therapy to Young Adults With Symptoms of Depression and Anxiety Using a Fully Automated Conversational Agent (Woebot): A Randomized Controlled Trial. JMIR mental health 4, 2 (June 2017), e19. https://doi.org/10.2196/mental.7785
  11. RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models. (2020). https://doi.org/10.48550/ARXIV.2009.11462 Publisher: arXiv Version Number: 2.
  12. Gilbert Herdt. 1989. Introduction: Gay and lesbian youth, emergent identities, and cultural scenes at home and abroad. Journal of Homosexuality 17, 1-2 (1989), 1–42. https://doi.org/10.1300/J082v17n01_01 Place: US Publisher: Haworth Press.
  13. Benjamin Mako Hill and Aaron Shaw. 2013. The Wikipedia Gender Gap Revisited: Characterizing Survey Response Bias with Propensity Score Estimation. PLoS ONE 8, 6 (June 2013), e65782. https://doi.org/10.1371/journal.pone.0065782
  14. Angela N. Hilton and Dawn M. Szymanski. 2011. Family dynamics and changes in sibling of origin relationship after lesbian and gay sexual orientation disclosure. Contemporary Family Therapy: An International Journal 33, 3 (2011), 291–309. https://doi.org/10.1007/s10591-011-9157-3 Place: Germany Publisher: Springer.
  15. Kate L.M. Hinrichs and Weston Donaldson. 2017. Recommendations for Use of Affirmative Psychotherapy With LGBT Older Adults. Journal of Clinical Psychology 73, 8 (2017), 945–953. https://doi.org/10.1002/jclp.22505 arXiv:https://onlinelibrary.wiley.com/doi/pdf/10.1002/jclp.22505
  16. Depression and self-harm from adolescence to young adulthood in sexual minorities compared with heterosexuals in the UK: a population-based cohort study. The Lancet Child & Adolescent Health 3, 2 (Feb. 2019), 91–98. https://doi.org/10.1016/S2352-4642(18)30343-2
  17. ChatGPT for good? On opportunities and challenges of large language models for education. Learning and Individual Differences 103 (2023), 102274. https://doi.org/10.1016/j.lindif.2023.102274
  18. Measuring Bias in Contextualized Word Representations. In Proceedings of the First Workshop on Gender Bias in Natural Language Processing. Association for Computational Linguistics, Florence, Italy, 166–172. https://doi.org/10.18653/v1/W19-3823
  19. Minhyeok Lee. 2023. A Mathematical Investigation of Hallucination and Creativity in GPT Models. Mathematics 11, 10 (May 2023), 2320. https://doi.org/10.3390/math11102320
  20. Holistic Evaluation of Language Models. Transactions on Machine Learning Research (2023). https://openreview.net/forum?id=iO4LZibEqW Featured Certification, Expert Certification.
  21. Evaluating the Experience of LGBTQ+ People Using Large Language Model Based Chatbots for Mental Health Support. arXiv preprint arXiv:2402.09260 (2024).
  22. Understanding the Benefits and Challenges of Using Large Language Model-based Conversational Agents for Mental Well-being Support. AMIA … Annual Symposium proceedings. AMIA Symposium 2023 (2023), 1105–1114.
  23. Ilan H. Meyer. 1995. Minority Stress and Mental Health in Gay Men. Journal of Health and Social Behavior 36, 1 (March 1995), 38. https://doi.org/10.2307/2137286
  24. Ilan H. Meyer. 2003. Prejudice, social stress, and mental health in lesbian, gay, and bisexual populations: conceptual issues and research evidence. Psychological Bulletin 129, 5 (Sept. 2003), 674–697. https://doi.org/10.1037/0033-2909.129.5.674
  25. Bonnie Moradi and Stephanie L. Budge. 2018. Engaging in LGBQ+ affirmative psychotherapies with all clients: Defining themes and practices. Journal of Clinical Psychology 74, 11 (Nov. 2018), 2028–2042. https://doi.org/10.1002/jclp.22687
  26. John E. Pachankis and Marvin R. Goldfried. 2004. Clinical Issues in Working With Lesbian, Gay, and Bisexual Clients. Psychotherapy: Theory, Research, Practice, Training 41, 3 (2004), 227–246. https://doi.org/10.1037/0033-3204.41.3.227
  27. Affirmative LGBT psychotherapy: Outcomes of a therapist training protocol. Psychotherapy 55, 1 (March 2018), 52–62. https://doi.org/10.1037/pst0000149
  28. Francesca Polletta. 1998. Contending stories: Narrative in social movements. Qualitative Sociology 21, 4 (1998), 419–446. https://doi.org/10.1023/A:1023332410633
  29. Brandon Andrew Robinson. 2018. Conditional Families and Lesbian, Gay, Bisexual, Transgender, and Queer Youth Homelessness: Gender, Sexuality, Family Instability, and Rejection. Journal of Marriage and Family 80, 2 (April 2018), 383–396. https://doi.org/10.1111/jomf.12466
  30. Non-Binary Clients’ Experiences of Psychotherapy: Uncomfortable and Affirmative Approaches. International journal of environmental research and public health 19, 22 (2022), 15339. https://doi.org/10.3390/ijerph192215339
  31. Family Rejection as a Predictor of Negative Health Outcomes in White and Latino Lesbian, Gay, and Bisexual Young Adults. Pediatrics 123, 1 (Jan. 2009), 346–352. https://doi.org/10.1542/peds.2007-3524
  32. Family Acceptance in Adolescence and the Health of LGBT Young Adults: Family Acceptance in Adolescence and the Health of LGBT Young Adults. Journal of Child and Adolescent Psychiatric Nursing 23, 4 (Nov. 2010), 205–213. https://doi.org/10.1111/j.1744-6171.2010.00246.x
  33. Elizabeth M Saewyc. 2011. Research on adolescent sexual orientation: Development, health disparities, stigma, and resilience. Journal of research on adolescence 21, 1 (2011), 256–272.
  34. Sexual orientation and symptoms of common mental disorder or low wellbeing: combined meta-analysis of 12 UK population health surveys. BMC psychiatry 16 (March 2016), 67. https://doi.org/10.1186/s12888-016-0767-z
  35. The Woman Worked as a Babysitter: On Biases in Language Generation. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). Association for Computational Linguistics, Hong Kong, China, 3407–3412. https://doi.org/10.18653/v1/D19-1339
  36. Coping With Sexual Orientation–Related Minority Stress. Journal of Homosexuality 65, 4 (March 2018), 484–500. https://doi.org/10.1080/00918369.2017.1321888
  37. Trevor Project. 2023. 2023 National Survey on LGBTQ Youth Mental Health. https://www.thetrevorproject.org/survey-2023/
  38. Black Lives Matter in Wikipedia: Collective Memory and Collaboration around Online Social Movements. In Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing (Portland, Oregon, USA) (CSCW ’17). Association for Computing Machinery, New York, NY, USA, 1400–1412. https://doi.org/10.1145/2998181.2998232
  39. Attention is All You Need. In Proceedings of the 31st International Conference on Neural Information Processing Systems (Long Beach, California, USA) (NIPS’17). Curran Associates Inc., Red Hook, NY, USA, 6000–6010.
  40. Enacted Stigma, Mental Health, and Protective Factors Among Transgender Youth in Canada. Transgender Health 2, 1 (Dec. 2017), 207–216. https://doi.org/10.1089/trgh.2017.0031
  41. An Evaluation of Generative Pre-Training Model-based Therapy Chatbot for Caregivers. ArXiv abs/2107.13115 (2021). https://api.semanticscholar.org/CorpusID:236469205
  42. Fine-Tuning Language Models from Human Preferences. http://arxiv.org/abs/1909.08593 arXiv:1909.08593 [cs, stat].
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (4)
  1. Yinru Long (2 papers)
  2. Zilin Ma (7 papers)
  3. Yiyang Mei (7 papers)
  4. Zhaoyuan Su (9 papers)
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets