· Trending Papers
Emergent Mind

Reliability Analysis of Psychological Concept Extraction and Classification in User-penned Text

(2401.06709)
Published Jan 12, 2024 in cs.CL and cs.AI

Abstract

The social NLP research community witness a recent surge in the computational advancements of mental health analysis to build responsible AI models for a complex interplay between language use and self-perception. Such responsible AI models aid in quantifying the psychological concepts from user-penned texts on social media. On thinking beyond the low-level (classification) task, we advance the existing binary classification dataset, towards a higher-level task of reliability analysis through the lens of explanations, posing it as one of the safety measures. We annotate the LoST dataset to capture nuanced textual cues that suggest the presence of low self-esteem in the posts of Reddit users. We further state that the NLP models developed for determining the presence of low self-esteem, focus more on three types of textual cues: (i) Trigger: words that triggers mental disturbance, (ii) LoST indicators: text indicators emphasizing low self-esteem, and (iii) Consequences: words describing the consequences of mental disturbance. We implement existing classifiers to examine the attention mechanism in pre-trained language models (PLMs) for a domain-specific psychology-grounded task. Our findings suggest the need of shifting the focus of PLMs from Trigger and Consequences to a more comprehensive explanation, emphasizing LoST indicators while determining low self-esteem in Reddit posts.

We're not able to analyze this paper right now due to high demand.

Please check back later (sorry!).

Sign up for a free account or log in to generate a summary of this paper:

Sign Up Now
or
Log In

We ran into a problem analyzing this paper.

Newsletter

Get summaries of trending comp sci papers delivered straight to your inbox:

Unsubscribe anytime.

References
  1. Incidence of social phobia and identification of its risk indicators: a model for prevention. Acta Psychiatrica Scandinavica, 119(1): 62–70.
  2. Optimal classifier for imbalanced data using Matthews Correlation Coefficient metric. PloS one, 12(6): e0177678.
  3. Non-suicidal self-injury in eating disorders: Prevalence, characteristics, DSM-5 proposed diagnostic criteria, and correlates. Journal of Affective Disorders Reports, 7: 100292.
  4. Risk and protective factors predicting multiple suicide attempts. Psychiatry research, 210(3): 957–961.
  5. Collaborators, G. . M. D.; et al. 2022. Global, regional, and national burden of 12 mental disorders in 204 countries and territories, 1990–2019: a systematic analysis for the Global Burden of Disease Study 2019. The Lancet Psychiatry, 9(2): 137–150.
  6. Cox, R. 1987. The rich harvest of Abraham Maslow. Motivation and personality, 245–271.

  7. Daws, R. 2020. Babylon Health lashes out at doctor who raised AI chatbot safety concerns. https://www.artificialintelligence-news.com/2020/02/26/babylon-health-doctor-ai-chatbot-safety-concerns/.

  8. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
  9. LonXplain: Lonesomeness as a Consequence of Mental Disturbance in Reddit Posts
  10. Guggenmoos-Holzmann, I. 1996. The meaning of kappa: probabilistic concepts of reliability and validity revisited. Journal of clinical epidemiology, 49(7): 775–782.
  11. DEBERTA: DECODING-ENHANCED BERT WITH DISENTANGLED ATTENTION. In International Conference on Learning Representations.
  12. Ethical challenges in data-driven dialogue systems
  13. Why do female domestic violence victims remain in or leave abusive relationships? A qualitative study. Journal of Aggression, Maltreatment & Trauma, 31(5): 677–694.
  14. Work and suicide: An interdisciplinary systematic literature review. Journal of Organizational Behavior, 43(2): 260–285.
  15. ClinicalBERT: Modeling Clinical Notes and Predicting Hospital Readmission
  16. MentalBERT: Publicly Available Pretrained Language Models for Mental Healthcare. In Proceedings of the Thirteenth Language Resources and Evaluation Conference, 7184–7190.
  17. A systematic review and meta-analysis of CBT interventions based on the Fennell model of low self-esteem. Psychiatry research, 267: 296–305.
  18. Suicide risk in chronic heart failure patients and its association with depression, hopelessness and self esteem. Journal of clinical neuroscience, 68: 51–54.
  19. ALBERT: A Lite BERT for Self-supervised Learning of Language Representations
  20. Do We Still Need Human Assessors? Prompt-Based GPT-3 User Simulation in Conversational AI. In Proceedings of the 4th Conference on Conversational User Interfaces, 1–6.
  21. The Interpersonal Needs Questionnaire: Statistical considerations for improved clinical application. Assessment, 27(3): 621–637.
  22. Potard, C. 2017. Self-esteem inventory (Coopersmith).
  23. Constructing a disease database and using natural language processing to capture and standardize free text clinical information. Scientific Reports, 13(1): 8591.
  24. ” Why should i trust you?” Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, 1135–1144.
  25. Rosenberg, M. 1965. Rosenberg self-esteem scale. Journal of Religion and Health.
  26. Low self-esteem and the formation of global self-performance estimates in emerging adulthood. Translational Psychiatry, 12(1): 272.
  27. DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter
  28. Knowledge-intensive language understanding for explainable AI. IEEE Internet Computing, 25(5): 19–24.
  29. Building and evaluating resources for sentiment analysis in the Greek language. Language resources and evaluation, 52(4): 1021–1044.
  30. Dreaddit: A Reddit Dataset for Stress Analysis in Social Media
  31. PsychBERT: a mental health language model for social media mental health behavioral analysis. In 2021 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), 1077–1082. IEEE.
  32. Mortality in mental disorders and global disease burden implications: a systematic review and meta-analysis. JAMA psychiatry, 72(4): 334–341.
  33. Want To Reduce Labeling Cost? GPT-3 Can Help. In Findings of the Association for Computational Linguistics: EMNLP 2021, 4195–4205.
  34. Rejection sensitivity, social withdrawal, and loneliness in young adults. Journal of Applied Social Psychology, 42(8): 1984–2005.
  35. Extending motivational interviewing to the treatment of major mental health problems: current directions and evidence. The Canadian Journal of Psychiatry.
  36. The FAIR Guiding Principles for scientific data management and stewardship. Scientific data, 3(1): 1–9.
  37. Adaptations of ROUGE and BLEU to Better Evaluate Machine Reading Comprehension Task. ACL 2018, 98.
  38. Explaining Models of Mental Health via Clinically Grounded Auxiliary Tasks. In CLPsych.

Show All 38