Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
149 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

A Dual-Prompting for Interpretable Mental Health Language Models (2402.14854v1)

Published 20 Feb 2024 in cs.CL and cs.AI

Abstract: Despite the increasing demand for AI-based mental health monitoring tools, their practical utility for clinicians is limited by the lack of interpretability.The CLPsych 2024 Shared Task (Chim et al., 2024) aims to enhance the interpretability of LLMs, particularly in mental health analysis, by providing evidence of suicidality through linguistic content. We propose a dual-prompting approach: (i) Knowledge-aware evidence extraction by leveraging the expert identity and a suicide dictionary with a mental health-specific LLM; and (ii) Evidence summarization by employing an LLM-based consistency evaluator. Comprehensive experiments demonstrate the effectiveness of combining domain-specific information, revealing performance improvements and the approach's potential to aid clinicians in assessing mental state progression.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (37)
  1. Large language models are few-shot clinical information extractors. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 1998–2022.
  2. Will affective computing emerge from foundation models and general ai? a first evaluation on chatgpt. IEEE Intelligent Systems, 38:2.
  3. Ethical research protocols for social media health research. In Proceedings of the first ACL workshop on ethics in natural language processing, pages 94–102.
  4. Detection of multiple mental disorders from social media with two-stream psychiatric experts. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 9071–9084, Singapore. Association for Computational Linguistics.
  5. Overview of the clpsych 2024 shared task: Leveraging large language models to identify evidence of suicidality risk in online posts. In Proceedings of the Ninth Workshop on Computational Linguistics and Clinical Psychology. Association for Computational Linguistics.
  6. Challenges to achieving universal health coverage throughout the world: a systematic review. Journal of preventive medicine and public health, 55(2):125.
  7. GPT3.int8(): 8-bit matrix multiplication for transformers at scale. In Advances in Neural Information Processing Systems.
  8. 8-bit optimizers via block-wise quantization. 9th International Conference on Learning Representations, ICLR.
  9. Methodological gaps in predicting mental health states from social media: triangulating diagnostic signals. In Proceedings of the 2019 chi conference on human factors in computing systems, pages 1–16.
  10. The mental health consequences of covid-19 and physical distancing: the need for prevention and early intervention. JAMA internal medicine, 180(6):817–818.
  11. Medalpaca–an open-source collection of medical conversational ai models and training data. arXiv preprint arXiv:2304.08247.
  12. Dirk Hovy and Shannon L Spruit. 2016. The social impact of natural language processing. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 591–598.
  13. Ethical dilemmas posed by mobile health and machine learning in psychiatry research. Bulletin of the World Health Organization, 98(4):270.
  14. Suicidal ideation and the subjective aspects of depression. Journal of affective disorders, 140(1):75–81.
  15. Solar 10.7 b: Scaling large language models with simple yet effective depth up-scaling. arXiv preprint arXiv:2312.15166.
  16. Bishal Lamichhane. 2023. Evaluation of chatgpt for nlp-based mental health applications. arXiv preprint arXiv:2303.15727.
  17. Less annotating, more classifying: Addressing the data scarcity issue of supervised machine learning with deep transfer learning and bert-nli. Political Analysis, 32(1):84–100.
  18. Detecting suicidality with a contextual graph neural network. In Proceedings of the eighth workshop on computational linguistics and clinical psychology, pages 116–125.
  19. Towards suicide prevention from bipolar disorder with temporal symptom-aware multitask learning. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 4357–4369.
  20. CAMEL: Communicative agents for ”mind” exploration of large language model society. In Thirty-seventh Conference on Neural Information Processing Systems.
  21. Improving mental health classifier generalization with pre-diagnosis data. In Proceedings of the International AAAI Conference on Web and Social Media, volume 17, pages 566–577.
  22. Chatgpt as a factual inconsistency evaluator for abstractive text summarization. arXiv preprint arXiv:2303.15621.
  23. Anshu Malhotra and Rajni Jindal. 2024. Xai transformer based approach for interpreting depressed and suicidal user behavior on online social networks. Cognitive Systems Research, 84:101186.
  24. An example of (too much) hyper-parameter tuning in suicide ideation detection. In Proceedings of the International AAAI Conference on Web and Social Media, volume 17, pages 1158–1162.
  25. Expert, crowdsourced, and machine assessment of suicide risk via online postings. In Proceedings of the fifth workshop on computational linguistics and clinical psychology: from keyboard to clinic, pages 25–36.
  26. Understanding factual errors in summarization: Errors, summarizers, datasets, error detectors. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 11626–11644, Toronto, Canada. Association for Computational Linguistics.
  27. Evaluating large language models on medical evidence summarization. npj Digital Medicine, 6(1):158.
  28. Towards an ethical framework for publishing twitter data in social research: Taking into account users’ views, online context and algorithmic estimation. Sociology, 51(6):1149–1168.
  29. Bloomberggpt: A large language model for finance. arXiv preprint arXiv:2303.17564.
  30. Expertprompting: Instructing large language models to be distinguished experts. arXiv preprint arXiv:2305.14688.
  31. Mental-llm: Leveraging large language models for mental health prediction via online text data. arXiv preprint arXiv:2307.14385.
  32. Towards interpretable mental health analysis with large language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 6056–6077.
  33. Mentalllama: Interpretable mental health analysis on social media with large language models. arXiv preprint arXiv:2309.13567.
  34. Extractive summarization via ChatGPT for faithful summary generation. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 3270–3278, Singapore. Association for Computational Linguistics.
  35. Natural language processing applied to mental illness detection: a narrative review. NPJ digital medicine, 5(1):46.
  36. Bertscore: Evaluating text generation with bert. In International Conference on Learning Representations.
  37. Clpsych 2019 shared task: Predicting the degree of suicide risk in reddit posts. In Proceedings of the sixth workshop on computational linguistics and clinical psychology, pages 24–33.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now