2000 character limit reached
ERD: A Framework for Improving LLM Reasoning for Cognitive Distortion Classification (2403.14255v1)
Published 21 Mar 2024 in cs.CL and cs.LG
Abstract: Improving the accessibility of psychotherapy with the aid of LLMs is garnering a significant attention in recent years. Recognizing cognitive distortions from the interviewee's utterances can be an essential part of psychotherapy, especially for cognitive behavioral therapy. In this paper, we propose ERD, which improves LLM-based cognitive distortion classification performance with the aid of additional modules of (1) extracting the parts related to cognitive distortion, and (2) debating the reasoning steps by multiple agents. Our experimental results on a public dataset show that ERD improves the multi-class F1 score as well as binary specificity score. Regarding the latter score, it turns out that our method is effective in debiasing the baseline method which has high false positive rate, especially when the summary of multi-agent debate is provided to LLMs.
- Graph of thoughts: Solving elaborate problems with large language models. arXiv preprint arXiv:2308.09687.
- Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901.
- Chateval: Towards better llm-based evaluators through multi-agent debate.
- Program of thoughts prompting: Disentangling computation from reasoning for numerical reasoning tasks.
- Empowering psychotherapy with large language models: Cognitive distortion detection through diagnosis of thought prompting. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 4295–4304, Singapore. Association for Computational Linguistics.
- Palm: Scaling language modeling with pathways. arXiv preprint arXiv:2204.02311.
- Improving factuality and reasoning in language models through multiagent debate.
- Large language models are zero-shot reasoners. Advances in neural information processing systems, 35:22199–22213.
- Chain of empathy: Enhancing empathetic response of large language models based on psychotherapy models. arXiv preprint arXiv:2311.04915.
- Encouraging divergent thinking in large language models through multi-agent debate.
- Chatcounselor: A large language models for mental health support. arXiv preprint arXiv:2309.15461.
- R OpenAI. 2023. Gpt-4 technical report. arXiv, pages 2303–08774.
- Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems, 35:27730–27744.
- Improving language understanding by generative pre-training.
- Language models are unsupervised multitask learners. OpenAI blog, 1(8):9.
- Gpt is an effective tool for multilingual psychological text analysis.
- A shoulder to cry on: Towards a motivational virtual assistant for assuaging mental agony. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2436–2449, Seattle, United States. Association for Computational Linguistics.
- A computational approach to understanding empathy expressed in text-based mental health support.
- Sagarika Shreevastava and Peter Foltz. 2021. Detecting cognitive distortions from patient-therapist interactions. In Proceedings of the Seventh Workshop on Computational Linguistics and Clinical Psychology: Improving Access, pages 151–158, Online. Association for Computational Linguistics.
- Large language models encode clinical knowledge. arXiv preprint arXiv:2212.13138.
- Towards expert-level medical question answering with large language models. arXiv preprint arXiv:2305.09617.
- An Analysis of Physician Resistance to Psychiatric Consultations. Archives of General Psychiatry, 37(9):1007–1012.
- Tell me, what are you most afraid of? exploring the effects of agent representation on information disclosure in human-chatbot interaction. In Artificial Intelligence in HCI: 4th International Conference, AI-HCI 2023, Held as Part of the 25th HCI International Conference, HCII 2023, Copenhagen, Denmark, July 23–28, 2023, Proceedings, Part II, page 179–191, Berlin, Heidelberg. Springer-Verlag.
- Chatbots and conversational agents in mental health: A review of the psychiatric landscape. The Canadian Journal of Psychiatry, 64(7):456–464. PMID: 30897957.
- Chain-of-thought prompting elicits reasoning in large language models. Advances in Neural Information Processing Systems, 35:24824–24837.
- A large-scale dataset for empathetic response generation. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 1251–1264, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
- Examining inter-consistency of large language models collaboration: An in-depth analysis via debate.
- Psycot: Psychological questionnaire as powerful chain-of-thought for personality detection. In The 2023 Conference on Empirical Methods in Natural Language Processing.
- Tree of thoughts: Deliberate problem solving with large language models. arXiv preprint arXiv:2305.10601.
- Judging llm-as-a-judge with mt-bench and chatbot arena.
- Sehee Lim (2 papers)
- Yejin Kim (35 papers)
- Chi-Hyun Choi (1 paper)
- Jy-yong Sohn (37 papers)
- Byung-Hoon Kim (21 papers)