Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
102 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Chain-of-Interaction: Enhancing Large Language Models for Psychiatric Behavior Understanding by Dyadic Contexts (2403.13786v2)

Published 20 Mar 2024 in cs.CL

Abstract: Automatic coding patient behaviors is essential to support decision making for psychotherapists during the motivational interviewing (MI), a collaborative communication intervention approach to address psychiatric issues, such as alcohol and drug addiction. While the behavior coding task has rapidly adapted machine learning to predict patient states during the MI sessions, lacking of domain-specific knowledge and overlooking patient-therapist interactions are major challenges in developing and deploying those models in real practice. To encounter those challenges, we introduce the Chain-of-Interaction (CoI) prompting method aiming to contextualize LLMs for psychiatric decision support by the dyadic interactions. The CoI prompting approach systematically breaks down the coding task into three key reasoning steps, extract patient engagement, learn therapist question strategies, and integrates dyadic interactions between patients and therapists. This approach enables LLMs to leverage the coding scheme, patient state, and domain knowledge for patient behavioral coding. Experiments on real-world datasets can prove the effectiveness and flexibility of our prompting method with multiple state-of-the-art LLMs over existing prompting baselines. We have conducted extensive ablation analysis and demonstrate the critical role of dyadic interactions in applying LLMs for psychotherapy behavior understanding.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (60)
  1. M. Freeman, “The world mental health report: transforming mental health for all,” World Psychiatry, vol. 21, pp. 391–392, 10 2022. [Online]. Available: https://onlinelibrary.wiley.com/doi/10.1002/wps.21018
  2. T. B. Moyers, T. Martin, J. K. Manuel, and W. R. Miller, “The motivational interviewing treatment integrity (miti) code: Version 2.0,” 2003.
  3. D. C. Atkins, M. Steyvers, Z. E. Imel, and P. Smyth, “Scaling up the evaluation of psychotherapy: evaluating motivational interviewing fidelity via statistical text classification,” Implementation Science, vol. 9, p. 49, 2014.
  4. M. Tanana, K. A. Hallgren, Z. E. Imel, D. C. Atkins, and V. Srikumar, “A comparison of natural language processing methods for automated coding of motivational interviewing,” Journal of Substance Abuse Treatment, vol. 65, pp. 43–50, 2016, motivational Interviewing in Substance Use Treatment. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0740547216000222
  5. X. Huang, L. Liu, K. Carey, J. Woolley, S. Scherer, and B. Borsari, “Modeling temporality of human intentions by domain adaptation,” in Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, E. Riloff, D. Chiang, J. Hockenmaier, and J. Tsujii, Eds.   Brussels, Belgium: Association for Computational Linguistics, Oct.-Nov. 2018, pp. 696–701. [Online]. Available: https://aclanthology.org/D18-1074
  6. J. Cao, M. Tanana, Z. Imel, E. Poitras, D. Atkins, and V. Srikumar, “Observing dialogue in therapy: Categorizing and forecasting behavioral codes,” in Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, A. Korhonen, D. Traum, and L. Màrquez, Eds.   Florence, Italy: Association for Computational Linguistics, Jul. 2019, pp. 5599–5611. [Online]. Available: https://aclanthology.org/P19-1563
  7. J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “BERT: Pre-training of deep bidirectional transformers for language understanding,” in Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), J. Burstein, C. Doran, and T. Solorio, Eds.   Minneapolis, Minnesota: Association for Computational Linguistics, Jun. 2019, pp. 4171–4186.
  8. L. Tavabi et al., “Multimodal automatic coding of client behavior in motivational interviewing,” in Proceedings of the 2020 International Conference on Multimodal Interaction, ser. ICMI ’20.   New York, NY, USA: Association for Computing Machinery, 2020, p. 406–413.
  9. J. Achiam, S. Adler, S. Agarwal, L. Ahmad, I. Akkaya, F. L. Aleman, D. Almeida, J. Altenschmidt, S. Altman, S. Anadkat et al., “Gpt-4 technical report,” 2023. [Online]. Available: https://arxiv.org/abs/2303.08774
  10. Ouyang et al., “Training language models to follow instructions with human feedback,” in Advances in Neural Information Processing Systems, S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, Eds., vol. 35.   New Orleans, Louisiana: Curran Associates, Inc., 2022, pp. 27 730–27 744. [Online]. Available: https://proceedings.neurips.cc/paper_files/paper/2022/file/b1efde53be364a73914f58805a001731-Paper-Conference.pdf
  11. H. Touvron et al., “Llama 2: Open foundation and fine-tuned chat models,” 2023. [Online]. Available: https://arxiv.org/abs/2307.09288
  12. K. Yang, T. Zhang, Z. Kuang, Q. Xie, S. Ananiadou, and J. Huang, “Mentallama: Interpretable mental health analysis on social media with large language models,” 2023. [Online]. Available: https://arxiv.org/abs/2309.13567
  13. S. Ji, T. Zhang, L. Ansari, J. Fu, P. Tiwari, and E. Cambria, “MentalBERT: Publicly available pretrained language models for mental healthcare,” in Proceedings of the Thirteenth Language Resources and Evaluation Conference, N. Calzolari, F. Béchet, P. Blache, K. Choukri, C. Cieri, T. Declerck, S. Goggi, H. Isahara, B. Maegaard, J. Mariani, H. Mazo, J. Odijk, and S. Piperidis, Eds.   Marseille, France: European Language Resources Association, Jun. 2022, pp. 7184–7190.
  14. J. M. Liu, D. Li, H. Cao, T. Ren, Z. Liao, and J. Wu, “Chatcounselor: A large language models for mental health support,” 2023. [Online]. Available: https://arxiv.org/abs/2309.15461
  15. J. Wei et al., “Chain-of-thought prompting elicits reasoning in large language models,” in Advances in Neural Information Processing Systems, S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, Eds., vol. 35.   Curran Associates, Inc., 2022, pp. 24 824–24 837. [Online]. Available: https://proceedings.neurips.cc/paper_files/paper/2022/file/9d5609613524ecf4f15af0f7b31abca4-Paper-Conference.pdf
  16. Y. Zhou, X. Geng, T. Shen, C. Tao, G. Long, J.-G. Lou, and J. Shen, “Thread of thought unraveling chaotic contexts,” 2023. [Online]. Available: https://arxiv.org/abs/2311.08734
  17. Y. Zhang, M. Zhu, Y. Gong, and R. Ding, “Optimizing science question ranking through model and retrieval-augmented generation,” International Journal of Computer Science and Information Technology, vol. 1, no. 1, pp. 124–130, 2023.
  18. Z. Jing, Y. Su, Y. Han, B. Yuan, H. Xu, C. Liu, K. Chen, and M. Zhang, “When large language models meet vector databases: A survey,” 2024. [Online]. Available: https://arxiv.org/abs/2402.01763
  19. E. Almazrouei et al., “The falcon series of open language models,” 2023. [Online]. Available: https://arxiv.org/abs/2311.16867
  20. A. Q. Jiang et al., “Mistral 7b,” 2023. [Online]. Available: https://arxiv.org/abs/2310.06825
  21. OpenAI, “Chatgpt: Optimizing language models for dialogue,” https://openai.com/blog/chatgpt/, 2022, accessed: 2023-07-24.
  22. B. Borsari et al., “In-session processes of brief motivational interventions in two trials with mandated college students.” Journal of consulting and clinical psychology, vol. 83, pp. 56–67, 2 2015.
  23. W. R. Miller, T. B. Moyers, D. Ernst, and P. Amrhein, “Manual for the motivational interviewing skill code (misc) version 2.1,” Substance Abuse and Addiction (CASAA), University of New Mexico, vol. 8, pp. 901–4, 2008.
  24. W. R. Miller and G. S. Rose, “Toward a theory of motivational interviewing.” The American psychologist, vol. 64, pp. 527–37, 9 2009.
  25. M. Magill and K. A. Hallgren, “Mechanisms of behavior change in motivational interviewing: do we understand how mi works?” Current opinion in psychology, vol. 30, pp. 1–5, 12 2019.
  26. K. Mishra, P. Priya, M. Burja, and A. Ekbal, “e-THERAPIST: I suggest you to cultivate a mindset of positivity and nurture uplifting thoughts,” in Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, H. Bouamor, J. Pino, and K. Bali, Eds.   Singapore: Association for Computational Linguistics, Dec. 2023, pp. 13 952–13 967. [Online]. Available: https://aclanthology.org/2023.emnlp-main.861
  27. T. Tran et al., “Multimodal analysis and assessment of therapist empathy in motivational interviews,” in Proceedings of the 25th International Conference on Multimodal Interaction, ser. ICMI ’23.   New York, NY, USA: Association for Computing Machinery, 2023, p. 406–415.
  28. A. Gagneur, “Respiratory syncytial virus: Motivational interviewing: A powerful tool to address vaccine hesitancy,” Canada Communicable Disease Report, vol. 46, no. 4, p. 93, 2020.
  29. S. Rollnick, W. R. Miller, C. C. Butler, and M. S. Aloia, “Motivational interviewing in health care: Helping patients change behavior,” COPD: Journal of Chronic Obstructive Pulmonary Disease, vol. 5, pp. 203–203, 1 2008.
  30. S. A. Cole, D. Sannidhi, Y. T. Jadotte, and A. Rozanski, “Using motivational interviewing and brief action planning for adopting and maintaining positive health behaviors,” Progress in Cardiovascular Diseases, vol. 77, pp. 86–94, 2023.
  31. W. Wang, V. W. Zheng, H. Yu, and C. Miao, “A survey of zero-shot learning: Settings, methods, and applications,” ACM Trans. Intell. Syst. Technol., vol. 10, no. 2, jan 2019.
  32. W. Yin, J. Hay, and D. Roth, “Benchmarking zero-shot text classification: Datasets, evaluation and entailment approach,” in Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), K. Inui, J. Jiang, V. Ng, and X. Wan, Eds.   Hong Kong, China: Association for Computational Linguistics, Nov. 2019, pp. 3914–3923. [Online]. Available: https://aclanthology.org/D19-1404
  33. T. B. Brown et al., “Language models are few-shot learners,” in Proceedings of the 34th International Conference on Neural Information Processing Systems, ser. NIPS’20.   Red Hook, NY, USA: Curran Associates Inc., 2020.
  34. X. Liu et al., “Large language models are few-shot health learners,” 2023. [Online]. Available: https://arxiv.org/abs/2305.15525
  35. T. Kojima, S. S. Gu, M. Reid, Y. Matsuo, and Y. Iwasawa, “Large language models are zero-shot reasoners,” 2023. [Online]. Available: https://arxiv.org/abs/2205.11916
  36. K. Cobbe et al., “Training verifiers to solve math word problems,” 2021. [Online]. Available: https://arxiv.org/abs/2110.14168
  37. Y. Bai et al., “Constitutional ai: Harmlessness from ai feedback,” 2022. [Online]. Available: https://arxiv.org/abs/2212.08073
  38. P. F. Christiano, J. Leike, T. Brown, M. Martic, S. Legg, and D. Amodei, “Deep reinforcement learning from human preferences,” in Advances in Neural Information Processing Systems, I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, Eds., vol. 30.   Curran Associates, Inc., 2017. [Online]. Available: https://proceedings.neurips.cc/paper_files/paper/2017/file/d5e2c0adad503c91f91df240d0cd4e49-Paper.pdf
  39. N. Stiennon et al., “Learning to summarize with human feedback,” in Advances in Neural Information Processing Systems, H. Larochelle, M. Ranzato, R. Hadsell, M. Balcan, and H. Lin, Eds., vol. 33.   Curran Associates, Inc., 2020, pp. 3008–3021. [Online]. Available: https://proceedings.neurips.cc/paper_files/paper/2020/file/1f89885d556929e98d3ef9b86448f951-Paper.pdf
  40. G. Penedo et al., “The refinedweb dataset for falcon llm: Outperforming curated corpora with web data, and web data only,” 2023. [Online]. Available: https://arxiv.org/abs/2306.01116
  41. T. Dao, D. Fu, S. Ermon, A. Rudra, and C. Ré, “Flashattention: Fast and memory-efficient exact attention with io-awareness,” in Advances in Neural Information Processing Systems, S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, Eds., vol. 35.   Curran Associates, Inc., 2022, pp. 16 344–16 359. [Online]. Available: https://proceedings.neurips.cc/paper_files/paper/2022/file/67d57c32e20fd0a7a302cb81d36e40d5-Paper-Conference.pdf
  42. A. Wang et al., “Superglue: A stickier benchmark for general-purpose language understanding systems,” in Advances in Neural Information Processing Systems, H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R. Garnett, Eds., vol. 32.   Curran Associates, Inc., 2019. [Online]. Available: https://proceedings.neurips.cc/paper_files/paper/2019/file/4496bf24afe7fab6f046bf4923da8de6-Paper.pdf
  43. M. Chen et al., “Evaluating large language models trained on code,” 2021. [Online]. Available: https://arxiv.org/abs/2107.03374
  44. A. Holtzman, J. Buys, L. Du, M. Forbes, and Y. Choi, “The curious case of neural text degeneration,” in International Conference on Learning Representations, 2020. [Online]. Available: https://openreview.net/forum?id=rygGQyrFvH
  45. X. Wang et al., “”my answer is c”: First-token probabilities do not match text answers in instruction-tuned language models,” 2024. [Online]. Available: https://arxiv.org/abs/2402.14499
  46. B. Xiao, C. Huang, Z. E. Imel, D. C. Atkins, P. Georgiou, and S. S. Narayanan, “A technology prototype system for rating therapist empathy from audio recordings in addiction counseling,” PeerJ Computer Science, vol. 2, p. e59, 4 2016.
  47. Z. L. Inbar and Elyoseph, “Suicide risk assessments through the eyes of chatgpt-3.5 versus chatgpt-4: Vignette study,” JMIR Ment Health, vol. 10, p. e51232, 9 2023.
  48. D. Can, P. G. Georgiou, D. C. Atkins, and S. S. Narayanan, “A case study: detecting counselor reflections in psychotherapy for addictions using linguistic features,” in Proc. Interspeech 2012, 2012, pp. 2254–2257.
  49. K. Singla et al., “Using prosodic and lexical information for learning utterance-level behaviors in psychotherapy.” Interspeech, vol. 2018, pp. 3413–3417, 9 2018.
  50. J. Gibson, D. Can, P. G. Georgiou, D. C. Atkins, and S. S. Narayanan, “Attention networks for modeling behaviors in addiction counseling,” in Interspeech 2017, 18th Annual Conference of the International Speech Communication Association, Stockholm, Sweden, August 20-24, 2017, F. Lacerda, Ed.   ISCA, 2017, pp. 3251–3255.
  51. J. Gibson, D. C. Atkins, T. A. Creed, Z. Imel, P. Georgiou, and S. Narayanan, “Multi-label multi-task deep learning for behavioral coding,” IEEE Transactions on Affective Computing, vol. 13, no. 1, pp. 508–518, 2022.
  52. K. Singla, Z. Chen, D. Atkins, and S. Narayanan, “Towards end-2-end learning for predicting behavior codes from spoken utterances in psychotherapy conversations,” in Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, D. Jurafsky, J. Chai, N. Schluter, and J. Tetreault, Eds.   Online: Association for Computational Linguistics, Jul. 2020, pp. 3797–3803. [Online]. Available: https://aclanthology.org/2020.acl-main.351
  53. D. Demszky, D. Yang, D. S. Yeager, C. J. Bryan, M. Clapper, S. Chandhok, J. C. Eichstaedt, C. Hecht, J. Jamieson, M. Johnson, M. Jones, D. Krettek-Cobb, L. Lai, N. JonesMitchell, D. C. Ong, C. S. Dweck, J. J. Gross, and J. W. Pennebaker, “Using large language models in psychology,” Nature Reviews Psychology, vol. 2, no. 11, pp. 688–701, 2023. [Online]. Available: https://doi.org/10.1038/s44159-023-00241-5
  54. T. He et al., “Towards a psychological generalist ai: A survey of current applications of large language models and future prospects,” 2023. [Online]. Available: https://arxiv.org/abs/2312.04578
  55. R. Lou, K. Zhang, and W. Yin, “A comprehensive survey on instruction following,” 2024. [Online]. Available: https://arxiv.org/abs/2303.10475
  56. R. Lou and W. Yin, “Toward zero-shot instruction following,” in Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics: Student Research Workshop, N. Falk, S. Papi, and M. Zhang, Eds.   St. Julian’s, Malta: Association for Computational Linguistics, Mar. 2024, pp. 50–60. [Online]. Available: https://aclanthology.org/2024.eacl-srw.5
  57. Y. Li, L. Zihan, K. Zhang, D. Ruilong, S. Jiang, and Y. Zhang, “Chatdoctor: A medical chat model fine-tuned on a large language model meta-ai (llama) using medical domain knowledge,” Cureus, vol. 15, no. 6, 2023. [Online]. Available: https://www.proquest.com/scholarly-journals/chatdoctor-medical-chat-model-fine-tuned-on-large/docview/2844019753/se-2
  58. T. Lai et al., “Psy-llm: Scaling up global mental health psychological services with ai-based large language models,” 2023. [Online]. Available: https://arxiv.org/abs/2307.11991
  59. H. Qi et al., “Supervised learning and large language model benchmarks on mental health datasets: Cognitive distortions and suicidal risks in chinese social media,” 2023. [Online]. Available: https://arxiv.org/abs/2309.03564
  60. Z. Chen, Y. Lu, and W. Wang, “Empowering psychotherapy with large language models: Cognitive distortion detection through diagnosis of thought prompting,” in Findings of the Association for Computational Linguistics: EMNLP 2023, H. Bouamor, J. Pino, and K. Bali, Eds.   Singapore: Association for Computational Linguistics, Dec. 2023, pp. 4295–4304. [Online]. Available: https://aclanthology.org/2023.findings-emnlp.284
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (4)
  1. Guangzeng Han (5 papers)
  2. Weisi Liu (7 papers)
  3. Xiaolei Huang (45 papers)
  4. Brian Borsari (1 paper)
Citations (15)

Summary

We haven't generated a summary for this paper yet.