Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
38 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
41 tokens/sec
o3 Pro
7 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Conversational Topic Recommendation in Counseling and Psychotherapy with Decision Transformer and Large Language Models (2405.05060v1)

Published 8 May 2024 in cs.CL

Abstract: Given the increasing demand for mental health assistance, AI, particularly LLMs, may be valuable for integration into automated clinical support systems. In this work, we leverage a decision transformer architecture for topic recommendation in counseling conversations between patients and mental health professionals. The architecture is utilized for offline reinforcement learning, and we extract states (dialogue turn embeddings), actions (conversation topics), and rewards (scores measuring the alignment between patient and therapist) from previous turns within a conversation to train a decision transformer model. We demonstrate an improvement over baseline reinforcement learning methods, and propose a novel system of utilizing our model's output as synthetic labels for fine-tuning a LLM for the same task. Although our implementation based on LLaMA-2 7B has mixed results, future work can undoubtedly build on the design.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize File Document Download Save Streamline Icon: https://streamlinehq.com PDF File Document Download Save Streamline Icon: https://streamlinehq.com Markdown Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (26)
  1. Latte: Language trajectory transformer. In 2023 IEEE International Conference on Robotics and Automation (ICRA), pages 7287–7294. IEEE.
  2. Decision transformer: Reinforcement learning via sequence modeling. In Neural Information Processing Systems.
  3. Towards augmenting crisis counselor training by improving message retrieval. Proceedings of the Sixth Workshop on Computational Linguistics and Clinical Psychology.
  4. Topic modeling in embedding spaces. Transactions of the Association for Computational Linguistics, 8:439–453.
  5. Addressing function approximation error in actor-critic methods. In International conference on machine learning, pages 1587–1596. PMLR.
  6. Off-policy deep reinforcement learning without exploration. In International conference on machine learning, pages 2052–2062. PMLR.
  7. Adam O Horvath and Leslie S. Greenberg. 1989. Development and validation of the working alliance inventory. Journal of Counseling Psychology, 36:223–233.
  8. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685.
  9. Ensemble method via ranking model for conversational modeling with subjective knowledge. In Proceedings of The Eleventh Dialog System Technology Challenge, pages 177–184.
  10. Clinical decision transformer: intended treatment recommendation through goal prompting. arXiv preprint arXiv:2302.00612.
  11. Continuous control with deep reinforcement learning. arXiv preprint arXiv:1509.02971.
  12. Baihan Lin. 2022. Computational inference in cognitive science: Operational, societal and ethical considerations. arXiv preprint arXiv:2210.13526.
  13. Compass: Computational mapping of patient-therapist alliance strategies with language modeling. arXiv preprint arXiv:2402.14701.
  14. Working alliance transformer for psychotherapy dialogue classification. arXiv preprint arXiv:2210.15603.
  15. Deep annotation of therapeutic working alliance in psychotherapy. In International workshop on health intelligence, pages 193–207. Springer.
  16. Supervisorbot: Nlp-annotated real-time recommendations of psychotherapy treatment strategies with deep reinforcement learning. In Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence, pages 7149–7153.
  17. Decision making as language generation. In NeurIPS 2022 Foundation Models for Decision Making Workshop.
  18. Distributed representations of words and phrases and their compositionality. In Neural Information Processing Systems.
  19. Pair: Prompt-aware margin ranking for counselor reflection scoring in motivational interviewing. In Conference on Empirical Methods in Natural Language Processing.
  20. Pretraining for language conditioned imitation with transformers.
  21. Self-critiquing models for assisting human evaluators. ArXiv, abs/2206.05802.
  22. Human–ai collaboration enables more empathic conversations in text-based peer-to-peer mental health support. Nature Machine Intelligence, 5:46–57.
  23. Learning to summarize with human feedback. Advances in Neural Information Processing Systems, 33:3008–3021.
  24. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971.
  25. Self-taught optimizer (stop): Recursively self-improving code generation. ArXiv, abs/2310.02304.
  26. Can offline reinforcement learning help natural language understanding? arXiv preprint arXiv:2212.03864.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (3)
  1. Aylin Gunal (4 papers)
  2. Baihan Lin (36 papers)
  3. Djallel Bouneffouf (73 papers)
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets