Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
119 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Socratic Reasoning Improves Positive Text Rewriting (2403.03029v1)

Published 5 Mar 2024 in cs.CL

Abstract: Reframing a negative into a positive thought is at the crux of several cognitive approaches to mental health and psychotherapy that could be made more accessible by LLM-based solutions. Such reframing is typically non-trivial and requires multiple rationalization steps to uncover the underlying issue of a negative thought and transform it to be more positive. However, this rationalization process is currently neglected by both datasets and models which reframe thoughts in one step. In this work, we address this gap by augmenting open-source datasets for positive text rewriting with synthetically-generated Socratic rationales using a novel framework called \textsc{SocraticReframe}. \textsc{SocraticReframe} uses a sequence of question-answer pairs to rationalize the thought rewriting process. We show that such Socratic rationales significantly improve positive text rewriting for different open-source LLMs according to both automatic and human evaluations guided by criteria from psychotherapy research.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (77)
  1. Gpt-4 technical report. arXiv preprint arXiv:2303.08774.
  2. Socratic question generation: A novel dataset, models, and evaluation. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 147–165, Dubrovnik, Croatia. Association for Computational Linguistics.
  3. A synthetic data generation framework for grounded dialogues. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 10866–10882, Toronto, Canada. Association for Computational Linguistics.
  4. Lowell Bautista. 2014. The socratic method as a pedagogical method in legal education. University of Wollongong Faculty of Law, Humanities and the Arts - Papers.
  5. Aaron T Beck. 1979. Cognitive therapy and the emotional disorders. Penguin.
  6. Dyslexia prediction from natural reading of Danish texts. In Proceedings of the 24th Nordic Conference on Computational Linguistics (NoDaLiDa), pages 60–70, Tórshavn, Faroe Islands. University of Tartu Library.
  7. Ralph Allan Bradley and Milton E. Terry. 1952. Rank analysis of incomplete block designs: I. the method of paired comparisons. Biometrika, 39:324.
  8. Therapist use of socratic questioning predicts session-to-session symptom change in cognitive therapy for depression. Behaviour research and therapy, 70:32–37.
  9. Observing dialogue in therapy: Categorizing and forecasting behavioral codes. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 5599–5611, Florence, Italy. Association for Computational Linguistics.
  10. Timothy A Carey and Richard J Mullan. 2004. What is socratic questioning? Psychotherapy: theory, research, practice, training, 41(3):217–226.
  11. REV: Information-theoretic evaluation of free-text rationales. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2007–2030, Toronto, Canada. Association for Computational Linguistics.
  12. Gaining wisdom from setbacks: Aligning large language models via mistake analysis. arXiv preprint arXiv:2310.10477.
  13. Identifying stable speech-language markers of autism in children: Preliminary evidence from a longitudinal telephony-based study. In Proceedings of the Eighth Workshop on Computational Linguistics and Clinical Psychology, pages 40–46, Seattle, USA. Association for Computational Linguistics.
  14. David A Clark. 2013. Cognitive restructuring. The Wiley handbook of cognitive behavioral therapy, pages 1–22.
  15. Qlora: Efficient finetuning of quantized llms. Advances in Neural Information Processing Systems, 36.
  16. Núria Gala and Johannes Ziegler. 2016. Reducing lexical complexity as a tool to increase text accessibility for children with dyslexia. In Proceedings of the Workshop on Computational Linguistics for Linguistic Complexity (CL4LC), pages 59–66, Osaka, Japan. The COLING 2016 Organizing Committee.
  17. The pile: An 800gb dataset of diverse text for language modeling. arXiv preprint arXiv:2101.00027.
  18. Detecting language impairments in autism: A computational analysis of semi-structured conversations with vector semantics. In Proceedings of the Society for Computation in Linguistics (SCiL) 2018, pages 12–22.
  19. Do models of mental health based on social media data generalize? In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 3774–3788, Online. Association for Computational Linguistics.
  20. Can synthetic text help clinical named entity recognition? a study of electronic health records in French. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 2320–2338, Dubrovnik, Croatia. Association for Computational Linguistics.
  21. Linguistic indicators of severity and progress in online text-based therapy for depression. In Proceedings of the Workshop on Computational Linguistics and Clinical Psychology: From Linguistic Signal to Clinical Reality, pages 7–16, Baltimore, Maryland, USA. Association for Computational Linguistics.
  22. Lora: Low-rank adaptation of large language models. In International Conference on Learning Representations.
  23. Text style transfer: A review and experimental evaluation. 24(1):14–45.
  24. Large language models can self-improve. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 1051–1068, Singapore. Association for Computational Linguistics.
  25. Mistral 7b. arXiv preprint arXiv:2310.06825.
  26. Discourse-level representations can improve prediction of degree of anxiety. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 1500–1511, Toronto, Canada. Association for Computational Linguistics.
  27. Nazmul Kazi and Indika Kahanda. 2019. Automatically generating psychiatric case notes from digital transcripts of doctor-patient conversations. In Proceedings of the 2nd Clinical Natural Language Processing Workshop, pages 140–148, Minneapolis, Minnesota, USA. Association for Computational Linguistics.
  28. Large language models are zero-shot reasoners. Advances in neural information processing systems, 35:22199–22213.
  29. Identifying therapist conversational actions across diverse psychotherapeutic approaches. In Proceedings of the Sixth Workshop on Computational Linguistics and Clinical Psychology, pages 12–23, Minneapolis, Minnesota. Association for Computational Linguistics.
  30. Synthetic data generation with large language models for text classification: Potential and limitations. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 10443–10461, Singapore. Association for Computational Linguistics.
  31. Generating synthetic clinical speech data through simulated ASR deletion error. In Proceedings of the RaPID Workshop - Resources and ProcessIng of linguistic, para-linguistic and extra-linguistic Data from people with various forms of cognitive/psychiatric/developmental impairments - within the 13th Language Resources and Evaluation Conference, pages 9–16, Marseille, France. European Language Resources Association.
  32. DExperts: Decoding-time controlled text generation with experts and anti-experts. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 6691–6706, Online. Association for Computational Linguistics.
  33. Measuring pointwise 𝒱𝒱\mathcal{V}caligraphic_V-usable information in-context-ly. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 15739–15756, Singapore. Association for Computational Linguistics.
  34. MathDial: A dialogue tutoring dataset with rich pedagogical properties grounded in math reasoning problems. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 5602–5621, Singapore. Association for Computational Linguistics.
  35. Opportunities and challenges in neural dialog tutoring. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 2357–2372, Dubrovnik, Croatia. Association for Computational Linguistics.
  36. Self-refine: Iterative refinement with self-feedback. Advances in Neural Information Processing Systems, 36.
  37. Training models to generate, recognize, and reframe unhelpful thoughts. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 13641–13660, Toronto, Canada. Association for Computational Linguistics.
  38. Examining the effects of thought records and behavioral experiments in instigating belief change. Journal of behavior therapy and experimental psychiatry, 43(1):540–547.
  39. William R Miller and Stephen Rollnick. 2012. Motivational interviewing: Helping people change. Guilford press.
  40. Quantifying the language of schizophrenia in social media. In Proceedings of the 2nd Workshop on Computational Linguistics and Clinical Psychology: From Linguistic Signal to Clinical Reality, pages 11–20, Denver, Colorado. Association for Computational Linguistics.
  41. Exploring online depression forums via text mining: A comparison of Reddit and a curated online forum. In Proceedings of the Fifth Social Media Mining for Health Applications Workshop & Shared Task, pages 70–81, Barcelona, Spain (Online). Association for Computational Linguistics.
  42. James C Overholser. 2018. Guided discovery: A clinical strategy derived from the socratic method. International Journal of Cognitive Therapy, 11:124–139.
  43. Christine A Padesky. 1993. Socratic questioning: Changing minds or guiding discovery. In A keynote address delivered at the European Congress of Behavioural and Cognitive Therapies, London, volume 24.
  44. Socratic pretraining: Question-driven pretraining for controllable summarization. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 12737–12755, Toronto, Canada. Association for Computational Linguistics.
  45. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 311–318, Philadelphia, Pennsylvania, USA. Association for Computational Linguistics.
  46. R. Paul and L. Elder. 2019. The Thinker’s Guide to Socratic Questioning. Thinker’s Guide Library. Foundation for Critical Thinking.
  47. Understanding and predicting empathic behavior in counseling therapy. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1426–1435, Vancouver, Canada. Association for Computational Linguistics.
  48. The art of SOCRATIC QUESTIONING: Recursive thinking with large language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 4177–4199, Singapore. Association for Computational Linguistics.
  49. Conditioning on dialog acts improves empathy style transfer. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 13254–13271, Singapore. Association for Computational Linguistics.
  50. A recipe for arbitrary text style transfer with large language models. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 837–848, Dublin, Ireland. Association for Computational Linguistics.
  51. Synthetic data augmentation for zero-shot cross-lingual question answering. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 7016–7030, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
  52. Predicting depression in screening interviews from latent categorization of interview prompts. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7–18, Online. Association for Computational Linguistics.
  53. Predictive linguistic features of schizophrenia. In Proceedings of the 6th Joint Conference on Lexical and Computational Semantics (*SEM 2017), pages 241–250, Vancouver, Canada. Association for Computational Linguistics.
  54. Learning to evaluate translation beyond English: BLEURT submissions to the WMT metrics 2020 shared task. In Proceedings of the Fifth Conference on Machine Translation, pages 921–927, Online. Association for Computational Linguistics.
  55. Measuring linguistic synchrony in psychotherapy. In Proceedings of the Eighth Workshop on Computational Linguistics and Clinical Psychology, pages 158–176, Seattle, USA. Association for Computational Linguistics.
  56. Cognitive reframing of negative thoughts through human-language model interaction. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 9977–10000, Toronto, Canada. Association for Computational Linguistics.
  57. Learning disentangled meaning and style representations for positive text reframing. In Proceedings of the 16th International Natural Language Generation Conference, pages 424–430, Prague, Czechia. Association for Computational Linguistics.
  58. Synthetic data generation and multi-task learning for extracting temporal information from health-related narrative text. In Proceedings of the Seventh Workshop on Noisy User-generated Text (W-NUT 2021), pages 260–273, Online. Association for Computational Linguistics.
  59. Sagarika Shreevastava and Peter Foltz. 2021. Detecting cognitive distortions from patient-therapist interactions. In Proceedings of the Seventh Workshop on Computational Linguistics and Clinical Psychology: Improving Access, pages 151–158, Online. Association for Computational Linguistics.
  60. Automatic generation of socratic subquestions for teaching math word problems. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 4136–4149, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
  61. Distilling reasoning capabilities into smaller language models. In Findings of the Association for Computational Linguistics: ACL 2023, pages 7059–7073, Toronto, Canada. Association for Computational Linguistics.
  62. Patrick E Shrout and Joseph L Fleiss. 1979. Intraclass correlations: uses in assessing rater reliability. Psychological bulletin, 86(2):420–428.
  63. Generate, delete and rewrite: A three-stage framework for improving persona consistency of dialogue generation. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5821–5831, Online. Association for Computational Linguistics.
  64. Recursive neural networks for coding therapist and patient behavior in motivational interviewing. In Proceedings of the 2nd Workshop on Computational Linguistics and Clinical Psychology: From Linguistic Signal to Clinical Reality, pages 71–79, Denver, Colorado. Association for Computational Linguistics.
  65. Style transfer for texts: Retrain, report errors, compare with rewrites. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3936–3945, Hong Kong, China. Association for Computational Linguistics.
  66. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288.
  67. Elsbeth Turcan and Kathy McKeown. 2019. Dreaddit: A Reddit dataset for stress analysis in social media. In Proceedings of the Tenth International Workshop on Health Text Mining and Information Analysis (LOUHI 2019), pages 97–107, Hong Kong. Association for Computational Linguistics.
  68. Not all emotions are created equal: the negativity bias in social-emotional development. Psychological bulletin, 134(3):383–403.
  69. Linguistic complexity loss in text-based therapy. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 4450–4459, Online. Association for Computational Linguistics.
  70. Chain-of-thought prompting elicits reasoning in large language models. Advances in Neural Information Processing Systems, 35:24824–24837.
  71. Chris Williams. 2001. Use of written cognitive–behavioural therapy self-help materials to treat depression. Advances in Psychiatric Treatment, 7(3):233–240.
  72. Transformers: State-of-the-art natural language processing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 38–45, Online. Association for Computational Linguistics.
  73. Hence, socrates is mortal: A benchmark for natural language syllogistic reasoning. In Findings of the Association for Computational Linguistics: ACL 2023, pages 2347–2367, Toronto, Canada. Association for Computational Linguistics.
  74. Are experts needed? on human evaluation of counselling reflection generation. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 6906–6930, Toronto, Canada. Association for Computational Linguistics.
  75. Depression and self-harm risk assessment in online forums. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2968–2978, Copenhagen, Denmark. Association for Computational Linguistics.
  76. Click: Controllable text generation with sequence likelihood contrastive learning. In Findings of the Association for Computational Linguistics: ACL 2023, pages 1022–1040, Toronto, Canada. Association for Computational Linguistics.
  77. Inducing positive perspectives with text reframing. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3682–3700, Dublin, Ireland. Association for Computational Linguistics.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (3)
  1. Anmol Goel (9 papers)
  2. Nico Daheim (24 papers)
  3. Iryna Gurevych (264 papers)

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now