ChatThero: LLM-Driven Addiction Recovery Chatbot

• ChatThero is an LLM-driven multi-agent conversational system that integrates evidence-based CBT and MI techniques to support addiction recovery.
• It employs dynamic patient modeling with structured synthetic profiles to simulate realistic therapeutic dialogues, enhancing patient motivation and empathy.
• The system uses a two-stage training pipeline—combining supervised fine-tuning and direct preference optimization—to improve dialogue efficiency and privacy-preserving evaluation.

ChatThero is an LLM-driven, multi-agent conversational system designed for addiction recovery and therapeutic support. The framework is characterized by adaptive dialogue grounded in cognitive behavioral therapy (CBT) and motivational interviewing (MI), dynamic patient modeling, a rigorous synthetic benchmark for evaluation, and privacy-preserving mechanisms for studying therapeutic interactions. Empirical results indicate ChatThero supports significant behavioral change, increases patient motivation, and achieves higher assessed empathy and responsiveness relative to powerful general-purpose models such as GPT-4o (Wang et al., 28 Aug 2025).

1. Multi-Agent Conversational Architecture

ChatThero models therapeutic dialogue as a multi-agent system comprising three principal agents:

• Patient Agent: Initialized from a structured synthetic patient profile that encodes trait-level characteristics (personality, substance use history, barriers). The profile dynamically evolves through a memory mechanism that tracks emotional states, coping strategies, and environmental stressors as the conversation progresses.
• Therapy Agent: Deploys adaptive CBT and MI interventions, selecting among a list of 18 evidence-based persuasive strategies. This agent interprets evolving patient signals and tailors responses contextually, supporting both affirmation and challenge, psychoeducation, and elicitation of patient motivation.
• Environment Agent: Introduces stochastic external events to simulate real-world stressors, allowing longitudinal modification of patient trajectories and calibration of therapeutic intensity.

This architecture enables simulation of realistic, multi-turn therapeutic exchanges, operationalizes context-sensitive selection of intervention strategies, and supports empirical paper of resilience and motivation mechanisms.

2. Integration of CBT and Motivational Interviewing

ChatThero’s therapeutic framework tightly couples evidence-based CBT and MI principles at multiple system components:

• Prompt Engineering: Synthetic dialogues used in supervised training encode explicit CBT techniques (e.g., cognitive restructuring, behavioral activation, identification of automatic negative thoughts) and MI strategies (e.g., open questioning, reflective listening, evocation of change talk).
• Adaptive Response: The system dynamically shifts its conversational stance based on real-time patient resistance (Easy, Medium, Hard). This operationalizes the contingency used in human MI protocols—such as scaling affirmations with patient readiness or escalating behavioral interventions for entrenched ambivalence.
• Strategy Diversity and Appropriateness: Diverse persuasive strategies are linked to observed improvements in patient motivation and confidence. For example, medium and hard resistance scenarios achieve higher gains when multiple intervention styles are applied in succession rather than repeated focus on a single technique.

Formally, the model selects system actions from the set of strategies $S = \{s_1, \dots, s_{18}\}$, with longitudinal tracking of utility via dialogue state sequence modeling (Wang et al., 28 Aug 2025).

3. Patient Modeling and Dynamic Synthetic Benchmark

Patient simulation is informed by structured profiles and dynamic memory, supporting a high-fidelity synthetic benchmark:

• Profile Generation: Synthetic patient attributes are derived from anonymized and ethically filtered data sources (e.g., Reddit SUD posts), encoded into feature vectors representing stable and evolving dimensions.
• Resistance Stratification: The benchmark organizes virtual patients into Easy, Medium, and Hard resistance categories. Each class is specified by probabilistic thresholds across skepticism, motivation volatility, and responsiveness.
• Dialogue Efficiency Metrics: Empirical evaluation compares ChatThero and GPT-4o in terms of turn count needed for satisfactory resolution, with ChatThero achieving a $26\%$ reduction for hard cases.

This synthetic evaluation environment allows rigorous, replicable measurement of both behavioral and conversational outcomes under privacy constraints.

4. Training Pipeline: Supervised Fine-Tuning and Direct Preference Optimization

ChatThero employs a two-stage training pipeline:

• Stage 1: Supervised Fine-Tuning (SFT) Fine-tuning is performed using synthetic multi-turn therapy dialogues ($\sim50$ turns per sequence), explicitly conditioned on patient profile and resistance level. This dataset reflects empirically observed clinical conversational flows, emphasizing empathic and motivational strategies.
• Stage 2: Direct Preference Optimization (DPO) Multiple candidate responses are sampled per dialogue turn. A ranking protocol—incorporating both automated (GPT-4o) and licensed clinical expert evaluations—optimizes selection based on pairwise preference ($P_{\text{preferred}}(R_i|R_j)$). This enhances empathy, appropriateness, and behavioral realism in the final output.

Resulting models are evaluated both quantitatively (motivation and confidence scores) and qualitatively (human clinician empathy and realism ratings).

5. Performance Metrics and Comparative Results

ChatThero demonstrates superior performance across multiple domains:

• Motivation Gain: Mean patient motivation score increases by $41.5\%$ post-interaction.
• Treatment Confidence: Average confidence in treatment rises by $0.49\%$.
• Turn Efficiency: Dialogue completion is achieved with $26\%$ fewer turns for challenging cases, relative to GPT-4o.
• Empathy, Responsiveness, Realism: Both LLM-as-a-Judge and licensed clinical experts rate ChatThero above GPT-4o in each category.

Correlation analysis (e.g., Figure 3, right panel) links strategy diversity to positive behavioral outcomes, especially in high-resistance patients. Core evaluation data is summarized as:

Metric	ChatThero	GPT-4o
Motivation gain	41.5%	lower
Confidence increase	0.49%	lower
Turns to resolution	26% fewer (hard)	baseline
Empathy (expert)	Higher	Lower

This suggests the explicit modeling of behavioral resistance and multi-strategy deployment substantively enhances therapeutic impact.

6. Privacy-Preserving Research and Clinical Translation

ChatThero’s synthetic framework allows rigorous, privacy-preserving paper of therapeutic conversation dynamics:

• Synthetic Patients: All benchmarking is performed with virtual agents derived from processed public data, avoiding direct use of real patient information.
• Replicability: Standardized agent initialization and controlled simulation facilitate cross-site and cross-institutional replication.
• Potential for Clinical Deployment: Results indicate that with further validation, ChatThero can support clinicians and researchers in scalable therapeutic interventions for SUDs, especially where access to human counselors is limited.

A plausible implication is that such frameworks could generalize to other behavioral health domains contingent on expansion of evidence-based strategy libraries and cultural adaptation.

7. Future Directions

Proposed future enhancements include:

• Realism: Integration of standardized patient actors and ongoing supervision by licensed clinicians.
• Cultural and Linguistic Generalization: Expansion to non-English and culturally distinct patient populations to validate universality of therapeutic impact.
• Longitudinal Outcome Studies: Tracking sustained behavioral change, trust, and alliance evolution over repeated interactions.
• Evaluation Protocol Refinement: Incorporating multimodal ratings and continuous quality assurance during implementation.

These directions underscore the systematic, research-grounded approach underlying ChatThero and its utility as a basis for further therapeutic chatbot development.

---

ChatThero thus represents a technically and clinically robust model for AI-supported behavior change, optimized through domain-specific conversational mechanisms, privacy-aware synthetic benchmarking, and rigorous empirical validation. Its effects on patient motivation, confidence, and dialogue quality mark a significant step for evidence-based AI interventions in addiction recovery contexts (Wang et al., 28 Aug 2025).