PsyCoTalk: Multi-Disorder Psychiatry Dialogue Corpus

• PsyCoTalk is a large-scale synthetic dialogue corpus designed for screening multiple psychiatric disorders in realistic clinical settings.
• It employs a rigorous multi-agent framework and Hierarchical Diagnostic State Machine to simulate detailed doctor–patient diagnostic interviews.
• The dataset integrates structured synthetic EMRs and multi-turn dialogues to support research on LLM-based diagnostic strategies and AI clinical decision support.

PsyCoTalk is a clinically grounded, large-scale, synthetic dialogue corpus designed for multi-disorder psychiatric screening. It integrates structured electronic medical records (EMRs) reflecting real-world psychiatric comorbidity profiles with multi-turn, doctor–patient dialogues that closely mimic genuine clinical diagnostic interviews. The dataset aims to advance research and development of dialogue-based LLMs and automated agents for comprehensive, stepwise screening of psychiatric comorbidity in accordance with DSM-5 protocols (Wan et al., 29 Oct 2025).

1. Dataset Scope and Purpose

PsyCoTalk is the first corpus constructed explicitly for multi-disorder (comorbidity) psychiatric dialogue modeling. The essential objective is to enable the training and evaluation of conversational agents—or LLMs—capable of screening for several co-occurring psychiatric conditions within a single diagnostic session.

Key features include:

• 502 synthetic EMRs, each simulating common comorbidity combinations among Major Depressive Disorder (MDD), Anxiety Disorder (AD), Bipolar Disorder (BD), and Attention-Deficit/Hyperactivity Disorder (ADHD).
• 3,000 multi-turn, clinician–patient dialogues (average: 45.9 turns per dialogue), all contextually generated from the EMRs using a rigorous multi-agent framework.
• Dialogue structure and annotation grounded in SCID-5-RV (Structured Clinical Interview for DSM-5 Research Version) interview protocols via a Hierarchical Diagnostic State Machine (HDSM) and an accompanying Diagnostic Context Tree (DCT).
• Dataset supports not only model training, but also the study of diagnostic reasoning, conversational strategies, and real-time AI triage or decision-support research (not for clinical deployment) (Wan et al., 29 Oct 2025).

2. Synthetic EMR Construction Pipeline

PsyCoTalk’s EMRs are generated through a multi-stage, clinically validated pipeline:

• Source corpus: PsySym, a Reddit-based dataset of 5,624 users self-reporting psychiatric diagnoses and symptoms.
• Filtering: Retention of users with at least 10 symptom posts and at least 20 distinct symptom types, removing inconsistent symptom-label pairs via a DSM-5–aligned symptom–disease graph.
• EMR template: Seven sections—Demographics, Chief Complaint, Medical Condition, Medical History, Personal History, Family History, Preliminary Diagnosis.
• Content generation strategies:
  • Chief Complaint and Medical Condition: Binary symptom/event vectors are extracted via dual classifiers trained on user posts, then summarized using a LLM (GPT-4o-mini).
  • Medical/Personal/Family History: Keyword-based categorization with segmented LLM-driven summarization.
  • Demographics: Extracted via keyword search (education, occupation, age) and rule-based approaches (gender, marital status, age).
• Validation: Distributions of age, gender, disease prevalence, and family history in synthetic EMRs were empirically matched to 1,000 de-identified real records. This ensures both clinical fidelity and population-level realism (Wan et al., 29 Oct 2025).

3. Diagnostic Dialogue Generation: Hierarchical State Machine and Context Tree

Dialogues are generated using a multi-agent approach that simulates semi-structured clinical interviews:

• Hierarchical Diagnostic State Machine (HDSM): Encodes SCID-5-RV protocols, with over 130 states spread across disorder-specific submachines:
  • MDD: High-Level (e.g., current episode, past history), Intermediate-Level (symptom clusters), and Basic-Level (binary symptom queries) states; terminal nodes encode mutually exclusive outcomes.
  • Similar substructures exist for BD (mania/hypomania), AD (current and past GAD), and ADHD.
  • State transitions are driven by patient responses aggregated within sub-state groups; thresholds define whether the conversation follows positive or negative diagnostic branches.
• Diagnostic Context Tree (DCT): Runs in parallel with HDSM, responsible for eliciting personal/family history and life experience narratives. The Experience Inquiry branch is dynamically triggered during turns via an agent-level function, with remaining nodes sampled in randomized order post-HDSM for natural conversational flow (Wan et al., 29 Oct 2025).

4. Dataset Statistics and Structural Properties

PsyCoTalk’s comprehensive statistics are as follows:

• EMRs: 502, spanning six core comorbidity types, including AD, MDD, BD, MDD, ADHD, AD, MDD, ADHD, MDD, AD, BD, MDD, and ADHD, AD.
• Dialogues: 3,000 multi-turn sessions, each averaging 45.9 turns. Doctor utterance mean length: 34.0 Chinese characters; patient: 43.5 characters.
• Disorders: 4 modeled (MDD, AD, BD, ADHD), yielding 51 distinct co-occurrence patterns across sessions.
• JSON Schema: Each record contains EMR, 10 fictitious experience narratives, a turn-annotated dialogue (with full state-machine and context-tree annotations at every turn), doctor profile, and final diagnosis (per-disorder Booleans).
• Annotation Granularity: HLS (High-Level State), ILS (Intermediate-Level State), BLS (Basic-Level State), context_tree_node per utterance (Wan et al., 29 Oct 2025).

5. Evaluation: Structural and Linguistic Realism

PsyCoTalk's realism is quantitatively benchmarked against real-world clinical transcripts:

• Dialogue-length and utterance-length distributions closely match those from real hospital transcripts: mean doctor turn length 34.0 (vs. 28.3 real), patient 43.5 (vs. 35.8 real). Template-like simulated corpora diverge markedly (e.g., MDD-5k: 91.1/162.8 chars).
• Diversity Measures:
  • Intra-EMR dialogue diversity via Jaccard overlap: Diversity = 0.647 ± 0.030.
  • Vocabulary-level: Normalized entropy (0.5974) and semantic diversity (0.7938) nearly equivalent to genuine dialogue (0.5880/0.8663), outperforming prior synthetic datasets.
• Diagnostic reasoning fidelity evidenced by diagnostic match accuracy (HDSM-guided system: 0.31 exact-match vs. 0.22 for zero-shot Qwen2.5-72B, $p=7\times10^{-6}$). Per-label F1: MDD 0.92, AD 0.81, ADHD 0.64, BD 0.40 (Wan et al., 29 Oct 2025).

6. Expert Validation Protocol

Quality assurance includes multi-faceted expert validation:

• Blind rating: Fifty dialogues scored by five licensed psychiatrists (≥7 years clinical experience) across six criteria (symptom coverage, two communication dimensions, two fluency criteria, realism). Mean scores out of 10: Professionalism 7.72, Doctor initiative 8.14, Patient responsiveness 8.24, Fluency 7.42/6.79, Realism 6.67.
• AB testing: Ten two-turn excerpts from PsyCoTalk, D⁴, MDD-5k, and real-world sessions presented in blind testing. Normalized realism scores: Real 6, PsyCoTalk 5, D⁴ 4, MDD-5k 1.
• Diagnostic performance: Exact-match diagnostic accuracy (0.31) exceeds zero-shot LLM baselines, with significantly improved per-label F1 metrics (Wan et al., 29 Oct 2025).

7. Applications, Comparative Analysis, and Resources

PsyCoTalk is optimized for development and evaluation of multi-disorder screening agents and clinical decision support research:

• Use cases: Dialogue agent training, LLM diagnostic strategy evaluation, conversational clinical decision-support, research in psychiatric comorbidity reasoning.
• Comparative context: Unlike prior dialogue corpora—which typically address single-disorder or lack clinical structure—PsyCoTalk mirrors realistic length, linguistic diversity, and structure, bridging the gap between naturalistic clinical interaction and synthetic data generation.
• Data accessibility: Each item includes rich annotations, comprehensive metadata, and explicit grounding in contemporary clinical protocols, establishing a new standard for psychiatric dialogue research (Wan et al., 29 Oct 2025).

A plausible implication is that such a dataset may facilitate robust benchmarking and research into LLM-based or agent-based diagnostic tools capable of nuanced, simultaneous multi-disorder screening—a core challenge in contemporary computational psychiatry.