DSM5AgentFlow: Multi-Agent DSM-5 Screening
- DSM5AgentFlow is a multi-agent LLM framework that simulates a DSM-5 Level-1 mental health screening interview using role-specific agents.
- It separates the workflow into therapist, client, and diagnostician roles to administer questionnaires, simulate responses, and produce transparent, criterion-based diagnoses.
- The system employs retrieval-augmented generation with synthetic data for privacy-preserving, auditable, and reproducible diagnostic screening research.
Searching arXiv for the primary paper and closely related agentic workflow work to ground the article. arXiv search query: (Ozgun et al., 15 Aug 2025) DSM5AgentFlow is a multi-agent LLM workflow for simulating a DSM-5 Level-1 mental health screening interview and turning that interview into an explainable provisional diagnosis. Introduced in “Trustworthy AI Psychotherapy: Multi-Agent LLM Workflow for Counseling and Explainable Mental Disorder Diagnosis” (Ozgun et al., 15 Aug 2025), it separates the screening process into role-specialized agents—therapist, client, and diagnostician—so that the system can administer the DSM-5 Level-1 Cross-Cutting Symptom Measure conversationally rather than as a static form, simulate realistic patient responses from configurable mental-health profiles without using real patient data, and produce a diagnosis with a transparent rationale explicitly tied to both the dialogue and DSM-5 criteria. The framework is presented as a trustworthy AI psychotherapy or screening workflow and, at the same time, as a complementary research tool rather than a deployable medical device (Ozgun et al., 15 Aug 2025).
1. Conceptual definition and scope
DSM5AgentFlow was proposed to address a mismatch between the promise of LLM agents and the requirements of mental health diagnosis. The underlying paper identifies four deficiencies in prior approaches: dependence on scarce and sensitive clinical datasets, reduction of diagnosis to one-shot classification or generalized interview automation, weak multi-turn conversational reasoning about what has and has not been covered, and opaque outputs that do not align clearly with expert clinical reasoning (Ozgun et al., 15 Aug 2025). The framework responds by making the workflow explicit, modular, criterion-grounded, and self-documenting.
The term “DSM5AgentFlow” can be misleading if read as a system that invents new questionnaires. The paper states that it is more accurate to say that the system generates conversational realizations of the questionnaire rather than new questionnaire content. The canonical source remains the DSM-5 Level-1 Cross-Cutting Symptom Measure; the generated component is the natural, empathetic spoken form used by the therapist agent. The screening logic is therefore constrained to all 23 items spanning 13 symptom domains: Depression, Anger, Mania, Anxiety, Somatic Symptoms, Suicidal Ideation, Psychosis, Sleep Problems, Memory, Repetitive Thoughts and Behaviors, Dissociation, Personality Functioning, and Substance Use (Ozgun et al., 15 Aug 2025).
The paper also frames DSM5AgentFlow as the first LLM-based agent workflow designed to autonomously generate DSM-5 Level-1 diagnostic questionnaires. More precisely, the methodological description supports a narrower reading: it is a workflow for autonomously administering and conversationally instantiating DSM-5 Level-1 screening while producing criterion-grounded explanations. This suggests that its novelty lies less in raw diagnostic prediction and more in the integration of standardized screening, role separation, retrieval grounding, and auditable reasoning.
2. Architecture and formal workflow
DSM5AgentFlow is organized as a three-agent pipeline. The therapist agent conducts the interview, the client agent plays a patient with a predefined profile, and the diagnostician agent reads the completed transcript, retrieves relevant DSM-5 passages, and produces a provisional diagnosis, explanation, and treatment recommendations (Ozgun et al., 15 Aug 2025). The workflow is formalized as Algorithm 1, “Multi-Agent Mental Health Diagnostic Workflow.”
The main procedure is:
It initializes conversation state and questionnaire coverage through:
and iterates until all questionnaire items have been addressed:
At each step, the system selects the next questionnaire item, formats a therapist prompt from the current history plus that item, generates the therapist’s question, appends it to the history, then formats a client prompt from the updated history plus the client profile, generates the client response, and appends that response. If the item has been sufficiently addressed, it is removed from pending and added to completed. After coverage is complete, the diagnostician retrieves DSM-5 passages relevant to the conversation:
It then generates a diagnosis and extracts four structured components:
The final output is:
From a systems perspective, the inputs are a questionnaire document, a client profile file, and a selected backbone LLM. The questionnaire can be loaded at runtime from PDF, TXT, or Markdown and includes section headings, item text, and scoring rules. The client profile is loaded from text files containing diagnostic priors, demographics, and conversational context. Runtime state includes conversation history, the pending and completed item sets, and the retrieved DSM-5 passage set. The implementation is modular, with an adapter layer supporting Ollama locally and Groq or OpenAI APIs in the cloud (Ozgun et al., 15 Aug 2025).
An important methodological caveat qualifies this formal design. Although the pseudocode is explicitly iterative, the limitations section states that the experimental conversations were produced in “one-shot generation” to reduce GPU cost, and that this “precludes true turn-by-turn adaptation.” The conceptual system is therefore a sequential multi-turn workflow, but the reported experiments do not fully instantiate live adaptive dialogue management.
3. Agent roles, prompting, and internal control
The therapist agent is the orchestration driver for the interview stage. Its function is not to repeat questionnaire items verbatim, but to administer the DSM-5 Level-1 instrument as a realistic and empathetic conversation. The paper specifies four design elements: initialization with the full questionnaire, iterative natural-language rephrasing of each item, coverage tracking to determine whether an item has been sufficiently addressed, and completion only when all items are covered (Ozgun et al., 15 Aug 2025). Its prompt constrains the role tightly: it must identify the questionnaire by name, explain purpose and confidentiality, mention the number of questions, begin with the first question, and avoid premature diagnostic assumptions. It is also instructed not to present the questionnaire as targeting a specific disorder unless that is literally the questionnaire’s name.
The client agent is a simulated patient generator controlled by a hidden profile. That profile can include primary disorder, optional comorbid modifiers, and context such as age, gender, recent life events, and coping style. These profiles are loaded from external files, which the paper presents as a mechanism for modular and reproducible persona generation. The prompt enforces first-person speech, in-character behavior, emotional authenticity, consistency with the hidden profile, and a prohibition on naming the disorder or revealing role-play status. The intended effect is that the diagnostician must infer the disorder from symptom expression and dialogue evidence rather than from an exposed label.
The diagnostician agent is the explainability and reasoning module. It returns exactly four parts: a compassionate summary paragraph, a diagnosis section, a reasoning section, and a “Recommended Next Steps/Treatment Options” section with numbered recommendations. It is also instructed to wrap medical terms in <med> tags, symptoms in <sym> tags, and patient quotes in <quote> tags. Those tags function as machine-readable anchors between claims, symptoms, and transcript evidence (Ozgun et al., 15 Aug 2025).
For diagnostic grounding, the diagnostician uses retrieval-augmented generation rather than parametric memory alone. The paper specifies retrieval of the top 5 most relevant DSM-5 passages using chunk sizes of 512 and 1024 tokens and the nomic-embed-text embedding model, with the transcript as query context. The rationale is to reduce unsupported inference and tie diagnostic reasoning to authoritative DSM-5 text.
The diagnostician’s reasoning is externalized rather than latent. The system maps client utterances to DSM-5 criteria, states why a criterion is supported or contradicted, and surfaces direct evidence from the transcript. The paper’s preferred form is an evidence-to-criterion-to-conclusion trace. Its representative Qwen-QWQ case study gives the following reasoning sequence: auditory hallucinations for 3 weeks satisfy Criterion A1; no mood episode longer than the psychosis supports criterion D; a normal toxicology panel excludes substance aetiology under Criterion E; the quoted patient statement indicates partial insight; and the conclusion is provisional schizophreniform disorder (Ozgun et al., 15 Aug 2025). This supports the paper’s claim that the workflow is internally self-documenting.
4. Explainability and the framework’s notion of trustworthiness
DSM5AgentFlow operationalizes trustworthiness primarily through transparency, grounding, role separation, and synthetic-data use. Transparency comes from criterion-linked rationales, direct evidence extraction from dialogue, use of quote and symptom tags, and retrieval of DSM-5 passages. Grounding comes from RAG over DSM-5 text. Role separation is implemented through prompt specialization across therapist, client, and diagnostician. Synthetic-data use avoids real patient identifiers and is presented as a privacy-preserving design choice (Ozgun et al., 15 Aug 2025).
The explainability mechanism is not limited to the presence of explanatory prose. The paper distinguishes between explanation text and auditable logic. In its structural comparison of one representative transcript per model, Qwen-QWQ combined 11 <sym> tags, 4 <quote> tags, DSM clauses A–E, and numbered reasoning steps; Mistral-Saba included 7 <sym> tags, 2 <quote> tags, and DSM clauses A–E but no step list; Llama-4 included 4 <sym> tags with no quotes, no DSM clauses, and no step list; GPT-4.1-Nano included 29 <sym> tags but no quotes, no DSM clauses, and no step list (Ozgun et al., 15 Aug 2025). The paper interprets this as evidence that criterion-level traceability matters more than the raw abundance of evidence markers.
The ethical framing is procedural rather than regulatory. The system uses only synthetic data, no IRB was required, and released data prohibit re-identification or clinical use without separate ethical approval. The authors explicitly state that outputs must not inform clinical decisions and that the framework is not a medical device. They also state that the design was intended to ensure adherence to ethical and legal standards, while acknowledging that the paper does not provide a detailed mapping to GDPR, HIPAA, MDR, or the EU AI Act (Ozgun et al., 15 Aug 2025).
A recurrent misconception is that explainability here means hidden chain-of-thought exposure. The paper rejects that framing. Reasoning is instead represented through structured text, retrieved DSM passages, criterion references, tagged symptoms, and transcript-linked evidence. This makes the output inspectable without claiming direct access to internal latent reasoning.
5. Experimental design and empirical findings
The experimental study uses a synthetic benchmark of 8,000 simulated therapist-client conversations, with 2,000 conversations per backbone model. The four backbone LLMs are Llama-4-scout-17b, Mistral-Saba-24b, Qwen-QWQ-32b, and GPT-4.1-Nano. Client profiles span 10 primary disorder categories: Adjustment Disorder, Anxiety, Bipolar Disorder, Depression, OCD, Panic Disorder, PTSD, Schizophrenia, Social Anxiety Disorder, and Substance Abuse, with additional comorbidities and demographic variation. Generation and retrieval or evaluation were parallelized across 4 worker threads, reducing estimated generation time from about 100 hours serially to about 24 hours (Ozgun et al., 15 Aug 2025).
The benchmark corresponds to three research questions: whether LLMs can simulate therapist-client conversations that complete the DSM-5 questionnaire, whether disorder predictions can be made by linking questionnaire responses to disorder descriptions, and whether the system can produce explainable and transparent diagnoses. Evaluation therefore spans conversational realism, diagnostic performance, and explainability.
For conversation quality, the paper uses BERTScore, Flesch Reading Ease, Flesch-Kincaid Grade, Gunning Fog Index, and an LLM-based rubric with five criteria: completeness of DSM-5 dimension coverage, clinical relevance and accuracy of questions, consistency and logical flow, diagnostic justification and explainability, and empathy or naturalness or professionalism. Metric results show moderate coherence across models. Llama-4-scout-17b achieved BERTScore 50.77%, FRE 61.67, FKG 7.01, and GFI 3.87; Mistral-Saba-24b achieved 51.30%, 49.58, 8.99, and 4.35; Qwen-QWQ-32b achieved 50.68%, 51.10, 8.70, and 4.21; GPT-4.1-Nano achieved 54.87%, 53.81, 8.96, and 5.23. In rubric evaluation, Llama-4 and Mistral-Saba achieved mean scores between 4.26 and 4.41 out of 5, Qwen-QWQ ranged from 3.64 to 4.23, and GPT-4.1-Nano ranged from 1.89 to 2.54 (Ozgun et al., 15 Aug 2025).
For diagnostic performance, the ranking reverses. Qwen-QWQ performs best overall, with 70% accuracy, 72% recall, and 77% F1. GPT-4.1-Nano follows with 73% F1 and the highest precision at 83%. Llama-4 and Mistral-Saba are lower, with accuracies of 52% and 57% and F1 scores of 65% and 63%. Per-disorder results show that Anxiety, Panic, PTSD, and Social Anxiety are comparatively easy across models, while Adjustment Disorder is hardest and Depression remains difficult. The confusion matrices show recurring misclassifications: Adjustment Disorder as Depression, Bipolar and Depression as one another, Social Anxiety as Anxiety, and Substance Abuse as Depression. The paper attributes these patterns partly to symptom overlap in the DSM-5 Level-1 questionnaire and lack of contextual or temporal granularity in the instrument itself (Ozgun et al., 15 Aug 2025).
A concise summary of the reported trade-off is as follows:
| Model | Conversational profile | Diagnostic profile |
|---|---|---|
| Llama-4-scout-17b | Most readable by FRE; strong rubric scores | 52% accuracy; 65% F1 |
| Mistral-Saba-24b | Strong rubric scores; paragraph-style rationale | 57% accuracy; 63% F1 |
| Qwen-QWQ-32b | Lower dialogue rubric than Llama/Mistral | Best overall: 70% accuracy, 77% F1 |
| GPT-4.1-Nano | Weak rubric scores | Highest precision at 83%; 73% F1 |
The explainability findings align with this trade-off. Qwen-QWQ was judged best for auditability because it combined evidence tags, direct quotes, DSM clause references, and numbered logic. Mistral-Saba produced correct but less auditable paragraph reasoning. Llama-4 was described as the most opaque. GPT-4.1-Nano produced many tags but weak logical structure. The paper’s broader interpretation is that conversation-oriented models produce better interviews, whereas reasoning-oriented models produce better diagnoses and more inspectable explanations (Ozgun et al., 15 Aug 2025).
6. Limitations, significance, and relation to adjacent research
The paper is explicit that DSM5AgentFlow’s evidence base remains synthetic. There are no real client transcripts, no human clinician evaluation study, and no expert annotations of transcript realism or rationale faithfulness beyond case-study discussion. The rubric-based evaluations are themselves performed by an LLM, which the authors identify as a limitation because evaluators may share biases or failure modes with the tested models. Text-only screening also misses nonverbal cues, and the DSM-5 Level-1 questionnaire may be too coarse to separate overlapping disorders such as Adjustment Disorder and Depression (Ozgun et al., 15 Aug 2025).
The implementation gap between conceptual workflow and reported experiments is also important. The formal system is item-by-item, stateful, and adaptive, but the experiments use one-shot generation for cost reasons. This means some claimed strengths belong more clearly to the architecture than to the instantiated benchmark. A plausible implication is that future versions would need true turn-by-turn execution, expert review of rationale quality, and richer multimodal evidence before any stronger claims about clinical realism could be sustained.
Within the broader literature, DSM5AgentFlow occupies an intermediate position between several adjacent strands. Automated prompt-refining multi-agent clinical screening workflows have been studied for note-level cognitive-concern detection rather than DSM diagnosis (Tian et al., 3 Feb 2025). Dynamic workflow frameworks emphasize runtime replanning and feedback-conditioned orchestration rather than fixed pipelines (Wang et al., 30 Sep 2025), while workflow-control systems such as FlowAgent formalize how LLMs can remain flexible under procedural constraints (Shi et al., 20 Feb 2025). In mental-health-specific settings, related work has explored DSM-5-informed text-analysis pipelines (Agarwal et al., 30 Sep 2025), neuro-symbolic synthetic diagnostic conversation generation (Yin et al., 2024), and honesty-aware synthetic psychiatric intake simulation (Zhang et al., 14 Jan 2026). Machine-actionable DSM knowledge representations have also been proposed through symptom profile generators that bridge narrative criteria and formal profile spaces (Kutil et al., 23 Nov 2025). This suggests that DSM5AgentFlow’s distinctive contribution is the integration of standardized DSM-5 Level-1 conversational administration, synthetic patient simulation, retrieval-grounded reasoning, and criterion-level explanation in a single reproducible workflow.
The framework’s open-source release and modular implementation reinforce that research orientation. The paper states that the datasets and implementations are fully open-sourced at https://github.com/mithatco/mental_health_multiagent (Ozgun et al., 15 Aug 2025). Its long-term significance therefore lies less in immediate clinical deployment than in establishing a reproducible architecture for privacy-preserving, inspectable, and role-separated mental-health screening with LLM agents.