MentraSuite: Reliable Mental Health Reasoning

• MentraSuite is a unified framework integrating MentraBench and Mindora to bolster reliable mental health reasoning with structured clinical steps.
• It operationalizes appraisal, diagnosis, intervention, abstraction, and verification to address common LLM reasoning errors in mental health contexts.
• A hybrid SFT–RL strategy with inconsistency-aware rewards enhances logical coherence and reduces hallucinations in complex mental health analyses.

Searching arXiv for the specified paper to ground the article in the primary source. MentraSuite is a unified framework for reliable mental-health reasoning with LLMs, comprising MentraBench, a comprehensive benchmark, and Mindora, a post-trained LLM. It is designed around clinically aligned, step-wise reasoning used in practice—appraisal, diagnosis, intervention planning, abstraction of evidence, and verification—while explicitly evaluating reasoning reliability through conciseness, coherence, hallucination avoidance, task understanding, and internal consistency. The framework is motivated by the observation that, in mental-health contexts, incomplete, inconsistent, or ungrounded LLM reasoning can misread self-reports, exaggerate symptoms, provide misleading feedback, or amplify anxiety; MentraSuite addresses these risks through benchmark design, structured reasoning trajectories, and a hybrid supervised fine-tuning and reinforcement learning procedure with inconsistency-aware reward shaping (Xiao et al., 10 Dec 2025).

1. Clinical motivation and problem formulation

Mental health disorders affect hundreds of millions globally, and the Web now serves as a primary medium for accessing support, information, and assessment. Within that setting, LLMs offer scalable and accessible assistance, but their deployment is risky when reasoning is incomplete, inconsistent, or ungrounded. The paper identifies several concrete failure modes: models may misread self-reports, exaggerate symptoms, provide misleading feedback, or amplify anxiety. At scale, such failures threaten trust and undermine AI’s social-good promise (Xiao et al., 10 Dec 2025).

A central premise of MentraSuite is that prior psychological LLMs emphasize emotional understanding and knowledge recall but do not adequately target clinically aligned, step-wise reasoning. The framework therefore centers five reasoning aspects that correspond to practical clinical processes: appraisal, diagnosis, intervention, abstraction, and verification. In the paper’s formulation, appraisal concerns recognizing maladaptive cognitions; diagnosis concerns condition judgment; intervention concerns selecting targeted strategies; abstraction concerns evidence synthesis; and verification concerns misinformation detection. This suggests a shift from affective responsiveness or exam-style QA toward reasoning workflows that more closely resemble structured clinical appraisal.

The alignment to clinical reasoning is explicit but bounded. Appraisal is mapped to cognitive distortion identification, diagnosis to condition classification from client narratives, intervention to selection among 13 counselor strategies, abstraction to synthesis from systematic reviews, and verification to evidence-based misinformation detection. The paper also notes that DSM-5/ICD mappings are not explicitly encoded; psychiatric QA and diagnosis datasets are instead drawn from medical exams and PubMed to approximate clinical knowledge coverage (Xiao et al., 10 Dec 2025).

2. MentraBench: benchmark design, tasks, and datasets

MentraBench evaluates five core reasoning aspects across six tasks and 13 datasets. It reports task performance and also scores reasoning quality. The six tasks are Cognitive Error Identification, Mental Health Condition Detection, Counseling Strategy Formulation, Psychiatry QA, Psychiatry Systematic Review Summarization, and Mental Health Misinformation Identification (Xiao et al., 10 Dec 2025).

The mapping from reasoning aspect to task is fixed in the benchmark design. Appraisal is operationalized as Cognitive-Pattern Reasoning through the Cognitive Error Identification task. Diagnosis is operationalized as Mental-Condition Reasoning through the Mental Health Condition Detection task. Intervention is operationalized as Therapeutic-Action Reasoning through the Counseling Strategy Formulation task. Multi-step reasoning is represented through Psychiatry QA. Abstraction is represented through Psychiatry Systematic Review Summarization. Verification is represented through Mental Health Misinformation Identification (Xiao et al., 10 Dec 2025).

The benchmark’s dataset inventory spans synthetic, real, weakly supervised, and expert-curated resources. Cognitive Error Identification uses CognitiveReframing, PatternReframe, and Therapist Q{paper_content}A. Mental Health Condition Detection uses DepSign, SWMH, and T-SID. Counseling Strategy Formulation uses PsyDTCorpus$_{M}$ for training and AnnoMI$_{M}$ for evaluation. Psychiatry QA uses MHQA, MedQA$_{M}$, MedMCQA$_{M}$, and PubMedQA$_{M}$. Psychiatry Systematic Review Summarization uses PSRS*. Mental Health Misinformation Identification uses MentalMisinfo (Xiao et al., 10 Dec 2025).

Task	Datasets	Metric
Cognitive Error Identification	CognitiveReframing; PatternReframe; Therapist Q{paper_content}A	MicroF1
Mental Health Condition Detection	DepSign; SWMH; T-SID	MicroF1
Counseling Strategy Formulation	PsyDTCorpus$_{M}$; AnnoMI$_{M}$	Jaccard
Psychiatry QA	MHQA; MedQA$_{M}$; MedMCQA$_{M}$; PubMedQA$_{M}$	MicroF1
Psychiatry Systematic Review Summarization	PSRS*	Recall
Mental Health Misinformation Identification	MentalMisinfo	MacroF1

Several dataset-specific properties are important for interpreting the benchmark. CognitiveReframing contains simulated negative thoughts and web self-reports annotated by 15 trained professionals and evaluates fine-grained cognitive distortion recognition such as All-or-Nothing and Overgeneralization. PatternReframe uses persona-based statements crafted to manifest specific distortions and labeled by five raters, emphasizing discrimination among closely related distortions. Therapist Q{paper_content}A consists of Kaggle-sourced therapist–client interactions annotated by clinical raters and evaluates recognition of distortions in real dialogue. DepSign, SWMH, and T-SID are public social-media datasets with weak supervision, and the benchmark explicitly requires models to avoid over-pathologizing and sycophancy. AnnoMI$_{M}$0 is curated from authorized motivational interviewing demonstration videos and annotated for 13 commonly taught strategies, including Clarification, Paraphrasing, Reflection of Feeling, Summarizing, Questioning Skills, Immediacy, Use of Silence, Self-Disclosure, Confrontation, Encouragement, Repetition, Interpretation, and Guidance. PSRS* is newly curated from Cochrane Library abstracts and evaluates synthesis of effect direction and certainty. MentalMisinfo uses transcribed video scripts from YouTube and BitChute and emphasizes detection of harmful, anecdotal, non-evidence-based claims (Xiao et al., 10 Dec 2025).

The benchmark also specifies usage and licensing constraints. Mental Health America website content is public, Kaggle “Therapist QA” is publicly available, social-media datasets come from Reddit and Twitter under weak supervision, PubMed and Cochrane abstracts are public, and PSRS contains no patient-level data. Items are verified by experts where indicated (Xiao et al., 10 Dec 2025).

3. Reasoning quality evaluation and clinically aligned assessment

A defining characteristic of MentraBench is that it evaluates not only end-task performance but also reasoning reliability. The five trajectory-level dimensions are reasoning conciseness, logical coherence, hallucination avoidance, task understanding, and internal consistency. Scoring uses a manual binary guideline: a dimension receives 1 if no error is present and 0 otherwise; the dimensions are then averaged to produce a trajectory score per case (Xiao et al., 10 Dec 2025).

The sampling procedure for trajectory evaluation is also prescribed. For each model and dataset, four representative cases are selected: two all-correct and two all-fail. The paper states that this is intended to ensure fairness. Reasoning trajectories are summarized by average across the five dimensions, while performance is reported per dataset and then averaged by task group and overall as Avg_all (Xiao et al., 10 Dec 2025).

The operational definitions of the five dimensions are illustrated with an appraisal example. For the prompt, “After walking in public, I thought ‘Am I insane?’ because I felt people were watching me,” the correct target is Labeling. Conciseness requires avoiding redundant exploration of unrelated distortions. Coherence requires step-wise focus on thought content rather than the situation. Hallucination avoidance prohibits inferring delusions or other symptoms not present. Task understanding requires identifying the distortion category rather than suggesting treatment or diagnosis. Internal consistency requires keeping the conclusion, Labeling, aligned with the final answer tags (Xiao et al., 10 Dec 2025).

This evaluation scheme is closely linked to the benchmark’s clinical orientation. The paper’s concern is not merely whether a model returns a correct label, but whether it produces a reasoning trajectory that remains bounded by the task, avoids unsupported psychiatric inference, and preserves alignment between intermediate reasoning and final judgment. A plausible implication is that MentraBench treats reliability as an observable property of generated trajectories rather than only as aggregate task accuracy.

4. Mindora: post-training architecture and reasoning trajectory generation

Mindora is the post-trained model component of MentraSuite. Its base policy is Qwen3-8B, and its auxiliary model for internal consistency detection is Qwen3-32B. The auxiliary model is used to flag internal contradictions or factual errors in generated reasoning (Xiao et al., 10 Dec 2025).

High-quality reasoning trajectories are constructed through a Reasoning Trajectory Generation (RTG) strategy. The first stage is difficulty filtering: zero-shot QA with Llama-3-8B-Instruct is run on training splits, and only samples answered incorrectly are retained. The stated purpose is to focus on challenging cases requiring deeper reasoning and to avoid dilution by easy, pattern-driven items (Xiao et al., 10 Dec 2025).

The second stage is iterative optimal path search with GPT-4o under verifier guidance. For a verifiable problem $_{M}$1 with ground truth $_{M}$2, GPT-4o generates an initial chain of thought $_{M}$3 and preliminary answer $_{M}$4. When $_{M}$5, refinement strategies include backtracking, exploring new paths, verification, and correction. Search terminates when the verifier confirms correctness; otherwise iterations are capped at $_{M}$6 per search attempt, with up to $_{M}$7 restarts, and failed trajectories are discarded (Xiao et al., 10 Dec 2025).

The third stage is structured formatting. Reasoning is placed in > ...</think>, with titled modules such as “### Symptom Analysis”, “### Differential Diagnosis”, and a mandatory “### Final Conclusion”. The answer is placed in <answer>...</answer>, ends with “Answer: [option/result]”, and must align logically with the Final Conclusion. The paper states that these constraints reduce redundancy and misalignment, including cases in which reasoning favors one diagnosis while the answer reports another (Xiao et al., 10 Dec 2025).

The RTG strategy is presented as a data-quality intervention as much as a prompting convention. The paper reports qualitative improvements in reduced step redundancy and backtracking and improved coherence and interpretability. In trajectory evaluation, Mindora_CHORD achieves $_{M}$8 across the five dimensions, which the paper interprets as confirming improved conciseness and consistency (Xiao et al., 10 Dec 2025).

5. Hybrid SFT–RL training and inconsistency-aware reward

Mindora is optimized through a hybrid supervised fine-tuning and reinforcement learning framework described as CHORD-style dynamic weighting with GRPO RL. The framework uses two data streams: an expert SFT dataset $_{M}$9 and an RL exploration dataset $_{M}$0. The training objective dynamically fuses $_{M}$1 and $_{M}$2 using a global weight $_{M}$3 and token-level weight $_{M}$4 (Xiao et al., 10 Dec 2025).

The paper provides the following training notation and objective, reproduced as written:

$_{M}$4

The inconsistency-detection reward is a multiplicative gate over format, length, consistency, and task-specific quality:

$_{M}$5

The components are specified as follows. FormatValid checks adherence to <think>...<answer>...</answer>. LengthValid requires the inner <think> trajectory length $_{M}$5 to lie in $_{M}$6, with $_{M}$7 tokens and $_{M}$8 tokens. Consistency uses the auxiliary model $_{M}$9 to detect factual inconsistencies or errors. $_{M}$0 depends on task type (Xiao et al., 10 Dec 2025).

The paper reproduces the task-specific quality definitions in LaTeX:

$_{M}$6

$_{M}$7

$_{M}$8

The adaptive global weight schedule is also specified explicitly. During warmup, with $_{M}$1, $_{M}$2 increases from $_{M}$3 to $_{M}$4. During decay, with $_{M}$5, it decreases accordingly. The exact LaTeX as reproduced in the paper is:

$_{M}$9

The token-wise SFT weighting is:

$_{M}$0

where $_{M}$6.

The total loss is:

$_{M}$1

The weighted SFT loss is:

$_{M}$2

The GRPO loss is:

$_{M}$3

with $_{M}$7, $_{M}$8, and $_{M}$9 (Xiao et al., 10 Dec 2025).

The optimization specifics are Adam with $_{M}$0, $_{M}$1, learning rate $_{M}$2, and checkpointing every 10 steps. SFT mini-batches use $_{M}$3; RL mini-batch size is dynamic; rollout uses temperature $_{M}$4 and $_{M}$5 samples per prompt. The paper notes that some LaTeX forms are not fully closed as reproduced and states that the RL objective follows a PPO-style clipped surrogate with normalized advantages, consistent with GRPO (Xiao et al., 10 Dec 2025).

6. Empirical results, ablations, and reproducibility

MentraSuite evaluates 20 LLMs spanning open-source, closed-source, and psychology-oriented variants, including GPT-4o, GPT-o4-mini, DeepSeek-R1/V3, Qwen-plus, QwQ-plus, LLaMA 4, LLaMA 3.3-70B, Qwen2.5-72B, Qwen3-32B/14B/8B, distilled variants, EmoLLM, and Psyche-R1. Mindora is reported in three variants: Mindora_SFT, Mindora_SFT+RL, and Mindora_CHORD (Xiao et al., 10 Dec 2025).

On overall average across all datasets, the reported Avg_all values are: Mindora_CHORD 0.6933, Mindora_SFT+RL 0.6548, Mindora_SFT 0.6367, GPT-o4-mini 0.6515, DeepSeek-R1 0.6505, DeepSeek-V3 0.6386, Qwen-plus 0.6387, Psyche-R1 0.5943, and the Qwen3-8B backbone 0.5729. The paper states that Mindora_CHORD achieves the highest average performance on MentraBench (Xiao et al., 10 Dec 2025).

Task-group averages show a more differentiated profile. For Appraisal (Avg1), Mindora_CHORD is 0.6408, DeepSeek-R1 is 0.6352, and GPT-o4-mini is 0.6026. For Diagnosis (Avg2), Mindora_SFT+RL is 0.7032, Mindora_CHORD is 0.6815, and DeepSeek-V3 is 0.6502. For Multi-step QA (Avg4), Mindora_CHORD is 0.7721, GPT-o4-mini is 0.7402, and DeepSeek-R1 is 0.7459. For Intervention on AnnoMI$_{M}$6, Mindora_CHORD is 0.4016 by Jaccard. For Abstraction on PSRS, Mindora_CHORD is 0.8379, while GPT-4o, DeepSeek-V3, and Qwen2.5-72B report 0.9065, 0.9296, and 0.9555 respectively. For Verification on MentalMisinfo, Mindora_CHORD is 0.7178, GPT-o4-mini is 0.7781, and Psyche-R1 is 0.6954 (Xiao et al., 10 Dec 2025).

Trajectory-quality evaluation further characterizes Mindora_CHORD. The reported scores are $_{M}$7, $_{M}$8, $_{M}$9, $_{M}$0, $_{M}$1, and $_{M}$2. Inter-annotator reliability is reported as Gwet AC1 0.9617, Cohen’s $_{M}$3 0.7986, and Consistency 0.9714 (Xiao et al., 10 Dec 2025).

Ablation findings indicate that Mindora_CHORD outperforms Mindora_SFT+RL, which in turn outperforms Mindora_SFT on Avg_all, and the paper attributes this to CHORD’s dynamic weighting and the consistency reward. The model also shows consistent gains over the backbone Qwen3-8B across all datasets. In a qualitative case study on cognitive error identification, a client thought “Am I insane?” after feeling watched. Many models reportedly misclassified based on the situation, treating feelings as facts, whereas Mindora focused on the thought content and correctly identified Labeling, demonstrating better task understanding and internal consistency (Xiao et al., 10 Dec 2025).

Reproducibility details are explicit. Code and data are available at https://github.com/elsa66666/MentraSuite. Closed-source models were accessed via official APIs, while open-source models were deployed on a single NVIDIA A800-SXM4-80GB GPU with default temperatures. Dataset preparation, splits, manually filtered psychiatric subsets, and prompt formatting aligned to Mindora’s structured schema are described as part of the experimental setup (Xiao et al., 10 Dec 2025).

7. Safety, limitations, and open directions

MentraSuite incorporates several safeguards. Structured formatting through <think>/<answer> is intended to reduce unstructured, misleading chains. The consistency reward penalizes contradictions and factual errors, and verification tasks assess evidence grounding. Length constraints are used to mitigate over-elaboration and redundancy that can confuse users (Xiao et al., 10 Dec 2025).

The paper also states clear limitations and intended use boundaries. MentraSuite evaluates reasoning reliability but does not replace clinicians, and outputs require clinician oversight in high-stakes settings. Fairness analyses across demographics are not reported. DSM-5/ICD mappings are not explicitly encoded, and the paper notes that clinical deployment would require validated diagnostic frameworks and escalation protocols, including for suicidality. It also emphasizes that, as with all LLMs, hallucination remains possible; structured outputs and verification help but are not foolproof (Xiao et al., 10 Dec 2025).

The stated contributions are threefold. First, MentraBench is presented as the first comprehensive benchmark covering five clinically grounded reasoning aspects with six tasks and 13 datasets, evaluating both task accuracy and reasoning quality. Second, Mindora combines hybrid SFT–RL with an LLM-based inconsistency detection reward and structured trajectory generation. Third, the RTG strategy uses difficulty filtering and iterative verifier-guided search to produce concise, coherent reasoning trajectories aligned with clinical logic. The empirical outcome reported is that Mindora_CHORD achieves the highest average performance across 20 LLMs on MentraBench together with superior trajectory reliability across the five dimensions (Xiao et al., 10 Dec 2025).

Several future directions are identified. These include multimodal inputs such as audio or video from counseling sessions, incorporation of longitudinal case context, stronger grounding via retrieval augmentation from clinical guidelines such as DSM-5/ICD summaries and NICE/Cochrane, integration into clinical workflows with decision support and escalation protocols, fairness and bias evaluation across demographics and languages, robustness to domain shift in informal social-media text, more explicit formal checkers for contradiction and hallucination detection beyond LLM-based auxiliary models, and reward shaping that accounts for calibration and uncertainty in diagnosis (Xiao et al., 10 Dec 2025). These proposals suggest that MentraSuite is positioned less as a finished clinical system than as an evaluation-and-training substrate for reliability-oriented mental-health reasoning research.