Psyche-R1: Psychological LLM with Empathy & Reasoning

• Psyche-R1 is a Chinese psychological LLM that integrates empathy, psychological expertise, and structured reasoning to generate reliable mental-health responses.
• It employs a hybrid training strategy combining supervised fine-tuning and Group Relative Policy Optimization on extensive, curated psychological and empathetic dialogue datasets.
• The model leverages a comprehensive data pipeline with hard-sample selection and multi-LLM cross-selection to refine its reasoning and improve response quality.

Psyche-R1 is a Chinese psychological LLM designed to unify three capabilities that its authors present as jointly necessary for reliable mental-health-oriented AI systems: empathy, psychological expertise, and reasoning. It is built on Qwen2.5-7B-Instruct and is accompanied by a data curation pipeline that produces more than 75k psychological questions with detailed rationales and 73k empathetic dialogues. The model is trained through a hybrid strategy that combines supervised fine-tuning with Group Relative Policy Optimization (GRPO), using a hard-sample selection procedure based on multi-LLM cross-selection. The reported goal is to move beyond psychological dialogue systems that emphasize emotional support alone by making structured reasoning a first-class component of psychological response generation (Dai et al., 14 Aug 2025).

1. Definition and positioning

Psyche-R1 is introduced as the first Chinese psychological LLM that jointly integrates empathy, psychological expertise, and reasoning. In the paper’s framing, psychological assistance is neither only a conversational-empathy task nor only a knowledge-retrieval task. It also requires structured inference over case narratives, concepts, and context, because reliable psychological responses must be emotionally appropriate, domain-grounded, and logically coherent.

This positioning distinguishes Psyche-R1 from two neighboring lines of work. One consists of psychological or counseling-oriented LLMs that focus primarily on empathetic dialogue and emotional support. The other consists of reasoning-augmented general-purpose models, such as DeepSeek-R1, whose strongest reported capabilities lie in domains such as mathematics and programming rather than psychological assistance. Psyche-R1 is presented as an attempt to bridge these lines by making reasoning supervision central rather than auxiliary (Dai et al., 14 Aug 2025).

A plausible implication is that Psyche-R1 should be understood less as a generic instruction-tuned chatbot for mental-health conversations and more as a domain-specific model whose training objective explicitly couples supportive interaction with rationale-bearing psychological inference.

2. Data construction and synthesis pipeline

The data pipeline is a central component of Psyche-R1. The paper describes four source types. Types I–III support the psychological reasoning question-answer corpus, while Type IV supports the empathetic dialogue corpus.

Source type	Role	Examples
Type I	Psychology knowledge source	Textbooks and curricular materials
Type II	Psychology QA source	Public Chinese educational platforms
Type III	Coverage supplementation	Distilled data from Qwen2.5-72B-Instruct
Type IV	Empathetic dialogue source	Yixinli, Jiandanxinli, Zhihu

For non-text materials, the pipeline uses OLMOCR, followed by manual curation by three trained volunteers. Text normalization standardizes Chinese and English punctuation and removes items such as emojis, emoticons, and links. Dialogue data are further filtered with LLM-based quality control so that responses lacking substantive advice are removed.

For the reasoning dataset, textbook and curricular materials are segmented into chunks, then an LLM generates questions and answers from those segments. Type III data are generated similarly but without segment-level contextual augmentation. This process yields about 200k generated QA pairs, which are then merged with Type II question-bank data into a pool of about 210k entries. The pool is reduced through min-hash locality-sensitive hashing, LLM-based ranking, few-shot LLM filtering, and manual review by 10 undergraduate and graduate students, leaving about 90k QA pairs.

Rationale generation is then performed through chain-of-thought prompting. For each QA pair $(q_i, a_i)$, the model produces a rationale-augmented sample $(q_i, r_i, \hat{a}_i)$. If $\hat{a}_i = a_i$, the rationale is accepted; otherwise regeneration is attempted up to $T$ times, after which the instance is discarded. The paper further applies iterative prompt–rationale optimization over $R = 3$ rounds. In each round, the current prompt is refined as

$P_i^* \leftarrow LLM(P, q_i, r_i),$

and a revised rationale and answer are generated as

$(r_i^*, \hat{a}_i^*) \leftarrow LLM(P_i^*, q_i).$

If $\hat{a}_i^* = a_i$, the updated prompt is retained; otherwise the procedure reverts to the previous prompt-rationale pair. This process yields approximately 75k high-quality rationale-enhanced instances (Dai et al., 14 Aug 2025).

In parallel, the empathetic dialogue pipeline refines platform-derived interactions along three dimensions: emotional resonance enhancement, evidence-based guidance, and solution-oriented support. The result is 73k empathetic dialogues.

3. Hard-sample selection and training corpus organization

Not all rationale-bearing questions are used in the same way. Psyche-R1 divides the reasoning data into challenging and non-challenging subsets through a multi-LLM cross-selection strategy. Three models—Qwen, Llama, and Phi—answer every question independently, and a question is marked as challenging if all three answer it incorrectly. This produces a hard subset of 19k instances.

The paper denotes the resulting datasets as follows:

• $\mathcal{D}_{pc}$: challenging rationale data
• $\mathcal{D}_{pr}$: remaining rationale data
• $(q_i, r_i, \hat{a}_i)$0: empathetic dialogues
• $(q_i, r_i, \hat{a}_i)$1: retained PsycoLLM multi-turn dialogues and knowledge-based QA
• $(q_i, r_i, \hat{a}_i)$2: 8,000 filtered high-quality questions from CPsyExam

These are combined into two training sets:

$(q_i, r_i, \hat{a}_i)$3

and

$(q_i, r_i, \hat{a}_i)$4

This organization encodes a training curriculum. The non-challenging data are used to establish broad empathetic response patterns, domain knowledge, and rationale style through supervised learning. The challenging subset is reserved for reinforcement learning so that reasoning improvement is concentrated on cases that defeated multiple reference LLMs. This suggests that the model’s reasoning specialization is not treated as a uniform property of the whole corpus, but as something sharpened on adversarially difficult psychological examples (Dai et al., 14 Aug 2025).

4. Training method and optimization objectives

Psyche-R1 uses a two-stage training procedure: full-parameter supervised fine-tuning followed by GRPO. The base model is Qwen2.5-7B-Instruct. No architecture modifications are reported.

During supervised fine-tuning, the input is $(q_i, r_i, \hat{a}_i)$5, and the target output is a rationale followed by an answer, $(q_i, r_i, \hat{a}_i)$6. The SFT loss is an autoregressive negative log-likelihood objective:

$(q_i, r_i, \hat{a}_i)$7

This stage is intended to improve empathetic response generation, psychological domain knowledge, and reasoning patterns learned from rationale supervision.

The second stage applies GRPO to $(q_i, r_i, \hat{a}_i)$8. The paper emphasizes reward design rather than the full policy-update mathematics. The model is required to output a reasoning process enclosed in > and ``, followed by the final answer. If that format is respected,

$(q_i, r_i, \hat{a}_i)$9

otherwise

$\hat{a}_i = a_i$0

The accuracy reward is defined as

$\hat{a}_i = a_i$1

and the final reward is

$\hat{a}_i = a_i$2

This reward design gives partial credit on multiple-answer questions and explicitly incentivizes structured reasoning traces. In effect, Psyche-R1 treats output formatting as part of the optimization target rather than a downstream prompting convention.

Implementation details are specified for both stages. SFT is conducted with LLaMA-Factory using learning rate $\hat{a}_i = a_i$3, batch size 256, and 2 epochs. GRPO is conducted with VeRL using learning rate $\hat{a}_i = a_i$4, batch size 128, and 2 epochs. All experiments are run on 8 RTX A6000 GPUs with 48GB memory each (Dai et al., 14 Aug 2025).

5. Evaluation claims and reported performance

The paper states that extensive experiments demonstrate the effectiveness of Psyche-R1 across several psychological benchmarks, and that the 7B Psyche-R1 achieves results comparable to 671B DeepSeek-R1. The excerpted material does not enumerate the benchmark names or reproduce their full metric tables, but the headline comparison is central to the model’s positioning.

The paper also reports an ablation pattern: SFT followed by RL outperforms SFT alone, especially on case tasks. This is presented as evidence that reinforcement learning on the hard-sample subset contributes materially to psychological reasoning performance rather than merely polishing already learned response style.

Because the model is framed as jointly optimizing empathy, expertise, and reasoning, these performance claims are intended to establish that domain-specialized training can partially offset scale disparities. A plausible implication is that the authors view reasoning-targeted data selection and reward shaping as a substitute, at least in part, for brute-force parameter scaling in the psychological domain (Dai et al., 14 Aug 2025).

6. Relation to adjacent frameworks and methodological limits

Psyche-R1 is a model and training pipeline, not an evaluation framework. In this respect it differs from PSYCHE, which is described as a framework for clinically relevant, ethically safe, cost-efficient, and quantitative evaluation of psychiatric assessment conversational agents through simulated patients and rubric-based scoring. PSYCHE should be interpreted as a benchmarking and patient-simulation setup rather than as a psychological LLM model itself (Lee et al., 3 Jan 2025).

This contrast is useful because it places Psyche-R1 within a broader ecosystem. Psyche-R1 focuses on model construction through curated rationale data, empathetic dialogue synthesis, SFT, and GRPO. PSYCHE focuses on evaluation methodology through construct-grounded patient simulation and rubric-based scoring. The two are therefore complementary rather than competing.

The paper on Psyche-R1 also leaves several technical boundaries explicit. It does not provide a formal safety framework or risk taxonomy. It does not report multiple Psyche-R1 model sizes or architecture variants. It does not include the full GRPO update equations, clipping rules, or baseline expressions, and it does not specify decoding hyperparameters such as temperature or top-$\hat{a}_i = a_i$5. These omissions do not alter the stated training pipeline, but they limit the extent to which the optimization dynamics can be reconstructed from the paper alone (Dai et al., 14 Aug 2025).

Taken together, Psyche-R1 represents a domain-specific attempt to merge rationale-based inference, empathetic dialogue generation, and psychological knowledge acquisition into a single Chinese 7B model. Its defining technical features are the 75k+ rationale-bearing psychological questions, the 73k empathetic dialogues, the multi-LLM cross-selection procedure for mining 19k hard questions, and the staged SFT-plus-GRPO training regime built on Qwen2.5-7B-Instruct.