Empathy-R1: Structured Empathy for Counseling

• The paper's main contribution is introducing a Chain-of-Empathy framework that integrates explicit, multi-layer reasoning with reinforcement learning to enhance long-form mental support responses.
• It employs a novel Chinese dataset, Empathy-QA, comprising over 40,000 questions and 168,000 responses to train and benchmark therapeutic dialogue systems.
• Empathy-R1 demonstrates improved BLEU-1, METEOR scores, and human preference metrics by combining supervised fine-tuning with GRPO-based reinforcement learning.

Searching arXiv for the primary paper and closely related work on empathetic reasoning and RL in dialogue. Empathy-R1 is a framework for long-form mental health support that integrates a Chain-of-Empathy (CoE) reasoning process with reinforcement learning to improve response quality on Long Counseling Texts (LCTs), particularly in a Chinese context (Yao et al., 18 Sep 2025). It is designed to address a specific limitation of existing LLMs: replies may be semantically fluent yet still lack the structured reasoning required for genuine psychological support. The framework combines a psychologically grounded multi-step reasoning scaffold, a new large-scale Chinese dataset named Empathy-QA, and a two-stage training procedure in which Supervised Fine-Tuning instills reasoning structure and reinforcement learning refines therapeutic relevance and contextual appropriateness (Yao et al., 18 Sep 2025). Within the recent literature on empathetic dialogue, Empathy-R1 belongs to a broader shift from surface-level affective mirroring toward explicit reasoning, structured empathy evaluation, and reward-based alignment (Ma et al., 2024, Yuan et al., 1 Dec 2025, Jia et al., 26 Jan 2026).

1. Definition and scope

Empathy-R1 is defined by two core commitments: interpretable reasoning and reinforcement learning for long-form mental health response generation (Yao et al., 18 Sep 2025). The framework is motivated by the observation that, in mental health support, fluency alone is insufficient. Effective assistance requires psychological depth, genuine empathy, and a reasoning process that can account for the help-seeker’s emotions, causes, and intentions before formulating a supportive reply (Yao et al., 18 Sep 2025).

The system is explicitly targeted at Long Counseling Texts, a setting in which user inputs are extended, context-rich, and often psychologically complex (Yao et al., 18 Sep 2025). This distinguishes it from many earlier empathetic response generation systems developed for shorter open-domain dialogue turns. A plausible implication is that Empathy-R1 treats empathy not as a single stylistic attribute of the final answer, but as a structured inferential process that precedes response generation.

In the literature, this design sits alongside several related but distinct approaches. EmpRL aligns empathy levels between generated and target responses through a reinforcement learning reward defined over emotional reaction, interpretation, and exploration (Ma et al., 2024). Kardia-R1 trains models for interpretable, stepwise empathetic cognition using rubric-based reinforcement learning over relevance, fluency, empathy, persona consistency, and safety (Yuan et al., 1 Dec 2025). ReEmpathy interleaves spoken response generation with reflective empathy-related reasoning in end-to-end spoken dialogue (Jia et al., 26 Jan 2026). Empathy-R1 differs in centering long-form Chinese mental health support and in organizing reasoning through the specific CoE scaffold (Yao et al., 18 Sep 2025).

2. Chain-of-Empathy reasoning paradigm

The central mechanism in Empathy-R1 is the Chain-of-Empathy reasoning paradigm, a structured framework inspired by cognitive-behavioral therapy and professional counseling practices (Yao et al., 18 Sep 2025). Rather than leaving empathetic behavior implicit in the model’s hidden states, CoE makes the analytic pathway explicit and interpretable.

The reasoning structure is four-layered:

Layer	Function
L1	Emotions and Context
L2	Causes and Beliefs
L3	Intent Analysis
L4	Response Strategy

At L1: Emotions and Context, the model recognizes the user’s core feelings and situational context, grounded in appraisal theory (Yao et al., 18 Sep 2025). At L2: Causes and Beliefs, it analyzes underlying causes such as cognitive biases or beliefs that generate the emotional state (Yao et al., 18 Sep 2025). At L3: Intent Analysis, it discerns the user’s primary goal, including validation, advice, or understanding (Yao et al., 18 Sep 2025). At L4: Response Strategy, it synthesizes a therapeutic response by combining the preceding analyses and drawing on techniques such as active listening, Rogers, and the NURSE model (Yao et al., 18 Sep 2025).

Implementation-wise, the model is prompted to reason step by step through these layers and to produce both an empathy_think rationale and a user-facing answer (Yao et al., 18 Sep 2025). The paper states that this is enforced through explicit system prompts and output format constraints. This places Empathy-R1 in a lineage of systems that make empathy operationally inspectable. IMAGINE, for example, explicitly models emotion cause extraction, knowledge extension, and communication mechanisms such as emotional reaction, interpretation, and exploration (Chen et al., 2022). ReEmpathy similarly uses explicit reflection traces during spoken response generation (Jia et al., 26 Jan 2026). Empathy-R1’s specific contribution is to adapt such explicit reasoning to long counseling texts and to frame it as a therapeutic scaffold (Yao et al., 18 Sep 2025).

3. Empathy-QA dataset

Empathy-R1 is supported by a new Chinese dataset, Empathy-QA, introduced as a large-scale resource for model development and evaluation in long-form mental health support (Yao et al., 18 Sep 2025). The paper positions this dataset as a response to the limitations of prior resources such as PsyQA, which are described as outdated and limited in scope (Yao et al., 18 Sep 2025).

Empathy-QA contains 40,959 user questions and 168,470 long-form responses (Yao et al., 18 Sep 2025). It covers contemporary topics, including AI-induced professional anxiety, and is filtered according to quality, length, and ethical standards (Yao et al., 18 Sep 2025). Each instance includes a question title, description, topic label, and multiple human responses (Yao et al., 18 Sep 2025). The dataset serves both as a pretraining corpus and as a benchmark (Yao et al., 18 Sep 2025).

This emphasis on domain-specific, large-scale, and culturally situated data is consistent with a broader trend in empathetic dialogue research. KardiaBench, used by Kardia-R1, grounds empathy in persistent user identity and evolving conversation through 178,080 QA pairs across 22,080 multi-turn conversations anchored to 671 real-world profiles (Yuan et al., 1 Dec 2025). Psyche-R1 similarly uses 73,000+ high-quality empathetic dialogues alongside reasoning-oriented psychological QA data for Chinese psychological modeling (Dai et al., 14 Aug 2025). Empathy-R1’s dataset contribution is narrower in task scope but specialized for long counseling texts and Chinese mental health support (Yao et al., 18 Sep 2025).

A plausible implication is that Empathy-QA does not merely expand scale; it also shifts the empirical target from short empathetic replies to sustained counseling-style generation, where contextual continuity and therapeutic appropriateness are central.

4. Training procedure and reward design

Empathy-R1 uses a two-stage training process consisting of Supervised Fine-Tuning followed by reinforcement learning with Group Relative Policy Optimization (GRPO) (Yao et al., 18 Sep 2025).

During Stage 1: Supervised Fine-Tuning, the goal is to teach the model the explicit CoE reasoning structure and output format (Yao et al., 18 Sep 2025). The training set includes a high-quality subset of 200 curated examples generated via Deepseek-R1 and reviewed by psychology professionals (Yao et al., 18 Sep 2025). Each sample contains the question, a human answer, and a detailed multi-layer CoE rationale with explicit tags (Yao et al., 18 Sep 2025). The paper states that SFT instills a predictable, interpretable reasoning scaffold, but can make outputs rigid without further adaptation (Yao et al., 18 Sep 2025).

During Stage 2: Reinforcement Learning, the framework applies GRPO to improve quality and empathy beyond structural adherence (Yao et al., 18 Sep 2025). For each input, the model generates a group of candidate outputs; each output receives a composite reward; and the advantage for sample $i$ in the group is normalized as

$A_i = \frac{r_i - \mu_{\{r\}}}{\sigma_{\{r\}}}$

(Yao et al., 18 Sep 2025).

The reward model has two components. The format reward enforces compliance with the full CoE structure and is binary:

$$r_\text{format} = \begin{cases} 1 & \text{if format correct} \ 0 & \text{otherwise} \end{cases}$$

(Yao et al., 18 Sep 2025).

The answer quality reward is trained using contrastive learning. For a query-response pair, embeddings are compared through cosine similarity, and triplet loss separates professional responses from irrelevant, generic, or mismatched-emotion negatives:

$$\mathcal{L}_{\text{contrastive}} = \max\left(0, \cos(E(q), E(a^-)) - \cos(E(q), E(a^+)) + m\right)$$

(Yao et al., 18 Sep 2025).

During RL, if cosine similarity exceeds a threshold, $r_\text{answer}=1$; otherwise it is 0, and the total reward is

$r_\text{total} = r_\text{format} + r_\text{answer}$

(Yao et al., 18 Sep 2025).

This reward design differs from other empathetic RL systems. EmpRL defines empathy reward through agreement between generated and target empathy levels across emotional reaction, interpretation, and exploration, while also adding a KL penalty for coherence (Ma et al., 2024). Kardia-R1 uses a unified reward combining format, emotion accuracy, and rubric reward with equal weights under GRPO (Yuan et al., 1 Dec 2025). Empathy-R1’s reward is narrower: structural correctness plus embedding-based answer quality (Yao et al., 18 Sep 2025). This suggests a more constrained but task-focused alignment objective for long-form counseling responses.

5. Evaluation and empirical performance

Empathy-R1 is evaluated using both automatic metrics and human preference judgments (Yao et al., 18 Sep 2025). The automatic metrics reported are BLEU-1 (B-1), Distinct-1 (D-1), ROUGE-L (R-L), METEOR, and Normalized Average (NAvg) (Yao et al., 18 Sep 2025). The paper states that Empathy-R1 achieves state-of-the-art in BLEU-1 and METEOR, indicating strong semantic and lexical alignment (Yao et al., 18 Sep 2025).

The most prominent result is the human evaluation outcome. On the Empathy-QA test set, Empathy-R1 achieves a Win@1 rate of 44.30% (Yao et al., 18 Sep 2025). On PsyQA, the reported Win@1 is 37.50% (Yao et al., 18 Sep 2025). The corresponding Mean Rank values are 2.60 on Empathy-QA and 2.62 on PsyQA (Yao et al., 18 Sep 2025). The paper describes these results as showing clear preference over strong baselines including Qwen3-8B without default thinking, CBT-LLM, EmoLLM, and Deepseek-R1 (Yao et al., 18 Sep 2025).

The paper also reports an ablation result on training strategy: SFT alone yields rigid or less nuanced responses, RL alone is less effective without the SFT-initialized structure, and the combination yields best results (Yao et al., 18 Sep 2025). A further qualitative ablation states that removing CoE structure leads to decreased BLEU-1 and Distinct-1, with responses becoming more repetitive and less insightful (Yao et al., 18 Sep 2025).

These findings align with a broader empirical pattern in empathetic dialogue research: structure plus explicit optimization tends to outperform either component in isolation. EmpRL reports that removing empathy reward or RL causes a steep drop in Emp-F1 and diversity (Ma et al., 2024). Kardia-R1 reports stronger outcomes when rubric-based RL is layered on top of SFT (Yuan et al., 1 Dec 2025). ReEmpathy reports that gains disappear when reflective alternation is ablated (Jia et al., 26 Jan 2026). Empathy-R1 extends this pattern to long-form mental health support (Yao et al., 18 Sep 2025).

6. Position within empathetic AI research

Empathy-R1 belongs to a family of systems that move beyond maximum-likelihood response generation toward explicit empathy modeling, structured reasoning, and alignment through task-specific reward signals. Earlier work such as IMAGINE integrated emotion cause extraction, knowledge graph–based knowledge extension, and communication mechanism prediction for empathetic response generation (Chen et al., 2022). EmpRL made empathy level alignment itself the RL objective (Ma et al., 2024). Kardia-R1 advanced interpretable, stepwise empathetic cognition through a rubric-LM judge and user-grounded data (Yuan et al., 1 Dec 2025). ReEmpathy introduced free-form reflective reasoning interleaved with spoken response generation (Jia et al., 26 Jan 2026). Psyche-R1 unified empathy, expertise, and reasoning in a Chinese psychological LLM using SFT and GRPO (Dai et al., 14 Aug 2025).

Relative to these systems, Empathy-R1 is distinguished by four properties stated in the paper: its focus on long counseling texts, its Chinese setting, its Chain-of-Empathy scaffold, and its combination of SFT plus GRPO guided by a dedicated reward model (Yao et al., 18 Sep 2025). Its interpretability claim is narrower than that of Kardia-R1, which explicitly segments outputs into understanding, reasoning, explicit emotion prediction, and empathetic response (Yuan et al., 1 Dec 2025), but broader than systems where empathy remains latent and only the final response is observed.

The framework also reflects a methodological convergence with recent empathy evaluation research. Multi-dimensional evaluation work has argued that empathy should be assessed from both expressed and perceived perspectives rather than from expression alone (Xu et al., 2024). ReEmpathy’s EmpathyEval similarly replaces single scalar signals with descriptive natural-language empathy assessment for spoken dialogue (Jia et al., 26 Jan 2026). Empathy-R1 does not introduce a similarly multi-dimensional evaluator, but its use of an explicit rationale plus human preference assessment indicates the same pressure toward richer supervision and evaluation (Yao et al., 18 Sep 2025).

A common misconception in this area is that empathetic response generation is primarily a matter of producing emotionally warm phrasing. The literature summarized here suggests otherwise. IMAGINE emphasizes emotion cause and communication mechanism (Chen et al., 2022). EmpRL distinguishes affective and cognitive empathy via emotional reaction, interpretation, and exploration (Ma et al., 2024). Kardia-R1 adds user understanding, contextual reasoning, emotion perception, persona consistency, and safety (Yuan et al., 1 Dec 2025). Empathy-R1 contributes to this line by making emotions, causes, intentions, and response strategy explicit within long-form mental health support (Yao et al., 18 Sep 2025).

7. Significance, limitations, and implications

The significance claimed for Empathy-R1 is twofold: improved performance and increased trustworthiness through interpretability (Yao et al., 18 Sep 2025). The framework is presented as enabling responses that are not boilerplate but instead reference specific user contexts and needs through four-layer analysis (Yao et al., 18 Sep 2025). In this sense, CoE is not merely a prompt template; it is the structural premise on which training, reward design, and evaluation are organized.

The paper also frames Empathy-R1 as filling a gap in Chinese-language mental health AI, describing Empathy-QA as the largest and most relevant corpus for long counseling texts to date (Yao et al., 18 Sep 2025). A plausible implication is that linguistic and cultural grounding are treated as part of empathetic competence rather than as peripheral deployment details.

At the same time, the reported design reveals several boundaries. The reward is based on format correctness and embedding similarity thresholding rather than on a richer rubric over empathy, safety, and persona consistency of the sort used in Kardia-R1 (Yuan et al., 1 Dec 2025). The evaluation emphasizes Win@1, Mean Rank, and lexical overlap metrics rather than multi-aspect human scoring. This suggests that Empathy-R1 prioritizes response preference and structural faithfulness, while leaving some dimensions of clinical or interpersonal evaluation less explicitly formalized.

More broadly, the framework illustrates a transition in empathetic AI from response imitation to reasoning-guided generation. This suggests a research trajectory in which empathy systems are increasingly expected to expose the intermediate stages by which they interpret emotion, attribute causes, infer user intent, and choose supportive strategies. Empathy-R1 exemplifies that trajectory for long-form mental health support in Chinese, using CoE as the organizing principle and reinforcement learning as the alignment mechanism (Yao et al., 18 Sep 2025).