Empathy-QA: Empathetic Conversational AI
- Empathy-QA is a research field that prioritizes empathetic alignment in AI by integrating psychological models with multi-dimensional evaluation techniques.
- It leverages diverse specialized datasets and innovative neural architectures, including reinforcement learning and adapter-based models, to enhance response quality.
- The field addresses challenges such as ensuring conversational continuity, balancing perceived versus expressed empathy, and managing cultural and safety nuances in interactions.
Empathy-QA denotes a line of question answering and conversational AI research in which response generation, evaluation, and benchmarking are organized around empathy as a primary target rather than as a stylistic by-product. In the literature, the term refers both to empathy-centric question answering and conversational systems more generally, and, in one concrete instantiation, to a large-scale, contemporary, Chinese-language dataset curated for the long-form mental health counseling domain, particularly single-turn Long Counseling Texts (LCTs) (Yao et al., 18 Sep 2025). The area integrates psychological theories of empathic concern, personal distress, cognitive empathy, affective empathy, and response appropriateness with neural generation, retrieval augmentation, reinforcement learning, user-centered evaluation, and safety auditing (Buechel et al., 2018, Xu et al., 2024, Randhawa et al., 14 Jan 2026).
1. Conceptual basis and scope
Empathy-QA research treats empathy as a multi-component construct rather than a single scalar property. One early computational foundation distinguishes empathic concern and personal distress using Batson’s Empathic Concern – Personal Distress Scale. In that formulation, empathic concern is other-oriented and marked by relatively positive affect, whereas personal distress is self-oriented and aversive; their moderate correlation, reported as Pearson , is taken as evidence that they are overlapping but distinct constructs (Buechel et al., 2018). Later work extends this decomposition by distinguishing cognitive empathy, affective empathy, and empathic response appropriateness, with cognitive empathy defined as correct understanding of another’s mental or emotional state, affective empathy as an appropriate affective response, and appropriateness treated as an outcome-focused dimension rather than empathy itself (Formanek et al., 2024).
A second conceptual shift concerns where empathy is measured. Multi-dimensional evaluation work argues that empathy in dialogue is collaborative and should be assessed from both the speaker’s and the listener’s perspective. On this view, expressed empathy is captured through communicative intents, while perceived empathy is assessed through Engagement, Understanding, Sympathy, and Helpfulness (Xu et al., 2024). SENSE-7 generalizes this user-centered perspective into a seven-dimensional taxonomy—the AI Empathic Behavior Scale—comprising Affective Understanding, Cognitive Understanding, Response Appropriateness, Prosocial Expression, Interest, Contextual Understanding, and Relational Continuity (Suh et al., 19 Sep 2025).
Empathy-QA also inherits concepts from broader empathy scholarship outside response generation. Work on physics faculty describes communication and contextual information as central mediators of empathetic action and introduces “dis-communication” as an impediment arising from what is not collected, not asked, or not acted upon (Merrill et al., 2024). The position paper on computer-mediated empathy similarly frames empathy as a dyadic interaction between empathizer and empathizee, emphasizing self-expression, self-reflection, perspective-taking, non-judgmental attitude, recognizing emotions, and communicating understanding as separate elements that technologies can support or hinder (Lee, 2019). This suggests that Empathy-QA systems are not only generation engines; they are also mechanisms for eliciting context, managing interactional repair, and making understanding legible to the recipient.
2. Corpora, datasets, and benchmarks
The empirical development of Empathy-QA has been driven by increasingly specialized corpora. Early text-based empathy prediction work introduced the first publicly available gold standard corpus for empathy prediction, built from self-reported writer states rather than third-party judgments. Participants read curated news articles, rated empathy and distress using multi-item scales, and then wrote responses; after filtering, the corpus contained 1,860 message-rating pairs, with split-half reliability for empathy and for distress (Buechel et al., 2018).
Later resources expanded both scale and domain specificity. SoulChatCorpus contains 2,300,248 multi-turn empathetic conversations across 12 topics, designed to resemble psychological counseling and to cover behaviors such as questioning, comfort, recognition, active listening, and emotional support (Chen et al., 2023). SENSE-7 contributes 695 real-world conversations from 109 participants interacting with four LLMs, with per-turn empathy annotations directly from the users and a 672-conversation anonymized release for research (Suh et al., 19 Sep 2025). KardiaBench adds persistent identity structure through 178,080 QA pairs across 22,080 multi-turn conversations anchored to 671 real-world profiles and spanning 32 emotion categories (Yuan et al., 1 Dec 2025).
A particularly important resource for the named dataset sense of Empathy-QA is the corpus introduced with Empathy-R1. It contains 40,959 user questions and 168,470 long-form responses, with 29 main topics, 182 subtopics, and an average of 4.11 responses per question. Average length statistics are 22.7 characters for question titles, 275.9 for question descriptions, and 775.6 for answers. The dataset is larger than PsyQA on questions, answers, and topical coverage, and its multiple responses per question enable multi-reference evaluation (Yao et al., 18 Sep 2025).
| Resource | Scope | Key properties |
|---|---|---|
| Writer-centric empathy corpus | 1,860 message-rating pairs | Self-reported empathy and distress; 418 news articles; multi-item scales (Buechel et al., 2018) |
| SoulChatCorpus | 2,300,248 multi-turn conversations | 12 topics; psychologist-like empathetic responses (Chen et al., 2023) |
| Empathy-QA | 40,959 questions; 168,470 responses | Long-form Chinese mental health counseling; 29 main topics; 182 subtopics (Yao et al., 18 Sep 2025) |
| SENSE-7 | 695 conversations; 672 released | User-annotated per-turn empathy in sustained human-AI conversations (Suh et al., 19 Sep 2025) |
| KardiaBench | 178,080 QA pairs | 22,080 multi-turn conversations grounded in 671 profiles (Yuan et al., 1 Dec 2025) |
Within Empathy-QA itself, a curated subset of 200 high-quality instances is used for reasoning supervision. For each, a base LLM generates a four-level Chain-of-Empathy (CoE) linking query to professional answer, and a psychology professional vets and edits the result for therapeutic validity and structural correctness. The four levels are L1 core emotions and user context, L2 underlying causes and beliefs, L3 user intent or communication goal, and L4 response strategy (Yao et al., 18 Sep 2025).
3. Modeling paradigms
Modeling in Empathy-QA spans prediction, retrieval, supervision with communication strategies, and reinforcement learning. In early empathy prediction, empathy and distress were formulated as regression tasks over continuous scores. Ridge regression, a feed-forward neural network, and a convolutional neural network were compared using FastText embeddings trained on Common Crawl; the CNN, with a single convolutional layer, filter sizes 1 to 3, 100 channels each, average pooling, a 128-unit dense layer, dropout, and early stopping, achieved the best results, with Pearson for empathy and for distress (Buechel et al., 2018).
For empathetic response generation, IMAGINE explicitly models three processes: emotion cause extraction, knowledge extension via ATOMIC-2020, and communication mechanisms. Its communication mechanism module predicts Emotion Reaction, Interpretation, and Exploration, while its knowledge component uses Affect, Behaviour, Physical, and Events relations to refine cause representations before decoding (Chen et al., 2022). LEMPEx likewise foregrounds communication elements, using dense passage retrieval for exemplar selection and synthetic labels for emotional presence, interpretation, exploration, and sentiment as auxiliary control signals (Majumder et al., 2021).
Retrieval-augmented large-model approaches further integrate theory-driven structure. APTNESS constructs an APT database from an empathetic emotional palette with 7 major emotion categories and 23 subcategories, yielding 230 factors, 2,415 situations, 9,663 dialogues, and approximately 19,896 responses. Retrieved examples are combined with emotional support strategies learned from ESConv and ExTES to improve both cognitive and affective empathy (Hu et al., 2024). SoulChat instead emphasizes scale and instruction tuning: ChatGLM-6B is fine-tuned on multi-turn dialogue histories whose targets cover questioning, comfort, recognition, listening, trust, sincerity, and emotional support (Chen et al., 2023).
Reinforcement learning introduces explicit empathy optimization. EmpRL uses a T5 generator and an empathy reward function grounded in three communication mechanisms—emotional reaction, interpretation, and exploration—optimized with proximal policy optimization. It also introduces Emp-F1 to measure similarity in empathy levels between generated and target responses (Ma et al., 2024). Empathy-R1 combines supervised fine-tuning on CoE-structured data with reinforcement learning guided by a dedicated reward model, targeting long-form mental health support on Empathy-QA. On the Empathy-QA test set it reports BLEU-1 $0.314$, METEOR $0.314$, ROUGE-L $0.045$, Distinct-1 $0.375$, Win@1 $44.30$, and Mean Rank 0 (Yao et al., 18 Sep 2025).
Context specialization has become a distinct paradigm. Empathetic expert adapters are LoRA-based adapters trained per task cluster to preserve context-appropriate empathy patterns over multiple turns; on the reported evaluation, the average empathy gap drops from 1 for the inherent LLM to 2 for the adapter, a 72.66% reduction, with measured empathy increasing by an average factor of 2.43 (Shayegani et al., 5 Nov 2025). In spoken dialogue, ReEmpathy alternates response chunks with internal reflective chunks using Empathetic Self-Reflective Alternating Inference, supervised by a descriptive evaluation model called EmpathyEval (Jia et al., 26 Jan 2026).
4. Evaluation methodologies
Evaluation in Empathy-QA is unusually heterogeneous because empathy is measured as prediction, perceived quality, behavioral alignment, and safety. In writer-centric prediction, the dominant metric is Pearson correlation between predicted scores and gold labels, using 10-fold cross-validation and a held-out development set (Buechel et al., 2018). In counseling-style generation, SoulChat combines automatic metrics—BLEU and ROUGE—with human evaluation under the CEHS framework: Content Naturalness, Empathy, Helpfulness, and Safety. Three psychology-trained raters score Con., Emp., and Hel. on 3 and Saf. on 4, with Fleiss’ 5 values of 0.489, 0.472, 0.532, and 1.0 respectively (Chen et al., 2023).
Multi-dimensional response evaluation broadens the target. One framework jointly annotates 16 communicative intents and four perceived empathy aspects—Engagement, Understanding, Sympathy, and Helpfulness—on 2,000 customer service dialogues. The analysis shows that perceived empathy is highly correlated with dialogue satisfaction, with Spearman’s 6 of 0.410 for Engagement, 0.396 for Understanding, 0.099 for Sympathy, and 0.580 for Helpfulness (Xu et al., 2024). SENSE-7 extends this to sustained human-AI conversations by collecting user ratings per turn and per conversation on seven empathy dimensions; its baseline GPT-4o classifier attains average Spearman 7, Accuracy 8, MAE 9, and Within-1 Accuracy 0 on the 672-conversation anonymized set (Suh et al., 19 Sep 2025).
A major empirical finding is that displayed empathy and perceived empathy do not coincide. In “The Illusion of Empathy,” 155 conversations drawn from human–human and human–chatbot settings show that chatbots are rated significantly lower than humans on general empathy and state empathy dimensions, even though they receive slightly higher ratings of conversation quality. GPT-4o annotations mirror this gap, whereas several language-based empathy models fail to recover it (Liu et al., 2024). This result has become central in Empathy-QA because it cautions against optimizing only for surface empathetic language.
Automatic measurement itself remains contested. Prompting-based measurement with GPT-4 and Flan-family models performs relatively poorly on public and internal empathy datasets, whereas instruction-finetuned Flan-T5 classifiers achieve stronger Macro F1, up to 78.3% on the internal Empeval dataset (Xu et al., 2024). In spoken dialogue, EmpathyEval moves from scalar-only judgments to natural-language assessments plus four regression scores—Need Support, Wording Appropriateness, Emotion Understanding, and Emotional Support—with reported high correlation to human MOS and GPT-judge scores (Jia et al., 26 Jan 2026).
5. Domain specialization and extensions
Mental health counseling is the most developed application domain. SoulChat is explicitly modeled on psychological counseling and fine-tunes responses closer to the expression of a psychological consultant (Chen et al., 2023). Empathy-QA targets single-turn Long Counseling Texts with long, substantive responses written by human counselors, while Kardia-R1 extends the setting to persistent user identity by training on KardiaBench, where each turn makes explicit <understanding>, <reasoning>, <emotion>, and <response> spans (Yuan et al., 1 Dec 2025).
Healthcare introduces anticipatory rather than reactive empathy modeling. The Empathy Applicability Framework classifies patient queries themselves according to whether Emotional Reactions and Interpretations are applicable, using real patient questions from HealthcareMagic and iCliniq. On human-consensus labels, fine-tuned RoBERTa reaches approximately 0.92 Accuracy, Macro-F1, and Weighted-F1 for emotional reactions and approximately 0.87 for interpretations, outperforming heuristic and zero-shot baselines (Randhawa et al., 14 Jan 2026). This reframes Empathy-QA as a pre-response triage problem: systems must decide whether empathy is warranted before they decide how to express it.
Speech and embodiment extend the same agenda beyond text. EchoMind evaluates Speech LLMs through sequential tasks for content understanding, vocal-cue perception, integrated reasoning, and response generation using semantically neutral scripts and controlled vocal styles; even top models remain below 4.0 on speech information relevance and achieve best-in-class Vocal Empathy Scores of 3.34 and 3.29 (Zhou et al., 26 Oct 2025). EmpathyAgent moves from verbal support to action, using 10,000 multimodal VirtualHome samples and evaluating scenario understanding, empathetic planning, and executable empathetic actions. On the empathetic action task, instruction-finetuned Llama3-8B substantially improves Overlap, TF-IDF, and LCS relative to the base model (Chen et al., 19 Mar 2025).
Safety-oriented benchmarking adds another specialization. EMPATH evaluates emotional-support chatbots through multilingual, multi-turn auditor-generated conversations scored on 19 metrics across crisis handling, therapeutic quality, conversational integrity, emotional safety, and cultural adaptation. Under its standard rubric, 93% of scores fall within plus or minus 1 across a second, cross-family judge, yet five-run test-retest experiments show that crisis-metric scores can swing from 2 to 10 on identical reruns (Sartori, 29 Jun 2026). In this formulation, run-to-run reliability is itself treated as a safety property.
6. Limitations, misconceptions, and open problems
A persistent misconception in Empathy-QA is that empathetic language alone is sufficient. Multiple studies argue otherwise. User-centered analyses show that a single poor turn can sharply lower overall empathy judgments and future use intent, and that Cognitive Understanding and Response Appropriateness often matter more than overt affective phrasing (Suh et al., 19 Sep 2025). The chatbot-perception study likewise shows that generic or polished responses can coincide with lower perceived empathy than human conversation partners (Liu et al., 2024). This suggests that empathy failures often arise from unmet expectations, loss of conversational continuity, or mismatched actionability rather than from absence of supportive wording.
Another open problem is the absence of a single correct empathetic response. Spoken-dialogue work explicitly argues that rigid supervised signals are fundamentally limited because there is no single “correct” response and a simple numerical score cannot fully capture the nuances of emotional expression or appropriateness (Jia et al., 26 Jan 2026). The intersectional empathy framework reaches a related conclusion from another direction: by varying protected attributes in otherwise identical prompt templates, it measures bias as variance in cognitive empathy, affective empathy, and response appropriateness across groups, but reports only small initial variance and calls for more refined statistical methods and evaluation samples (Formanek et al., 2024).
Long-horizon robustness is also unresolved. Empathetic expert adapters outperform system prompts because prompt effects diminish as conversations lengthen, and adapter-based control better preserves empathy patterns across turns (Shayegani et al., 5 Nov 2025). EMPATH shows that system-level safety cannot be inferred from one-off averages because identical seeds may yield materially different conversations on rerun (Sartori, 29 Jun 2026). The implication for Empathy-QA is that evaluation must be conversation-level, multi-turn, and reliability-aware.
Finally, the field remains strongly shaped by culture, annotation practice, and domain assumptions. EAF highlights implicit distress, clinical-severity ambiguity, and contextual hardship as persistent annotation challenges and recommends multi-annotator modeling, clinician-in-the-loop calibration, and culturally diverse annotation (Randhawa et al., 14 Jan 2026). Empathy in academic spaces and computer-mediated communication further emphasizes that empathy depends on contextual information, communication mechanisms, and what interlocutors choose not to surface (Merrill et al., 2024, Lee, 2019). In that sense, Empathy-QA remains a moving target: not a single benchmark problem, but a family of tasks concerned with measuring, eliciting, and sustaining contextually appropriate understanding and support across text, speech, action, and safety-critical interaction.
