LongEmotion Benchmark: Evaluating Long-Context EI

• LongEmotion is a benchmark that evaluates the emotional intelligence of large language models in extended, realistic contexts through tasks like emotion classification, detection, QA, conversation, summary, and self-expression.
• It challenges models with lengthy, noisy inputs by requiring them to recognize, track, and express subtle emotional cues distributed across diverse and complex scenarios.
• Empirical results show that retrieval augmentation techniques (RAG and CoEM) improve performance by isolating sparse emotional evidence from long, context-rich datasets.

Searching arXiv for LongEmotion and closely related long-context emotion benchmarks and methods. arxiv_search(query="LongEmotion measuring emotional intelligence of LLMs in long-context interaction", max_results=5) arxiv_search(query="emotional support trajectory benchmark long-term LLMs BEL ETV ECP", max_results=5) arxiv_search(query="MemEmo evaluating emotion in memory systems of agents HLME", max_results=5) LongEmotion is a benchmark introduced to measure the emotional intelligence of LLMs in long-context, realistic interaction settings. It is motivated by the claim that existing benchmarks overemphasize short, explicit, and relatively clean emotional inputs, while practical emotionally intelligent systems must recognize, track, summarize, and express emotion across lengthy, diverse, and often noisy contexts. LongEmotion therefore evaluates six long-context tasks—Emotion Classification, Emotion Detection, Emotion QA, Emotion Conversation, Emotion Summary, and Emotion Expression—with an average input length of 8,777 tokens overall, and with long-form generation required for Emotion Expression (Liu et al., 9 Sep 2025).

1. Conceptual scope and motivation

LongEmotion is positioned against two limitations in prior evaluation practice. First, many earlier emotional intelligence benchmarks are described as relying on short texts with obvious emotional cues, which understates the difficulty of real interaction, where emotion may be subtle, distributed across many turns, and mixed with irrelevant content. Second, existing long-context benchmarks are said to focus mainly on general reasoning, retrieval, or comprehension rather than emotional intelligence. In this sense, LongEmotion defines a specific evaluation regime for long-context EI rather than a general long-context language benchmark (Liu et al., 9 Sep 2025).

The benchmark treats emotional intelligence as a multi-component capability. Its task design explicitly spans emotion perception, emotional understanding, emotionally informed reasoning, emotionally supportive dialogue, summarization of emotionally salient material, and long-form emotional self-expression. This distribution of tasks matters because the paper argues that EI is not exhausted by recognition accuracy; it also includes the ability to sustain emotional coherence over time, apply emotional knowledge, and generate emotionally consistent outputs under long-context constraints (Liu et al., 9 Sep 2025).

A central implication of this framing is that LongEmotion is not merely a sentiment benchmark with longer inputs. It is intended to test whether a model can remain emotionally grounded when relevant cues are sparse, delayed, partially obscured by noise, or embedded in extended interaction. The benchmark’s use of counseling-style dialogue, psychological literature, pathology reports, and structured emotional narration places it closer to practical emotionally intelligent assistance than to short-form affect classification (Liu et al., 9 Sep 2025).

2. Task suite and dataset construction

LongEmotion contains six tasks designed to cover both long-input understanding and long-output generation. The benchmark overview reported for each task is as follows (Liu et al., 9 Sep 2025).

Task	Construction and scale	Primary metric
Emotion Classification	EmoBench snippets inserted into BookCorpus; avg length 16,691; 200 samples	Accuracy
Emotion Detection	Covid-worry segments arranged as $N+1$ emotional segments with one mismatch; avg length 4,106; 136 samples	Accuracy
Emotion QA	Questions grounded in long-context psychological literature; avg length 11,207; 120 QA pairs	F1
Emotion Conversation	CPsyCoun dialogues expanded into four stages; avg length 4,856; 100 samples	GPT-4o with 12 counseling metrics
Emotion Summary	CPsyCounR pathology reports expanded to long context; avg length 3,129; 150 samples	GPT-4o on factual consistency, completeness, and clarity
Emotion Expression	EmotionBench scenarios for long-form emotional self-narrative; avg generated length 8,546; 428 samples	GPT-4o on six dimensions

Emotion Classification is formulated as a long-context “needle in a haystack” problem. Emotional snippets from EmoBench are inserted into passages from BookCorpus, with manual adjustment to preserve syntactic and contextual coherence and with proper nouns modified to avoid identity overlap. The task is to predict the emotional category of the target entity despite substantial irrelevant context (Liu et al., 9 Sep 2025).

Emotion Detection presents $N+1$ emotional segments, where $N$ share one emotion and one segment differs. The segments are grouped from Covid-worry by emotion label and then contrastively composed. This tests whether a model can identify a subtle emotional outlier among emotionally similar segments rather than merely detect overt sentiment polarity (Liu et al., 9 Sep 2025).

Emotion QA is grounded in long-context psychological literature. The construction pipeline consists of expert-written questions, refinement of reference answers for clarity and F1 compatibility, and filtering of examples deemed too ambiguous or too trivial. The result is a dataset of 120 QA pairs aimed at emotionally informed factual understanding over long context (Liu et al., 9 Sep 2025).

Emotion Conversation is based on CPsyCoun and expands 100 emotionally rich dialogues into four stages: Reception and Inquiry, Diagnostic, Consultation, and Consolidation and Ending. Evaluation occurs at three checkpoints per stage—quarter point, halfway point, and three-quarter point—so the task measures preservation of emotional consistency and counseling quality over the course of a long interaction rather than at a single terminal response (Liu et al., 9 Sep 2025).

Emotion Summary uses CPsyCounR pathology reports and requires models to summarize causes, symptoms, treatment process, illness characteristics, and treatment effects. Emotion Expression places the model in a specified emotional situation and requires a long-form emotional self-narrative structured into Immediate Reaction, Cognitive Appraisal, Emotional and Physiological Expression, Regulation Strategies, and Reflective Integration, with a preceding psychometric self-assessment such as PANAS (Liu et al., 9 Sep 2025).

3. Evaluation protocol and emotionally aware retrieval frameworks

LongEmotion compares three evaluation settings: Base, RAG, and CoEM. The Base setting is standard prompt-based evaluation. The RAG setting is Retrieval-Augmented Generation, but with an important modification: retrieval does not rely on an external knowledge base. Instead, the retrieval source is the conversation context and the LLM itself, which is used as a source of emotional reasoning or enrichment. This design is intended to remain grounded in the given interaction rather than outsource the task to external corpora (Liu et al., 9 Sep 2025).

In the RAG setup, the context is chunked, similarity is computed between the query and the chunks, top chunks are retrieved, and the output is generated from the retrieved material. Task-specific parameters differ, and for Emotion Conversation the paper reports that RAG is used only in the fourth stage because the first three stages are relatively short. The stated intuition is that retrieval reduces noise by filtering the long context down to emotionally relevant evidence (Liu et al., 9 Sep 2025).

CoEM, or Collaborative Emotional Modeling, is the paper’s more elaborate framework. It decomposes processing into five stages: Chunking, Initial Ranking, Multi-Agent Enrichment, Re-Ranking, and Emotional Ensemble Generation. Initial Ranking and Re-Ranking are performed by CoEM-Rank; Multi-Agent Enrichment is performed by CoEM-Sage; and final task output is produced by CoEM-Core. The retrieval model is bge-m3, while CoEM-Sage uses GPT-4o for Emotion Classification, Emotion Detection, Emotion QA, and Emotion Expression, and DeepSeek-V3 for Emotion Conversation-4 and Emotion Summary (Liu et al., 9 Sep 2025).

The distinguishing claim of CoEM is that it augments retrieval with emotional analysis, psychological priors, and limited knowledge injection. The paper explicitly states that the knowledge injection is limited and not intended to leak task answers. RAG is described as comprising Chunking, Re-Ranking, and Emotional Ensemble Generation, whereas CoEM uses the full five-stage pipeline. This makes CoEM an iterative retrieval-plus-reasoning framework rather than a pure retrieval filter (Liu et al., 9 Sep 2025).

The evaluation protocol also varies by task type. Emotion Classification and Emotion Detection use accuracy; Emotion QA uses F1; and Emotion Conversation, Emotion Summary, and Emotion Expression use GPT-4o as an automatic judge with task-specific rubrics. For Emotion Conversation, the paper uses 12 counseling metrics derived from CBT, ACT, Humanistic Therapy, Existential Therapy, and Satir Family Therapy. The reported alignment between GPT-4o and human annotators is Pearson correlation $= 0.934$ with $p = 0.066$ (Liu et al., 9 Sep 2025).

4. Empirical findings and model-level patterns

The paper’s main empirical result is that both RAG and CoEM generally improve performance over the Base prompting method on most tasks. The gains are especially marked in tasks that require the model to isolate sparse emotional evidence from long noisy context. For example, GPT-4o-mini on Emotion Classification improves from 28.50 in Base to 38.33 with RAG and 48.00 with CoEM, while on Emotion Detection it improves from 16.42 in Base to 21.57 with RAG, with CoEM at 20.59. DeepSeek-V3 on Emotion Classification improves from 44.00 in Base to 52.17 with RAG and 54.33 with CoEM (Liu et al., 9 Sep 2025).

The results are not uniformly monotonic across tasks. The paper notes that Emotion QA and Emotion Summary are more context-faithful tasks, and that excessive emotional enrichment can introduce harmful noise. In those settings, RAG often helps by identifying relevant content, whereas CoEM can reduce F1 or summary quality when knowledge injection alters details. This is one of the benchmark’s important findings because it shows that emotionally aware augmentation is beneficial only when it remains compatible with faithfulness constraints (Liu et al., 9 Sep 2025).

Emotion Conversation is treated as a particularly revealing task because later stages require stronger emotional continuity. The paper reports that models such as Qwen3-8B and Llama3.1-8B-Instruct sometimes outperform GPT-4o on the conversation task, which is used to argue that EI performance is not simply correlated with overall model scale or general capability. This observation directly supports the benchmark’s claim that emotional intelligence is a distinct evaluation axis rather than a byproduct of general language-model strength (Liu et al., 9 Sep 2025).

The comparative case study across GPT-4o-mini, GPT-4o, and GPT-5 further sharpens that point. GPT-5 is described as strongest in general reasoning and in the use of psychological theory, but also as more rigid, mechanical, or less empathetic. GPT-4o-mini is described as often more human-like and richer in emotional texture, while weaker in theory application and structured reasoning. GPT-4o is presented as more balanced between theoretical structure and expressive naturalness. In Emotion QA, GPT-5 may lose F1 because it paraphrases rather than remaining literal; in Emotion Expression, GPT-4o-mini may sound more like a real person, whereas GPT-5 is more comprehensive but sometimes too formal (Liu et al., 9 Sep 2025).

These findings support a substantive claim about benchmark design: EI in LLMs is multi-dimensional. A model may be analytically strong yet emotionally less natural, or emotionally vivid yet less theoretically grounded. LongEmotion is therefore structured to expose such trade-offs rather than collapse them into a single short-form empathy score (Liu et al., 9 Sep 2025).

5. Position within long-context emotion research

LongEmotion belongs to a broader movement from short, static affect evaluation toward long-horizon emotional modeling, but it occupies a specific niche. It measures emotional intelligence under long-context interaction through six tasks averaging 8,777 tokens, whereas "Detecting Emotional Dynamic Trajectories: An Evaluation Framework for Emotional Support in LLMs" shifts from snapshot-based evaluation to trajectory-based assessment in 40-turn dialogues with 328 emotional contexts and 1,152 disturbance events, using trajectory-level metrics such as BEL, ETV, and ECP (Tan et al., 12 Nov 2025). The two benchmarks are adjacent rather than interchangeable: LongEmotion emphasizes long-context EI task diversity, while trajectory-based support evaluation emphasizes state evolution and stabilization over time.

LongEmotion is also complementary to emotion-aware memory benchmarks. "MemEmo: Evaluating Emotion in Memory Systems of Agents" introduces HLME to evaluate emotional information extraction, emotional memory updating, and emotional memory question answering in prolonged interaction, and reports that no evaluated memory system is robust across all three tasks (Liu et al., 27 Feb 2026). This comparison situates LongEmotion as a benchmark for end-task EI behavior under long context, whereas MemEmo isolates the memory-system substrate required for affect-aware persistence.

A further connection lies with adaptive emotional reasoning. "Emotion-o1: Adaptive Long Reasoning for Emotion Understanding in LLMs" argues that fixed-length chain-of-thought is mismatched to the heterogeneous complexity of emotion tasks and proposes variable-length reasoning trained with supervised fine-tuning and reinforcement learning, yielding consistent improvements across sentiment, emotion, humor, and sarcasm tasks (Song et al., 28 May 2025). This suggests that LongEmotion-style performance, especially on QA, Conversation, and Summary, is likely sensitive to reasoning depth control rather than only to retrieval.

The benchmark also resonates with long-sequential and privacy-constrained emotion analysis outside text-only interaction. "EALD-MLLM: Emotion Analysis in Long-sequential and De-identity videos with Multi-modal LLM" argues that long sequential videos reveal more authentic emotions than short snippets and that Non-Facial Body Language is an important identity-free cue (Li et al., 2024). That work reinforces the broader thesis shared by LongEmotion: emotionally realistic evaluation requires longer temporal horizons and more challenging conditions than conventional short-form affect classification.

6. Limitations, misconceptions, and open problems

LongEmotion explicitly acknowledges several limitations. The paper states that future work should evaluate more open-source and closed-source models; some tasks, especially Emotion Conversation, rely on expanded or synthetic generation pipelines; multiple tasks depend on GPT-4o as an automatic judge; CoEM can inject harmful noise, particularly in QA and Summary; and the benchmark still cannot fully capture all real-world emotional complexity (Liu et al., 9 Sep 2025).

One common misconception is that stronger theoretical reasoning necessarily yields better emotional intelligence. The GPT case study argues against this. GPT-5 often performs well on theory-driven metrics, yet the benchmark reports that it can appear rigid or less empathetic, while GPT-4o-mini can sound more human-like despite weaker structured reasoning. LongEmotion therefore treats theoretical correctness and emotionally natural expression as related but non-identical properties (Liu et al., 9 Sep 2025).

A second misconception is that long-context emotion evaluation can be reduced to retrieval. The benchmark shows that retrieval is helpful, but not sufficient. RAG often improves by reducing noise, yet CoEM demonstrates that emotionally informed enrichment can further help on many tasks. At the same time, CoEM can also hurt context-faithful tasks when enrichment distorts details. The benchmark’s results therefore argue for a balance between grounding, emotional reasoning, and faithfulness rather than a single universal enhancement mechanism (Liu et al., 9 Sep 2025).

A broader implication is that LongEmotion advances realistic EI evaluation without exhausting it. It does not directly evaluate trajectory-level emotional regulation, parameterized emotional memory updating, or multimodal long-sequential cues. Instead, it provides a long-context task suite for measuring how well models recognize, reason about, support, summarize, and express emotion under noisy extended contexts. In that sense, it is best understood as a benchmark for practical long-context emotional intelligence, and as part of a larger research agenda connecting long reasoning, emotional memory, trajectory-based support, and long-horizon affective interaction (Liu et al., 9 Sep 2025).