Emotion Transcription in Conversation (ETC)

• ETC is the generation of free-form emotion descriptions from conversational context, expanding beyond fixed categorical labels.
• The task uses context-to-text methods, balancing rich descriptive representation with evaluation metrics like BLEU, ROUGE, and fine-grained faithfulness scores.
• ETC benchmarks reveal challenges in modeling implicit emotions and encourage integration with multimodal, speaker-aware techniques for richer understanding.

Searching arXiv for the cited ETC/ERC papers to ground the article in the current literature. I’ll look up the relevant arXiv records now. Emotion Transcription in Conversation (ETC) is a dialogue-emotion task in which a model generates a natural-language description of a speaker’s emotional state for each utterance, conditioned on the conversational context, rather than assigning only a categorical label (Tanaka et al., 7 Mar 2026). It is positioned as a response to the limitations of conventional Emotion Recognition in Conversation (ERC), where categorical or dimensional annotations can fail to represent subtle, mixed, context-dependent, or culturally specific affective states (Tanaka et al., 7 Mar 2026). Within the broader ERC literature, ETC also intersects with work on context-sensitive speaker modeling, commonsense inference, multimodal fusion, and transfer learning, because all of these lines of research address the underlying problem of inferring emotion from conversational structure rather than from isolated utterances alone (Fu et al., 2024).

1. Conceptual definition and relation to ERC

ETC is defined as generating “natural language descriptions that accurately reflect speakers' emotional states within conversational contexts” (Tanaka et al., 7 Mar 2026). In the ETC formulation, the relevant question is not only which emotion category applies to an utterance, but “How would we describe the speaker’s internal emotional state in natural language, given the conversational context?” (Tanaka et al., 7 Mar 2026). The task therefore changes the output space from a fixed inventory of labels to free-form text.

Traditional ERC, by contrast, is formulated as utterance-level emotion prediction in a dialogue. One formulation represents a conversation as a sequence $U=[(u_1,p_1),(u_2,p_2),\dots,(u_n,p_n)]$, with the goal of predicting an emotion label $e_i$ for each utterance $u_i$ (Fu et al., 2024). A multimodal variant writes the conversation as $C = [(u_1,y_1),(u_2,y_2),\dots,(u_K,y_K)]$, where each utterance $u_k=\{S_k,T_k\}$ includes speech and transcript modalities (Dutta et al., 20 Jan 2025). ETC preserves the context dependence central to ERC, but replaces label prediction with context-to-text generation (Tanaka et al., 7 Mar 2026).

The ETC paper identifies three explicit differences from traditional ERC: output format, expressiveness, and evaluation (Tanaka et al., 7 Mar 2026). ERC outputs a label; ETC outputs a natural-language description. ERC is constrained by a fixed label inventory; ETC can express mixed, nuanced, and implicit emotions. ERC is usually evaluated with classification metrics; ETC is evaluated as a generation task using text-generation metrics and a fine-grained faithfulness score (Tanaka et al., 7 Mar 2026). This suggests ETC is not merely a relabeling of ERC, but a reformulation that treats emotional understanding as descriptive inference.

2. Motivation: expressive emotion representation in dialogue

The ETC benchmark is motivated by the claim that existing ERC methods “predominantly employ categorical or dimensional emotion annotations, which often fail to adequately represent complex, subtle, or culturally specific emotional nuances” (Tanaka et al., 7 Mar 2026). The paper emphasizes that real emotions in dialogue are frequently subtle, mixed or blended, context-dependent, and sometimes culturally specific (Tanaka et al., 7 Mar 2026). Because ETC uses free-form descriptions, it can represent “what the speaker feels,” “why they feel it,” “how several emotions coexist,” and whether the state is tied to intention, empathy, relief, or shock (Tanaka et al., 7 Mar 2026).

This concern is consistent with earlier ERC research that treats conversation as a context-sensitive inference problem. TL-ERC argues that an utterance’s emotion often depends on earlier utterances, speaker identity, self-influence, inter-speaker influence, conversation topic, and latent pragmatic factors (Hazarika et al., 2019). DialogueTRM likewise distinguishes individual context and conversational context, formalizing the difference between same-speaker history and all-speaker history (Mao et al., 2020). LaERC-S, described in the provided details as CKERC, further argues that emotion should be inferred from the speaker’s latent mental state, intended effect, and likely reactions from others, rather than from text alone (Fu et al., 2024).

A common misconception is that ETC is simply a more verbose form of classification. The ETC paper argues otherwise by showing that emotion transcriptions differ lexically from utterances themselves: utterances tend to contain event or action words, while emotion transcriptions tend to contain internal feeling words and nuanced descriptors (Tanaka et al., 7 Mar 2026). This supports the view that ETC is not paraphrasing dialogue turns, but modeling the speaker’s internal state.

3. Formal task specification and dataset design

The ETC paper formalizes a dialogue as

$D = ((u_1, s_1), (u_2, s_2), \dots, (u_N, s_N)),$

where $u_i$ is the $i$-th utterance and $s_i$ is the speaker of that utterance, with $N = 10$ in the dataset (Tanaka et al., 7 Mar 2026). For any position $e_i$0, the context up to that utterance is

$e_i$1

and the model $e_i$2 is trained to generate

$e_i$3

where $e_i$4 is a natural-language description of the emotional state of speaker $e_i$5 at the time of uttering $e_i$6 (Tanaka et al., 7 Mar 2026). The task is therefore explicitly a context-to-text generation problem.

The dataset is a Japanese text-based dialogue corpus collected using CrowdWorks (Tanaka et al., 7 Mar 2026). It uses a speaker-listener setup inspired by EmpatheticDialogues, with one participant assigned the Speaker role and the other the Listener role (Tanaka et al., 7 Mar 2026). Each dialogue contains 5 turns per person, for 10 utterances total, with alternating turns beginning from the Speaker (Tanaka et al., 7 Mar 2026). Speakers were given one of 32 emotion labels, described as Japanese translations of the labels used in EmpatheticDialogues, and were instructed to talk about a personal experience related to that emotion (Tanaka et al., 7 Mar 2026).

After each utterance, both participants wrote a free-form natural-language description of their own emotional state at that moment (Tanaka et al., 7 Mar 2026). The paper defines an emotion transcription as “a verbalization of the internal emotional state or intention a participant held at the time of their utterance” (Tanaka et al., 7 Mar 2026). The resulting corpus contains 1,002 dialogues, 10,020 utterances / emotion transcriptions, and 199 unique crowdworkers (Tanaka et al., 7 Mar 2026). The average utterance length is 42.72 characters, with 44.64 for Speakers and 40.80 for Listeners, while the average emotion transcription length is 28.89 characters, with 28.92 for Speakers and 28.86 for Listeners (Tanaka et al., 7 Mar 2026).

The dialogues were roughly balanced across the 32 emotion prompts, with about 30–32 dialogues per emotion label (Tanaka et al., 7 Mar 2026). This suggests the corpus was designed to diversify affective starting conditions while preserving a fixed interactional structure.

4. Annotation scheme, corpus analysis, and empirical properties

Because free-form transcriptions are difficult to analyze quantitatively, the ETC dataset additionally annotates each transcription with emotion categories (Tanaka et al., 7 Mar 2026). The category inventory contains 7 labels: joy, sadness, fear, anger, surprise, disgust, and neutral, corresponding to Ekman’s six basic emotions plus neutral (Tanaka et al., 7 Mar 2026). Annotation is multi-label rather than single-label, so one transcription may express multiple emotions (Tanaka et al., 7 Mar 2026).

Each transcription was labeled by 3 annotators who saw the full dialogue context and the transcription (Tanaka et al., 7 Mar 2026). For each of the 7 categories, annotators answered whether it was present. Final labels were assigned by majority vote: a label was assigned if 2 or more of 3 annotators agreed, and if no category reached agreement, the transcription was labeled neutral only (Tanaka et al., 7 Mar 2026). The paper reports an overall Fleiss’ kappa of $e_i$7, interpreted as moderate agreement (Tanaka et al., 7 Mar 2026).

The corpus statistics show a strong neutral component together with substantial non-neutral affect. The overall distribution is Neutral 50.3%, Joy 25.0%, Sadness 9.8%, Fear 6.2%, Anger 3.2%, Surprise 3.5%, and Disgust 4.8% (Tanaka et al., 7 Mar 2026). Speakers show more labeled emotion than listeners, while surprise appears more often for listeners than speakers (Tanaka et al., 7 Mar 2026). About 5.6% of non-neutral transcriptions contain multiple emotion labels (Tanaka et al., 7 Mar 2026). This multi-emotion rate provides direct evidence for the ETC claim that dialogue emotions can be mixed or overlapping.

Aspect	ETC dataset property
Dialogue structure	5 turns per person, 10 utterances total
Emotion transcription	Free-form natural-language description after each utterance
Category annotation	7-label multi-label annotation with majority vote
Corpus size	1,002 dialogues; 10,020 utterances / emotion transcriptions
Inter-annotator agreement	Fleiss’ kappa $e_i$8
Multi-emotion transcriptions	About 5.6% of non-neutral transcriptions

The annotation design also preserves a bridge back to ERC. Because each transcription is associated with category labels, the dataset supports both natural-language generation and conventional classification-style analysis (Tanaka et al., 7 Mar 2026). A plausible implication is that ETC can function both as a richer benchmark and as a resource for studying the relation between free-form affect descriptions and standard emotion inventories.

5. Benchmark models and evaluation methodology

The ETC benchmark evaluates GPT-4.1 and Llama-3.1-Swallow-8B-Instruct-v0.3 in zero-shot and 4-shot settings, and additionally evaluates supervised fine-tuning of Llama-3.1 on the ETC dataset (Tanaka et al., 7 Mar 2026). The dataset split is 80% train, 10% validation, and 10% test (Tanaka et al., 7 Mar 2026). Fine-tuning was performed on 2 NVIDIA A100 80GB GPUs with batch size 8 for 2 epochs, using 4-bit quantization and PagedAdamW8bit, with validation every 200 steps (Tanaka et al., 7 Mar 2026). The learning rate was selected from $e_i$9, warm-up steps from $u_i$0, and the best setting was $u_i$1 with 300 warm-up steps (Tanaka et al., 7 Mar 2026).

Evaluation uses BLEU, ROUGE, and BERTScore, together with a fine-grained faithfulness evaluation inspired by FActScore (Tanaka et al., 7 Mar 2026). In the faithfulness protocol, the emotion transcription is decomposed into atomic units, each containing one piece of emotional information, and each unit is judged as Supported, Not Supported, or Neutral (Tanaka et al., 7 Mar 2026). Precision is the proportion of predicted atomic units supported by the ground truth, Recall is the proportion of ground-truth atomic units supported by the prediction, and F1 is the harmonic mean of Precision and Recall (Tanaka et al., 7 Mar 2026). If the ground-truth transcription has no emotional description, it is excluded from this metric; if the model outputs nothing while the reference has emotion content, both Precision and Recall are set to 0 (Tanaka et al., 7 Mar 2026).

The ETC results show that fine-tuning Llama-3.1 on the dataset improves performance across all automatic metrics (Tanaka et al., 7 Mar 2026). The fine-tuned model is best on BLEU, ROUGE, BERTScore, Precision, and F1, with a best reported F1 of 14.29 (Tanaka et al., 7 Mar 2026). The fine-grained evaluation further shows that zero-shot models, especially GPT-4.1, tend to generate more atomic units, which increases Recall but lowers Precision (Tanaka et al., 7 Mar 2026). The reported average atomic-unit counts are 1.35 for ground truth, 1.39 for fine-tuned Llama-3.1, 1.99 for Llama-3.1 zero-shot, and 3.39 for GPT-4.1 zero-shot (Tanaka et al., 7 Mar 2026). This suggests that ETC models must balance informativeness against overgeneration.

A central finding is that current models still struggle with implicit emotional states (Tanaka et al., 7 Mar 2026). The paper presents a case in which the literal dialogue concerns a negative event, but the true emotion transcription is happiness because of the partner’s empathetic response; most models focus on the explicit negative cues and miss the emergent positive emotion (Tanaka et al., 7 Mar 2026). This is consistent with the ETC claim that affect in dialogue can emerge from interaction rather than from lexical polarity alone.

6. Technical antecedents in ERC: speaker modeling, multimodality, and transfer

Although ETC is defined as a generation task, several ERC lines of work provide technical antecedents for the kinds of inference ETC requires.

LaERC-S, described in the provided details as CKERC, uses LLM-generated commonsense conditioned on dialogue history to improve utterance-level emotion prediction (Fu et al., 2024). It centers on ATOMIC relations in the mental-state category, specifically xIntent, xReact, and oReact, and uses Llama-2-7B-Chat to generate commonsense descriptions from the historical utterances preceding a target utterance (Fu et al., 2024). The framework is trained in two stages: speaker commonsense identification with loss

$u_i$2

followed by ERC fine-tuning with

$u_i$3

It reports weighted-F1 scores of 72.40 on IEMOCAP, 42.08 on EmoryNLP, and 69.27 on MELD, compared with InstructERC scores of 71.39, 41.37, and 69.15 respectively, for an average weighted-F1 of 61.25 versus 60.64 (Fu et al., 2024). In ETC terms, this work is relevant because it treats emotion as an inference over mental state, speaker intention, and listener reaction rather than as a surface-text label.

MERITS-L extends the ETC-related logic to multimodal ERC by exploiting transcribed emotional speech (Dutta et al., 20 Jan 2025). It first uses Whisper-large-v3 to transcribe MSP-PODCAST, then prompts GPT-3.5 Turbo for positive, negative, or neutral pseudo-labels, and fine-tunes RoBERTa-large on the resulting pseudo-labeled transcripts (Dutta et al., 20 Jan 2025). The downstream model combines text embeddings with CARE speech embeddings through a three-stage hierarchical process: utterance-level unimodal classification, conversation-level contextual modeling with a Bi-GRU with self-attention, and co-attention-based multimodal fusion (Dutta et al., 20 Jan 2025). On IEMOCAP, MELD, and CMU-MOSI, the final Audio+Text Stage III model reports weighted F1 of 86.48, 66.02, and 86.81 respectively (Dutta et al., 20 Jan 2025). The paper further reports that GPT-3.5-supervised RoBERTa improves over plain pre-trained RoBERTa by +8.22% on IEMOCAP, +5.01% on MELD, and +21.9% on CMU-MOSI (Dutta et al., 20 Jan 2025). This indicates that transcript-based supervision can be a strong intermediate representation for conversational emotion modeling.

DialogueTRM approaches the same problem from multimodal context modeling (Mao et al., 2020). It represents conversational input as

$u_i$4

distinguishes individual context $u_i$5 from conversational context $u_i$6, and uses a Hierarchical Transformer plus Multi-Grained Interactive Fusion (Mao et al., 2020). Text is modeled with a context-dependent sequential Transformer, while visual and acoustic modalities use a context-free setting that behaves effectively as feed-forward (Mao et al., 2020). DialogueTRM reports improvements over prior SOTA of +4.3% ACC and +7.1% F1 on IEMOCAP, and +10.4% ACC and +4.5% F1 on MELD (Mao et al., 2020). For ETC, the significance lies in its explicit treatment of modality-specific temporal behavior and speaker-specific versus full-conversation context.

TL-ERC provides a transfer-learning perspective (Hazarika et al., 2019). It pretrains a Hierarchical Recurrent Encoder-Decoder on large conversational corpora and transfers the context encoder to an ERC classifier with BERT-based utterance representations (Hazarika et al., 2019). On IEMOCAP, weighted F-score rises from 53.8 with random initialization and 55.1 with BERT only to 58.8 with TL-ERC + Ubuntu context and 58.5 with TL-ERC + Cornell context (Hazarika et al., 2019). The method is also evaluated under 10%, 25%, 50%, and 100% data conditions and is reported to be more robust in low-data settings (Hazarika et al., 2019). This suggests that conversational pretraining may be especially relevant for ETC, where supervision is richer but harder to scale than categorical labels.

7. Research significance, limitations, and open directions

ETC reframes dialogue emotion understanding as generation of internal-state descriptions rather than classification over a fixed label set (Tanaka et al., 7 Mar 2026). The benchmark demonstrates that supervised fine-tuning improves performance, but also that performance remains low overall and that implicit emotions remain difficult even for strong models (Tanaka et al., 7 Mar 2026). The ETC paper therefore positions the task as a difficult benchmark rather than a solved extension of ERC.

Several limitations are explicit in the ETC benchmark. The current dataset is Japanese-only, so generalization to other languages and cultures remains open (Tanaka et al., 7 Mar 2026). The authors also point to the need for better modeling of implicit emotion, advanced prompting and training such as chain-of-thought prompting, RLHF, and contrastive learning, speaker modeling through speaker embeddings, personality traits, and dynamic speaker state, greater use of human evaluation, and multimodal ETC with audio and visual cues (Tanaka et al., 7 Mar 2026). These proposed directions align with the ERC literature summarized above: speaker characteristics in LaERC-S (Fu et al., 2024), transcript-based multimodal learning in MERITS-L (Dutta et al., 20 Jan 2025), differentiated modality fusion in DialogueTRM (Mao et al., 2020), and conversational transfer in TL-ERC (Hazarika et al., 2019).

A further misconception is that ETC supersedes ERC. The ETC paper instead makes the dataset usable for conventional ERC-style classification experiments by adding emotion category labels to each transcription (Tanaka et al., 7 Mar 2026). ETC is therefore better understood as an expanded formulation that can connect back to ERC quantitatively while preserving a richer descriptive target. A plausible implication is that ETC may serve as a testbed for studying whether models that can describe emotional states in natural language also acquire more faithful latent representations for downstream ERC.

In that sense, ETC occupies a distinctive position in dialogue affect analysis. It retains ERC’s emphasis on utterance-level prediction in context, but demands a representation of emotion that is compositional, context-sensitive, interaction-aware, and linguistically explicit (Tanaka et al., 7 Mar 2026). The benchmark’s broader significance lies in making those requirements measurable.