ProEmoTrans: Zero-Shot Emotion Transfer Framework

• ProEmoTrans is a prototype-based emotion transfer framework that enables zero-shot emotion recognition by transferring both semantic and sequential information from seen to unseen labels.
• It leverages LLM-enhanced emotion descriptions and parameter-free Gaussian Self-Attention to capture subtle, implicit emotion expressions in sequential dialogues.
• Attention Viterbi Decoding combined with CRF-based transition modeling effectively bridges different emotion taxonomies, resulting in improved performance on benchmark datasets.

ProEmoTrans is a prototype-based emotion transfer framework introduced for Unseen Emotion Recognition in Conversation (UERC), a zero-shot formulation in which a model is trained on conversations labeled with seen emotions and tested on conversations whose labels belong to a disjoint unseen emotion set (Peng et al., 27 Aug 2025). It is designed for the setting where standard Emotion Recognition in Conversation (ERC) becomes brittle because emotion taxonomies differ across datasets and because conversational emotion is both implicit and sequential. The framework addresses this by combining prototype-based label semantics, LLM-enhanced emotion descriptions, parameter-free context aggregation, and transition-aware decoding, thereby transferring both semantic and sequential knowledge from seen to unseen emotions.

1. Problem formulation and conceptual motivation

ProEmoTrans is built on the observation that most ERC systems assume a closed label set: training and test labels come from the same categories. The UERC task removes that assumption. A conversation is written as

$U=\{(u_1,t_1),(u_2,t_2),\dots,(u_N,t_N)\},$

where $u_i$ is the $i$-th utterance and $t_i$ is its speaker. The training data are denoted $D_s$ with seen emotion set $E_s$, the test data are denoted $D_u$ with unseen emotion set $E_u$, and the task imposes

$E_s \cap E_u = \varnothing.$

The goal is to learn from conversations labeled with seen emotions and predict the emotion of each utterance in conversations whose labels come from unseen classes (Peng et al., 27 Aug 2025).

The framework is motivated by three difficulties identified for UERC. First, implicit emotion expression makes label semantics underdetermined; a dictionary definition alone is often too weak to represent how an emotion is actually expressed in dialogue. Second, utterance encoding in long conversations is difficult; standard parameterized context modules such as LSTM, GCN, or GAT can overfit to seen emotions and degrade transfer. Third, emotion transition mismatch arises because emotions have sequential dependencies, but transition patterns learned on seen labels cannot be directly reused at test time when the labels are unseen (Peng et al., 27 Aug 2025).

This formulation makes ProEmoTrans distinct from ordinary zero-shot text classification. The framework does not only align utterances with unseen label descriptions; it also attempts to transfer the Markovian flow nature of emotions from seen to unseen labels through a decoding procedure that incorporates conversation-level transition structure (Peng et al., 27 Aug 2025).

2. Prototype-based emotion transfer and LLM-enhanced descriptions

The central representational device in ProEmoTrans is the emotion prototype. Instead of training a classifier tied to a fixed output space, the framework encodes each emotion label into a vector representation and matches utterance representations against these prototypes in a shared embedding space. This is intended to make unseen labels addressable at test time, provided that they too can be encoded as prototypes (Peng et al., 27 Aug 2025).

For each seen emotion word $e_i^{see}\in E_s$, the framework constructs an input sequence from three sources: the emotion label itself, a dictionary definition from Wiktionary, and LLM-generated implicit examples. The resulting prototype input is

$u_i$0

This sequence is encoded by a prototype encoder: $u_i$1 yielding the prototype set

$u_i$2

where $u_i$3. Unseen emotion prototypes are built analogously as

$u_i$4

where $u_i$5 (Peng et al., 27 Aug 2025).

The enrichment step is called LLM-enhanced Emotion Description (LED). Its prompt is explicitly given as:

“Write two sentences expressing [MASK]'s emotions.”

In experiments, the framework uses ChatGPT-3.5 to generate these implicit descriptions. The role of LED is to bridge the gap between explicit lexical definitions and the more indirect forms through which emotion appears in conversation. In the paper’s framing, this is especially important for complex unseen labels such as powerful, whose conversational realization may not be recoverable from a dictionary gloss alone (Peng et al., 27 Aug 2025).

Prototype matching is performed through a similarity matrix between utterance representations and seen prototypes: $u_i$6 with element-wise form

$u_i$7

The paper states that this is inspired by InfoNCE, since it pulls matched utterance–prototype pairs together and pushes mismatched pairs apart (Peng et al., 27 Aug 2025).

3. Context encoding, sequence modeling, and Attention Viterbi Decoding

Utterance encoding begins by representing each utterance independently: $u_i$8 Let

$u_i$9

To incorporate context without introducing additional trainable context modules, ProEmoTrans uses Gaussian Self-Attention (GSA), a parameter-free mechanism intended to preserve local conversational structure while reducing overfitting to seen emotions (Peng et al., 27 Aug 2025).

For utterance $i$0, the Gaussian attention score is

$i$1

where $i$2. The updated utterance representation is

$i$3

The effect is distance-aware aggregation: nearby utterances receive more focus, while distant ones are down-weighted. The framework presents this as a deliberate alternative to parameterized LSTM, GCN, or GAT context modules, which performed worse in UERC (Peng et al., 27 Aug 2025).

After prototype similarity computation, the model applies a CRF over seen emotions to capture emotion dependencies within the conversation. At training time, this uses the seen-label transition matrix. At test time, however, direct CRF decoding is impossible because the target labels are unseen. ProEmoTrans therefore introduces improved Attention Viterbi Decoding (AVD) to transfer seen emotion transitions to unseen emotions (Peng et al., 27 Aug 2025).

AVD first derives probabilities over seen labels during dynamic programming traversal, then injects this transition-aware seen-emotion information back into each utterance representation: $i$4 The final prediction for an unseen emotion is obtained by nearest-neighbor search against unseen prototypes: $i$5 This mechanism is the point at which the framework’s notion of emotion transfer becomes precise: it transfers semantic and transition information learned on seen emotions to embeddings that can be matched against unseen emotion prototypes (Peng et al., 27 Aug 2025).

4. Experimental protocol, results, and ablations

ProEmoTrans is evaluated on three benchmark datasets—IEMOCAP, EmoryNLP, and MELD—using only the text modality. The paper reports all six source–target settings: $i$6, $i$7, $i$8, $i$9, $t_i$0, and $t_i$1. When training on one dataset and testing on another, the remaining dataset is used for validation (Peng et al., 27 Aug 2025).

Implementation details are explicit. The backbone encoder is BERT-base-uncased; the LLM for emotion descriptions is ChatGPT-3.5; optimization uses AdamW with batch size 4, learning rate $t_i$2, 10 epochs, 100 warmup steps, Gaussian variance $t_i$3, and temperature $t_i$4. Evaluation uses weighted-average F1 computed only on unseen emotions, with results averaged over 5 random seeds (Peng et al., 27 Aug 2025).

The comparison includes four baseline categories: feature-based ERC models (DialogueGCN, DialogueCRN, DualGAT), prototype or contrastive methods (CPTC, SACL-LSTM, SCCL, EACL), and LLM baselines (Llama-3.1-8B, Qwen-2.5-7B, GPT-4o, DeepSeek-V3). ProEmoTrans achieves the best reported results in all six settings. The weighted-F1 scores are 37.27 for $t_i$5, 32.36 for $t_i$6, 28.34 for $t_i$7, 20.73 for $t_i$8, 38.59 for $t_i$9, and 35.64 for $D_s$0, for an average of 32.16. The reported improvements over the best baseline across the six settings are 11.58%, 6.10%, 4.24%, 2.05%, 3.44%, and 5.88% (Peng et al., 27 Aug 2025).

The ablation study identifies LED as the dominant factor. Removing LED reduces average weighted F1 from 32.16 to 17.86, a drop of 14.30 points. Using only one generated description yields 20.68 average weighted F1; using three descriptions gives 32.49, only +0.33 over two descriptions. Removing GSA reduces average weighted F1 to 31.10, and replacing GSA with standard self-attention reduces it to 30.63. Removing CRF/AVD reduces average weighted F1 to 25.23, a drop of 6.93, indicating that transition modeling and AVD are crucial for UERC (Peng et al., 27 Aug 2025).

Additional analyses reinforce the same pattern. The best Gaussian variance is 0.5; as $D_s$1 increases, GSA approaches standard self-attention and performance decreases. Using larger backbones improves performance somewhat—RoBERTa-large reaches 34.83 average weighted F1 versus 32.16 for BERT-base—but the gains are described as modest relative to inference cost. For similarity functions, contrastive similarity performs best; replacing it reduces average weighted F1 by 1.64 with Euclidean distance, 0.67 with cosine similarity, and 1.53 with dot product. The paper also reports that ProEmoTrans performs best in 16-shot ERC on IEMOCAP, EmoryNLP, and MELD, suggesting utility beyond the zero-shot UERC setting (Peng et al., 27 Aug 2025).

5. Position within emotion-transition research

ProEmoTrans belongs to a broader shift from static emotion labeling toward emotion dynamics, but its target problem is narrower and more specific than several adjacent lines of work. In ProEmoTrans, “emotion transfer” refers to transferring seen emotion transitions to unseen emotions during conversational recognition, not transferring affect between speech signals, not generating emotionally improved dialogue, and not benchmarking multimodal reasoning (Peng et al., 27 Aug 2025).

A direct precursor in dialogue modeling is the construction of positively transitioned (PT) sentiment data for open-domain chatbots. That work defines PT segments as multi-turn dialogue snippets in which sentiment moves in a more positive direction over the conversation, filters data by agreement between a sentiment predictor and an emoji-sentiment mapper, and fine-tunes DialoGPT-large to obtain Emily. The key relevance is the emphasis on emotion transition modeling across turns rather than isolated utterance sentiment, although the objective there is supportive response generation rather than unseen-label recognition (Wang et al., 2022).

A different conversational line models emotional trajectories with Affect Control Theory (ACT), Word2vec, and Emoji2vec. That framework maps words and emojis into EPA space, applies impression change equations, and tracks deflection over time in customer–chatbot messaging. Its output is a sequence of emotional states and deflection values rather than zero-shot label prediction, but it shares with ProEmoTrans the assumption that conversational emotion is fundamentally a state-transition process (Mostafavi et al., 2021).

Several speech-centered systems also pursue transition-aware or transfer-based affect modeling, but at different levels of abstraction. Emotion Neural Transducer and FENT model fine-grained temporal emotion localization in speech using an emotion lattice and joint ASR training; AffectEcho transfers affect from a reference utterance through a vector-quantized codebook of 25 emotion embeddings; TRACE-EVC formulates emotional voice conversion as instruction-guided relative emotional voice conversion, using a source-anchored rectified flow to predict affective displacement from natural-language instructions; and EmoTransCap introduces discourse-level Emotion Transition-Aware Speech Captioning together with EmoTransSpeech, a transition-rich bilingual dataset and a Multi-Task Emotion Transition Recognition (MTETR) model (Shen et al., 2024, Viswanath et al., 2023, Zhang et al., 4 Jul 2026, Xu et al., 29 Apr 2026).

At the evaluation level, EmoTrans is a benchmark for understanding, reasoning, and predicting emotion transitions in multimodal videos. It defines four tasks—Emotion Change Detection (ECD), Emotion State Identification (ESI), Emotion Transition Reasoning (ETR), and Next Emotion Prediction (NEP)—and shows that current multimodal LLMs struggle with fine-grained emotion dynamics, especially in socially complex multi-person scenarios. This benchmark is conceptually aligned with ProEmoTrans in its focus on emotion as a temporal process, but it is an evaluation framework for video-based multimodal reasoning rather than a conversational UERC model (Hu et al., 25 Apr 2026).

6. Limitations, interpretations, and likely extensions

The paper identifies two explicit limitations. First, the LED prompting strategy is manually crafted and not automatically optimized. Second, the method is restricted to a text-only setting and does not use multimodal cues such as facial expressions or audio, which could be valuable for emotion understanding (Peng et al., 27 Aug 2025).

These limitations clarify what ProEmoTrans does and does not solve. It is not a general theory of unseen affect modeling across modalities; it is a first benchmark system for transferring prototype semantics and transition structure within textual ERC. The paper therefore characterizes it as a strong baseline for preliminary exploration rather than a definitive solution (Peng et al., 27 Aug 2025).

A plausible implication is that future UERC systems may combine ProEmoTrans-style prototype transfer with the multimodal and discourse-level treatment of emotion dynamics seen in adjacent work. Video benchmarks now evaluate person-specific emotion transition reasoning, and discourse-level speech pipelines already construct data rich in 1 to 3 emotional transitions and use transition-aware recognition modules for captioning and controllable synthesis (Hu et al., 25 Apr 2026, Xu et al., 29 Apr 2026). This suggests a natural research direction: extending unseen emotion recognition from text-only conversation into multimodal, temporally grounded settings where emotional evidence is distributed across language, prosody, and visual behavior.

Within the current literature, ProEmoTrans is best understood as the first explicit formulation of UERC together with a modular solution that targets the three core obstacles of the task: semantic mismatch between labels and actual emotional expression, context modeling without overfitting, and transfer of transition knowledge from seen to unseen emotions. Its significance lies less in introducing a new backbone architecture than in defining a new recognition regime and showing that prototype learning, LLM-enhanced semantics, parameter-free context aggregation, and transition-aware decoding can be combined into an effective zero-shot conversational emotion framework (Peng et al., 27 Aug 2025).