---
title: 'AffectGPT-R1: RL for Open-Vocab Emotion Recog'
url: https://www.emergentmind.com/topics/affectgpt-r1
type: topic
---

# AffectGPT-R1: RL for Open-Vocab Emotion Recog

Searching arXiv for the cited AffectGPT papers and related reinforcement-learning OV-MER work.
AffectGPT-R1 is a reinforcement learning framework for open-vocabulary multimodal emotion recognition (OV-MER). It was introduced as a later extension of the AffectGPT line, with the specific aim of aligning training with emotion-wheel-based evaluation rather than relying only on token-level supervision. In this formulation, a multimodal large language model receives video-centered affective evidence and generates natural-language emotion outputs, while reinforcement learning uses emotion-wheel metrics as rewards to optimize semantically meaningful emotion prediction in the open-vocabulary regime [2508.01318].

## 1. Naming, lineage, and scope

The name belongs to a broader AffectGPT lineage, but its meaning changed across successive papers. In 2024, “AffectGPT” denoted a two-stage framework for Explainable Multimodal Emotion Recognition (EMER), built around EMER-Coarse, EMER-Fine, and a Video-LLaMA-based architecture trained to generate emotion-related descriptions from audio, video, and subtitles [2407.07653]. In early 2025, “AffectGPT” referred to a different but related framework centered on the MER-Caption dataset, the AffectGPT multimodal model, and the MER-UniBench benchmark for multimodal emotion understanding with natural-language outputs [2501.16566].

The foundational 2025 AffectGPT paper explicitly stated that it did not use the term “AffectGPT-R1” anywhere, and described only AffectGPT, MER-Caption, MER-Caption+, and MER-UniBench [2501.16566]. The formal designation “AffectGPT-R1” appears later, in the paper titled “AffectGPT-R1: Leveraging Reinforcement Learning for Open-Vocabulary Emotion Recognition,” where it denotes a reinforcement-learning extension of the AffectGPT framework for OV-MER [2508.01318].

Within this lineage, AffectGPT-R1 is best understood not as an unrelated architecture, but as an alignment layer over the AffectGPT foundation. It preserves the multimodal large-language-model orientation of AffectGPT while changing the optimization target from token-level imitation to reward-driven optimization of emotion-wheel similarity.

## 2. Problem formulation and motivation

AffectGPT-R1 is defined for Open-Vocabulary Multimodal Emotion Recognition, a setting in which predicted emotions are not restricted to a predefined label set. This differs from conventional multimodal emotion recognition, where models are trained as discriminative classifiers over small closed taxonomies such as six or seven basic emotions. OV-MER instead allows free-form emotion words or phrases, which makes generative multimodal language models a natural fit [2508.01318].

The immediate motivation is objective mismatch. AffectGPT, like other generative OV-MER systems, was trained with standard token-level loss on descriptive emotion data. However, OV-MER evaluation is based on emotion wheels and related set-level matching procedures rather than exact textual overlap. The AffectGPT paper had already formalized fine-grained emotion recognition through open-vocabulary prediction on OV-MERD+, with synonym normalization and emotion-wheel grouping, and used the set-level score $F_s$ as the primary metric for fine-grained recognition [2501.16566]. AffectGPT-R1 argues that token-level cross-entropy is only an indirect proxy for these evaluation criteria and cannot directly optimize them because emotion-wheel metrics depend on discrete language outputs and non-differentiable matching procedures [2508.01318].

This design responds to a broader pattern in affective computing with large language models. Earlier work had shown that GPT-style systems could exhibit strong zero-shot affective processing, including Valence–Arousal–Dominance prediction, latent mapping between situations and emotion words, and basic OCC-style appraisal reasoning [2309.01664]. A wider benchmark study also found that GPT-4 performed strongly on many sentiment-, emotion-, and toxicity-related tasks, but fell short on problems with implicit signals such as engagement measurement and personality assessment [2308.13911]. AffectGPT-R1 can be read as a multimodal and reinforcement-learning response to those limitations: rather than treating affect as ordinary text generation, it optimizes directly for the semantic structure used in OV-MER evaluation.

## 3. Base architecture and two-stage training pipeline

AffectGPT-R1 uses the same architecture as AffectGPT for its cold-start stage. In the AffectGPT implementation, the visual encoder is frozen CLIP ViT-L, the acoustic encoder is frozen HuBERT-L, and the language backbone is Qwen2.5, frozen except for a LoRA module; trainable components include the LoRA parameters, the audio and video projectors, and the pre-fusion module for audio–video integration [2501.16566]. AffectGPT’s central architectural idea is pre-fusion: cross-modal fusion is performed before projection into LLM token space, rather than being left entirely to the language model. Two variants were proposed—Q-Former-based pre-fusion and a simpler attention-based pre-fusion—and both improved over the no-pre-fusion baseline, with the attention-based version slightly outperforming Q-Former in the reported setting [2501.16566].

AffectGPT-R1 then adds a two-stage post-training pipeline. The first stage is a supervised “cold-start” stage on MER-Caption+, where the model learns both multimodal emotion understanding and a structured output format. The second stage is reinforcement learning on MER2025-OV, where labels are available but emotion descriptions are not, and the model is optimized against emotion-wheel-based rewards [2508.01318].

| Stage | Dataset | Role |
|---|---|---|
| Cold start | MER-Caption+ | 31,327 samples; emotion descriptions and labels |
| RL fine-tuning | MER2025-OV | 1,000 non-overlapping samples; emotion labels only |
| Evaluation | OV-MERD+ | 532-sample OV-MER benchmark |

The cold-start stage teaches the model to emit two explicitly separated regions:

```xml
<think> ... </think>
<answer> ... </answer>
```

In the original formulation, the `<think>` segment contains explanatory reasoning or emotion-related description, and the `<answer>` segment contains the final open-vocabulary emotion label or phrase [2508.01318]. This format is not merely cosmetic. It provides a parsing boundary for reward computation and separates descriptive reasoning from the final label string used in evaluation.

## 4. Reinforcement learning, GRPO, and reward design

The reinforcement-learning stage treats OV-MER as a sequence-generation problem with non-differentiable rewards. Given multimodal input and a prompt, the model samples candidate outputs; each output is scored by a reward function that combines format compliance with emotion-wheel accuracy. In the original AffectGPT-R1 paper, the combined reward is

$$
R(o, y \mid v, q) = R_{\text{accuracy}}(o, y \mid v, q) + \beta\, R_{\text{format}}(o \mid v, q),
$$

where the accuracy term is the official emotion-wheel metric computed from the extracted `<answer>` content, and the format term checks whether the output strictly follows the required `<think></think><answer></answer>` structure [2508.01318].

Optimization uses Group Relative Policy Optimization (GRPO), a PPO-style method that removes the critic and computes relative advantages within a sampled group of outputs. For a group of rewards $\{r_1,\dots,r_G\}$, the group-normalized advantage is

$$
A_i = \frac{r_i - \mathrm{mean}(r_1,\dots,r_G)}{\mathrm{std}(r_1,\dots,r_G)}.
$$

This replacement of critic-based advantages by group-relative normalization is intended to reduce memory overhead while preserving stable policy updates [2508.01318].

A later analytical extension, “AffectGPT-RL,” substantially deepened the reward analysis. It introduced additional rewards beyond accuracy and format: an alignment reward for consistency between reasoning and answer, a dual reward comparing extracted reasoning emotions directly against ground-truth labels, and a perception reward based on pairwise judging of reasoning quality. It also documented reward hacking: because emotion-wheel evaluation is set-based and tolerant to synonym expansion, the model can improve reward by outputting overly long lists of emotion words. To counter this, the paper proposed length penalties $P_1$, $P_2$, and $P_3$, and reported that $P_2$ and $P_3$ suppressed overlong outputs without harming performance [2605.06126].

One of the most consequential later findings concerns explicit reasoning. Although the original AffectGPT-R1 formulation emphasized `<think>` and `<answer>`, AffectGPT-RL reported that RL without a thinking segment outperformed RL with a thinking segment on OV-MERD+, and interpreted this as evidence that noisy or weak reasoning traces can distract optimization away from the core semantic target [2605.06126]. This does not invalidate the original format, but it reframes the role of explicit reasoning in affective generation: chain-of-thought-style structure is useful only when the reasoning supervision itself is reliable.

## 5. Empirical performance and benchmark position

The baseline AffectGPT system already reported strong multimodal emotion-understanding performance. On MER-UniBench, which aggregates basic emotion recognition, sentiment analysis, and fine-grained emotion recognition from generated text, AffectGPT achieved a mean score of 74.77 across all tasks and reported an OV-MERD+ fine-grained score of 62.52 [2501.16566].

AffectGPT-R1 improved the OV-MERD+ score to 66.35, compared with 62.52 for AffectGPT, an absolute gain of 3.83 points on that benchmark [2508.01318]. The paper interprets this as evidence that directly optimizing emotion-wheel metrics via reinforcement learning is more effective than optimizing token-level likelihood alone when the evaluation target is semantic rather than lexical.

Subsequent work in AffectGPT-RL further analyzed where these gains come from. It reported that accuracy reward and dual reward were the most effective single-reward choices, that combining accuracy with format could improve over accuracy alone, and that combining too many rewards led to reward interference. It also examined generalization beyond OV-MER and reported gains on basic emotion recognition tasks, while noting that optimization for OV-MER and basic emotion prediction could slightly reduce performance on some sentiment-analysis datasets [2605.06126].

These results sharpen the distinction between the three major stages in the AffectGPT line. The 2024 EMER-oriented AffectGPT showed that large weakly supervised descriptive datasets could make explainable multimodal emotion recognition feasible at scale [2407.07653]. The 2025 AffectGPT system supplied a stronger multimodal foundation with MER-Caption, MER-Caption+, pre-fusion multimodal integration, and MER-UniBench [2501.16566]. AffectGPT-R1 then reoriented training toward the benchmark’s own semantic geometry by optimizing emotion-wheel reward directly [2508.01318].

## 6. Limitations, interpretations, and broader significance

The original AffectGPT-R1 paper is explicit about several limitations. It reports limited hyperparameter tuning, application only to OV-MER, evaluation restricted to OV-MERD+, and no extensive ablations on reward components or RL configuration [2508.01318]. It also notes broader concerns typical of affective computing systems, including possible cultural skew in emotion data and the risks inherent in inferring emotions from video and speech [2508.01318]. The later AffectGPT-RL analysis adds a more technical limitation: not all reasoning-style additions help, and some reward terms—especially perception-style judging—can introduce noise rather than useful supervision [2605.06126].

The broader significance of AffectGPT-R1 lies in objective alignment. Earlier affective large-language-model studies had demonstrated that general models could perform meaningful affective processing under prompting alone, including continuous affect prediction and appraisal-style reasoning, but those studies did not solve the mismatch between open-vocabulary generation and semantic emotion evaluation [2309.01664]. AffectGPT-R1 addresses that mismatch by using reinforcement learning to optimize the same non-differentiable emotion-wheel structure that governs evaluation [2508.01318].

A plausible implication is that AffectGPT-R1 marks a transition from descriptive multimodal emotion understanding to metric-aligned multimodal emotion optimization. In that sense, it is less a standalone architecture than a specific training doctrine: start from a multimodal AffectGPT backbone, represent emotions in open vocabulary, and then use reinforcement learning to move the model toward the semantic topology encoded by emotion wheels. The later AffectGPT-RL study suggests that this doctrine extends beyond OV-MER to parts of MER-UniBench, but also that affective reasoning benefits only when rewards, reasoning traces, and output constraints are chosen with unusual care [2605.06126].

Source: https://www.emergentmind.com/topics/affectgpt-r1