- The paper introduces a large-scale, viewer-centric video emotion dataset and a novel DAR-R1 framework for dynamic affective reasoning.
- It employs a two-stage approach combining supervised fine-tuning and reinforcement learning to achieve precise temporal segmentation and causal emotion explanations.
- Empirical results demonstrate significant gains over baselines, highlighting the method's potential to reshape dynamic affective computing research.
Benchmarking Dynamic Affective Reasoning: A Comprehensive Assessment of the DAR Dataset and DAR-R1 Methodology
Introduction
"Benchmarking Dynamic Affective Reasoning: A Viewer-Centric Video Emotion Dataset" (2607.10238) addresses critical deficiencies in video-based affective computing, particularly the disregard for the dynamic, causally driven trajectory of viewer emotions over time. The authors introduce the Dynamic Affective Reasoning (DAR) benchmark: a large-scale, temporally dense, viewer-centric dataset annotated with fine-grained emotional states and causal rationales. They further propose DAR-R1, a two-stage multimodal LLM (MLLM) adaptation framework leveraging supervised fine-tuning and reinforcement learning to align model outputs with fine-grained affective dynamics. The work rigorously redefines video emotion analysis from static classification to dynamic, event-driven, and causally interpretable prediction, establishing quantitative and qualitative baselines for the emergent research direction of dynamic affective reasoning.
Motivations and Framework
Existing video emotion recognition methods typically apply static, coarse-grained labels to entire clips, predominantly focusing on character emotions or facial affect. This neglects the time-evolving and subjective nature of emotional response, as well as the causal interplay of consecutive visual events that precipitate and shape viewer affect. Prior datasets (e.g., DFEW [19], VCE [29], MER [24]) fail to provide:
- Temporally explicit emotion shift boundaries
- Fine-grained viewer-centric emotion taxonomies
- Segmented, temporally grounded causal explanations
To remedy these deficits, DAR draws on the psychological substrate of Affective Events Theory, which postulates that emotions are dynamic, cumulative reactions to discrete events within a narrative context.
DAR Dataset Construction
The DAR dataset comprises 15,087 videos and 36,908 affectively segmented events, sourced from the VCE dataset and expanded via automated, then validated, annotation protocols. Its methodological pipeline involves:
- Event-Aligned Segmentation: High-level semantic event proposals (via Gemini-2.5-Pro) are reconciled with frame-level cut detection (PySceneDetect), with semantic/visual boundaries snapped to within 0.5s precision. Segment integrity is verified using InternVL3.5.
- Incremental Differential Captioning: Qwen3-VL, conditioned on prior segment descriptions, generates incremental captions focused on visual/contentual changes. Visual entity grounding is validated with Grounding DINO.
- Stream-of-Affect Reasoning Generation: Causal explanations are constructed for each segment, linking visual evidence and context to emotion triggers, following AET logic. Segment merges are performed if affect is sustained across events.
- Dual-Consistency Verification: Emotion-reason pairs are independently scored by InternVL3.5 and Qwen3-omni across five axes (visual grounding, causal logic, viewer centricity, temporal consistency, answer consistency). Feedback-driven rewrites enforce high annotation reliability.
Error auditing reveals low incidence rates for segmentation (>0.5s, 2.4%), emotion label (3.1%), grounding (2.2%), and non-viewer-centricity (1.3%), reflecting rigorous quality control.
The dataset's 27 emotion categories enable nuanced modeling of affect transitions (e.g., intense transitions such as Craving→Satisfaction and Anxiety→Fear are quantitatively dominant). Reasoning rationales average 118.8 words, far outperforming standard benchmarks in richness and contextual argumentation.
Dynamic Affective Reasoning Tasks
DAR delineates three hierarchical tasks:
- Affective Segmentation: Precisely localizing temporal boundaries where the viewer's dominant emotion shifts.
- Fine-grained Emotion Classification: Assigning one of 27 viewer-centric emotions to each segment.
- Affective Reasoning: Generating causal chain explanations for why a given event transition induces a particular emotion in the observer.
These tasks collectively challenge models to move beyond static or character-centric affect recognition toward temporally grounded, causal, and subjective emotional modeling.
DAR-R1: Model and Training Paradigm
The baseline model, DAR-R1, utilizes a two-stage process:
Supervised Fine-Tuning (SFT)
A Qwen2.5-VL-3B backbone is adapted to the DAR schema, learning to output segment-aligned emotion and rationale tuples via next-token prediction on ground-truth structured targets.
Group Relative Policy Optimization (GRPO)
To overcome SFT's limitations in precision and causal interpretability, RL-based fine-tuning is employed. GRPO, a critic-free variant of PPO, samples multiple candidate outputs per prompt, using group mean rewards as baseline. The reward function incorporates:
- Structural constraints (JSON format, timestamp monotonicity)
- Segment count fidelity
- Temporal segmentation overlap (mean IoU, boundary snap)
- Emotion label correctness (label agreement and minimum segment overlap)
- Reasoning quality (length, uniqueness via Jaccard distance between segment rationales)
Reward ablations confirm the necessity of each component for stabilizing RL training and boosting accuracy across all task axes.
Empirical Results
Quantitatively, DAR-R1 demonstrates consistent, strong improvements over both emotion-focused (Videmo, EmotionLlama, AffectGPT) and generalist (Video-ChatGPT, PandaGPT, Video-LLaVA, InternVL) MLLMs:
| Model |
SC-Acc (%) |
mIoU (%) |
Emo-Acc (%) |
Reasoning Score (Avg/5) |
| Best Baseline (Qwen3-VL-4B) |
29.2 |
48.2 |
17.9 |
2.7 |
| DAR-SFT |
37.7 |
47.6 |
25.8 |
3.0 |
| DAR-R1 |
41.5 |
52.3 |
28.6 |
3.3 |
Human and LLM-based evaluation confirm that DAR-R1 delivers significantly higher grounding, causal logic, and viewer-centricity (Avg 4.2/5), marking a distinct performance gap relative to both generic and emotion-specialized alternatives.
Qualitative examples show that DAR-R1 is superior in localizing affect shifts, correctly identifying high-arousal events (e.g., transitions from anticipation to pain), and grounding its explanations in temporally aligned visual evidence. In contrast, baseline models produce temporally invalid, generic, or character-centric rationales, disregarding critical viewer-centric causal logic.
Out-of-distribution evaluation (e.g., TSL temporal sentiment localization) shows non-trivial generalization gains, supporting the claim that dynamic, causally structured annotation and reward-guided training is robust to domain shift.
Implications, Limitations, and Future Directions
Pragmatically, this work establishes DAR as the new reference standard for benchmarking dynamic affective reasoning, providing both the data scale and annotation granularity necessary for rigorous model development. The task protocols and RL reward decomposition should serve as templates for future research in dynamic, event-driven affect modeling—especially for agent scenarios demanding continuous, contextually adaptive emotional intelligence.
Theoretically, DAR paves the way for a reorientation toward temporally and causally explicit modeling in affective computing—converging psychological theory (AET) with scalable ML and RL methodologies. This enables new lines of inquiry:
- Integration of memory-based or online learning architectures to further model cumulative affect history
- Extension to multi-viewer perspectives or group affect dynamics
- Domain transfer to other dynamic subjective reasoning tasks (e.g., continuous sentiment, trust calibration, engagement prediction)
- Development of explicit causal graph or program synthesis models grounded in temporally segmented event logic
Limitations remain in annotation expense, potential propagative bias from LLM-driven annotation stages, and the challenge of modeling extremely rare or highly subjective emotions not covered by the VCE taxonomy. Furthermore, the performance ceiling of current MLLMs, even after domain-specific RL finetuning, highlights the continued need for architectural and algorithmic advancement in long-range, context-dependent multimodal inference.
Conclusion
"Benchmarking Dynamic Affective Reasoning: A Viewer-Centric Video Emotion Dataset" (2607.10238) establishes a new paradigm for emotion recognition research via its large-scale, event-segmented, causally annotated dataset and reinforcement learning aligned baseline. The shift from static, taxonomic labeling to dynamic, viewer-centric, and causally interpretable modeling is empirically validated as nontrivial and essential for the next generation of affective intelligence in AI agents. This dataset and methodology provide the necessary benchmark and toolkit for future advances in multimodal affect understanding, setting a rigorous standard for both dataset construction and model evaluation in dynamic emotion analysis.