Large Emotional World Model

Abstract: World Models serve as tools for understanding the current state of the world and predicting its future dynamics, with broad application potential across numerous fields. As a key component of world knowledge, emotion significantly influences human decision-making. While existing LLMs have shown preliminary capability in capturing world knowledge, they primarily focus on modeling physical-world regularities and lack systematic exploration of emotional factors. In this paper, we first demonstrate the importance of emotion in understanding the world by showing that removing emotionally relevant information degrades reasoning performance. Inspired by theory of mind, we further propose a Large Emotional World Model (LEWM). Specifically, we construct the Emotion-Why-How (EWH) dataset, which integrates emotion into causal relationships and enables reasoning about why actions occur and how emotions drive future world states. Based on this dataset, LEWM explicitly models emotional states alongside visual observations and actions, allowing the world model to predict both future states and emotional transitions. Experimental results show that LEWM more accurately predicts emotion-driven social behaviors while maintaining comparable performance to general world models on basic tasks.

• The paper introduces a novel framework that integrates affective reasoning with physical state transitions using a dual-path causal approach via the EWH dataset.
• The model achieves up to an 8% accuracy increase on subjective tasks by explicitly incorporating emotion into world state predictions.
• Integrating emotional signals enables robust simulation of human behavior in complex social settings without compromising objective task performance.

Large Emotional World Model: Unifying Emotion and World Dynamics

Introduction and Background

World models have become central in the simulation and reasoning of real-world dynamics by building internal representations that predict environmental state transitions. Traditional world models have primarily focused on encoding and forecasting physical and logical regularities of the environment, as epitomized by large-scale generative video and LLMs. However, they have a fundamental limitation: an inability to faithfully capture and predict emotion-driven human behaviors, which frequently violate rational, physics-based expectations.

The "Large Emotional World Model" (LEWM) directly addresses this crucial gap by integrating affective reasoning into world modeling architectures. The paper empirically demonstrates that excising emotional information from state representations sharply degrades predictive performance on behaviorally subjective tasks—highlighting the pivotal regulatory role of emotion in human cognition. LEWM is constructed atop the newly proposed Emotion-Why-How (EWH) dataset, which encodes state–action–emotion tuples in a multimodal context, facilitating the modeling of both the causal origins of actions and the propagation of emotional state transitions.

Motivation: The Necessity of Emotion in World Modeling

A core insight of this work is the inadequacy of current world models for capturing non-physical, emotion-driven behavioral patterns. The authors illustrate this by analyzing error modes in conventional physics-based models, demonstrating that certain human decisions—such as impulsive spending to alleviate distress—directly contradict cost/utility maximization, and therefore elude prediction by classical frameworks.

Figure 1: A purely physics-driven world model systematically fails to anticipate emotion-driven deviations in human behavior.

To validate the influence of emotion, the paper introduces an emotion-filtering module that systematically removes affective cues from inputs. Models trained with this ablated data exhibit up to an 8% accuracy drop on subjective tasks while showing only minor degradation on objective, knowledge-based benchmarks. This empirical observation provides compelling evidence that emotional context fundamentally modulates high-level cognitive processing in world models.

Methodology: EWH Dataset and LEWM Architecture

Emotion-Why-How (EWH) Dataset

The EWH dataset is constructed for the explicit modeling of emotion-influenced world state transitions within real-world behavioral contexts. Each sample is a temporally anchored tuple consisting of synchronized multimodal state data (visual, audio, image), an explicit action description, and paired pre- and post-transition emotional states. Critical to the dataset design is the weakly supervised extraction of emotion-driven behaviors using Large Multimodal Model (LMM) analysis of video, with automatic generation of fine-grained action and emotion annotations.

Figure 2: Schematic representation of the EWH dataset, showcasing its multimodal structure and integration of action and emotion labels.

This dataset enables dual-path causal reasoning: (1) the observable state-action-state pathway and (2) the latent emotional pathway that mediates behavioral state transitions. This dual-path design supports models capable of theory-of-mind inference—reasoning not only about "how" observable transitions occur, but also "why" agents undertake particular actions given their emotional states.

Large Emotional World Model (LEWM) Architecture

LEWM's architecture encodes multimodal world states, actions, and emotional states into a fused latent space. State transitions are factorized along an emotion-first causal pathway: the model first infers the evolution of the internal affective state, then conditions environmental transitions on this updated emotional context.

Figure 3: LEWM model architecture, illustrating separate emotional and state transition modules with shared latent representations.

Formally, given $(\mathcal{S}_t, \mathcal{E}_t, A_t)$, the architecture predicts $$p(\mathcal{S}_{t+1}, \mathcal{E}_{t+1}|\mathcal{S}_t, \mathcal{E}_t, A_t)$$, explicitly decomposing the transition as $$p(\mathcal{E}_{t+1}|h_t)p(\mathcal{S}_{t+1}|h_t, \mathcal{E}_{t+1})$$. The model includes specialized transition heads for emotion and environment, jointly optimized via state and emotion prediction objectives, plus an emotion-consistency regularizer that penalizes implausible divergences in prediction under minor emotional perturbations.

Experimental Evaluation

The empirical analysis evaluates (1) the impact of emotion filtering and (2) the benefits of emotion-aware modeling across subjective and objective benchmarks. Experiments are based on standardized datasets for emotion recognition (MELD), common sense inference (HellaSwag), and general knowledge (MMLU), as well as the newly constructed EWH dataset.

Figure 4: Comparative model performance showing accuracy differentials across affective and objective reasoning tasks.

Key results include:

• Emotion-filtered models display up to an 8% accuracy drop and 10% decrease in F1 on subjective tasks, while experiencing marginal (~1–3%) losses on objective reasoning, indicating that affect acts as a significant bias in human-centric inference.
• LEWM achieves superior performance in emotion-driven behavior prediction, establishing the effectiveness of affective integration, while maintaining competitive accuracy on general world modeling tasks.
• The integration of emotional signals does not degrade performance on standard benchmarks, contradicting claims that emotion-aware architectures necessarily entail a trade-off with general predictive power.

Theoretical and Practical Implications

The findings establish that emotion is not merely ancillary but fundamentally constitutive for accurate prediction and simulation of human social dynamics. By formalizing the interaction between emotion, action, and state transition, LEWM expands the scope of world models from rigidly physical environments to richly social, affect-driven domains. This architecture opens new directions for the development of AI agents capable of reliable interaction in real-world social systems, agent-based simulation, and human-robot interaction scenarios.

The proposed emotion-consistency regularization leverages weak supervision from automatically extracted affect labels, suggesting a scalable path to incorporate internal state tracking into future large world models. Architectures based on LEWM principles could be adapted for autonomous driving, assistive robotics, and interactive simulators where human behavior frequently departs from rational utility models.

Conclusion

The "Large Emotional World Model" paper provides a comprehensive framework for integrating affective reasoning into world models. The combination of the EWH dataset and the LEWM architecture demonstrates significant improvements in modeling and prediction of emotion-driven behaviors without sacrificing generalization to objective tasks. This work substantiates the centrality of emotion in human-aligned world understanding and offers a technically rigorous foundation for future research in simulation, behavior forecasting, and the development of theory-of-mind-capable AI.