Cognition–Affect Integrated Model of Emotion

• Cognition–affect integrated model is defined as the bidirectional interaction between cognitive processes (like intention, appraisal, and memory) and affective mechanisms (such as core affect and physiological states) that together generate emotion.
• It employs computational pipelines integrating modules for action generation, emotion prediction, and cognitive appraisal, with empirical validation via metrics like F1 scores and BLEU-1 scores.
• The model underpins applications in dialogue systems, robotics, personalized affective computing, and cognitive neuroscience by facilitating real-time, context-sensitive emotion inference.

A cognition–affect integrated model of emotion posits that emotion is neither a purely affective process nor a function of cognitive appraisal alone, but the emergent property of tightly coupled, bidirectional interactions between cognitive structures (intention, goals, beliefs, appraisal, memory, planning, attention, action selection) and affective mechanisms (core affect, physiological states, anticipatory signaling, value/arousal systems). Recent computational frameworks formalize these interactions at architectural, algorithmic, and functional levels, providing both mathematical formalisms and empirically validated workflow pipelines.

1. Theoretical Foundations and Core Components

Cognition–affect integration arises from a convergence of psychological, neurocomputational, and machine learning perspectives. Early appraisal theories (OCC, Scherer, Smith & Lazarus) emphasized discrete appraisal checks (goal relevance, expectedness, agency), but strong empirical evidence demonstrates that affect alone is insufficient for emotion generation without concurrent engagement of domain-general cognitive systems (Mishra et al., 2019). Newer models (e.g., CogIntAc) formalize the interplay among:

• Intention ($I$): The internal cognitive motive, classically grounded in goal-directed behavior theories, serving as the driver of action.
• Emotional Expectation (%%%%1%%%%): The agent’s anticipated affective outcome if intention succeeds.
• Action ($A$): Observable behaviors (e.g., utterance sequences) that externalize intention and emotional expectation.
• Emotional Reaction ($E^{\mathrm{react}}$): The agent’s retrospective affective outcome, modulated by the actual action outcome.

Alternative models further distinguish among:

• Core affect: Evolutionarily ancient value–arousal signals (valence, arousal), subcortically instantiated.
• Cognitive modules: Autobiographical memory, social context, self-referential appraisal, and task-level planning, all situated in large-scale cortical networks (Mishra et al., 2019, Rosenbloom et al., 2024).

Table: Core Constructs Across Models

Model	Cognitive Component	Affective Component	Interface Mechanism
CogIntAc	Intention, action gen/inf	Expectation, reaction	Multi-head/fusion
RL-Appraisal	Value, policy, Q-update	Appraisal (novelty, power)	Classifier on appraisal
Common Model+	WM, LTM, procedural cycle	Emotion vector, appraisal	Bidirectional modulation

2. Formalization and Mathematical Structure

Models specify a closed cognitive–affective loop using various mathematical frameworks:

• CogIntAc Pipeline (Peng et al., 2022):
  • Action generation: $A_s \sim f_{\mathrm{act}}(I_s, E_s^{\mathrm{exp}})$
  • Intention inference: $$P(I_s | A_s) = \mathrm{Softmax}(\mathrm{MLP}_{\mathrm{enc}}(h_s) \oplus \alpha_{\mathrm{int}})$$
  • Emotion prediction: $$P(E_s^{\mathrm{react}} | I_s, A_r) = \mathrm{Softmax}(\mathrm{MLP}_{\mathrm{fuse}}( h_r, \mathrm{ReLU}(W_e\cdot[h_s;\alpha_{\mathrm{int}}]) ))$$
• RL-Based Appraisal (Zhang et al., 2023):
  • Appraisal checks computed from RL signals:
  • Suddenness: $$A_{s,t} = 1 - \hat T(s_t,a_t,s_{t+1}) / \sum_{s''} \hat T(s_t,a_t,s'')$$
  • Goal relevance: $A_{gr,t} = |\alpha\,\delta_t|$
  • Goal conduciveness: $A_{gc,t}= (\mathrm{clip}(\delta_t, -1, 1) + 1)/2$
  • Power: $$A_{p,t} = (\overline{Q}(s_{t+1}, \cdot) - \min Q(s_{t+1},\cdot))$$
• Hierarchical Bayes (Zhong et al., 2016):
  • Sensorimotor–emotion inference: $P(S|A,E) \propto P(S)P(A|S)$
  • Action generation: $P(A|E,S) \propto P(E|A,S)P(A|S)$
• Meta-monitoring architecture (Jin, 15 Sep 2025):
  • Deviations: $$\delta_g(t) = P_g(t) - T_g(t);\quad |\delta_g| > \theta_g \implies$$ emotion label assigned through a parameterized mapping.

All approaches emphasize recurrent, bidirectional coupling: affective signals (e.g., TD error, physiological vector, surprise) recursively update cognitive states (intention, planning, attention priorities) and vice versa.

3. Computational Architecture and Learning Algorithms

Architectures are realized as either distributed non-modular systems or compositionally modular pipelines.

• CogIntAc (Peng et al., 2022): Three modules—action abduction (BiLSTM/PLM + IntDic), emotion prediction (multi-task MLP for $E^{\mathrm{react}}$ and satisfaction), action generation (BART/GPT2, conditioned on inferred intention and affective template). Loss functions include cross-entropy for intention and emotion, and NLL for generation, aggregated as $L = L_1 + L_2 + L_3$.
• Integrated RL–Appraisal (Zhang et al., 2023): MDP-based agent learning via Q-learning; appraisal features mapped to emotion via SVM classifier.
• Hierarchical Sensorimotor (Zhong et al., 2016): Two-level RNNPB network—low-level (sensorimotor) and high-level (parametric bias representing emotion), optimized via BPTT with slow PB update.
• Common Model Extensions (Rosenbloom et al., 2024): Two added modules—Emotion (vector-based, cycle-synchronous, bidirectional modulatory links) and Metacognitive Assessment; integration with Perception, WM, both LTMs, and Motor systems.

Affective modulation pervades each computation cycle, biasing selection rules, attention, memory retrieval, and action amplification.

4. Empirical Validation and Benchmarks

Empirical studies span NLP dialogue, robotics, human–agent interaction, and behavioral simulation.

• CogIntAc Results (Peng et al., 2022):
  • Dataset: “CogIEA” (~2,100 dialogues, 7 intention classes, 6 emotion classes, binary satisfaction).
  • Action abduction: $F_1=72.7\%$; human ceiling $\sim$90.5%
  • Emotion prediction: $F_1=63.5\%$ (emotion), $89.7\%$ (satisfaction)
  • Action generation: BLEU-1 $\approx27.1$, human coherence $\approx2.67/3$ (scale)
  • All tasks show improved performance with explicit modeling of cognitive–affective triad.
• RL-Appraisal Model (Zhang et al., 2023):
  • $R^2=0.65$–$0.92$ correspondence of appraisal-derived emotion intensities to human free/forced-choice vignette labeling.
  • Discrete fine-grain emotions (anxiety, desperation, irritation, rage) reliably predicted by appraisal vector classifier.
• SensAI+Expanse (Henriques et al., 2020):
  • Individualized XGBoost models attain $F_1 > 0.9$ for 31/49 users in real-world emotional valence prediction.

Validation occurs both at the level of quantitative fit to human-label distributions and qualitatively via interpretability (template-based explanations, behavioral consistency).

5. Functional Implications, Limitations, and Interpretability

Integrating cognition and affect yields architectural and functional benefits:

• Robustness: Shared cognitive–affective pathways prioritize salient, urgent, or opportunity-associated signals (Pessoa, 2019).
• Resource Management: Unified “executive” and modulation resources avoid duplicative computation and enable context-appropriate adaptation.
• Interpretability: Explicit representation of intention, emotion, and satisfaction labels, and template-based rationales ("the speaker is happy because their request was satisfied") (Peng et al., 2022).
• Naturalistic Social Behavior: Enables human-like flexibility and context sensitivity in dialogue, robot control, and human–agent interfaces (Pessoa, 2019, Peng et al., 2022).

Increased integration introduces computational complexity (computing and tuning overlapped dynamic networks), challenges for model tuning and scaling, and reduced interpretability during failure states (since cognitive and emotional pathways are non-modular and highly entangled) (Pessoa, 2019).

6. Application Domains and Future Developments

Cognition–affect integrated models catalyze applications in:

• Conversational agents: Enhanced intent/action/emotion inference, emotion-aware dialogue generation, and explainable behavior (Peng et al., 2022).
• Adaptive human–robot interaction: Real-time modulation of perception, action, and planning for robust, context-sensitive engagement (Pessoa, 2019, Zhong et al., 2016).
• Personalized affective computing: Individualized valence prediction, real-world emotion tracking, and context-responsive empathy scoring (Henriques et al., 2020).
• Cognitive neuroscience: Framework for testing core hypotheses about bidirectional brain–body–context emotion generation, with layered laminar and network-specific predictions (Mishra et al., 2019).

Future work emphasizes multi-turn (multi-episode) modeling of evolving intentions and affect, deeper commonsense/world-state grounding of affective inference, online adaptation in real-time interactive systems, and formal convergence bridges between competing appraisal, discrete, and dimensional models via explicit mapping of cognitive update routines to emotion labels (Peng et al., 2022, Jin, 15 Sep 2025, Rosenbloom et al., 2024).