Dual-Speed Emotion Dynamics

• Dual-Speed Emotion Dynamics is a modeling paradigm that separates fast, stimulus-driven emotional responses from slower, context-integrated mood adjustments.
• It leverages computational architectures such as dual-branch LSTMs, transformer layers, and differential equations to capture multi-scale affective dynamics.
• The approach enhances applications in speech emotion recognition, dynamic music analysis, text-to-speech synthesis, and agent-based simulations by improving prediction and synthesis accuracy.

Dual-speed emotion dynamics refers to the principled modeling, analysis, or control of affective states evolving at multiple characteristic time scales. This paradigm emerges in computational affective science, speech and music emotion recognition, text-to-speech systems, multi-agent simulation, and social interaction modeling, where emotion is neither monolithic nor solely a fast-responding system. Rather, emotion displays both rapid, stimulus-driven changes and slower, contextually integrated, or homeostatic adjustments. Models operationalizing dual-speed emotion dynamics typically instantiate distinct “fast” and “slow” update processes—often via parallel learnable branches, multi-scale attention, or explicit dynamical system layers—enabling richer description, prediction, and generation of affective behaviors across modalities and timescales.

1. Mathematical and Computational Foundations

Mathematically, dual-speed emotion dynamics decomposes affective state updates into at least two constituent processes, each characterized by its own rate or integration window. In the context of agent-based modeling for online emotion, this is formalized as a set of coupled differential equations for valence $v(t)$ and arousal $a(t)$, where internal relaxation (exponential decay to baseline with rates $\gamma_v$, $\gamma_a$) and external fast driving (stimulus-dependent forcing $\mathcal{F}_v$, $\mathcal{F}_a$) jointly determine the evolution: $$\frac{dv}{dt} = -\gamma_v\bigl(v(t)-b\bigr) + \mathcal{F}_v\bigl(h(t),v(t)\bigr) + A_v \xi_v(t)$$

$$\frac{da}{dt} = -\gamma_a\bigl(a(t)-d\bigr) + \mathcal{F}_a\bigl(h(t),a(t)\bigr) + A_a \xi_a(t)$$

Here, $b$, $d$ are baseline values, and $h(t)$ denotes external emotional input. The separation of $(\gamma_v, \gamma_a)$ from the external forcing terms ensures distinct time constants for natural emotion decay versus content-driven response (Garcia et al., 2016).

Recurrent and transformer-based architectures in speech or music emotion recognition implement analogous dual-scale processing: parallel branches ingest spectrograms at different resolutions or processed with attention spans reflecting fast (short-window) or slow (long-window) context, whose outputs are later fused (Wang et al., 2019, Zhang et al., 2024).

2. Dual-Speed Emotion Dynamics across Modalities

Speech Emotion Recognition (SER)

Dual-speed modeling in SER leverages multiple spectral representations to decouple rapid affective cues (e.g., onsets, pitch shifts) from slower prosodic patterns. In the Dual-Sequence LSTM (DS-LSTM) architecture, raw audio is transformed into two mel-spectrogram sequences at different time-frequency scales and processed in a custom six-gate LSTM cell. This cell dynamically balances updates from the time-rich (fast) and frequency-rich (slow) streams, allowing the system to capture both abrupt and sustained emotional cues. A parallel MFCC-based LSTM branch complements the multi-scale processing. During inference, the outputs are averaged, resulting in state-of-the-art unimodal SER performance with remarkable gains over single-scale baselines (Wang et al., 2019).

Dynamic Music Emotion Recognition (DMER)

DSAML for DMER introduces a dual-scale feature extractor: one branch, using a fixed global encoder (ImageBind), aggregates long-term context, while a local adapter captures segment-level (short-term) changes. A transformer layer is deployed with two attention masks, enforcing short (local) and long (global) temporal dependencies, and their results are merged through late fusion. Auxiliary losses ensure proper scale separation. This is key for aligning model outputs with the multi-scale nature of musical emotion, where local features reflect transient events and global features represent overarching mood. Ablation studies confirm the necessity of both streams for optimal performance (Zhang et al., 2024).

Text-to-Speech (TTS) and Fine-Grained Dynamics

Emo-FiLM models dual-speed emotion dynamics by annotating each spoken word with high-resolution emotion vectors, mapped from frame-level acoustic analytics. This supports both global (sentence-wide) and local (word-by-word) control during TTS synthesis via Feature-wise Linear Modulation (FiLM) layers, modulating embedding trajectories in accordance with fine-grained emotional annotations. Objective and subjective metrics show that the explicit modeling of both scales produces speech outputs with accurately tracked global and intra-sentence emotional shifts (Wang et al., 20 Sep 2025).

Social Simulation and Emotionally Stateful Agents

In LLM–driven multi-agent platforms, dual-speed emotion dynamics governs agent affect through the composition of:

• Fast appraisal: rapid PAD (Pleasure-Arousal-Dominance) state adjustments in response to conversational input ($\Delta s^{\text{fast}}$, capped per dimension).
• Slow reflection: periodic updates integrating retrieved, memory-tagged affective events; these produce larger cumulative PAD shifts, building mood over time. A decay term ensures continuous relaxation to neutrality, while thresholds on memory poignancy gate slow updates. Trajectories manifest jittery turn-level variation superseded by smooth mood arcs, especially in high-capacity models capable of leveraging this structure for enhanced continuity and believability (Fu et al., 25 Jan 2026).

3. Methodological Instantiations

Several architectural and algorithmic strategies for dual-speed modeling have been adopted:

Domain	Fast Process	Slow Process	Fusion Mechanism
SER (DS-LSTM)	Fine-res mel-spectrograms (short windows)	Coarse-res mel-spectrograms, MFCCs (long windows)	Gated LSTM cell, inference avg.
DMER (DSAML)	Local adapter/short-window attention	Global encoder/long-window attention (ImageBind)	Sigmoid fusion of branches
TTS (Emo-FiLM)	Word-level emotion2vec embeddings	Sentence/global reference	FiLM-modulated text embeddings
Social Sim (Sentipolis)	Turn-level appraisal	Periodic, aggregation-driven reflection (memory-driven)	Additive PAD updates
Online Inter. (Garcia et al., 2016)	Instantaneous stimulus in ODE	Relaxation to baseline (ODE timescale)	Summative in dynamics

Each approach grounds the separation of timescales in domain-specific priors (acoustic windowing, transformer masking, memory integration) but is unified by the explicit or implicit management of at least two characteristic speeds.

4. Empirical Findings and Cross-Domain Evaluation

Across modalities, empirical studies validate the superiority of dual-speed models over single-scale or stateless baselines:

• In SER, DS-LSTM achieves 72.7% WA (weighted accuracy) and 73.3% UA (unweighted accuracy) on IEMOCAP, exceeding previous unimodal models by 6.8% WA (Wang et al., 2019).
• For DMER, DSAML attains CCC(a)=0.402 and CCC(v)=0.117 on the DEAM dataset, and ablations removing either attention scale yield notable drops. Personalized meta-learning further extends this to rapid adaptation for new listeners (Zhang et al., 2024).
• In fine-grained TTS, Emo-FiLM substantially reduces emotion trajectory DTW error (–9.1% relative) and delivers highest subjective and objective scores on both global and dynamic speech emotion tasks (Wang et al., 20 Sep 2025).
• In LLM-driven social agents, dual-speed emotion dynamics more than doubles emotional continuity, increases communication and empathy, and, in higher-capacity models, raises believability. In smaller models, excessive weighting of the slow (mood) pathway can reduce naturalness, suggesting a capacity-dependent trade-off (Fu et al., 25 Jan 2026).
• In agent-based online-interaction models, internal relaxation timescales (τ_v ≈ 2.7 min, τ_a ≈ 2.4 min) and immediate stimulus-driven valence/arousal jointly explain empirical timecourses; agent simulations display bursty cascades and exponential decays matching human data (Garcia et al., 2016).

5. Theoretical Context and Interpretation

Dual-speed emotion dynamics aligns with psychological appraisal theories such as EMA, which postulate rapid appraisal-driven emotional reactions modulated by slower, integrative mood or reflection mechanisms (e.g., memory tagging, Poignancy-weighted aggregation). The separation of time constants facilitates both immediate behavioral adaptation and the maintenance of long-horizon emotional coherence, addressing issues such as “emotional amnesia” and weak inter-turn affective linking in LLM agents (Fu et al., 25 Jan 2026), as well as speech/musical perception phenomena (abrupt versus sustained emotional cues).

A plausible implication is that architectures explicitly encoding multi-scale affective processing outperform single-scale models not only in discrete classification or regression benchmarks but also in generating nuanced and temporally coherent affective dynamics, especially evident under interactive or personalized evaluation paradigms.

6. Limitations and Open Challenges

Not all studies provide isolated ablation or per-transition analyses on the specific contributions of dual-speed components. For example, Sentipolis does not report ablation isolating the effect of dual-speed dynamics alone, making it difficult to quantify their distinct contribution in the broader system (Fu et al., 25 Jan 2026). In fine-grained TTS, while ablation on modulation and annotation components is performed, the interaction between global and fine-grained control remains an open area for deeper analysis (Wang et al., 20 Sep 2025).

Model capacity and data annotation granularity impose practical constraints: in LLM-based agents, dual-speed modeling can overdrive lower-capacity backbones, impairing realism. In music and speech, segment size and window choices critically affect the separation of temporal scales.

7. Future Directions

Anticipated developments include:

• Systematic, multi-domain ablation studies to more precisely attribute gains to dual-speed components.
• Expansion to higher-order architectures, e.g., triple-scale or continuous-scale attention masks.
• Broader personalization, leveraging meta-learning and annotator-conditioned dynamics across modalities (Zhang et al., 2024).
• Unified benchmarks for affective temporal dynamics, such as the construction and adoption of datasets like FEDD for dynamic TTS (Wang et al., 20 Sep 2025).
• Mechanistic interpretation of neurocomputational correlates underlying discrete versus integrated emotion dynamics modeled in silico.

The dual-speed paradigm is thus a foundational principle for modeling, recognizing, generating, and simulating emotion-rich behavior across human and artificial systems.