Collaborative Emotional Modeling (CoEM)

• Collaborative Emotional Modeling (CoEM) is a computational framework that treats emotions as dynamic, shared objects rather than static labels.
• It integrates diverse methods like appraisal theory, multi-agent coordination, and collaborative filtering to boost AI’s emotional intelligence.
• CoEM is applied in dialogue systems, embodied interactions, and affective data recovery to improve adaptive modeling and emotional reasoning.

Collaborative Emotional Modeling (CoEM) denotes a family of computational approaches in which emotional state is treated as a shared, structured, and revisable object rather than as a one-shot label prediction. Across recent work, CoEM does not denote a single architecture. It appears as appraisal-grounded coordination between cognitive and emotional reasoning inside one assistant, as multi-agent critique around a shared case formulation, as participant-taught embodied interaction, as selective symbolic alignment between embodied agents, as retrieval–enrichment–generation for long-context emotional intelligence, as collaborative filtering over sparse affective reports, and as persona–stimulus–reaction analysis in agent societies (Zhang et al., 17 Mar 2026, Han et al., 20 Jun 2026, Tütüncü et al., 26 Sep 2025, Zhang et al., 10 May 2026, Liu et al., 9 Sep 2025, Jolly et al., 2021, Hasan et al., 19 May 2026).

1. Conceptual scope and major variants

A central distinction in the literature is whether collaboration is internal, inter-agent, or cross-participant. In EmoLLM, collaboration is primarily internal: a single assistant coordinates cognitive reasoning (IQ) and emotional reasoning (EQ) through an explicit Appraisal Reasoning Graph (ARG), while reverse-perspective reasoning simulates user-side consequences during training; inference remains single-agent rather than multi-assistant (Zhang et al., 17 Mar 2026). In MindTailor, by contrast, collaboration is explicitly multi-agent: three counselor agents grounded in cognitive reframing, unconditional positive regard, and solution-focused counseling iteratively critique a draft, and a meta-synthesizer selects the most therapeutically important revisions (Han et al., 20 Jun 2026). In LongEmotion, CoEM is defined operationally as a five-stage hybrid retrieval-generation pipeline implemented by distinct roles—CoEM-Rank, CoEM-Sage, and CoEM-Core—so that long-context emotionally salient evidence is selected, enriched, and aggregated under controlled flow (Liu et al., 9 Sep 2025).

A second distinction concerns who participates in the construction of the emotion model. Commonaiverse treats emotion inference as a co-created, embodied, and culturally situated process in which participants explicitly teach the system what emotions look like in motion, the system aggregates multiple perspectives on that motion, and the outputs are returned in real time through adaptive audiovisual feedback (Tütüncü et al., 26 Sep 2025). The study on co-constructed emotion between embodied agents defines collaboration as the emergence of a shared emotional lexicon through selective communication: agents do not homogenize their private bodily states, but align symbolic categories so that interaction becomes mutually intelligible (Zhang et al., 10 May 2026). The Moltbook analysis shifts the focus again, using CoEM to characterize, align, and jointly reason about emotions across many interacting agents through structured emotion profiles and the Persona–Stimulus–Reaction domain (Hasan et al., 19 May 2026).

A third variant is statistical rather than dialogic. The collaborative filtering work reconstructs sparse emotional measurements by leveraging structured covariation across individuals; the “collaboration” lies in using other people’s ratings to recover an individual’s missing states without assuming that individual differences are noise (Jolly et al., 2021). CogIntAc provides an interactional cognitive precursor in which two Individual Activities Models are connected through an interaction chain linking intention, emotional expectation, emotional reaction, and action across interlocutors (Peng et al., 2022). The survey of collaborative affective computing places these heterogeneous designs under a broader taxonomy ranging from structured collaboration to autonomous collaboration, and interprets them through Dual Process Theory: fast, intuitive affect perception coupled with slower, deliberative reasoning and controlled generation (Lai et al., 2 Jun 2025).

2. Representational substrates and formal models

A defining characteristic of CoEM systems is the use of explicit intermediate structures. EmoLLM grounds collaboration in appraisal theory through ARG, a directed acyclic graph with node types $V=\{F,N,A,E,S\}$ for contextual facts, inferred needs/goals, appraisals, emotional state, and response strategy, and edges $(F,N)\rightarrow A$, $A\rightarrow E$, and $(F,N,A,E)\rightarrow S$. Its response model is factorized as

$$p_\theta(y_t \mid x_{\le t}) = \sum_{F_t,N_t,A_t,E_t,S_t} p_\theta(F_t \mid x_{\le t}) \cdot p_\theta(N_t \mid x_{\le t}) \cdot p_\theta(A_t \mid F_t, N_t) \cdot p_\theta(E_t \mid A_t) \cdot p_\theta(S_t \mid F_t, N_t, A_t, E_t) \cdot p_\theta(y_t \mid x_{\le t}, F_t, N_t, A_t, E_t, S_t),$$

which makes the binding between IQ and EQ explicit and inspectable (Zhang et al., 17 Mar 2026).

Other systems define different shared objects. MindTailor uses a history-grounded case formulation with four dimensions—Seeker Profile, Underlying Concern, Historical Context, and Response Blueprint—so that collaboration is organized around a clinically meaningful representation of the person rather than around free-form textual debate (Han et al., 20 Jun 2026). CogIntAc formalizes interaction through an intention–emotion–action chain, operationalizing how speaker intention shapes listener intention, how listener action feeds back into speaker emotional reaction, and how satisfaction functions as a cognitively grounded explanation of emotion (Peng et al., 2022). Moltbook decomposes each interaction into Persona, Stimulus, and Reaction, embeds emotion in Valence–Arousal–Dominance space, and models reactions as a Gaussian mixture,

$$r \sim p(r \mid C_c, S_p, P_a) = \sum_{c=1}^{K} \pi_c N(r \mid \mu_c, \Sigma_c),$$

thereby accommodating multimodal reactions conditioned jointly on agent baseline and conversational context (Hasan et al., 19 May 2026).

Embodied and long-context CoEM systems introduce further representational layers. Commonaiverse uses MoveNet in TensorFlow Lite to estimate 17 full-body keypoints, applies temporal smoothing, interpolation, dynamic calibration, and confidence filtering, and extracts speed, amplitude and range of motion, movement frequency, and inter-person proximity/spacing; these feed a modular multi-recommender architecture and a two-axis visualization “based on Russell/Scherer” (Tütüncü et al., 26 Sep 2025). LongEmotion represents long interactions as semantically coherent or token-bounded chunks, ranked by dense retrieval with

$s(c_i,q) = \text{cosine}(e(c_i), e(q)),$

then re-ranked after emotional enrichment so that only the most affectively aligned content reaches generation (Liu et al., 9 Sep 2025). In collaborative filtering, emotion is represented as a sparse user-by-item matrix, and missing values are recovered from structurally similar users, for example through user-based $k$NN:

$$\hat{r}_{ui} = \frac{\sum_{v \in \mathcal{N}_u(i)} s(u,v)\, r_{vi}}{\sum_{v \in \mathcal{N}_u(i)} |s(u,v)|}.$$

This formulation treats emotional individuality as signal that can be exploited rather than averaged away (Jolly et al., 2021).

3. Learning, adaptation, and collaboration protocols

CoEM systems differ sharply in how collaboration is learned or orchestrated. EmoLLM uses a staged training program. Stage I combines continued pretraining with affective knowledge and ARG-guided supervised initialization, including structured prefixes of the form "> g_t || x≤t" and teacher-provided ARG traces with a gating label. Stage II performs multi-turn reinforcement learning in a role-play environment with reverse-perspective reasoning, where the policy predicts user-side updates in needs, appraisals, and emotion, receives turn-level rewards for cognitive reliability, ARG trace quality, reverse-perspective plausibility, and overthinking control, and is optimized with GRPO (Zhang et al., 17 Mar 2026). This design makes collaboration both structure-aware and explicitly prospective: the model is rewarded not only for what it says, but for the predicted emotional consequence of saying it.

MindTailor exemplifies a different regime: it is training-free and relies on iterative critique at inference time. The system retrieves the top-$k$ relevant historical posts with text-embedding-3-small and cosine similarity, constructs a case formulation, produces an initial peer-support draft, and then runs four rounds of specialist critique. The meta-synthesizer prioritizes safety, empathy gaps, alignment to formulation and needs, actionability, and style, while selecting at most two high-priority improvements per round to avoid degradation from over-editing (Han et al., 20 Jun 2026). LongEmotion is likewise prompt-based rather than fine-tuned; its CoEM pipeline uses chunking, initial ranking, enrichment by CoEM-Sage, re-ranking, and emotional ensemble generation, with task-specific variants such as pairwise dissimilarity for odd-one-out emotion detection and summary-based re-ranking for multi-turn conversation (Liu et al., 9 Sep 2025).

Embodied and statistical systems rely more on online adaptation. Commonaiverse adapts in session through participant-taught exemplars, longitudinal tracking, dynamic calibration, and confidence-thresholded sensing, but does not specify optimization formulas or model-update rules (Tütüncü et al., 26 Sep 2025). The collaborative filtering work trains non-negative matrix factorization with stochastic gradient descent, optionally using small temporal dilation kernels to exploit autocorrelation in continuous affective reports (Jolly et al., 2021). In co-constructed embodied-agent emotion, communication itself drives learning: each agent infers multimodal latents with a PoE-MVAE and GMM, then aligns symbolic categories using the Metropolis–Hastings Naming Game, where selective acceptance of proposed signs reshapes category assignments and posterior parameters without forcing latent homogenization (Zhang et al., 10 May 2026). The survey of collaborative affective computing summarizes these strategies as structured collaboration—planning, tool use, retrieval, reflection, memory—or autonomous collaboration, including negotiation, voting, role-playing, and coordinator–agent organizations (Lai et al., 2 Jun 2025).

4. Empirical evidence and benchmark behavior

The strongest quantitative evidence for CoEM in dialogue appears in EmoLLM. On a Qwen3-8B backbone, the model improves emotional support, technical assistance, medical consultation, and academic peer review dialogue simultaneously. Relative to the base model, representative results are: ED with SR 92.1% (+3.8%), AT 1.87 (−38.3%), ES 4.86 (+6.1%), EA 4.92 (+28.1%); MSD with SR 83.2% (+7.2%), AT 2.86 (−12.5%), ES 4.17 (+1.0%), EA 3.71 (+8.8%); MedD with SR 95.3% (+6.8%), AT 2.18 (−25.3%), ES 4.08 (+3.3%), EA 4.59 (+28.6%); and ICLR with SR 96.2% (+15.3%), AT 1.21 (−26.2%), ES 4.21 (+11.1%), EA 3.95 (+3.1%). Full EmoLLM also improves factual accuracy over base Qwen3-8B—80.3% on ED, 95.4% on MSD, 91.9% on MedD, and 86.7% on ICLR—while ablations show that removing ARG, Stage I, or reverse-perspective rewards degrades performance (Zhang et al., 17 Mar 2026).

MindTailor reports consistent gains in personalized emotional support. In LLM-as-a-Judge pairwise comparisons, win rates against ES-VR exceed 79% across six backbones, and the method also consistently beats MentalAgora and a Vanilla prompt with history. In expert human evaluation on Qwen2.5-14B outputs, it achieves Empathy 4.20, Personalization 3.48, Understanding 3.82, the highest average 3.81, and best overall rank 2.12. In a user study with 50 seekers on their own posts, it leads on all six dimensions, including Empathy 3.98, Personalization 3.70, Trustworthiness/Safety 3.80, average 3.72, and best overall rank 2.04. Ablations show that removing refinement, removing both case formulation and refinement, swapping history, or dropping any formulation dimension reduces performance (Han et al., 20 Jun 2026).

LongEmotion presents a more conditional picture. CoEM consistently enhances performance on several long-context emotional-intelligence tasks, especially Emotion Classification and some Emotion Detection settings, but can hurt document-grounded tasks such as Emotion QA and Emotion Summary when enrichment shifts details. Exact examples include EC for GPT-4o-mini, where Accuracy rises from 28.50 to 48.00 under CoEM, and EC for Qwen3-8B, where it rises from 38.50 to 52.83. For MC-4, gains are smaller but consistent, such as GPT-4o moving from 3.77 to 3.81 and DeepSeek-V3 from 3.99 to 4.34. By contrast, QA F1 for GPT-4o falls from 50.12 in Base to 47.24 under CoEM, and ES scores for all listed models decline under CoEM (Liu et al., 9 Sep 2025).

Outside dialogue, the evidence supports more specialized claims. In co-constructed embodied emotion, MHNG produces the highest inter-agent agreement; in the Original/Original condition, Cohen’s Kappa reaches $(F,N)\rightarrow A$0, ARI improves, and DBS is lowest under MHNG, while TopSim shows no consistent latent convergence, indicating that alignment concentrates at the symbolic layer rather than the perceptual latent representation (Zhang et al., 10 May 2026). In sparse affect measurement, collaborative filtering accurately recovers missing data across images, videos, and trust-game decisions and outperforms mean imputation most strongly when inter-individual variability is high; on emotional videos, NNMF with 5s dilation yields 15–20% error reduction versus mean across sparsity levels (Jolly et al., 2021). CogIntAc validates its interactional framework on action abduction, emotion prediction, and action generation, with RoBERTa_large + IntDic achieving F1 = 72.71 on action abduction and the best “+All” emotion-prediction variant reaching F1 = 63.47 for emotion and 89.72 for satisfaction (Peng et al., 2022). The Moltbook study is descriptive rather than benchmarked, but it reports neutral dominance across bios, posts, and comments, more emotional diversity in comments, and a PSR typology in which Type 3 stimulus-driven behavior is the largest proportion among classified agents (Hasan et al., 19 May 2026).

5. Domains of use and operational settings

One major CoEM domain is dialogue systems that must balance factual reliability with emotional appropriateness. EmoLLM explicitly evaluates emotional support, technical assistance, medical consultation, and academic peer review discourse, showing that appraisal-grounded strategy selection can be deployed outside canonical counseling tasks (Zhang et al., 17 Mar 2026). MindTailor addresses a more specific history-aware support setting on Reddit, where present distress is interpreted through prior post history and refined through strategy-specialized counselors (Han et al., 20 Jun 2026). LongEmotion generalizes the problem to long-context Emotion Classification, Emotion Detection, Emotion QA, Emotion Conversation, Emotion Summary, and Emotion Expression, with an average input length of 8,777 tokens and long-form generation for Emotion Expression (Liu et al., 9 Sep 2025). The survey situates these systems within broader affective understanding and affective generation workloads, including ABSA, ERC, ESC, emotional storytelling, and multimodal affect recognition (Lai et al., 2 Jun 2025).

A second domain is embodied interaction. Commonaiverse uses full-body motion tracking and adaptive audiovisual feedback in a closed-loop installation, shifting emotion modeling away from facial-only recognition toward participant-driven, co-authored movement interpretation (Tütüncü et al., 26 Sep 2025). The co-constructed-emotion study extends embodiment to agent–agent communication using visual, auditory, and simulated interoceptive inputs, with shared emotional meaning emerging through selective symbolic exchange rather than direct state sharing (Zhang et al., 10 May 2026). In human–robot collaboration, the cobot study examines whether production rhythm modulates emotional state and experiential locus of control. It finds no difference in emotional state or ELoC across Slow, Fast, and Adaptive conditions, but performance changes strongly—12.90±1.11 assemblies in C1, 17.84±2.34 in C2, and 16.19±1.70 in C3—and preferences split across fast and adaptive pacing, suggesting that rhythm is a powerful control variable even when average affect remains stable (Mondellini et al., 2024).

A third domain is measurement and analysis of collective or longitudinal affect. Collaborative filtering enables minimally disruptive recovery of dense emotional ratings from sparse self-report in experimental settings, including static affective images, moment-by-moment valence during autobiographical stories, and reciprocity decisions in a hidden multiplier trust game (Jolly et al., 2021). Moltbook uses CoEM to model large-scale emotional dynamics in an agent-only social network by assigning structured emotion profiles to personas, posts, and comments and then analyzing PSR alignment and stability across contexts (Hasan et al., 19 May 2026). The persona-ensemble study on emotionally integrated collective intelligence extends CoEM into decision aggregation: it fine-tunes DarkIdol-Llama-3.1-8B with GoEmotions, instantiates 15,064 persona configurations, and examines how emotional diversity changes response patterns and aggregation efficiency in a distance-estimation task (Kadiyala et al., 5 Mar 2025).

6. Limitations, safety, and likely research trajectory

The literature repeatedly emphasizes that CoEM performance depends on scaffolds that may themselves be fragile. EmoLLM relies heavily on LLM-based evaluators and simulators, uses simulated role-play rather than real users, and notes that ARG reasoning traces are scaffolds for training rather than guaranteed faithful explanations of internal model states (Zhang et al., 17 Mar 2026). MindTailor is computationally expensive, uses only three counseling strategies, and reports failure modes such as quote manipulation, risk underestimation, and temporal hallucinations (Han et al., 20 Jun 2026). LongEmotion shows clear task dependence: enrichment can improve emotionally salient retrieval while simultaneously lowering strict source alignment in QA or Summary tasks (Liu et al., 9 Sep 2025). Commonaiverse does not report formal user-study metrics or baselines, and its movement-centric design can miss subtle or overlapping emotions and may introduce ableist exclusion if not carefully adapted (Tütüncü et al., 26 Sep 2025). In the cobot study, the adaptive condition is Wizard-of-Oz rather than autonomously sensed, and the reduced ICI and ELoC reliabilities constrain interpretability (Mondellini et al., 2024). The Moltbook analysis is limited by incomplete PSR triples, translation noise, and dependence on silver emotion labels from a pretrained model (Hasan et al., 19 May 2026).

Safety and ethics are not ancillary concerns in this area. EmoLLM explicitly states that it is not a substitute for professional support in high-stakes domains such as mental health and medicine, and warns that modeling user emotion can increase over-trust or unintended influence, requiring transparency, guardrails, and human oversight (Zhang et al., 17 Mar 2026). MindTailor addresses safety through a risk scan in the case formulation, priority ordering in meta-synthesis, and prompts that discourage clinical overreach in peer contexts (Han et al., 20 Jun 2026). Commonaiverse frames its design against surveillance and extraction logics, emphasizes private space, participant ownership via QR-linked session data, and advocates decolonizing affective AI by foregrounding localized, situated knowledge (Tütüncü et al., 26 Sep 2025). The survey identifies broader unresolved issues: hallucination mitigation, cultural sensitivity, evaluation rigor beyond surface NLG metrics, privacy of affective data, multimodal efficiency, and the cost and alignment of autonomous agent teams (Lai et al., 2 Jun 2025).

Current work suggests several converging future directions. One is richer theory-grounded representation: EmoLLM recommends broader, cross-cultural appraisal taxonomies, explicit theory-of-mind modules, inverse planning, and multi-agent CoEM protocols with shared ARGs (Zhang et al., 17 Mar 2026). MindTailor points toward additional counseling strategies such as motivational interviewing, ACT, and DBT, along with stronger supervision for crisis detection and safer meta-synthesis (Han et al., 20 Jun 2026). The embodied-agent literature suggests replacing simulated interoception with measured physiology and extending communication channels beyond discrete signs to prosody or facial-expression vectors (Zhang et al., 10 May 2026). Commonaiverse indicates a need for formal fairness audits, decentralized or privacy-preserving stewardship, and benchmarks built around co-created, motion-based affect data (Tütüncü et al., 26 Sep 2025). Across the field, a plausible implication is that CoEM will increasingly be defined not by any single model class, but by explicit intermediate emotional objects, collaboration protocols for revising those objects, and evaluation regimes that test whether the resulting systems remain both affectively aligned and operationally reliable across cultures, modalities, and interaction horizons (Lai et al., 2 Jun 2025).