Affective Recommender Systems

• Affective recommender systems are personalized models that incorporate users' affective states—emotions, moods, and attitudes—in addition to traditional behavioral signals.
• They leverage multi-modal signal acquisition methods such as text analysis, EEG, and acoustic features to infer dynamic user feelings and refine recommendations.
• Hybrid approaches balance stable long-term preferences with transient emotional cues, promoting ethical, privacy-aware designs across diverse application domains.

Affective recommender systems are recommender systems that improve personalization by aligning recommendations not only with what users have liked in the past, but also with their affective states: attitudes, emotions, moods, and combinations of these. In this formulation, recommendation is no longer modeled only as stable utility estimation from ratings, clicks, browsing logs, purchases, or metadata; it also becomes a problem of modeling how the user feels before, during, and after interaction. A psychology-grounded survey organizes the area into four main categories—attitude aware, emotion aware, mood aware, and hybrid—and treats this distinction as foundational because different affective states differ in duration, intensity, and cognitive involvement (Hasan et al., 27 Aug 2025).

1. Conceptual foundations and taxonomy

The most explicit organizing framework for the field is the four-part taxonomy grounded in Scherer’s typology of affective states. In that taxonomy, attitudes are stable evaluative dispositions such as liking, disliking, loving, hating, valuing, and desiring; emotions are intense, short-lived responses to specific stimuli; moods are more diffuse and longer-lasting than emotions; and hybrid systems combine multiple affective-state types, including attitude+emotion, emotion+mood, and the special case of serendipity, which is interpreted as the combination of the emotion of surprise with a positive evaluative attitude toward the outcome (Hasan et al., 27 Aug 2025).

This taxonomy clarifies a distinction that is often blurred in the literature. In attitude-aware systems, the central object is usually evaluative polarity or aspect-level sentiment extracted from reviews, comments, or social text. In emotion-aware systems, the emphasis shifts to stimulus-linked user states, which may be represented categorically, dimensionally, or as latent embeddings. In mood-aware systems, recommendation is conditioned on more persistent state, often in music or wellness applications. Hybrid systems attempt to model the interaction of these layers rather than assuming that a single affective variable is sufficient (Hasan et al., 27 Aug 2025).

A plausible implication is that affective recommendation is best understood as a multi-timescale personalization problem. Stable evaluative structure is useful for long-term alignment, transient emotion is useful for turn-level or session-level adaptation, and mood is useful for broader contextual shaping. The survey explicitly argues that most published systems still isolate one state type and that hybrid affective models remain underdeveloped (Hasan et al., 27 Aug 2025).

2. Affective signal acquisition and representation

The dominant signal source in the literature is text. Reviews, comments, tweets, posts, dialogue utterances, lyrics, and item descriptions are used to infer sentiment, emotion labels, or latent affective embeddings through lexicon-based methods, traditional classifiers, deep sequence models, and language-model-based pipelines. This is particularly visible in attitude-aware recommendation, where text reviews remain the main substrate for explicit affect extraction (Hasan et al., 27 Aug 2025).

A representative text-centric emotion-aware design is the Affective Index proposed for text-based Emotion Aware Recommender Systems. There, subjective passages are mapped into normalized probabilistic values over Ekman’s six emotions, yielding a six-dimensional affective profile:

$$\mathbf{AI}(x) = [p_e(x)]_{e \in E}, \qquad \sum_{e \in E} p_e(x)=1,$$

with

$$E = \{\text{happiness, sadness, anger, fear, surprise, disgust}\}.$$

User profiles are then formed by averaging the Affective Indices of consumed items, and the Affective Index Indicator is a similarity list derived from those profiles, typically using cosine similarity or nearest-neighbor comparison (Leung et al., 2023).

A closely related movie-setting formulation uses movie emotion vectors (MVECs) inferred from movie overviews and user emotion vectors (UVECs) obtained by averaging the vectors of watched movies. In that architecture, both users and items are embedded in the same affective space, and recommendation becomes a similarity problem over those embeddings. The paper’s operational emotion label set is seven-dimensional and includes neutral, happy/joy, sad/sadness, hate, anger, disgust, and surprise, although the paper itself is not fully consistent about the final taxonomy (Leung et al., 2020).

Beyond text, the field uses multimodal and physiological signals when richer affect modeling is required. EEG-SVRec introduces an EEG dataset for short-video recommendation with 30 participants and 3,657 interactions, pairing behavioral logs with six Multidimensional Affective Engagement Scores: valence, arousal, immersion, interest, visual, and auditory. EEG features are extracted as differential entropy over five frequency bands from 62 channels, yielding a 310-dimensional per-second representation (Zhang et al., 2024). In large-scale social media ranking, affective response is modeled through a compact 15-class taxonomy that includes adoring, connected, constructively-angered, destructively-angered, entertained, excited, grateful, informed, inspired, neutral, relaxed, saddened, scared, surprised, and touched, learned from nearly 820k annotated posts and auxiliary behavioral signals (Dwivedi-Yu et al., 2022).

These signal choices reflect two broad strategies. One strategy infers affect from user-generated or item-centered text and treats affect as a semantic property of users and items. The other measures or approximates affect through behavior, multimodal sensing, or physiological traces and treats it as a state variable. The survey identifies both as important, but also notes that multimodal and hybrid affective representations remain comparatively scarce (Hasan et al., 27 Aug 2025).

3. Recommendation models and affect injection mechanisms

Affective information enters recommender pipelines in several recurrent ways: as feature augmentation for user/item representations, as context for ranking, as a similarity space for matching, as a constraint or weight in latent-factor learning, and as a re-ranking criterion for diversity, novelty, or serendipity (Hasan et al., 27 Aug 2025).

The simplest class of models uses affective profile matching. In text-based EARS, recommendation can be reconstructed as similarity between a user affective profile and an item affective profile, with the user profile defined as the centroid of consumed-item affective indices and candidate ranking performed by cosine similarity over those vectors (Leung et al., 2023). In movie recommendation, UVEC/MVEC systems use the same principle, but with item emotion profiles derived from movie overviews and user profiles averaged from watch histories (Leung et al., 2020).

A more elaborate multimodal formulation appears in emotion-driven recommendation for AI-generated content. The proposed Multi-Modal Emotion and Intent Recognition Model uses pretrained encoders—ViT for visual input, Wav2Vec2 for acoustic input, and BERT-base for text—followed by a BERT-based Cross-Modal Transformer and attention-based fusion:

$$F = \alpha_v \hat{V} + \alpha_a \hat{A} + \alpha_t \hat{T}.$$

The fused representation supports joint emotion-intent prediction and personalized ranking through a contextual matching score

$S(u,c) = \langle F_u, E_c \rangle,$

where $F_u$ is the user’s fused emotion-intent embedding and $E_c$ is the content embedding of an AI-generated item (Hu et al., 25 Nov 2025).

Cross-domain affective recommendation extends the same idea across modalities. In art therapy, music-driven preference elicitation is used to recommend paintings. The simplest method, Haydn, compares source-domain music and target-domain paintings only in valence-arousal space; Mozart learns a shared latent space with contrastive alignment guided by continuous affective similarity; and Salieri combines acoustic or visual features with GPT-4o-generated semantic descriptions before cross-domain retrieval (Yilma et al., 18 Jul 2025). A plausible implication is that affect can act as a transfer principle even when the elicitation modality and the recommendation modality differ.

Mood-aware music recommendation adopts a lighter-weight geometric approach. Both songs and user-selected mood are embedded in energy-valence space, and candidate songs are ranked or sampled by distance to the target mood. The deterministic baseline uses squared Euclidean distance,

$D[i] = (V[i]-v)^2 + (E[i]-e)^2,$

while the probabilistic variant converts distance into a Boltzmann-like sampling distribution

$$W[i] = \exp\left(-\frac{D[i]}{k}\right), \qquad P[i] = \frac{W[i]}{\sum_{j=1}^{n} W[j]}.$$

This architecture separates stable taste, handled through candidate generation from listening history, from transient affective need, handled through mood-conditioned re-ranking (Zeng et al., 11 Jun 2026).

At the other end of the design spectrum, Affective Music Recommendation under ethical constraints treats affect as the optimization target itself. AMRS trains a rollout-based world model to jointly predict engagement, binary rating, and self-reported valence and arousal, then fine-tunes a recommender policy offline with Direct Preference Optimization against a configurable utility:

$$R_t = \lambda_r \hat r_t + \lambda_e \hat e_t + \lambda_v \hat v_t + \lambda_a \hat a_t.$$

In the reported experiments, the utility is affect-only, with $\lambda_v = \lambda_a = 0.5$ and $$E = \{\text{happiness, sadness, anger, fear, surprise, disgust}\}.$$0, so policy optimization explicitly targets predicted valence and arousal rather than engagement (Chan et al., 27 May 2026).

4. Conversational, human-centered, and privacy-aware affective recommendation

Affective recommendation is not limited to static ranking; it also appears as dialogue policy, explanation style, memory design, and privacy architecture. Conversational recommender systems are especially important here because affect can influence both what is recommended and how the recommendation is communicated.

An explicit example is the Empathetic Conversational Recommender. ECR introduces empathy as the capacity to capture and express emotions and decomposes the problem into emotion-aware item recommendation and emotion-aligned response generation. User utterances are annotated into nine main emotion labels—like, curious, happy, grateful, negative, neutral, nostalgia, agreement, and surprise—and these labels are used to refine entity representations for ranking. Response generation is trained on emotional IMDb reviews with retrieval-augmented prompts, and the framework is evaluated not only with ranking metrics but also with emotional intensity, emotional persuasiveness, logic persuasiveness, informativeness, lifelikeness, and direct satisfaction judgments (Zhang et al., 2024).

A broader conversational framing appears in Recommendation-as-Experience. That framework treats affect not as a narrow classification target, but as an interactional aim: recommendation should align emotionally and socially with the user. The state representation encodes domain profile, item value, user traits, autonomy preferences, and dialogue history, and a policy maps that state to a three-way aim vector over educative, explorative, and affective behavior. In this view, affective recommendation becomes a dialogue-control problem as much as a ranking problem (Mahmud et al., 12 Jan 2026).

Human-centered assistant frameworks occupy an adjacent position. RAH introduces an LLM-based assistant with Perceive, Learn, Act, Critic, and Reflect agents to mediate between the user and the recommender system, emphasizing user control, burden reduction, privacy, and personality alignment. It does not directly model mood, emotion, or sentiment trajectories, so it is not an affective recommender in the strict sense; however, it is relevant because it treats subjective fit, psychological comfort, privacy, and user agency as first-class objectives in the recommendation loop (Shu et al., 2023).

Privacy-aware designs also appear in explicitly affective systems. The text-based EARS paper advocates a separation-of-responsibility model in which users retain control over emotional profile data through an emotion ID, while service providers update affective indices only in memory and refrain from persistent storage of the emotional profile. This is an architectural privacy mechanism based on non-retention rather than cryptographic privacy, differential privacy, or a formal adversarial model (Leung et al., 2023). That design highlights a recurrent tension in affective recommendation: the more affect is made actionable, the more emotionally sensitive data become part of the personalization pipeline.

5. Application areas and empirical evidence

The field spans a wide range of domains, including music, movies, tourism, education, healthcare, e-commerce, social media, short video, art therapy, and AI-generated content (Hasan et al., 27 Aug 2025). Reported results are heterogeneous because the literature mixes conceptual proposals, offline benchmarks, user studies, and production-scale deployments.

Domain	Representative system	Reported evidence
AIGC	MMEI	F1 91.4; MAP 0.872; NDCG 0.894; HR@10 0.923; engagement time +15.2%; satisfaction +11.8% (Hu et al., 25 Nov 2025)
Social media	Affective response embedding	More than 8% AUC-ROC loss reduction in a survey-prediction module; more than 0.6% decreases in surfaced violating content; around 0.4% improvement in likes (Dwivedi-Yu et al., 2022)
Music	AMRS	Valence $$E = \{\text{happiness, sadness, anger, fear, surprise, disgust}\}.$$1, arousal $$E = \{\text{happiness, sadness, anger, fear, surprise, disgust}\}.$$2, both $$E = \{\text{happiness, sadness, anger, fear, surprise, disgust}\}.$$3, with no distributional collapse seen in greedy baselines (Chan et al., 27 May 2026)
Books	ACRec	HR@10 15.6 and NDCG@10 10.1 in full-item ranking with A+AC+C descriptions; Top-1 accuracy 67.6 in 20-item ranking (Hasan et al., 26 May 2025)
Short video	EEG-SVRec benchmarks	Significant improvement with EEG on multiple feedback-prediction tasks, including Like, Immersion, Valence, Arousal, and AudioPref in several models (Zhang et al., 2024)
Mood-aware music	Energy-valence reranking	Mean rating 3.59 versus 2.67 for the control, with $$E = \{\text{happiness, sadness, anger, fear, surprise, disgust}\}.$$4 in a single-blind user study (Zeng et al., 11 Jun 2026)

These results illustrate several distinct evidential styles. Some papers report classical recommender metrics together with affective or online engagement outcomes. The AIGC system improves both recognition metrics and ranking metrics, and also reports user-level online gains in engagement time and satisfaction (Hu et al., 25 Nov 2025). The social media system integrates affective response as a learned embedding inside a production ranker and reports both offline module gains and online shifts in integrity-related and engagement-related metrics (Dwivedi-Yu et al., 2022). AMRS uses a world model as both simulator and safety filter, reporting predicted gains in valence and arousal under a strict cold-start protocol while monitoring coverage, entropy, intra-list diversity, and Gini concentration (Chan et al., 27 May 2026).

Other application areas remain more demonstrative. The tourist recommender for Hiroshima sightseeing integrates Emotion Generating Calculations with Google search popularity and TF-IDF scores through

$$E = \{\text{happiness, sadness, anger, fear, surprise, disgust}\}.$$5

but its evidence is scenario-based rather than benchmark-driven (Ichimura et al., 2018). The design-aesthetics recommender asks new users for a wanted affect through Kansei adjectives and reports average recommendation accuracy of 92% for “Feminine” and 64% for “Technological,” again in a small proof-of-concept setting (Benaissa et al., 2023). Group recommendation work based on the Affective Aware Pseudo Association Method shows how group profiles can be formed and re-ranked in affective space, but it does not provide standard comparative recommender metrics (Leung et al., 2021).

A plausible conclusion from these domain studies is that affective recommendation currently exhibits two coexisting research cultures. One culture integrates affect into industrial or large-scale learning pipelines and evaluates downstream ranking impact with strong metrics. The other uses affect to propose new personalization principles—emotional fit, group mood dynamics, therapeutic resonance, wanted affect, or serendipity—even when evaluation remains exploratory.

6. Limitations, controversies, and future directions

Several limitations recur across the literature. The first is conceptual. The survey argues that the field often relies on an imprecise “folk vocabulary” of affective states and calls for more precise terminology grounded in cognitive and social psychology, especially when distinguishing sentiment, attitude, emotion, and mood (Hasan et al., 27 Aug 2025). This concern is not merely terminological: it affects dataset design, supervision choice, model timescale, and the interpretation of what a recommender is actually optimizing.

The second is representational. Emotion labels are subjective, context-dependent, and culturally variable. The text-based EARS paper emphasizes that emotional labels are difficult to collect because the same passage may evoke different emotions across users, contexts, cultures, times, and situations, and it explicitly raises the unresolved question of whose emotion is being represented when a model assigns probabilities to content (Leung et al., 2023). In mood-aware music systems, explicit mood input is interpretable and controllable, but it introduces burden, self-report bias, and ambiguity between current mood and desired mood (Zeng et al., 11 Jun 2026).

The third is evidential. Many affective recommenders remain conceptual or weakly benchmarked. The tourist system, the group recommendation system, and the text-based privacy-oriented EARS framework all illustrate important design ideas but do not provide the kind of controlled train/test evaluation or large-scale user validation that would isolate the contribution of affect from general model capacity (Ichimura et al., 2018, Leung et al., 2021, Leung et al., 2023). Even stronger multimodal systems can leave causal attribution unresolved: the AIGC paper does not provide ablations that isolate the effect of emotion and intent modeling from the effect of richer multimodal user modeling in general (Hu et al., 25 Nov 2025).

The fourth is ethical and operational. Rich sensing and affect optimization raise privacy and manipulation concerns. The social media system explicitly models viewer affective response at production scale (Dwivedi-Yu et al., 2022); EEG-SVRec shows that physiological data can improve prediction but also makes clear that EEG is expensive, intrusive, and privacy-sensitive (Zhang et al., 2024); AMRS argues that online experimentation on emotion can be ethically untenable in clinical populations and therefore shifts optimization offline into a world-model-and-stress-testing pipeline (Chan et al., 27 May 2026). This suggests that affective recommendation must often be designed under constraints that do not arise in ordinary click-optimization settings.

The most consistent future directions are likewise shared. The survey emphasizes hybrid models that leverage multiple types of affective states across different modalities, the development of large-scale affect-aware datasets, and the replacement of imprecise affect terminology with more psychologically grounded vocabularies (Hasan et al., 27 Aug 2025). The text-based EARS paper calls for benchmark datasets, validated emotion representations, and calibration studies for LLM-based affect detection (Leung et al., 2023). Cross-domain and multimodal work suggests that affect can serve as a bridge across modalities rather than only as a side feature (Yilma et al., 18 Jul 2025). Offline optimization under safety constraints suggests that deployed affective systems may increasingly need simulator-based evaluation, policy stress testing, and explicit guardrails before exposure to real users (Chan et al., 27 May 2026).

Taken together, these directions suggest that affective recommender systems are moving away from the narrow idea of “emotion-aware ranking” toward a broader formulation in which attitudes, emotions, moods, multimodal affective signals, privacy constraints, conversational adaptation, and user control are all part of the personalization problem. The strongest papers in the area no longer treat affect as an ornament on top of conventional recommendation; they treat it as a modeling principle that changes what counts as relevance, what counts as utility, and what counts as a satisfactory recommendation experience (Hasan et al., 27 Aug 2025).