EmoHeal: Personalized Digital Wellness

• EmoHeal is an end-to-end digital wellness system that converts free-form emotional text into personalized, three-stage therapeutic audiovisual experiences based on music retrieval.
• It employs fine-grained emotion recognition using a multilingual XLM-RoBERTa classifier combined with a theory-driven emotion–music knowledge graph to map emotions to musical parameters.
• The system’s retrieval architecture uses over 600 curated 3-minute 4K video clips and an immersive web interface to deliver guided, supportive narratives that enhance mental wellness.

EmoHeal is an end-to-end, theory-driven digital wellness system that converts a user’s free-form emotional text into a personalized therapeutic audiovisual experience centered on music retrieval and delivery. It is presented as a response to a gap in mainstream digital mental wellness tools, which are described as largely static content libraries requiring users to self-diagnose, browse pre-recorded materials, and choose interventions manually. EmoHeal instead combines multilingual fine-grained emotion recognition, a music-therapy-informed emotion–music knowledge graph, multimodal retrieval with CLaMP3, and an immersive web interface to deliver personalized, three-stage supportive narratives organized as “match-guide-target” (Wan et al., 19 Sep 2025).

1. Conceptual framing and therapeutic rationale

EmoHeal is motivated by the claim that digital mental wellness depends on responding to nuanced emotional states rather than only to coarse labels such as positive, negative, happy, or sad. The system is explicitly designed around the idea that therapeutic benefit requires the user to feel accurately recognized and “understood,” and that many existing systems risk an “emotional mismatch” by offering content that is soothing in the abstract but poorly aligned with the user’s immediate affective condition (Wan et al., 19 Sep 2025).

Its theoretical foundation is drawn from music therapy and music psychology, especially GEMS and the iso-principle. In EmoHeal, the iso-principle is operationalized as a three-stage “match-guide-target” narrative: the system first matches the user’s current state, then provides intermediate material, and finally guides toward a calmer endpoint. The paper repeatedly describes the output as “personalized, three-stage supportive narratives,” positioning the system as a guided micro-intervention rather than a recommendation engine (Wan et al., 19 Sep 2025).

This structure also defines the scope of personalization. Personalization arises from the user’s own text and from the system’s fine-grained interpretation of that text, rather than from long-term user modeling, listening history, physiological sensing, or reinforcement from repeated use. The system is therefore personalized at the session level and text-driven rather than longitudinal or adaptive (Wan et al., 19 Sep 2025).

2. Fine-grained emotion computation

The first module is fine-grained emotion computation using a fine-tuned XLM-RoBERTa-base classifier. Architecturally, the model uses the backbone encoder plus a linear classification head of size $768 \times 27$ on top of the model’s $[CLS]$ token representation, producing a 27-dimensional probability vector

$e \in \mathbb{R}^{27}.$

Each dimension corresponds to one of the 27 fine-grained emotion classes defined by the GoEmotions dataset: admiration, amusement, anger, annoyance, approval, caring, confusion, curiosity, desire, disappointment, disapproval, disgust, embarrassment, excitement, fear, gratitude, grief, joy, love, nervousness, optimism, pride, realization, relief, remorse, sadness, and surprise (Wan et al., 19 Sep 2025).

The model is fine-tuned on a custom multi-corpus dataset combining the English GoEmotions dataset with the NLPCC-2014 Chinese Emotion Analysis Dataset. Because NLPCC-2014 has a coarser label space, the authors created a programmatic label mapping from coarse Chinese labels to multi-label GoEmotions targets; the paper’s example maps a coarse label such as joy to a multi-hot target in which joy, amusement, and excitement are all activated. This is the mechanism by which multilingual capability is achieved despite non-identical taxonomies (Wan et al., 19 Sep 2025).

Training uses Focal Loss,

$$L_{cls} = -\alpha_{t}(1-p_{t})^{\gamma}\log(p_{t}),$$

with focusing parameter

$\gamma = 2.$

Optimization uses AdamW with learning rate $2\times10^{-5}$ and 5-fold cross-validation. Reported validation performance is Macro-F1 = 0.64 and Weighted-F1 = 0.71. The paper does not provide per-class scores, thresholding strategy, tokenization settings, or explicit preprocessing beyond multilingual user text input, so the classifier is specified more clearly at the architectural and objective level than at the inference-policy level (Wan et al., 19 Sep 2025).

The practical significance of this module is empirical as well as conceptual. The paper reports a strong correlation between perceived emotion recognition accuracy and supportive outcome,

$r = 0.72,\quad p < 0.001,$

which is the central quantitative support for the claim that fine-grained recognition is therapeutically relevant rather than a merely cosmetic technical refinement (Wan et al., 19 Sep 2025).

3. Emotion–music knowledge graph and the “match-guide-target” narrative

The central symbolic layer in EmoHeal is the emotion–music knowledge graph, which translates the 27-dimensional emotion vector into six musically meaningful parameters: tempo, mode, timbre, harmony, register, and density. Its output is a six-parameter vector $p$, where tempo is continuous, such as 60–120 BPM, and the remaining dimensions are represented through musically meaningful contrasts such as major/minor, bright/dark, consonant/dissonant, high/low, and sparse/dense (Wan et al., 19 Sep 2025).

The graph uses a two-tier inference design. In the first tier, if one primary emotion exceeds an empirically chosen high-intensity threshold,

$\tau = 0.7,$

the system applies an expert rule of the form

$\text{IF } e_i > \tau \text{ THEN } p_j = v.$

This rule-based path is intended for unambiguous emotional states and provides fast, interpretable mappings (Wan et al., 19 Sep 2025).

If no single emotion exceeds the threshold, the state is treated as a blended or complex emotion. The system then uses a handcrafted, theory-driven weight matrix

$[CLS]$0

with weights normalized to $[CLS]$1, to compute

$[CLS]$2

where $[CLS]$3 and $[CLS]$4 is a normalization function. The paper frames this as smooth blending rather than categorical switching, and explicitly characterizes the layer as interpretable and theory-grounded in neuro-symbolic terms (Wan et al., 19 Sep 2025).

The output of this mapping is not a raw control signal shown directly to the user. Instead, the six parameters are rendered into a natural-language description that becomes the query for retrieval. The paper does not reproduce the exact template, but it states that the prompt is deterministic and parameter-driven. This matters because the therapeutic narrative in EmoHeal is built from the sequence linking detected emotion, mapped musical attributes, and selected audiovisual clips; it is not generated as long-form therapeutic prose (Wan et al., 19 Sep 2025).

4. Retrieval architecture, audiovisual corpus, and interface

EmoHeal uses retrieval rather than generation in order to preserve content quality and reduce computational cost. The audiovisual library contains over 600 3-minute 4K video clips drawn from commercially licensed long-form films by Nature Relaxation. Curation follows environmental psychology, especially Attention Restoration Theory. Long videos are segmented automatically by scene-boundary detection using color histograms, then “calm segments” are identified based on low motion magnitude via optical flow, and these are partitioned into non-overlapping 3-minute clips (Wan et al., 19 Sep 2025).

For each clip, the audio track is embedded offline using a pretrained VGGish model and temporally average-pooled into a fixed 128-dimensional embedding. These embeddings are indexed in Faiss using an Inverted File (IVF) index. At retrieval time, the natural-language prompt produced from the six music parameters is encoded by the CLaMP3 text encoder, which places text and music/audio in a shared multimodal embedding space. Matching is then performed by cosine similarity, selecting clips $[CLS]$5 that maximize

$[CLS]$6

where $[CLS]$7 is the CLaMP3 text embedding and $[CLS]$8 is the indexed audio embedding. The system returns the top 3 most similar videos (Wan et al., 19 Sep 2025).

The interface is a web application with an HTML5/CSS3/JavaScript frontend using Bootstrap and a Python Flask REST API backend. The user-facing design follows Calm Technology principles through a dark theme, low cognitive load, progressive disclosure, and ephemeral data handling. The paper’s emphasis here is not merely aesthetic: the interface is part of the supportive narrative logic, aiming to avoid overwhelming users in vulnerable emotional states while maintaining immersion and presence (Wan et al., 19 Sep 2025).

5. Empirical evaluation

The empirical study is a within-subjects evaluation with $[CLS]$9 participants recruited mainly from Queen Mary University of London through snowball sampling via university-affiliated WhatsApp and WeChat student groups. The sample had mean age 26.2 years with SD = 4.8, and included 22 female and 18 male participants; 60.0% reported English as their primary language and 40.0% Chinese. The study excluded participants with severe mental health conditions requiring clinical intervention (Wan et al., 19 Sep 2025).

In the protocol, each participant described their current mood in text, watched the system-generated 3-minute therapeutic video, and completed a post-session questionnaire. Outcomes were measured on 5-point Likert scales from 1 (“Strongly Disagree”) to 5 (“Strongly Agree”). The key descriptive results are summarized below.

Measure	Reported value
Positive mood impact	$e \in \mathbb{R}^{27}.$0
Perceived emotion accuracy	$e \in \mathbb{R}^{27}.$1
Overall atmosphere	$e \in \mathbb{R}^{27}.$2
Multimodal coherence	$e \in \mathbb{R}^{27}.$3

All one-sample $e \in \mathbb{R}^{27}.$4-tests were conducted against the neutral midpoint of 3.0, with all $e \in \mathbb{R}^{27}.$5. In addition, 85.0% of participants rated emotion accuracy as 4 or 5, 87.5% rated mood improvement as 4 or 5, and 78% explicitly mentioned feeling “understood” by the system. A second correlation,

$e \in \mathbb{R}^{27}.$6

linked text length with perceived emotion accuracy, suggesting that richer self-expression may improve perceived responsiveness (Wan et al., 19 Sep 2025).

The evaluation is intentionally framed as preliminary. The study is single-session, non-clinical, and based on self-report rather than physiological, behavioral, or clinical endpoints. The paper also notes that it does not include ablations against coarse-emotion classifiers, non-personalized retrieval, random retrieval, or existing wellness applications. The findings therefore support perceived supportive effect and perceived recognition accuracy in a brief intervention setting, but do not establish long-term efficacy or comparative clinical benefit (Wan et al., 19 Sep 2025).

6. Limitations, boundaries, and relation to adjacent research

EmoHeal is explicitly presented as a mental wellness support tool, not a substitute for care. Its limitations are both methodological and architectural. The sample is relatively homogeneous and mainly student-based; the study is non-clinical; the classifier depends partly on heuristic cross-dataset label mapping; the knowledge graph’s weight matrix is handcrafted; and personalization remains session-level and text-driven rather than longitudinal, multimodal, or feedback-adaptive (Wan et al., 19 Sep 2025).

Several adjacent systems clarify what EmoHeal does not yet do. “Empathy-R1” introduces a Chain-of-Empathy reasoning scaffold with reinforcement learning for long-form Chinese mental health support, focusing on transparent analysis of emotions, causes, intentions, and response strategy rather than retrieval-based audiovisual intervention (Yao et al., 18 Sep 2025). “HealMe” treats cognitive reframing as a structured multi-turn psychotherapy dialogue rather than as content retrieval, emphasizing situation-thought-feeling decomposition and guided alternative interpretation (Xiao et al., 2024). “EmoStage” adds inference-only perspective-taking and counseling phase recognition to regulate when empathy, exploration, or problem-solving is appropriate in supportive dialogue (Qi et al., 24 Jun 2025).

Other work points to extensions beyond text-only session personalization. “EmojiHeroVR” shows that facial expression recognition under head-mounted display occlusion is feasible in VR, which suggests one route toward affect-aware immersive wellness settings (Ortmann et al., 2024). “PHemoNet” and Hyper-MML address multimodal physiological emotion recognition from EEG and peripheral signals, indicating possible future integration of involuntary biosignals into emotion-aware support pipelines (Lopez et al., 2024, Kang et al., 28 Feb 2025). “EmoAgent” reframes emotional support as a safety problem, using simulated vulnerable users, PHQ-9, PDI, and PANSS to assess deterioration and adding an intermediary safeguard agent to reduce harmful interaction patterns (Qiu et al., 13 Apr 2025). “MemEmo” further shows that current memory systems remain weak at extracting, updating, and querying emotional history over long interactions, implying that persistent emotionally grounded personalization is still an open systems problem (Liu et al., 27 Feb 2026).

Taken together, these relations suggest a broader interpretation of EmoHeal’s contribution. It does not attempt to solve clinical dialogue, multimodal sensing, long-term emotional memory, or crisis-safe intervention in a single model. Instead, it provides a concrete blueprint for operationalizing fine-grained affect recognition, interpretable music-therapy mapping, and retrieval-based supportive narratives inside a scalable digital wellness pipeline (Wan et al., 19 Sep 2025).