---
title: 'PMEmo: Music Emotion Recognition Benchmark'
url: https://www.emergentmind.com/topics/pmemo
type: topic
---

# PMEmo: Music Emotion Recognition Benchmark

PMEmo is a music emotion recognition dataset centered on valence–arousal annotation and repeatedly used as a benchmark for multimodal affective computing, static Music Emotion Recognition (MER), and Dynamic Music Emotion Recognition (DMER). Across subsequent studies, it is treated both as a corpus that combines music/audio with synchronized electrodermal activity (EDA) and as an audio-only benchmark for dimensional or binarized emotion prediction. In the physiological-signal literature, PMEmo is described as “the currently largest dataset with EDA and music signals for emotion recognition,” with 7962 EDA signal pieces from 457 subjects and music stimuli drawn from 794 songs; in later MER work, it is also used as a 794-song dataset with static and dynamic valence–arousal labels, sometimes under full-song processing and sometimes under excerpt-based protocols [1908.03692] [2008.09743] [2502.03979] [2412.19200].

## 1. Corpus identity and composition

PMEmo is consistently associated with 794 songs, but later papers describe the musical corpus in different ways. One line of work states that music stimuli were drawn from 794 songs and that chorus excerpts were used as emotion stimuli, with each song clip annotated by at least 10 subjects [1908.03692]. A later multimodal study likewise states that “the chorus excerpts clipped from 794 pop songs are collected as the emotion elicitation” and reports 7,962 EDA signal segments from 457 subjects after discarding bad cases [2008.09743]. By contrast, a unified multitask MER study lists PMEmo as 794 full-length Western pop songs and explicitly marks its “Length” as “full” in the experiment table [2502.03979]. Another study describes PMEmo as 794 annotated chorus excerpts from Chinese pop music, originally released with synchronized EDA signals [2512.13998], while a distribution-shift analysis identifies the source material as songs from the Billboard Hot 100, iTunes Top 100 (USA), and UK Top 40 Singles, with manually selected chorus excerpts whose durations range from 11 to 88 seconds [2510.04688].

This divergence is itself informative. It suggests that PMEmo functions less as a single frozen benchmark protocol than as a shared underlying resource whose musical unit can be interpreted as full track, chorus excerpt, or audio-only excerpt depending on downstream methodology. What remains stable across the literature is the dataset’s role as a large-scale valence–arousal resource for popular music, with strong downstream interest in both audio content and physiological response [1908.03692] [2510.04688].

## 2. Annotation structure and label conventions

PMEmo includes two annotation granularities. In the large-scale EDA studies, annotations include dynamic arousal/valence at 2 Hz and a static valence–arousal value per clip [1908.03692]. The DMER work adopts Russell’s two-dimensional valence–arousal space and states that all datasets used provide valence–arousal values every 0.5 seconds (2 Hz), with unstable annotations from the first 15 seconds removed before training and evaluation [2412.19200]. Static labels are central to several MER formulations: the unified multitask framework uses only the static valence–arousal annotations for PMEmo [2502.03979], and the joint-learning study on music and generalized sounds uses static song- or chorus-level labels rather than frame-level or continuous labels [2408.02009].

The literature also applies multiple label conventions to PMEmo. In the subject-independent EDA classification line, the original PMEmo task is described as regression to continuous static valence–arousal, but later work converts it into binary classification by assigning high/low labels via per-subject thresholds computed with k-means clustering over each subject’s static annotations; label 1 indicates high valence/arousal and label 0 indicates low valence/arousal [1908.03692]. A subsequent multimodal framework states that PMEmo valence and arousal originally range from 1 to 9 and again uses subject-wise k-means thresholding to produce binary classes [2008.09743]. In cross-dataset regression, one study states that PMEmo’s original labels are on a continuous scale from 0 to +1 and normalizes all datasets independently to [-1, +1] [2510.04688], whereas the unified multitask framework normalizes all dimensional labels to the range 1–9 across DEAM, PMEmo, and EmoMusic [2502.03979].

These heterogeneous label treatments are not merely cosmetic. They define distinct tasks: continuous regression, binary classification, static MER, dynamic MER, and subject-specific physiological recognition. A further structural limitation emerges in personalized modeling: PMEmo provides the mean and standard deviation of dynamic annotations, but it does not include individual annotator data, so per-annotator meta-learning cannot be applied to it [2412.19200].

## 3. Modalities and preprocessing pipelines

PMEmo is multimodal in origin. In the physiological studies, it contains music/audio and EDA, with EDA sampled at 50 Hz [1908.03692] [2008.09743]. The Res-SIN baseline applies convex optimization-based EDA, or cvxEDA, to decompose each observed signal $y$ into tonic, phasic, and noise components,
$$
y = t + r + \epsilon,
$$
and rewrites this as
$$
y = MA^{-1}p + B\lambda + Cd + \epsilon,
$$
where $r = MA^{-1}p$ is the phasic component and $t = B\lambda + Cd$ is the tonic component [1908.03692]. That work models static versus dynamic emotion changes through tonic and phasic EDA, transforms origin, phasic, and tonic channels into images, and uses a $3 \times 50 \times T$ representation rearranged into gray images of size $T \times 50$ before bilinear resizing to $224 \times 224$ [1908.03692].

RTCAN-1D retains the same origin–phasic–tonic decomposition but processes PMEmo as a three-channel 1D temporal signal rather than as images. It applies z-score normalization before decomposition, discards the first 3 seconds of each trial, length-aligns the EDA by linear interpolation, and feeds the normalized origin, phasic, and tonic channels into a residual temporal and channel attention network [2008.09743]. By contrast, the earlier Res-SIN work discards the first 15 seconds of each EDA sequence to allow an orientation period before processing [1908.03692]. The DMER work on PMEmo similarly removes the first 15 seconds of annotations and slices log Mel-spectrogram sequences at 2 Hz for moment-level prediction [2412.19200].

Audio-only MER papers adopt distinct preprocessing strategies. The unified multitask framework extracts MERT-v1-95M embeddings from 30-second audio segments, concatenates the 5th and 6th layers, randomly selects a contiguous series of segments during training, and averages their embeddings; chord progressions and key are computed over the whole song [2502.03979]. DAMER processes a 60-second window from 15 s to 75 s after track onset, using both a 128-filter Mel spectrogram and an 84-filter cochleagram [2512.13998]. The data-distribution study takes a random 25-second segment from each annotated clip, pads clips shorter than 25 seconds, extracts a 4,800-dimensional Jukebox-5B embedding from layer 36, and concatenates it with a 72-dimensional chroma descriptor [2510.04688]. The joint music-and-generalized-sound study instead extracts the 6,375-dimensional openSMILE ComParE 2013 feature set and places PMEmo and IADS-E into a single common feature space [2408.02009].

## 4. Benchmark roles across research programs

PMEmo has supported several distinct research programs. In large-scale affective computing, it is the substrate for subject-independent emotion recognition from EDA under strong inter-subject variability. Res-SIN establishes what is described as the first attempt to classify large-scale subject-independent emotion with 7962 pieces of EDA signals from 457 subjects, using EDA signal images and music-derived features [1908.03692]. RTCAN-1D extends this multimodal direction by fusing subject-specific EDA with external music features, adding channel-temporal attention and 1D residual modeling to address overfitting and underfitting when hundreds of subjects are involved [2008.09743].

In audio-only MER, PMEmo is used as a dimensional regression benchmark with static valence–arousal labels. The unified multitask framework places PMEmo alongside DEAM and EmoMusic in the dimensional branch of a shared backbone, with selective updates so that PMEmo batches update only the dimensional branch and receive dimensional knowledge-distillation signals from a PMEmo-specific teacher [2502.03979]. The joint-learning study uses PMEmo as the music domain in a multi-domain setup with generalized sounds from IADS-E, arguing for a common emotional space built from the same ComParE 2013 descriptors [2408.02009].

PMEmo also appears in dynamic sequence modeling. DSAML uses PMEmo only for traditional DMER, not for personalized dynamic music emotion recognition, precisely because PMEmo lacks per-annotator labels [2412.19200]. Finally, PMEmo is central to analyses of distribution shift: the cross-dataset generalization study treats it as a mainstream, chart-oriented corpus whose audio and annotation distributions differ notably from DEAM, EmoMusic, and especially classical datasets such as WTC and WCMED [2510.04688]. This suggests that PMEmo is both a benchmark and a diagnostic instrument for probing genre bias, annotation heterogeneity, and the limits of in-domain success.

## 5. Reported baselines and empirical performance

For subject-independent binary classification with multimodal EDA and music, Res-SIN reports fold-averaged binary accuracies of 73.65% for arousal and 73.43% for valence on PMEmo when using three-channel mixed signals and multi-feature fusion [1908.03692]. RTCAN-1D improves these large-scale PMEmo results to Valence—Accuracy 79.68%, F1 82.45%; Arousal—Accuracy 83.76%, F1 86.12% under subject-independent 10-fold cross-validation with disjoint subject folds [2008.09743]. DAMER, in an audio-only binary formulation that thresholds continuous labels at zero, reports Arousal: ACC = 85.98%, F1 = 90.58%, AUC = 74.33, and Valence: ACC = 77.61%, F1 = 85.54%, AUC = 79.71 on a stratified 70%/30% split [2512.13998].

For static dimensional regression, the unified multitask framework reports PMEmo single-dataset performance of $R^2_V = 0.5360$ and $R^2_A = 0.7772$ when using MERT plus chord/key features, with gains to $R^2_V = 0.5473$ and $R^2_A = 0.7940$ when trained jointly on MTG-Jamendo, DEAM, EmoMusic, and PMEmo [2502.03979]. The joint music-and-generalized-sound study reports, for PMEmo-only AutoML training, Valence: RMSE = 0.223, $R^2 = 0.525$ and Arousal: RMSE = 0.193, $R^2 = 0.727$; when trained on IADS-E (with music) plus PMEmo, the same framework reports Valence: RMSE = 0.152, $R^2 = 0.780$ and Arousal: RMSE = 0.137, $R^2 = 0.861$ [2408.02009].

For dynamic sequence prediction, DSAML evaluates PMEmo on a test set of 40 songs after discarding 122 songs shorter than 25 seconds and using 632 songs for training. It reports Arousal—CCC 0.186, PCC 0.386, RMSE 0.112; Valence—CCC 0.108, PCC 0.260, RMSE 0.122, outperforming CRNN, DNN, and MCRNN on CCC and RMSE and surpassing DAMFF on CCC while remaining lower than DAMFF on PMEmo valence PCC [2412.19200].

The strongest cautionary results arise in out-of-distribution testing. When PMEmo is the test set in the Jukebox-based cross-dataset analysis, models trained on other datasets achieve Avg $R^2$ values of 0.04 from EmoMusic, -0.12 from DEAM, 0.02 from WTC, and -0.62 from WCMED, whereas in-domain training on PMEmo yields Avg $R^2 = 0.61$ with arousal 0.72 and valence 0.51 [2510.04688]. The converse is also poor: training on PMEmo does not generalize well to EmoMusic, DEAM, WTC, or WCMED [2510.04688]. The plausible implication is that PMEmo supports strong in-domain modeling while simultaneously exposing the fragility of cross-dataset MER.

## 6. Protocol variability, limitations, and reproducibility

PMEmo does not have a single standard evaluation protocol across the literature. The subject-independent EDA studies use 10-fold cross-validation and report averaged accuracy, F1-score, precision, and recall, but one of them does not explicitly detail whether folds are strictly disjoint by subject [1908.03692], while the later RTCAN-1D study explicitly uses subjects split into 10 disjoint folds with one fold for validation, one for testing, and eight for training [2008.09743]. The unified multitask framework states that PMEmo does not provide official train/validation/test splits and therefore applies a random 70%/15%/15% partition into 536 training songs, 116 validation songs, and 115 test songs [2502.03979]. DAMER uses a stratified 70%/30% train/test split [2512.13998]. The distribution-gap study adopts an 8:1:1 split for in-distribution experiments and uses the entire dataset when PMEmo serves only as an out-of-distribution test set [2510.04688]. The joint music-and-sound study uses 5-fold cross-validation with stratified folds built via Ward clustering of continuous arousal–valence labels [2408.02009].

Several limitations recur. PMEmo’s lack of per-annotator identities prevents personalized meta-learning, even though its dynamic labels are useful for traditional DMER [2412.19200]. Binary formulations collapse continuous affect into two classes; subject-specific thresholds help accommodate subject specificity, but fine-grained modeling is not explored in the baseline EDA classification work [1908.03692]. Later work notes that music features can be contradictory across subjects because the same music can evoke opposite emotions, even when those features improve convergence and accuracy [1908.03692]. In cross-dataset analysis, PMEmo is characterized as concentrated on chart music, with strong dataset-level clustering in learned audio embeddings and poor cross-genre generalization, especially relative to classical corpora [2510.04688]. Static, chorus-focused labels may not capture emotional dynamics or represent the whole song [2510.04688], and some frameworks explicitly leave PMEmo’s dynamic annotations unused [2502.03979].

Reproducibility is uneven but improving. The RTCAN-1D study releases code at `https://github.com/guanghaoyin/RTCAN-1D` [2008.09743]. The joint-learning study releases an end-to-end pipeline at `https://github.com/LIMUNIMI/MusicSoundEmotions` [2408.02009]. DSAML provides a project page and code at `https://littleor.github.io/PDMER` [2412.19200]. DAMER states that Memo2496 and source code are publicly available via Figshare and the authors’ data portal, and that the PMEmo experiments follow the same pipeline and settings [2512.13998]. Taken together, these resources position PMEmo as a benchmark whose scientific value lies not only in its scale and multimodality, but also in how clearly it exposes unresolved issues in split standardization, label heterogeneity, genre bias, and the boundary between individualized and generalized emotion modeling.

Source: https://www.emergentmind.com/topics/pmemo