Quantifying Valence and Arousal in Text with Multilingual Pre-trained Transformers (2302.14021v1)

Abstract: The analysis of emotions expressed in text has numerous applications. In contrast to categorical analysis, focused on classifying emotions according to a pre-defined set of common classes, dimensional approaches can offer a more nuanced way to distinguish between different emotions. Still, dimensional methods have been less studied in the literature. Considering a valence-arousal dimensional space, this work assesses the use of pre-trained Transformers to predict these two dimensions on a continuous scale, with input texts from multiple languages and domains. We specifically combined multiple annotated datasets from previous studies, corresponding to either emotional lexica or short text documents, and evaluated models of multiple sizes and trained under different settings. Our results show that model size can have a significant impact on the quality of predictions, and that by fine-tuning a large model we can confidently predict valence and arousal in multiple languages. We make available the code, models, and supporting data.

• The paper presents a method for quantifying continuous emotional dimensions (valence and arousal) in text using multilingual pre-trained Transformer models like XLM-RoBERTa on a large dataset spanning 13 languages.
• The study found that larger multilingual Transformer models, like XLM-RoBERTa large, significantly outperform smaller models in predicting valence and arousal across languages.
• Predicting arousal proved more challenging than valence, and while models generalize to unseen languages, zero-shot performance shows some degradation compared to fine-tuning.

The paper "Quantifying Valence and Arousal in Text with Multilingual Pre-trained Transformers" explores the application of dimensional emotion analysis in contrast to the categorical approaches typically used in sentiment analysis. It focuses on predicting emotional dimensions of valence and arousal from textual data using multilingual Transformer models. This approach leverages pre-trained models adapted for multilingual context, particularly focusing on models like DistilBERT and XLM-RoBERTa, which are fine-tuned for the task of regression rather than classification.

The paper compiles an extensive dataset of 34 publicly available psycho-linguistic resources, annotated for valence and arousal, spanning 13 languages, creating a comprehensive multilingual corpus for training and evaluating these models. The authors highlight the potential of dimensional emotion analysis for providing nuanced understanding by measuring affective states on continuous scales, specifically in a two-dimensional valence-arousal space.

The key findings indicate that model size significantly affects prediction quality, with larger models like XLM-RoBERTa large outperforming smaller ones in terms of achieving higher Pearson correlation and lower error metrics such as RMSE and MAE for both valence and arousal across datasets. The results illustrate the capability of these models to generalize across languages when fine-tuned with multilingual training datasets. However, predicting arousal tends to be more challenging compared to valence, corresponding to observed variability in human annotations.

The paper further explores the impact of different loss functions on training models, ranging from standard Mean Squared Error (MSE) to more complex ones like the Concordance Correlation Coefficient Loss (CCCL) and Robust Loss (RL), identifying MSE as the simplest yet effective option for training. Additionally, experiments in zero-shot settings confirm the ability of these models to generalize to languages not encountered during fine-tuning, albeit with some degradation in performance.

Ultimately, this paper provides extensive empirical evidence supporting the utility of pre-trained multilingual Transformers for predicting affective dimensions in text, alongside valuable insights into optimizing performance through model and dataset considerations. The accompanying code, trained models, and datasets are made publicly available, fostering further exploration and application of dimensional emotion analysis in multilingual contexts.