Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
41 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
41 tokens/sec
o3 Pro
7 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

LastResort at SemEval-2024 Task 3: Exploring Multimodal Emotion Cause Pair Extraction as Sequence Labelling Task (2404.02088v1)

Published 2 Apr 2024 in cs.CL, cs.SD, and eess.AS

Abstract: Conversation is the most natural form of human communication, where each utterance can range over a variety of possible emotions. While significant work has been done towards the detection of emotions in text, relatively little work has been done towards finding the cause of the said emotions, especially in multimodal settings. SemEval 2024 introduces the task of Multimodal Emotion Cause Analysis in Conversations, which aims to extract emotions reflected in individual utterances in a conversation involving multiple modalities (textual, audio, and visual modalities) along with the corresponding utterances that were the cause for the emotion. In this paper, we propose models that tackle this task as an utterance labeling and a sequence labeling problem and perform a comparative study of these models, involving baselines using different encoders, using BiLSTM for adding contextual information of the conversation, and finally adding a CRF layer to try to model the inter-dependencies between adjacent utterances more effectively. In the official leaderboard for the task, our architecture was ranked 8th, achieving an F1-score of 0.1759 on the leaderboard.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (37)
  1. Muhammad Abdul-Mageed and Lyle Ungar. 2017. EmoNet: Fine-grained emotion detection with gated recurrent neural networks. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 718–728, Vancouver, Canada. Association for Computational Linguistics.
  2. wav2vec 2.0: A framework for self-supervised learning of speech representations. Advances in neural information processing systems, 33:12449–12460.
  3. Chatbot-based emotion management for distributed teams: A participatory design study. Proceedings of the ACM on Human-Computer Interaction, 4(CSCW2):1–30.
  4. Iemocap: interactive emotional dyadic motion capture database. Language Resources and Evaluation, 42:335–359.
  5. Wavlm: Large-scale self-supervised pre-training for full stack speech processing. IEEE Journal of Selected Topics in Signal Processing, 16(6):1505–1518.
  6. Emotion cause detection with linguistic constructions. In Proceedings of the 23rd International Conference on Computational Linguistics (Coling 2010), pages 179–187, Beijing, China. Coling 2010 Organizing Committee.
  7. GoEmotions: A dataset of fine-grained emotions. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4040–4054, Online. Association for Computational Linguistics.
  8. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805.
  9. ECPE-2D: Emotion-cause pair extraction based on joint two-dimensional representation, interaction and prediction. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 3161–3170, Online. Association for Computational Linguistics.
  10. Simon D’Alfonso. 2020. Ai in mental health. Current Opinion in Psychology, 36:112–117.
  11. Paul Ekman et al. 1999. Basic emotions. Handbook of cognition and emotion, 98(45-60):16.
  12. Imagebind: One embedding space to bind them all. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 15180–15190.
  13. Emotion cause extraction, a challenging task with corpus construction. In Social Media Processing, pages 98–109, Singapore. Springer Singapore.
  14. A sentiment-and-semantics-based approach for emotion detection in textual conversations. arXiv preprint arXiv:1707.06996.
  15. Deberta: Decoding-enhanced bert with disentangled attention. arXiv preprint arXiv:2006.03654.
  16. Bidirectional lstm-crf models for sequence tagging.
  17. The kinetics human action video dataset.
  18. Conditional random fields: Probabilistic models for segmenting and labeling sequence data.
  19. A text-driven rule-based system for emotion cause detection. In Proceedings of the NAACL HLT 2010 Workshop on Computational Approaches to Analysis and Generation of Emotion in Text, pages 45–53, Los Angeles, CA. Association for Computational Linguistics.
  20. Towards building a social emotion detection system for online news. Future Generation Computer Systems, 37:438–448. Special Section: Innovative Methods and Algorithms for Advanced Data-Intensive Computing Special Section: Semantics, Intelligent processing and services for big data Special Section: Advances in Data-Intensive Modelling and Simulation Special Section: Hybrid Intelligence for Growing Internet and its Applications.
  21. Ecpec: Emotion-cause pair extraction in conversations. IEEE Transactions on Affective Computing, 14(3):1754–1765.
  22. Mvitv2: Improved multiscale vision transformers for classification and detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4804–4814.
  23. Si-lstm: Speaker hybrid long-short term memory and cross modal attention for emotion recognition in conversation. arXiv preprint arXiv:2305.03506.
  24. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692.
  25. The semaine database: Annotated multimodal records of emotionally colored conversations between a person and a limited agent. IEEE Transactions on Affective Computing, 3(1):5–17.
  26. Endang Wahyu Pamungkas. 2019. Emotionally-aware chatbots: A survey.
  27. MELD: A multimodal multi-party dataset for emotion recognition in conversations. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 527–536, Florence, Italy. Association for Computational Linguistics.
  28. Recognizing emotion cause in conversations. CoRR, abs/2012.11820.
  29. Long short-term memory based recurrent neural network architectures for large vocabulary speech recognition. arXiv preprint arXiv:1402.1128.
  30. Towards emotion-and time-aware classification of tweets to assist human moderation for suicide prevention. In Proceedings of the International AAAI Conference on Web and Social Media, volume 15, pages 609–620.
  31. Multimodal emotion-cause pair extraction in conversations. CoRR, abs/2110.08020.
  32. Multimodal emotion-cause pair extraction in conversations. IEEE Transactions on Affective Computing, 14(3):1832–1844.
  33. Semeval-2024 task 3: Multimodal emotion cause analysis in conversations. In Proceedings of the 18th International Workshop on Semantic Evaluation (SemEval-2024).
  34. Effective inter-clause modeling for end-to-end emotion-cause pair extraction. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 3171–3181, Online. Association for Computational Linguistics.
  35. Rui Xia and Zixiang Ding. 2019a. Emotion-cause pair extraction: A new task to emotion analysis in texts. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 1003–1012.
  36. Rui Xia and Zixiang Ding. 2019b. Emotion-cause pair extraction: A new task to emotion analysis in texts. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 1003–1012, Florence, Italy. Association for Computational Linguistics.
  37. Jeewoo Yun and Jungkun Park. 2022. The effects of chatbot service recovery with emotion words on customer satisfaction, repurchase intention, and positive word-of-mouth. Frontiers in psychology, 13:922503.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (4)
  1. Suyash Vardhan Mathur (3 papers)
  2. Akshett Rai Jindal (2 papers)
  3. Hardik Mittal (2 papers)
  4. Manish Shrivastava (62 papers)
Citations (1)
View on Semantic Scholar