Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
125 tokens/sec
GPT-4o
47 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

From Multilingual Complexity to Emotional Clarity: Leveraging Commonsense to Unveil Emotions in Code-Mixed Dialogues (2310.13080v1)

Published 19 Oct 2023 in cs.CL and cs.AI

Abstract: Understanding emotions during conversation is a fundamental aspect of human communication, driving NLP research for Emotion Recognition in Conversation (ERC). While considerable research has focused on discerning emotions of individual speakers in monolingual dialogues, understanding the emotional dynamics in code-mixed conversations has received relatively less attention. This motivates our undertaking of ERC for code-mixed conversations in this study. Recognizing that emotional intelligence encompasses a comprehension of worldly knowledge, we propose an innovative approach that integrates commonsense information with dialogue context to facilitate a deeper understanding of emotions. To achieve this, we devise an efficient pipeline that extracts relevant commonsense from existing knowledge graphs based on the code-mixed input. Subsequently, we develop an advanced fusion technique that seamlessly combines the acquired commonsense information with the dialogue representation obtained from a dedicated dialogue understanding module. Our comprehensive experimentation showcases the substantial performance improvement obtained through the systematic incorporation of commonsense in ERC. Both quantitative assessments and qualitative analyses further corroborate the validity of our hypothesis, reaffirming the pivotal role of commonsense integration in enhancing ERC.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (64)
  1. Emotinet: A knowledge base for emotion detection in text built on the appraisal theories. In Natural Language Processing and Information Systems: 16th International Conference on Applications of Natural Language to Information Systems, NLDB 2011, Alicante, Spain, June 28-30, 2011. Proceedings 16, pages 27–39. Springer.
  2. Multi-modal sarcasm detection and humor classification in code-mixed conversations. IEEE Transactions on Affective Computing, 14(2):1363–1375.
  3. Two headed dragons: multimodal fusion and cross modal transactions. In 2021 IEEE International Conference on Image Processing (ICIP), pages 2893–2897. IEEE.
  4. COMET: Commonsense transformers for automatic knowledge graph construction. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4762–4779, Florence, Italy. Association for Computational Linguistics.
  5. Luís Caires and Hugo Torres Vieira. 2010. Conversation types. Theoretical Computer Science, 411(51):4399–4440. European Symposium on Programming 2009.
  6. Incorporating structured emotion commonsense knowledge and interpersonal relation into context-aware emotion recognition. Applied Intelligence, 53(4):4201–4217.
  7. Ze-Jing Chuang and Chung-Hsien Wu. 2004. Multi-modal emotion recognition from speech and text. In International Journal of Computational Linguistics & Chinese Language Processing, Volume 9, Number 2, August 2004: Special Issue on New Trends of Speech and Language Processing, pages 45–62.
  8. Alan S Cowen and Dacher Keltner. 2017. Self-report captures 27 distinct categories of emotion bridged by continuous gradients. PNAS, 114(38):E7900–E7909.
  9. Eeg-based emotion recognition using an end-to-end regional-asymmetric convolutional neural network. Knowledge-Based Systems, 205:106243.
  10. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805.
  11. Marta Dynel. 2009. Beyond a joke: Types of conversational humour. Language and Linguistics Compass, 3(5):1284–1299.
  12. Paul Ekman. 1992. An argument for basic emotions. Cognition & emotion, 6(3-4):169–200.
  13. A multi-modal eliza using natural language processing and emotion recognition. In Text, Speech and Dialogue: 6th International Conference, TSD 2003, České Budéjovice, Czech Republic, September 8-12, 2003. Proceedings 6, pages 394–399. Springer.
  14. Cosmic: Commonsense knowledge for emotion identification in conversations. arXiv preprint arXiv:2010.02795.
  15. COSMIC: COmmonSense knowledge for eMotion identification in conversations. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 2470–2481, Online. Association for Computational Linguistics.
  16. Dialoguegcn: A graph convolutional neural network for emotion recognition in conversation. arXiv preprint arXiv:1908.11540.
  17. Auli Hakulinen. 2009. Conversation types. The pragmatics of interaction, pages 55–65.
  18. Icon: interactive conversational memory network for multimodal emotion detection. In EMNLP, pages 2594–2604.
  19. Conversational memory network for emotion recognition in dyadic dialogue videos. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 2122–2132, New Orleans, Louisiana.
  20. Conversational transfer learning for emotion recognition. Information Fusion, 65:1–12.
  21. Emotion detection in code-mixed roman urdu - english text. ACM Trans. Asian Low-Resour. Lang. Inf. Process., 22(2).
  22. Real-time emotion recognition via attention gated hierarchical memory network. In AAAI, volume 34, pages 8002–8009.
  23. Higru: Hierarchical gated recurrent units for utterance-level emotion recognition. arXiv preprint arXiv:1904.04446.
  24. Automatic sarcasm detection: A survey. ACM Comput. Surv., 50(5).
  25. Gabriele Kasper and Johannes Wagner. 2014. Conversation analysis in applied linguistics. Annual Review of Applied Linguistics, 34:171–212.
  26. Muril: Multilingual representations for indian languages. CoRR, abs/2103.10730.
  27. Klaus Krippendorff. 2011. Computing krippendorff’s alpha-reliability.
  28. When did you become so smart, oh wise one?! sarcasm explanation in multi-modal multi-party dialogues. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 5956–5968, Dublin, Ireland. Association for Computational Linguistics.
  29. Explaining (sarcastic) utterances to enhance affect understanding in multimodal dialogues.
  30. Joosung Lee and Wooin Lee. 2022. CoMPM: Context modeling with speaker’s pre-trained memory tracking for emotion recognition in conversation. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 5669–5679, Seattle, United States. Association for Computational Linguistics.
  31. Enhancing emotion inference in conversations with commonsense knowledge. Knowledge-Based Systems, 232:107449.
  32. Research on textual emotion recognition incorporating personality factor. In 2007 IEEE International Conference on Robotics and Biomimetics (ROBIO), pages 2222–2227. IEEE.
  33. Bieru: bidirectional emotional recurrent unit for conversational sentiment analysis. arXiv preprint arXiv:2006.00492.
  34. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692.
  35. A multi-view network for real-time emotion recognition in conversations. Knowledge-Based Systems, 236:107751.
  36. Yukun Ma and Bin Ma. 2022. Multimodal sentiment analysis on unaligned sequences via holographic embedding. In ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 8547–8551. IEEE.
  37. Detecting offensive speech in conversational code-mixed dialogue on social media: A contextual dataset and benchmark experiments. Expert Systems with Applications, 215:119342.
  38. Speech emotion recognition using amplitude modulation parameters and a combined feature selection procedure. Knowledge-Based Systems, 63:68–81.
  39. Saif Mohammad. 2018. Obtaining reliable human ratings of valence, arousal, and dominance for 20,000 English words. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 174–184, Melbourne, Australia. Association for Computational Linguistics.
  40. Long dialogue emotion detection based on commonsense knowledge graph guidance. IEEE Transactions on Multimedia, pages 1–15.
  41. Rosalind W. Picard. 1997. Affective Computing. MIT Press, Cambridge, MA, USA.
  42. A review of affective computing: From unimodal analysis to multimodal fusion. Information fusion, 37:98–125.
  43. Context-dependent sentiment analysis in user-generated videos. In ACL, pages 873–883.
  44. Emotion recognition in conversation: Research challenges, datasets, and recent advances. IEEE Access, 7:100943–100953.
  45. Atomic: An atlas of machine commonsense for if-then reasoning. Proceedings of the AAAI Conference on Artificial Intelligence, 33(01):3027–3035.
  46. Emotion detection in hinglish(hindi+english) code-mixed social media text. Procedia Computer Science, 171:1346–1352. Third International Conference on Computing and Network Communications (CoCoNet’19).
  47. Dialogxl: All-in-one xlnet for multi-party conversation emotion recognition. arXiv preprint arXiv:2012.08695.
  48. Conceptnet 5.5: An open multilingual graph of general knowledge. In Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, AAAI’17, page 4444–4451. AAAI Press.
  49. Andi Suciati and Indra Budi. 2020. Aspect-based sentiment analysis and emotion detection for code-mixed review. International Journal of Advanced Computer Science and Applications, 11(9).
  50. Naf’an Tarihoran and Iin Ratna Sumirat. 2022. The impact of social media on the use of code mixing by generation z. International Journal of Interactive Mobile Technologies (iJIM), 16(7):54–69.
  51. Mary WJ Tay. 1989. Code switching and code mixing as a communicative strategy in multilingual discourse. World Englishes, 8(3):407–417.
  52. Code-mixing: A brief survey. In 2018 International conference on advances in computing, communications and informatics (ICACCI), pages 2382–2388. IEEE.
  53. Context- and sentiment-aware networks for emotion recognition in conversation. IEEE Transactions on Artificial Intelligence, 3(5):699–708.
  54. William Turnbull. 2003. Language in action: Psychological models of conversation. Routledge.
  55. Anshul Wadhawan and Akshita Aggarwal. 2021. Towards emotion recognition in hindi-english code-mixed data: A transformer based approach.
  56. Edda Weigand. 2010. Language as dialogue. 7(3):505–515.
  57. Context-aware self-attention networks. Proceedings of the AAAI Conference on Artificial Intelligence, 33(01):387–394.
  58. Hybrid curriculum learning for emotion recognition in conversation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 11595–11603.
  59. Intelligent facial emotion recognition using moth-firefly optimization. Knowledge-Based Systems, 111:248–267.
  60. Care: Commonsense-aware emotional response generation with latent concepts. Proceedings of the AAAI Conference on Artificial Intelligence, 35(16):14577–14585.
  61. Knowledge-enriched transformer for emotion detection in textual conversations. In EMNLP-IJCNLP, pages 165–176, Hong Kong, China. Association for Computational Linguistics.
  62. Knowledge-enriched transformer for emotion detection in textual conversations. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 165–176, Hong Kong, China.
  63. Leveraging bilingual-view parallel translation for code-switched emotion detection with adversarial dual-channel encoder. Knowledge-Based Systems, 235:107436.
  64. Aligning books and movies: Towards story-like visual explanations by watching movies and reading books. CoRR, abs/1506.06724.
Citations (22)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now