Papers
Topics
Authors
Recent
Search
2000 character limit reached

Dual-Constrained Dynamical Neural ODEs for Ambiguity-aware Continuous Emotion Prediction

Published 31 Jul 2024 in cs.AI | (2407.21344v1)

Abstract: There has been a significant focus on modelling emotion ambiguity in recent years, with advancements made in representing emotions as distributions to capture ambiguity. However, there has been comparatively less effort devoted to the consideration of temporal dependencies in emotion distributions which encodes ambiguity in perceived emotions that evolve smoothly over time. Recognizing the benefits of using constrained dynamical neural ordinary differential equations (CD-NODE) to model time series as dynamic processes, we propose an ambiguity-aware dual-constrained Neural ODE approach to model the dynamics of emotion distributions on arousal and valence. In our approach, we utilize ODEs parameterised by neural networks to estimate the distribution parameters, and we integrate additional constraints to restrict the range of the system outputs to ensure the validity of predicted distributions. We evaluated our proposed system on the publicly available RECOLA dataset and observed very promising performance across a range of evaluation metrics.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (24)
  1. Y. Wang, W. Song, W. Tao, A. Liotta, D. Yang, X. Li, S. Gao, Y. Sun, W. Ge, W. Zhang et al., “A systematic review on affective computing: Emotion models, databases, and recent advances,” Information Fusion, 2022.
  2. J. A. Russell, “A circumplex model of affect.” Journal of personality and social psychology, vol. 39, no. 6, p. 1161, 1980.
  3. H. Gunes and M. Pantic, “Automatic, dimensional and continuous emotion recognition,” International Journal of Synthetic Emotions (IJSE), vol. 1, no. 1, pp. 68–99, 2010.
  4. V. Sethu, E. M. Provost, J. Epps, C. Busso, N. Cummins, and S. Narayanan, “The ambiguous world of emotion representation,” arXiv preprint arXiv:1909.00360, 2019.
  5. H. Gunes and B. Schuller, “Categorical and dimensional affect analysis in continuous input: Current trends and future directions,” Image and Vision Computing, vol. 31, no. 2, pp. 120–136, 2013.
  6. T. Dang, V. Sethu, J. Epps, and E. Ambikairajah, “An investigation of emotion prediction uncertainty using gaussian mixture regression.” in INTERSPEECH, 2017, pp. 1248–1252.
  7. J. Wu, T. Dang, V. Sethu, and E. Ambikairajah, “A novel sequential monte carlo framework for predicting ambiguous emotion states,” in ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).   IEEE, 2022, pp. 8567–8571.
  8. T. Mani Kumar, E. Sanchez, G. Tzimiropoulos, T. Giesbrecht, and M. Valstar, “Stochastic process regression for cross-cultural speech emotion recognition,” Proc. Interspeech 2021, pp. 3390–3394, 2021.
  9. B. Zhang, G. Essl, and E. Mower Provost, “Predicting the distribution of emotion perception: capturing inter-rater variability,” in Proceedings of the 19th ACM International Conference on Multimodal Interaction, 2017, pp. 51–59.
  10. J. Han, Z. Zhang, Z. Ren, and B. Schuller, “Exploring perception uncertainty for emotion recognition in dyadic conversation and music listening,” Cognitive Computation, vol. 13, pp. 231–240, 2021.
  11. J. Wu, T. Dang, V. Sethu, and E. Ambikairajah, “Belief mismatch coefficient (bmc): A novel interpretable measure of prediction accuracy for ambiguous emotion states,” in 2023 11th International Conference on Affective Computing and Intelligent Interaction (ACII).   IEEE, 2023, pp. 1–8.
  12. P. Kuppens and P. Verduyn, “Emotion dynamics,” Current Opinion in Psychology, vol. 17, pp. 22–26, 2017.
  13. S. P. Yadav, S. Zaidi, A. Mishra, and V. Yadav, “Survey on machine learning in speech emotion recognition and vision systems using a recurrent neural network (rnn),” Archives of Computational Methods in Engineering, vol. 29, no. 3, pp. 1753–1770, 2022.
  14. T. Dang, V. Sethu, and E. Ambikairajah, “Dynamic multi-rater gaussian mixture regression incorporating temporal dependencies of emotion uncertainty using kalman filters,” in 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).   IEEE, 2018, pp. 4929–4933.
  15. T. Dang, A. Dimitriadis, J. Wu, V. Sethu, and E. Ambikairajah, “Constrained dynamical neural ode for time series modelling: A case study on continuous emotion prediction,” in ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).   IEEE, 2023, pp. 1–5.
  16. D. Bose, V. Sethu, and E. Ambikairajah, “Parametric distributions to model numerical emotion labels,” Proc. Interspeech 2021, pp. 4498–4502, 2021.
  17. Z. Zhang, J. Han, E. Coutinho, and B. Schuller, “Dynamic difficulty awareness training for continuous emotion prediction,” IEEE Transactions on Multimedia, vol. 21, no. 5, pp. 1289–1301, 2018.
  18. D. Bose, V. Sethu, and E. Ambikairajah, “Continuous emotion ambiguity prediction: Modeling with beta distributions,” IEEE Transactions on Affective Computing, no. 01, pp. 1–12, 2024.
  19. F. Ringeval, A. Sonderegger, J. Sauer, and D. Lalanne, “Introducing the recola multimodal corpus of remote collaborative and affective interactions,” in 2013 10th IEEE international conference and workshops on automatic face and gesture recognition (FG).   IEEE, 2013, pp. 1–8.
  20. F. Ringeval, B. Schuller, M. Valstar, R. Cowie, and M. Pantic, “Avec 2015: The 5th international audio/visual emotion challenge and workshop,” in Proceedings of the 23rd ACM international conference on Multimedia, 2015, pp. 1335–1336.
  21. M. Schmitt and B. Schuller, “Openxbow: introducing the passau open-source crossmodal bag-of-words toolkit,” 2017.
  22. M. Schmitt, F. Ringeval, and B. W. Schuller, “At the border of acoustics and linguistics: Bag-of-audio-words for the recognition of emotions in speech.” in Interspeech, 2016, pp. 495–499.
  23. Z. Huang, T. Dang, N. Cummins, B. Stasak, P. Le, V. Sethu, and J. Epps, “An investigation of annotation delay compensation and output-associative fusion for multimodal continuous emotion prediction,” in Proceedings of the 5th International Workshop on Audio/Visual Emotion Challenge, 2015, pp. 41–48.
  24. R. T. Chen, Y. Rubanova, J. Bettencourt, and D. K. Duvenaud, “Neural ordinary differential equations,” Advances in neural information processing systems, vol. 31, 2018.

Summary

No one has generated a summary of this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Sign Up to Summarize

Paper to Video (Beta)

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Collections

Sign up for free to add this paper to one or more collections.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up