Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
129 tokens/sec
GPT-4o
28 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Pain Analysis using Adaptive Hierarchical Spatiotemporal Dynamic Imaging (2312.06920v1)

Published 12 Dec 2023 in cs.CV

Abstract: Automatic pain intensity estimation plays a pivotal role in healthcare and medical fields. While many methods have been developed to gauge human pain using behavioral or physiological indicators, facial expressions have emerged as a prominent tool for this purpose. Nevertheless, the dependence on labeled data for these techniques often renders them expensive and time-consuming. To tackle this, we introduce the Adaptive Hierarchical Spatio-temporal Dynamic Image (AHDI) technique. AHDI encodes spatiotemporal changes in facial videos into a singular RGB image, permitting the application of simpler 2D deep models for video representation. Within this framework, we employ a residual network to derive generalized facial representations. These representations are optimized for two tasks: estimating pain intensity and differentiating between genuine and simulated pain expressions. For the former, a regression model is trained using the extracted representations, while for the latter, a binary classifier identifies genuine versus feigned pain displays. Testing our method on two widely-used pain datasets, we observed encouraging results for both tasks. On the UNBC database, we achieved an MSE of 0.27 outperforming the SOTA which had an MSE of 0.40. On the BioVid dataset, our model achieved an accuracy of 89.76%, which is an improvement of 5.37% over the SOTA accuracy. Most notably, for distinguishing genuine from simulated pain, our accuracy stands at 94.03%, marking a substantial improvement of 8.98%. Our methodology not only minimizes the need for extensive labeled data but also augments the precision of pain evaluations, facilitating superior pain management.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (62)
  1. M. Lynch. Pain as the fifth vital sign. Journal of Intravenous Nursing, 24(2):85 – 94, 2001. Cited by: 66.
  2. John Smith and et al. Understanding pain: Mechanisms, evaluation, and management. Journal of Pain Research, 2019.
  3. The facial action coding system. Human Interaction Lab, 2002.
  4. The structure, reliability and validity of pain expression: Evidence from patients with shoulder pain. Pain, 139(2):267–274, 2008.
  5. Painful monitoring: automatic pain monitoring using the unbc-mcmaster shoulder pain expression archive database. Image and Vision Computing, 30(3):197–205, 2012.
  6. Continuous pain intensity estimation from facial expressions. In Advances in Visual Computing, pages 368–377, 2012.
  7. Signature verification using a ’siamese’ time delay neural network. In Advances in Neural Information Processing Systems, pages 737–744, 1994.
  8. Recurrent convolutional neural network regression for continuous pain intensity estimation in video. In Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pages 84–92, 2016.
  9. Deep pain: Exploiting long short-term memory networks for facial expression classification. IEEE transactions on cybernetics, 52(5):3314–3324, 2017.
  10. Facial Action Coding System: A Technique for the Measurement of Facial Movements. Consulting Psychologist Press, 1978.
  11. Automatic coding of facial expressions displayed during posed and genuine pain. Image and Vision Computing, 27(12):1797–1803, 2009.
  12. Cross-database evaluation of pain recognition from facial video. In ISPA, pages 181–186, 2019.
  13. Unsupervised domain adaptation for facial expression recognition using generative adversarial networks. Computational Intelligence and Neuroscience, pages 1–10, 2018.
  14. Hao Yan. Transfer subspace learning for cross-dataset facial expression recognition. Neurocomputing, 208:165–173, 2016.
  15. Li Jing and Yu Tian. Self-supervised visual feature learning with deep neural networks: a survey. IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI), 2020.
  16. Self-supervised pain intensity estimation from facial videos via statistical spatiotemporal distillation. Pattern Recognition Letters, 140:26–33, 2020.
  17. Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems, pages 1097–1105, 2012.
  18. Large-scale video classification with convolutional neural networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1725–1732, 2014.
  19. Better exploiting motion for better action recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2555–2562, 2014.
  20. Dynamic image networks for action recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 3034–3042, 2016.
  21. Hidden conditional random fields for gesture recognition. In 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’06), volume 2, pages 1521–1527. IEEE, 2006.
  22. Personalized automatic estimation of self-reported pain intensity from facial expressions. In 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pages 2318–2327, 2017.
  23. Multi-modal data fusion for pain intensity assessment and classification. In 2017 Seventh International Conference on Image Processing Theory, Tools and Applications (IPTA), pages 1–6, 2017.
  24. A spatiotemporal convolutional neural network for automatic pain intensity estimation from facial dynamics. International Journal of Computer Vision, 127(10):1413–1425, 2019.
  25. Pain intensity estimation based on a spatial transformation and attention cnn. Plos One, 15(6):e0232412, 2020.
  26. Personalized automatic estimation of self-reported pain intensity from facial expressions. In Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pages 70–79, 2017.
  27. Two-stream convolutional networks for action recognition in videos. Advances in neural information processing systems, 27, 2014.
  28. Painful data: The unbc-mcmaster shoulder pain expression archive database. In 2011 IEEE International Conference on Automatic Face & Gesture Recognition (FG), pages 57–64. IEEE, 2011.
  29. Registration invariant representations for expression detection. In 2010 International Conference on Digital Image Computing: Techniques and Applications, pages 255–261. IEEE, 2010.
  30. Automatic detection of pain intensity. In Proceedings of the 14th ACM international conference on Multimodal interaction, pages 47–52, 2012.
  31. Long short-term memory. Neural computation, 9(8):1735–1780, 1997.
  32. A deep attention transformer network for pain estimation with facial expression video. In Chinese Conference on Biometric Recognition, pages 112–119. Springer, 2021.
  33. Learning deep facial expression features from image and optical flow sequences using 3d cnn. The Visual Computer, 34:1461–1475, 2018.
  34. Deep domain adaptation with ordinal regression for pain assessment using weakly-labeled videos. Image and Vision Computing, 110:104167, 2021.
  35. Pain intensity estimation using deep spatiotemporal and handcrafted features. IEICE TRANSACTIONS on Information and Systems, 101(6):1572–1580, 2018.
  36. Learning spatiotemporal features with 3d convolutional networks. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), December 2015.
  37. Learning to detect genuine versus posed pain from facial expressions using residual generative adversarial networks. In 2019 14th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2019), pages 1–8. IEEE, 2019.
  38. Dynamic image networks for action recognition. Institute of Electrical and Electronics Engineers, 2016.
  39. Modeling video evolution for action recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 5378–5387, 2015.
  40. Rank pooling for action recognition. IEEE transactions on pattern analysis and machine intelligence, 39(4):773–787, 2016.
  41. Discriminative hierarchical rank pooling for activity recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 1924–1932, 2016.
  42. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016.
  43. How many bits does it take for a stimulus to be salient? In Proc. IEEE CVPR, pages 5501–5510, 2015.
  44. Speech recognition with deep recurrent neural networks. CoRR, abs/1303.5778, 2013.
  45. Attention is all you need. In Advances in neural information processing systems, pages 5998–6008, 2017.
  46. Show, attend and tell: Neural image caption generation with visual attention. In International conference on machine learning, pages 2048–2057, 2015.
  47. Hierarchical attention networks for document classification. In Proceedings of the 2016 conference of the North American chapter of the association for computational linguistics: human language technologies, pages 1480–1489, 2016.
  48. University of Northern British Columbia and McMaster University. Unbc-mcmaster shoulder pain expression archive. https://www.cs.ubc.ca/~bdm/shoulderpain.html, 2005.
  49. E biovid heat pain database. In 2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG), pages 1–6. IEEE, 2013.
  50. Going deeper with convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1–9, 2015.
  51. Francois Chollet. Xception: Deep learning with depthwise separable convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1251–1258, 2017.
  52. Efficientnetv3: An improved baseline for model scaling. arXiv preprint arXiv:2104.00298, 2021.
  53. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929, 2020.
  54. Swin transformer: Hierarchical vision transformer using shifted windows. arXiv preprint arXiv:2103.14030, 2021.
  55. Gunnar Farnebäck. Two-frame motion estimation based on polynomial expansion. In Scandinavian conference on image analysis, pages 363–370. Springer, 2003.
  56. Deep pain: Exploiting long short-term memory networks for facial expression classification. IEEE Transactions on Cybernetics, 52(5):3314–3324, 2022.
  57. Deep weakly supervised domain adaptation for pain localization in videos. In 2020 15th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2020), pages 473–480. IEEE, 2020.
  58. On pain assessment from facial videos using spatio-temporal local descriptors. In 2016 Sixth International Conference on Image Processing Theory, Tools and Applications (IPTA), pages 1–6. IEEE, 2016.
  59. Bilateral ordinal relevance multi-instance regression for facial action unit intensity estimation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 7034–7043, 2018.
  60. Multi-task neural networks for personalized pain recognition from physiological signals. In 2017 Seventh International Conference on Affective Computing and Intelligent Interaction Workshops and Demos (ACIIW), pages 181–184. IEEE, 2017.
  61. Temporal stochastic softmax for 3d cnns: An application in facial expression recognition. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 3029–3038, 2021.
  62. Automatic pain assessment with facial activity descriptors. IEEE Transactions on Affective Computing, 8(3):286–299, 2016.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now