Papers
Topics
Authors
Recent
Search
2000 character limit reached

Resolve Domain Conflicts for Generalizable Remote Physiological Measurement

Published 11 Apr 2024 in cs.CV | (2404.07855v1)

Abstract: Remote photoplethysmography (rPPG) technology has become increasingly popular due to its non-invasive monitoring of various physiological indicators, making it widely applicable in multimedia interaction, healthcare, and emotion analysis. Existing rPPG methods utilize multiple datasets for training to enhance the generalizability of models. However, they often overlook the underlying conflict issues across different datasets, such as (1) label conflict resulting from different phase delays between physiological signal labels and face videos at the instance level, and (2) attribute conflict stemming from distribution shifts caused by head movements, illumination changes, skin types, etc. To address this, we introduce the DOmain-HArmonious framework (DOHA). Specifically, we first propose a harmonious phase strategy to eliminate uncertain phase delays and preserve the temporal variation of physiological signals. Next, we design a harmonious hyperplane optimization that reduces irrelevant attribute shifts and encourages the model's optimization towards a global solution that fits more valid scenarios. Our experiments demonstrate that DOHA significantly improves the performance of existing methods under multiple protocols. Our code is available at https://github.com/SWY666/rPPG-DOHA.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (61)
  1. Emotion recognition based on eeg using lstm recurrent neural network. International Journal of Advanced Computer Science and Applications, 8(10), 2017.
  2. Unsupervised skin tissue segmentation for remote photoplethysmography. Pattern Recognition Letters, 124:82–90, 2019.
  3. An open framework for remote-PPG methods and their assessment. IEEE Access, pages 1–1, 2020.
  4. Are all negatives created equal in contrastive instance discrimination? arXiv preprint arXiv:2010.06682, 2020.
  5. Deepphys: Video-based physiological measurement using convolutional attention networks. In ECCV, 2018.
  6. Domain generalized rppg network: Disentangled feature learning with domain permutation and domain augmentation. In Proceedings of the Asian Conference on Computer Vision, pages 807–823, 2022.
  7. Robust pulse rate from chrominance-based rppg. IEEE Transactions on Biomedical Engineering, 60(10):2878–2886, 2013.
  8. Sharpness-aware minimization for efficiently improving generalization. CoRR, abs/2010.01412, 2020.
  9. Unsupervised domain adaptation by backpropagation. In International conference on machine learning, pages 1180–1189. PMLR, 2015.
  10. Eta-rppgnet: Effective time-domain attention network for remote heart rate measurement. IEEE Transactions on Instrumentation and Measurement, 70:1–12, 2021.
  11. Vitamon: Measuring heart rate variability using smartphone front camera. In Mi Zhang, editor, SenSys 2019 - Proceedings of the 17th Conference on Embedded Networked Sensor Systems, SenSys 2019 - Proceedings of the 17th Conference on Embedded Networked Sensor Systems, pages 1–14. Association for Computing Machinery, Nov. 2019. Publisher Copyright: © 2019 ACM.; 17th ACM Conference on Embedded Networked Sensor Systems, SenSys 2019 ; Conference date: 10-11-2019 Through 13-11-2019.
  12. Feature stylization and domain-aware contrastive learning for domain generalization. In Proceedings of the 29th ACM International Conference on Multimedia, MM ’21, page 22–31, New York, NY, USA, 2021. Association for Computing Machinery.
  13. Pain recognition with camera photoplethysmography. In 2017 Seventh International Conference on Image Processing Theory, Tools and Applications, pages 1–5. IEEE, 2017.
  14. Davis E King. Max-margin object detection. arXiv preprint arXiv:1502.00046, 2015.
  15. Out-of-distribution generalization via risk extrapolation (rex). In International Conference on Machine Learning, pages 5815–5826. PMLR, 2021.
  16. Meta-rppg: Remote heart rate estimation using a transductive meta-learner. In European Conference on Computer Vision, pages 392–409. Springer, 2020.
  17. Measuring pulse rate with a webcam — a non-contact method for evaluating cardiac activity. In 2011 Federated Conference on Computer Science and Information Systems, pages 405–410, 2011.
  18. Gradient harmonized single-stage detector. In Proceedings of the AAAI conference on artificial intelligence, volume 33, pages 8577–8584, 2019.
  19. Face liveness detection by rppg features and contextual patch-based cnn. In Proceedings of the 2019 3rd international conference on biometric engineering and applications, pages 61–68, 2019.
  20. Multi-task temporal shift attention networks for on-device contactless vitals measurement. Advances in Neural Information Processing Systems, 33:19400–19411, 2020.
  21. Sgdr: Stochastic gradient descent with warm restarts. arXiv preprint arXiv:1608.03983, 2016.
  22. Dual-gan: Joint bvp and noise modeling for remote physiological measurement. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12404–12413, 2021.
  23. Neuron structure modeling for generalizable remote physiological measurement, 2023.
  24. Sparseppg: Towards driver monitoring using camera-based vital signs estimation in near-infrared. In Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pages 1272–1281, 2018.
  25. Domain generalization via gradient surgery, 2021.
  26. Daniel McDuff. Applications of camera-based physiological measurement beyond healthcare. In Contactless Vital Signs Monitoring, pages 165–177. Elsevier, 2022.
  27. Daniel McDuff. Camera measurement of physiological vital signs. ACM Computing Surveys, 55(9):1–40, 2023.
  28. Visualization of complex health data on mobile devices. In Proceedings of the 2016 ACM Workshop on Multimedia for Personal Health and Health Care, pages 31–34, 2016.
  29. Towards stable test-time adaptation in dynamic wild world. arXiv preprint arXiv:2302.12400, 2023.
  30. Synrhythm: Learning a deep heart rate estimator from general to specific. In 2018 24th International Conference on Pattern Recognition, pages 3580–3585, 2018.
  31. Vipl-hr: A multi-modal database for pulse estimation from less-constrained face video. 2018.
  32. Rhythmnet: End-to-end heart rate estimation from face via spatial-temporal representation. IEEE Transactions on Image Processing, 29:2409–2423, 2020.
  33. Video-based remote physiological measurement via cross-verified feature disentangling. In European Conference on Computer Vision, 2020.
  34. Learning explanations that are hard to vary. arXiv preprint arXiv:2009.00329, 2020.
  35. Non-contact, automated cardiac pulse measurements using video imaging and blind source separation. Optics express, 18(10):10762–10774, 2010.
  36. Advancements in noncontact, multiparameter physiological measurements using a webcam. IEEE Transactions on Biomedical Engineering, 58(1):7–11, 2011.
  37. Ubfc-phys: A multimodal database for psychophysiological studies of social stress. IEEE Transactions on Affective Computing, 2021.
  38. Generalizing across domains via cross-gradient training. arXiv preprint arXiv:1804.10745, 2018.
  39. Pulse transit time: an appraisal of potential clinical applications. Thorax, 54(5):452–457, 1999.
  40. Heart rate estimation from facial videos using a spatiotemporal representation with convolutional neural networks. IEEE Transactions on Instrumentation and Measurement, 69(10):7411–7421, 2020.
  41. Visual heart rate estimation with convolutional neural network. In Proceedings of the british machine vision conference, Newcastle, UK, pages 3–6, 2018.
  42. Non-contact video-based pulse rate measurement on a mobile service robot. In The 23rd IEEE International Symposium on Robot and Human Interactive Communication, pages 1056–1062, 2014.
  43. Lstm for dynamic emotion and group emotion recognition in the wild. In Proceedings of the 18th ACM international conference on multimodal interaction, pages 451–457, 2016.
  44. Contrast-phys: Unsupervised video-based remote physiological measurement via spatiotemporal contrast. In European Conference on Computer Vision, pages 492–510. Springer, 2022.
  45. Neuron coverage-guided domain generalization. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(1):1302–1311, 2022.
  46. Siamese-rppg network: Remote photoplethysmography signal estimation from face videos. In Proceedings of the 35th annual ACM symposium on applied computing, pages 2066–2073, 2020.
  47. Wearable emotion recognition system based on gsr and ppg signals. In Proceedings of the 2nd international workshop on multimedia for personal health and health care, pages 53–59, 2017.
  48. Remote plethysmographic imaging using ambient light. Optics express, 16(26):21434–21445, 2008.
  49. Domain generalization via frequency-domain-based feature disentanglement and interaction. In Proceedings of the 30th ACM International Conference on Multimedia, MM ’22, page 4821–4829, New York, NY, USA, 2022. Association for Computing Machinery.
  50. Generalizing to unseen domains: A survey on domain generalization. IEEE Transactions on Knowledge and Data Engineering, 2022.
  51. Semantic data augmentation based distance metric learning for domain generalization. In Proceedings of the 30th ACM International Conference on Multimedia, MM ’22, page 3214–3223, New York, NY, USA, 2022. Association for Computing Machinery.
  52. Algorithmic principles of remote ppg. IEEE Transactions on Biomedical Engineering, 64(7):1479–1491, 2016.
  53. Eulerian video magnification for revealing subtle changes in the world. ACM transactions on graphics, 31(4):1–8, 2012.
  54. Image enhancement for remote photoplethysmography in a low-light environment. In 2020 15th IEEE International Conference on Automatic Face and Gesture Recognition, pages 1–7. IEEE, 2020.
  55. rppg-based spoofing detection for face mask attack using efficientnet on weighted spatial-temporal representation. In 2021 IEEE International Conference on Image Processing, pages 3872–3876. IEEE, 2021.
  56. Gradient surgery for multi-task learning, 2020.
  57. Benchmarking joint face spoofing and forgery detection with visual and physiological cues. arXiv preprint arXiv:2208.05401, 2022.
  58. Remote photoplethysmograph signal measurement from facial videos using spatio-temporal networks. 2019.
  59. Remote heart rate measurement from highly compressed facial videos: An end-to-end deep learning solution with video enhancement. In 2019 IEEE/CVF International Conference on Computer Vision, pages 151–160, 2019.
  60. Physformer: Facial video-based physiological measurement with temporal difference transformer. arXiv preprint arXiv:2111.12082, 2021.
  61. Matthew D Zeiler. Adadelta: an adaptive learning rate method. arXiv preprint arXiv:1212.5701, 2012.
Citations (9)
View on Semantic Scholar

Summary

No one has generated a summary of this paper yet.

Sign Up to Summarize

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Generate Now

Collections

Sign up for free to add this paper to one or more collections.

Sign Up

Tweets

Sign up for free to view the 1 tweet with 0 likes about this paper.

Sign Up for Free