Papers
Topics
Authors
Recent
Detailed Answer
Quick Answer
Concise responses based on abstracts only
Detailed Answer
Well-researched responses based on abstracts and relevant paper content.
Custom Instructions Pro
Preferences or requirements that you'd like Emergent Mind to consider when generating responses
Gemini 2.5 Flash
Gemini 2.5 Flash 70 tok/s
Gemini 2.5 Pro 45 tok/s Pro
GPT-5 Medium 34 tok/s Pro
GPT-5 High 37 tok/s Pro
GPT-4o 102 tok/s Pro
Kimi K2 212 tok/s Pro
GPT OSS 120B 466 tok/s Pro
Claude Sonnet 4 39 tok/s Pro
2000 character limit reached

UbiHR: Resource-efficient Long-range Heart Rate Sensing on Ubiquitous Devices (2410.19279v1)

Published 25 Oct 2024 in eess.SP and cs.AI

Abstract: Ubiquitous on-device heart rate sensing is vital for high-stress individuals and chronic patients. Non-contact sensing, compared to contact-based tools, allows for natural user monitoring, potentially enabling more accurate and holistic data collection. However, in open and uncontrolled mobile environments, user movement and lighting introduce. Existing methods, such as curve-based or short-range deep learning recognition based on adjacent frames, strike the optimal balance between real-time performance and accuracy, especially under limited device resources. In this paper, we present UbiHR, a ubiquitous device-based heart rate sensing system. Key to UbiHR is a real-time long-range spatio-temporal model enabling noise-independent heart rate recognition and display on commodity mobile devices, along with a set of mechanisms for prompt and energy-efficient sampling and preprocessing. Diverse experiments and user studies involving four devices, four tasks, and 80 participants demonstrate UbiHR's superior performance, enhancing accuracy by up to 74.2\% and reducing latency by 51.2\%.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (85)
  1. External validation of a deep learning model for breast density classification. ESR—European Society of Radiology: Vienna, Austria (2023).
  2. Power spectrum analysis of heart rate fluctuation: a quantitative probe of beat-to-beat cardiovascular control. science 213, 4504 (1981), 220–222.
  3. On the Impact of Lateralization in Physiological Signals from Wearable Sensors. In Adjunct Proceedings of the 2022 ACM International Joint Conference on Pervasive and Ubiquitous Computing and the 2022 ACM International Symposium on Wearable Computers. 472–477.
  4. Unsupervised skin tissue segmentation for remote photoplethysmography. Pattern Recognition Letters 124 (2019), 82–90.
  5. Long short-term memory deep-filter in remote photoplethysmography. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. 306–307.
  6. RTrPPG: An ultra light 3DCNN for real-time remote photoplethysmography. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2146–2154.
  7. 3D convolutional neural networks for remote pulse rate measurement and mapping from facial video. Applied Sciences 9, 20 (2019), 4364.
  8. Modeling adoption of intelligent agents in medical imaging. International Journal of Human-Computer Studies 168 (2022), 102922.
  9. Guard Your Heart Silently: Continuous Electrocardiogram Waveform Monitoring with Wrist-Worn Motion Sensor. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 6, 3 (2022), 1–29.
  10. Hung-Chang Chang. 2023. Noncontact Lie detection system involving photoplethysmography and heart rate variability. International Journal of Pattern Recognition and Artificial Intelligence 37, 07 (2023), 2350006.
  11. Weixuan Chen and Daniel McDuff. 2018. Deepphys: Video-based physiological measurement using convolutional attention networks. In Proceedings of the european conference on computer vision (ECCV). 349–365.
  12. LGI-rPPG-Net: A shallow encoder-decoder model for rPPG signal estimation from facial video streams. Biomedical Signal Processing and Control 89 (2024), 105687.
  13. Captivates: A smart eyeglass platform for across-context physiological measurement. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 5, 3 (2021), 1–32.
  14. Captivates: A Smart Eyeglass Platform for Across-Context Physiological Measurements. GetMobile: Mobile Computing and Communications 27, 2 (2023), 18–22.
  15. Efficient remote photoplethysmography with temporal derivative modules and time-shift invariant loss. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2182–2191.
  16. Active shape models-their training and application. Computer vision and image understanding 61, 1 (1995), 38–59.
  17. Heart Rate Estimation from Noisy PPGs Using 1D/2D Conversion and Transfer Learning. In Adjunct Proceedings of the 2022 ACM International Joint Conference on Pervasive and Ubiquitous Computing and the 2022 ACM International Symposium on Wearable Computers. 163–167.
  18. Gerard De Haan and Vincent Jeanne. 2013. Robust pulse rate from chrominance-based rPPG. IEEE transactions on biomedical engineering 60, 10 (2013), 2878–2886.
  19. Gerard De Haan and Arno Van Leest. 2014. Improved motion robustness of remote-PPG by using the blood volume pulse signature. Physiological measurement 35, 9 (2014), 1913.
  20. Resting heart rate: A physiological predicator of lie detection ability. Physiology & behavior 186 (2018), 10–15.
  21. Task Force of the European Society of Cardiology the North American Society of Pacing Electrophysiology. 1996. Heart rate variability: standards of measurement, physiological interpretation, and clinical use. Circulation 93, 5 (1996), 1043–1065.
  22. Heart rate trend forecasting during high-intensity interval training using consumer wearable devices. In Proceedings of the 27th Annual International Conference on Mobile Computing and Networking. 855–857.
  23. Resting heart rate in cardiovascular disease. Journal of the American College of Cardiology 50, 9 (2007), 823–830.
  24. Prognostic value of 24-hour blood pressure variability. Journal of hypertension 11, 10 (1993), 1133–1137.
  25. Blood oxygen level–dependent MRI allows metabolic description of tissue at risk in acute stroke patients. Stroke 37, 7 (2006), 1778–1784.
  26. Efficient real-time camera based estimation of heart rate and its variability. In Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops. 0–0.
  27. Physiological parameter monitoring of drivers based on video data and independent vector analysis. In 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 4374–4378.
  28. Radiant: Better rppg estimation using signal embeddings and transformer. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. 4976–4986.
  29. Design and development of a heart rate measuring device using fingertip. In International Conference on Computer and Communication Engineering (ICCCE’10). IEEE, 1–5.
  30. Longitudinal monitoring of IL-6 and CRP in inflammatory bowel disease using IBD-AWARE. Biosensors and Bioelectronics: X 16 (2024), 100435.
  31. Enabling Smartphone-based Estimation of Heart Rate. arXiv preprint arXiv:1912.08910 (2019).
  32. Deep learning with time-frequency representation for pulse estimation from facial videos. In 2017 IEEE international joint conference on biometrics (IJCB). IEEE, 383–389.
  33. Robust heart rate estimation with spatial–temporal attention network from facial videos. IEEE Transactions on Cognitive and Developmental Systems 14, 2 (2021), 639–647.
  34. A novel one-stage framework for visual pulse rate estimation using deep neural networks. Biomedical Signal Processing and Control 66 (2021), 102387.
  35. Improved pulse detection from head motions using DCT. In 2014 international conference on computer vision theory and applications (VISAPP), Vol. 3. IEEE, 118–124.
  36. Estimating the Degree of Mental State using Heart Rate while Studying. In Adjunct Proceedings of the 2021 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2021 ACM International Symposium on Wearable Computers. 126–130.
  37. A review on flexible wearables-Recent developments in non-invasive continuous health monitoring. Sensors and Actuators A: Physical (2024), 114993.
  38. LETS-a Wearable Heart and Lung Monitoring Device for the Diagnosis of Cardiac and Respiratory Diseases. In Adjunct Proceedings of the 2022 ACM International Joint Conference on Pervasive and Ubiquitous Computing and the 2022 ACM International Symposium on Wearable Computers. 60–62.
  39. Recent developments and future directions of wearable skin biosignal sensors. Advanced Sensor Research 3, 2 (2024), 2300118.
  40. Physiological Indices to Predict Driver Situation Awareness in VR. In Adjunct Proceedings of the 2023 ACM International Joint Conference on Pervasive and Ubiquitous Computing & the 2023 ACM International Symposium on Wearable Computing. 40–45.
  41. DistancePPG: Robust non-contact vital signs monitoring using a camera. Biomedical optics express 6, 5 (2015), 1565–1588.
  42. Meta-rppg: Remote heart rate estimation using a transductive meta-learner. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XXVII 16. Springer, 392–409.
  43. M Lewandowska et al. [n. d.]. Measuring Pulse Rate with a Webcam-a Non-contact Method for Evaluating Cardiac Activity; 2011. In Proc. of the Federated Conf. on Computer Science and Information Systems. 405–410.
  44. Learning motion-robust remote photoplethysmography through arbitrary resolution videos. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 37. 1334–1342.
  45. EchoPFL: Asynchronous Personalized Federated Learning on Mobile Devices with On-Demand Staleness Control. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 8, 1 (2024), 1–22.
  46. Tsm: Temporal shift module for efficient video understanding. In Proceedings of the IEEE/CVF international conference on computer vision. 7083–7093.
  47. AdaDeep: A usage-driven, automated deep model compression framework for enabling ubiquitous intelligent mobiles. IEEE Transactions on Mobile Computing 20, 12 (2020), 3282–3297.
  48. AdaSpring: Context-adaptive and runtime-evolutionary deep model compression for mobile applications. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 5, 1 (2021), 1–22.
  49. AdaEnlight: Energy-aware low-light video stream enhancement on mobile devices. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 6, 4 (2023), 1–26.
  50. Multi-task temporal shift attention networks for on-device contactless vitals measurement. Advances in Neural Information Processing Systems 33 (2020), 19400–19411.
  51. Efficientphys: Enabling simple, fast and accurate camera-based cardiac measurement. In Proceedings of the IEEE/CVF winter conference on applications of computer vision. 5008–5017.
  52. Mobilephys: Personalized mobile camera-based contactless physiological sensing. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 6, 1 (2022), 1–23.
  53. Lightweight and interpretable convolutional neural network for real-time heart rate monitoring using low-cost video camera under realistic conditions. Biomedical Signal Processing and Control 87 (2024), 105461.
  54. Dual-gan: Joint bvp and noise modeling for remote physiological measurement. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 12404–12413.
  55. Scamps: Synthetics for camera measurement of physiological signals. Advances in Neural Information Processing Systems 35 (2022), 3744–3757.
  56. Facial video-based photoplethysmography to detect HRV at rest. International journal of sports medicine (2015), 474–480.
  57. Synrhythm: Learning a deep heart rate estimator from general to specific. In 2018 24th international conference on pattern recognition (ICPR). IEEE, 3580–3585.
  58. Rhythmnet: End-to-end heart rate estimation from face via spatial-temporal representation. IEEE Transactions on Image Processing 29 (2019), 2409–2423.
  59. Video-based remote physiological measurement via cross-verified feature disentangling. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part II 16. Springer, 295–310.
  60. Paolo Palatini and Stevo Julius. 1997. Heart rate and the cardiovascular risk. Journal of hypertension 15, 1 (1997), 3–17.
  61. HeartTrack: Convolutional neural network for remote video-based heart rate monitoring. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops. 288–289.
  62. Local group invariance for heart rate estimation from face videos in the wild. In Proceedings of the IEEE conference on computer vision and pattern recognition workshops. 1254–1262.
  63. Advancements in noncontact, multiparameter physiological measurements using a webcam. IEEE transactions on biomedical engineering 58, 1 (2010), 7–11.
  64. Non-contact, automated cardiac pulse measurements using video imaging and blind source separation. Optics express 18, 10 (2010), 10762–10774.
  65. Ubfc-phys: A multimodal database for psychophysiological studies of social stress. IEEE Transactions on Affective Computing 14, 1 (2021), 622–636.
  66. Physiological parameter monitoring from optical recordings with a mobile phone. IEEE Transactions on Biomedical Engineering 59, 2 (2011), 303–306.
  67. A pulse rate estimation algorithm using PPG and smartphone camera. Journal of medical systems 40 (2016), 1–6.
  68. Large language models encode clinical knowledge. Nature 620, 7972 (2023), 172–180.
  69. PulseGAN: Learning to generate realistic pulse waveforms in remote photoplethysmography. IEEE Journal of Biomedical and Health Informatics 25, 5 (2021), 1373–1384.
  70. Visual heart rate estimation with convolutional neural network. In Proceedings of the british machine vision conference, Newcastle, UK. 3–6.
  71. Estimating stress in online meetings by remote physiological signal and behavioral features. In Adjunct Proceedings of the 2022 ACM International Joint Conference on Pervasive and Ubiquitous Computing and the 2022 ACM International Symposium on Wearable Computers. 216–220.
  72. A robust real time system for remote heart rate measurement via camera. In 2015 IEEE International Conference on Multimedia and Expo (ICME). IEEE, 1–6.
  73. Constrained independent component analysis approach to nonobtrusive pulse rate measurements. Journal of biomedical optics 17, 7 (2012), 077011–077011.
  74. Remote plethysmographic imaging using ambient light. Optics express 16, 26 (2008), 21434–21445.
  75. EmoTracer: A Wearable Physiological and Psychological Monitoring System With Multi-modal Sensors. In Adjunct Proceedings of the 2022 ACM International Joint Conference on Pervasive and Ubiquitous Computing and the 2022 ACM International Symposium on Wearable Computers. 444–449.
  76. Algorithmic principles of remote PPG. IEEE Transactions on Biomedical Engineering 64, 7 (2016), 1479–1491.
  77. Synthetic generation of face videos with plethysmograph physiology. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 20587–20596.
  78. Vision-based heart rate estimation via a two-stream cnn. In 2019 IEEE international conference on image processing (ICIP). IEEE, 3327–3331.
  79. Non-contact, synchronous dynamic measurement of respiratory rate and heart rate based on dual sensitive regions. Biomedical engineering online 16 (2017), 1–21.
  80. Remote photoplethysmography for heart rate measurement: A review. Biomedical Signal Processing and Control 88 (2024), 105608.
  81. Doctorglm: Fine-tuning your chinese doctor is not a herculean task. arXiv preprint arXiv:2304.01097 (2023).
  82. Autohr: A strong end-to-end baseline for remote heart rate measurement with neural searching. IEEE Signal Processing Letters 27 (2020), 1245–1249.
  83. Remote photoplethysmograph signal measurement from facial videos using spatio-temporal networks. arXiv preprint arXiv:1905.02419 (2019).
  84. Physformer: Facial video-based physiological measurement with temporal difference transformer. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 4186–4196.
  85. Personalized health monitoring via vital sign measurements leveraging motion sensors on AR/VR headsets. In Proceedings of the 20th Annual International Conference on Mobile Systems, Applications and Services. 529–530.
List To Do Tasks Checklist Streamline Icon: https://streamlinehq.com

Collections

Sign up for free to add this paper to one or more collections.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
Dice Question Streamline Icon: https://streamlinehq.com

Follow-Up Questions

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Don't miss out on important new AI/ML research

See which papers are being discussed right now on X, Reddit, and more:

Explore Trending Papers

“Emergent Mind helps me see which AI papers have caught fire online.”

Philip

Philip

Creator, AI Explained on YouTube