2000 character limit reached
rFaceNet: An End-to-End Network for Enhanced Physiological Signal Extraction through Identity-Specific Facial Contours (2403.09034v2)
Published 14 Mar 2024 in cs.CV
Abstract: Remote photoplethysmography (rPPG) technique extracts blood volume pulse (BVP) signals from subtle pixel changes in video frames. This study introduces rFaceNet, an advanced rPPG method that enhances the extraction of facial BVP signals with a focus on facial contours. rFaceNet integrates identity-specific facial contour information and eliminates redundant data. It efficiently extracts facial contours from temporally normalized frame inputs through a Temporal Compressor Unit (TCU) and steers the model focus to relevant facial regions by using the Cross-Task Feature Combiner (CTFC). Through elaborate training, the quality and interpretability of facial physiological signals extracted by rFaceNet are greatly improved compared to previous methods. Moreover, our novel approach demonstrates superior performance than SOTA methods in various heart rate estimation benchmarks.
- “Remote plethysmographic imaging using ambient light.,” Optics express, vol. 16, no. 26, pp. 21434–21445, 2008.
- “Self-adaptive matrix completion for heart rate estimation from face videos under realistic conditions,” in CVPR, 2016, pp. 2396–2404.
- “Algorithmic principles of remote ppg,” IEEE Transactions on Biomedical Engineering, vol. 64, no. 7, pp. 1479–1491, 2016.
- “Advancements in noncontact, multiparameter physiological measurements using a webcam,” IEEE transactions on biomedical engineering, vol. 58, no. 1, pp. 7–11, 2010.
- “Robust pulse rate from chrominance-based rppg,” IEEE Transactions on Biomedical Engineering, vol. 60, no. 10, pp. 2878–2886, 2013.
- “Local group invariance for heart rate estimation from face videos in the wild,” in CVPR, 2018, pp. 1254–1262.
- “Improved motion robustness of remote-ppg by using the blood volume pulse signature,” Physiological measurement, vol. 35, no. 9, pp. 1913, 2014.
- “Rhythmnet: End-to-end heart rate estimation from face via spatial-temporal representation,” TIP, vol. 29, pp. 2409–2423, 2019.
- “Robust remote heart rate estimation from face utilizing spatial-temporal attention,” in FG 2019. IEEE, 2019, pp. 1–8.
- Hao Lu and Hu Han, “Nas-hr: Neural architecture search for heart rate estimation from face videos,” Virtual Reality & Intelligent Hardware, vol. 3, no. 1, pp. 33–42, 2021.
- “Video-based remote physiological measurement via cross-verified feature disentangling,” in ECCV 2020. Springer, 2020, pp. 295–310.
- “Dual-gan: Joint bvp and noise modeling for remote physiological measurement,” in CVPR, 2021, pp. 12404–12413.
- “Deepphys: Video-based physiological measurement using convolutional attention networks,” in ECCV, 2018, pp. 349–365.
- “Multi-task temporal shift attention networks for on-device contactless vitals measurement,” Advances in Neural Information Processing Systems, vol. 33, pp. 19400–19411, 2020.
- “Efficientphys: Enabling simple, fast and accurate camera-based cardiac measurement,” in IEEE/CVF winter conference on applications of computer vision, 2023, pp. 5008–5017.
- “Domain generalized rppg network: Disentangled feature learning with domain permutation and domain augmentation,” in ACCV, 2022, pp. 807–823.
- “Grad-cam: Visual explanations from deep networks via gradient-based localization,” in ICCV, 2017, pp. 618–626.
- “Elimrec: Eliminating single-modal bias in multimedia recommendation,” in Proceedings of the 30th ACM International Conference on Multimedia, 2022, pp. 687–695.
- “Self-supervised learning for multimedia recommendation,” IEEE Transactions on Multimedia, 2022.
- “Strategy-aware bundle recommender system,” in Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval, 2023, pp. 1198–1207.
- “Toda: Target-oriented diffusion attacker against recommendation system,” arXiv preprint arXiv:2401.12578, 2024.
- “Physformer++: Facial video-based physiological measurement with slowfast temporal difference transformer,” International Journal of Computer Vision, vol. 131, no. 6, pp. 1307–1330, 2023.
- “Physformer: Facial video-based physiological measurement with temporal difference transformer,” in CVPR, 2022, pp. 4186–4196.
- “Quo vadis, action recognition? a new model and the kinetics dataset,” in CVPR, 2017, pp. 6299–6308.
- “Remote photoplethysmograph signal measurement from facial videos using spatio-temporal networks,” arXiv preprint arXiv:1905.02419, 2019.
- “Autohr: A strong end-to-end baseline for remote heart rate measurement with neural searching,” IEEE Signal Processing Letters, vol. 27, pp. 1245–1249, 2020.
- “Leveraging multimodal features and item-level user feedback for bundle construction,” in Proceedings of the 17th ACM International Conference on Web Search and Data Mining, 2024, pp. 510–519.
- “Multi-task learning using uncertainty to weigh losses for scene geometry and semantics,” in CVPR, 2018, pp. 7482–7491.
- “Vipl-hr: A multi-modal database for pulse estimation from less-constrained face video,” in ACCV 2018. Springer, 2019, pp. 562–576.
- “Unsupervised skin tissue segmentation for remote photoplethysmography,” Pattern Recognition Letters, vol. 124, pp. 82–90, 2019.
- “Non-contact video-based pulse rate measurement on a mobile service robot,” in The 23rd IEEE International Symposium on Robot and Human Interactive Communication. IEEE, 2014, pp. 1056–1062.
- “rppg-toolbox: Deep remote ppg toolbox,” arXiv preprint arXiv:2210.00716, vol. 6, pp. 13, 2022.