Phase-shifted remote photoplethysmography for estimating heart rate and blood pressure from facial video

Abstract: Human health can be critically affected by cardiovascular diseases, such as hypertension, arrhythmias, and stroke. Heart rate and blood pressure are important biometric information for the monitoring of cardiovascular system and early diagnosis of cardiovascular diseases. Existing methods for estimating the heart rate are based on electrocardiography and photoplethyomography, which require contacting the sensor to the skin surface. Moreover, catheter and cuff-based methods for measuring blood pressure cause inconvenience and have limited applicability. Therefore, in this thesis, we propose a vision-based method for estimating the heart rate and blood pressure. This thesis proposes a 2-stage deep learning framework consisting of a dual remote photoplethysmography network (DRP-Net) and bounded blood pressure network (BBP-Net). In the first stage, DRP-Net infers remote photoplethysmography (rPPG) signals for the acral and facial regions, and these phase-shifted rPPG signals are utilized to estimate the heart rate. In the second stage, BBP-Net integrates temporal features and analyzes phase discrepancy between the acral and facial rPPG signals to estimate SBP and DBP values. To improve the accuracy of estimating the heart rate, we employed a data augmentation method based on a frame interpolation model. Moreover, we designed BBP-Net to infer blood pressure within a predefined range by incorporating a scaled sigmoid function. Our method resulted in estimating the heart rate with the mean absolute error (MAE) of 1.78 BPM, reducing the MAE by 34.31 % compared to the recent method, on the MMSE-HR dataset. The MAE for estimating the systolic blood pressure (SBP) and diastolic blood pressure (DBP) were 10.19 mmHg and 7.09 mmHg. On the V4V dataset, the MAE for the heart rate, SBP, and DBP were 3.83 BPM, 13.64 mmHg, and 9.4 mmHg, respectively.

• The paper presents a dual-site, phase-shifted rPPG framework that significantly enhances the accuracy of heart rate and blood pressure estimation from facial videos.
• It employs a two-stage deep learning pipeline with DRP-Net and BBP-Net, using spatial-temporal attention and signal bounding for robust feature extraction.
• Empirical evaluations on MMSE-HR and V4V datasets show marked improvements in MAE and RMSE, underscoring its potential for unobtrusive clinical and telehealth applications.

Phase-shifted Remote Photoplethysmography for Estimating Heart Rate and Blood Pressure from Facial Video

Introduction and Motivation

"Phase-shifted remote photoplethysmography for estimating heart rate and blood pressure from facial video" (2401.04560) addresses the challenge of contactless, simultaneous heart rate (HR) and blood pressure (BP) estimation, leveraging facial video. Traditional ECG, PPG, and oscillometric cuff-based methods—while clinically dominant—require physical contact, which limits their deployment in continuous and ambulatory settings. The work proposes a framework that directly models temporal and phase shifts inherent in remote PPG (rPPG) signals extracted from facial regions versus classic acral (e.g., finger) PPG, introducing a dual-site, phase-aware approach to non-contact physiological monitoring.

Methodology

Pipeline Overview

The proposed method is a two-stage deep learning pipeline comprising:

1. DRP-Net (Dual Remote Photoplethysmography Network): Extracts phase-shifted rPPG signals from facial video sequences at both acral-like (projected) and facial anatomical locations.
2. BBP-Net (Bounded Blood Pressure Network): Consumes outputs from DRP-Net, leveraging both the rPPG signals and their phase discrepancy to predict SBP/DBP, using strong temporal modeling and explicit bounding of outputs via a scaled sigmoid constraint.

   Figure 1: An end-to-end pipeline for extracting physiological parameters, with blue and red denoting ground-truth and predicted signals.

Data Augmentation and Preprocessing

To mitigate class imbalance and enrich the training distribution, the framework adopts temporal upsampling and downsampling via frame interpolation (FILM-Net), generating augmented bradycardia and tachycardia cases from limited video data. ROIs are extracted using MTCNN, normalized, and processed into pseudo-PPG reference signals from synchronous ABP recordings, following detrending and strict bandpass filtering.

Figure 2: Strategy for increasing bradycardia and tachycardia diversity through frame interpolation and sampling.

DRP-Net Architecture

DRP-Net is a 3D CNN incorporating atrous convolutions for expansive spatiotemporal receptive fields, feeding into Siamese-structured heads that independently predict "facial" and "acral" rPPG signals. Twin spatial and temporal attention mechanisms bias the feature aggregation toward skin regions with strong physiological signals and temporally consistent (artifact-minimized) periods, respectively.

Figure 3: DRP-Net structure with parallel spatial/temporal attention and dual prediction heads producing phase-aligned signals.

Losses for optimizing DRP-Net incorporate both frequency-domain (PSD-based) and time-domain objectives, including a peak-valley matching term ($L_{pv}$) to enforce physiological plausibility.

BBP-Net Architecture

BBP-Net employs multi-scale feature fusion (MSF) blocks with depthwise separable convolutions, Hardswish activations, and BAM for attention. Both raw and derivative (VPG, APG) physiological signals (from both predicted sites) are used as input. The network constrains outputs with a temperature-controlled scaled sigmoid, ensuring physiologically realistic ranges for SBP/DBP. Training objectives utilize Huber loss and explicit time-domain alignment via reconstructed ABP signals.

Figure 4: BBP-Net architecture: stacks of MSF blocks extract multi-temporal dependencies from phase-shifted multi-modal signals.

Experimental Results

Evaluations utilize the MMSE-HR and V4V public datasets, which provide paired facial video and ABP data. Heart rate estimation is measured via MAE, RMSE, and $r$, while BP estimation is assessed with MAE and RMSE for both SBP/DBP.

Heart Rate Estimation:

DRP-Net achieves substantial improvement over prior methods, e.g., on MMSE-HR, facial-rPPG yields $\text{MAE}=1.78$ BPM, $\text{RMSE}=4.27$ BPM, $r=0.95$, surpassing the best baseline (PhysFormer++) [2.71, 5.15, 0.93]. Similar trends are observed on V4V. Pearson correlation plots (Figure 5) show tight coupling between predicted and ground-truth HR.

Figure 5: High linearity in Pearson correlation between estimated and actual heart rates on MMSE-HR.

Signal waveform analysis demonstrates phase shifts and high morphological fidelity to reference PPG signals.

Figure 6: Overlaid predicted facial, acral, and true pseudo-PPG, highlighting phase and amplitude matching.

Blood Pressure Estimation:

BBP-Net yields MAE/RMSE of 10.19/13.01 mmHg (SBP) and 7.09/8.86 mmHg (DBP) on MMSE-HR—significant reductions over prior camera-based and hybrid methods. Bland-Altman plots (Figure 7) indicate tight error bounds and minimal bias.

Figure 7: Bland-Altman analysis of SBP (left) and DBP (right) reveals high agreement between predictions and references.

On V4V, BBP-Net consistently outperforms AlexNet, ResNet, LSTM, and recent neural combination models, even when all baselines utilize strong learning architectures. Notably, using both facial and acral rPPG (phase-shifted pair) as BBP-Net input dominates single-location approaches, with DBP error improvements exceeding 2 mmHg.

Ablation Studies

Multiple ablations confirm:

• Data Augmentation: Frame-interpolation-based HR variants directly benefit model generalization, lowering MAE by over 1 BPM.
• Loss Functions: Including $L_{pv}$ improves HR estimation by about 2 BPM.
• Window Length: Longer input sequences enhance HR accuracy, though with increased inference latency.
• Phase-shifted Input: Dual-site input to BBP-Net is strictly superior for BP estimation versus either facial or acral alone (DBP MAE: 7.09 mmHg vs 8.67/9.50 mmHg).
• Output Bounding: Scaled sigmoid in BBP-Net is critical—the removal yields SBP MAE degradation of ~9 mmHg.

Implications and Future Directions

This dual-site, phase-aware method offers a paradigm shift for contactless vital sign monitoring. Practically, the approach is conducive to deployment in real-world clinical and telehealth applications, particularly for populations where continuous, unobtrusive monitoring is essential. Theoretically, the explicit modeling of phase shifts in pulse wave propagation expands the utility of facial video as a physiological surrogate, supporting more robust biomarker extraction and multi-task architectures.

Challenges remain, including broader clinical validation, real-world deployment with diverse demographic/lighting confounds, and dataset limitations (e.g., absence of facial-site PPG for hard phase verification). The framework can serve as a basis for extending non-contact monitoring to further cardiovascular indices, arrhythmia detection, and even high-fidelity waveform reconstruction.

Conclusion

The work systematically introduces a phase-shifted, dual-site rPPG modeling paradigm with a novel deep learning pipeline, demonstrating strong advancements in camera-based HR and BP estimation. Empirical evidence shows state-of-the-art performance across established datasets, robust ablation support, and methodical compliance with international BP validation standards. Future directions include dataset expansion for facial PPG and adaptation to varied acquisition conditions, marking critical steps toward practical, ubiquitous remote vital sign monitoring systems.