Remote Photoplethysmography (rPPG)

• Remote photoplethysmography (rPPG) is a non-contact optical method that quantifies blood volume changes to estimate heart rate.
• It utilizes ROI detection, spatial averaging, and temporal filtering to extract subtle color variations in skin pixels from video data.
• Critical system parameters—frame rate irregularities, rolling shutter effects, and temporal window sizes—must be controlled to ensure measurement accuracy.

Remote photoplethysmography (rPPG) is a non-contact optical technique for quantifying physiological signals—most commonly heart rate—by measuring subtle temporal variations in the color intensity of human skin as recorded by a standard camera. This method extends photoplethysmography (PPG) beyond the traditional use of contact sensors, providing a flexible, unobtrusive, and scalable approach to physiological monitoring across numerous domains including medicine, biometrics, and human–computer interaction.

1. Principles of Remote Photoplethysmography

At its core, rPPG exploits the periodic changes in blood volume within the microvascular bed of the skin, which modulate the absorption and scattering of light. In video-based rPPG, these fluctuations manifest as minute color changes—primarily in the green and adjacent spectral bands—across skin pixels in each frame. The canonical processing pipeline involves:

• Region-of-interest (ROI) detection (skin segmentation or facial landmark alignment)
• Spatial averaging of color values over selected ROIs across frames
• Temporal filtering to isolate the cardiac band (typically 0.6–4 Hz)
• Signal extraction methods including color space projection (e.g., chrominance- or plane-orthogonal-to-skin (POS)-based)
• Subsequent analysis (e.g., frequency-domain peak detection, HRV computation)

Mathematical formalism for heart rate (HR) estimation from inter-beat intervals (RR) is often expressed as:

• For a window with $N$ beats:

$HR(W^s) = \frac{N-1}{\sum_{i=1}^{N-1} RR_i}$

where $W^s$ is a temporal window of length $s$.

2. Underexplored Experimental Factors: Frame Rate, Shutter, and Temporal Windowing

Although macroscopic influences—subject motion, skin tone variability, video compression, illumination—are commonly acknowledged, precise rPPG estimation reliability also depends on factors often overlooked in experimental and comparative studies (Mironenko et al., 2020):

1. Irregular Frame Rate

• rPPG algorithms frequently assume a constant acquisition frame rate, critical for the application of Fourier-based spectral analyses. Significant variability could, in principle, distort the time–frequency representation of the blood volume pulse (BVP).
• Empirical assessment with six smartphones demonstrated that amplitude and spectral discrepancies between interpolation methods that account for actual frame timestamps and those that assume ideal equidistant sampling are negligibly small—orders of magnitude below the signal’s main components.
• Under typical smartphone-induced frame rate jitter, the rPPG output is not materially degraded; however, device- and codec-wise differences may warrant additional investigation in future work.

2. Rolling Shutter Effect

• Most consumer cameras utilize a rolling shutter mechanism, capturing frames line by line. This induces temporal phase shifts of the pulse signal across different spatial image regions, mainly along the sensor’s rolling direction (often horizontal).
• Experiments using a four-quadrant LED arrangement and multi-phone recording revealed measurable phase shifts of up to 0.02 s across the horizontal axis (corresponding to ~10° at 1.43 Hz HR), an effect of similar magnitude to physiologically meaningful pulse transit time differences.
• The phase offset was not constant but oscillated over tens of seconds, a temporal dynamic that can confound spatial phase difference analyses for vascular research or biometric matching.
• This sensor artifact necessitates either algorithmic correction, reporting, or explicit control in sensitive rPPG analyses, especially those interpreting phase lags.

3. Size of Temporal Windows

• HR and related metrics are commonly calculated over finite time windows. Variation in the analysis window size can result in discrepancies up to 10% (absolute error of 4–10 bpm for a typical human HR).
• When comparing methods or benchmarking against reference ECG/PPG, identical windowing parameters or transparent reporting is critical; otherwise, variations in window duration represent a systematic error source that may overstate or mask algorithmic improvements.

3. Methodological Frameworks and Quantitative Impact

The quantification of these “minor” factors was established through structured experiments:

Factor	Impact Magnitude	Experimental Approach	Implication
Irregular Frame Rate	Negligible (<< signal amp.)	Dual interpolation (with/without timestamps), spectral & time–domain comparison	Device-specific; of minor impact under tested conditions
Rolling Shutter Effect	Up to 0.02 s phase shift (~10° HR phase)	Multi-region modulated LED, phase computation across axes	Sensor artifact can match physiological effect, must be controlled or corrected
Temporal Window Size	Up to 10% HR error	Varied window lengths on Fantasia ECG database, sliding differences analysis	Window durations must be matched/reported for fair method comparison

Conclusions drawn from these results include the recommendation for standardized reporting and, where feasible, correction of sensor-induced artifacts.

4. Practical Implications for rPPG Algorithm Development

The nuanced consideration of these factors yields several actionable guidelines:

• rPPG algorithm evaluations and inter-method comparisons must explicitly document temporal window sizes for both estimated and ground-truth reference signals, to avoid spurious differences driven by windowing effects rather than algorithmic advances.
• Correction for rolling shutter artifacts—whether via sensor orientation metadata, spatial result weighting, or algorithmic compensation—should be incorporated in workflows where phase relationships or spatial distribution of BVP are interpreted.
• Although frame rate irregularities were not pronounced in the tested smartphones, comprehensive studies across more diverse hardware, environmental conditions, and video codecs are warranted to establish generality.
• Mathematical modeling, such as the explicit use of

$HR(W^s) = \frac{N-1}{\sum_{i=1}^{N-1} RR_i}$

facilitates sensitivity analyses regarding the effect of non-uniform sampling and window size, offering a principled basis for protocol design and error estimation.

5. Reporting Standards and Future Research Directions

The findings underscore a set of community best practices and open research avenues:

• Standardization of evaluation protocols, particularly concerning window sizing and sensor reporting, is vital for reproducible rPPG research.
• Broader multi-device and multi-codec investigations are needed to robustly characterize the impact of smartphone/frame rate irregularities.
• Development of algorithmic modules that explicitly estimate or correct rolling shutter–induced phase shifts would mitigate cross-device variability and align non-contact PPG measurements with physiological ground truth.
• Researchers are encouraged to design protocols and benchmarks that dissociate genuine physiological differences from sensor- and processing-induced artifacts.

6. Conclusion

While attention in rPPG research has historically centered on primary factors such as motion, skin tone, and compression, rigorous error reduction and interpretability demand scrutiny of secondary influences. The experimental evidence reviewed here demonstrates that:

• Frame rate irregularities—if appropriately timestamp-corrected—have negligible effect on rPPG output.
• The rolling shutter effect can introduce phase distortions that reach or exceed physiologically meaningful thresholds.
• Variations in analysis window size have a substantial and systematic effect on derived HR metrics.

Accordingly, refined rPPG methods require both hardware-level artifact control and harmonized signal processing workflows. This attention to methodological details facilitates more accurate, transparent, and comparative physiological measurement, supporting the maturation of rPPG as both a scientific and practical tool (Mironenko et al., 2020).