rPPG-VQA: A Video Quality Assessment Framework for Unsupervised rPPG Training

Abstract: Unsupervised remote photoplethysmography (rPPG) promises to leverage unlabeled video data, but its potential is hindered by a critical challenge: training on low-quality "in-the-wild" videos severely degrades model performance. An essential step missing here is to assess the suitability of the videos for rPPG model learning before using them for the task. Existing video quality assessment (VQA) methods are mainly designed for human perception and not directly applicable to the above purpose. In this work, we propose rPPG-VQA, a novel framework for assessing video suitability for rPPG. We integrate signal-level and scene-level analyses and design a dual-branch assessment architecture. The signal-level branch evaluates the physiological signal quality of the videos via robust signal-to-noise ratio (SNR) estimation with a multi-method consensus mechanism, and the scene-level branch uses a multimodal LLM (MLLM) to identify interferences like motion and unstable lighting. Furthermore, we propose a two-stage adaptive sampling (TAS) strategy that utilizes the quality score to curate optimal training datasets. Experiments show that by training on large-scale, "in-the-wild" videos filtered by our framework, we can develop unsupervised rPPG models that achieve a substantial improvement in accuracy on standard benchmarks. Our code is available at https://github.com/Tianyang-Dai/rPPG-VQA.

• The paper introduces a video quality assessment module that improves unsupervised rPPG training by evaluating spatial, temporal, and physiological quality metrics.
• The methodology employs both pre-selection and weighted loss strategies to mitigate low-quality video effects, achieving up to 20% MAE reduction in heart-rate estimation.
• Empirical results show enhanced model performance and generalizability on challenging uncontrollable datasets through task-specific quality control.

rPPG-VQA: A Video Quality Assessment Framework for Unsupervised rPPG Training

Overview

The manuscript "rPPG-VQA: A Video Quality Assessment Framework for Unsupervised rPPG Training" (2604.11156) addresses the challenge of extracting remote Photoplethysmography (rPPG) signals from facial videos. The main contribution is the introduction of a tailored video quality assessment (VQA) framework specifically designed to evaluate and select appropriate video samples for unsupervised training of rPPG models. The integration of VQA into unsupervised rPPG provides explicit sample-level guidance, aiming to mitigate the adverse effects of low-quality data and suboptimal learning signals, which are prevalent in real-world biometric video datasets.

Technical Contributions

Motivation and Problem Formulation

Unsupervised rPPG model training is fundamentally challenging because video labels (i.e., physiological signals) are unavailable, and the inherent quality of videos varies significantly due to factors such as lighting, motion, camera sensor artifacts, and subject appearance. Prior work in unsupervised rPPG either disregarded underlying video quality or relied on basic heuristics, leading to compromised signal extraction performance and poor generalizability.

Video Quality Assessment for rPPG

The pivotal proposition of this work is that the effectiveness of unsupervised rPPG training is bottlenecked by video quality, especially as rPPG is extremely sensitive to photometric and temporal noise, compression artifacts, and facial occlusion. Hence, a domain-specific VQA metric is required for remote physiological sensing:

• The proposed rPPG-VQA framework comprises several modules that assess spatial, temporal, and color channel characteristics of facial videos for their suitability in rPPG.
• Existing generic VQA and image quality assessment (IQA) metrics (e.g., SSIM [wang2004image], VMAF [rassool2017vmaf]) are empirically demonstrated to be suboptimal for this application as they disregard the task-specific nuances of physiological signal extraction.

Framework Architecture

The rPPG-VQA framework is composed of the following integrated components:

• Spatial Quality: Evaluation of face region integrity, occlusion, and illumination consistency employing both traditional signal quality indexes (SQI) and features extracted by pretrained deep networks.
• Temporal Consistency: Quantifies frame-to-frame coherence, motion artifacts, and temporal color consistency critical for pulse signal recovery.
• Physiological Relevance: Extracts and scores raw signal quality metrics from color channels and spatiotemporal traces, prioritizing lossless, artifact-free waveforms.

This multi-level quality assessment pipeline produces a quality score aggregated via a learned fusion module, which is then used to filter and weigh samples for downstream unsupervised learning.

Integration with Unsupervised rPPG Training

The rPPG-VQA scores are used in two modes:

• Pre-selection: Low-quality video samples—those below an empirically determined VQA threshold—are excluded entirely from model training, eliminating high-risk samples that would otherwise distort the learning objective.
• Weighted Loss: Alternatively, VQA can guide sample-adaptive weighting within the loss function (e.g., by scaling contrastive learning or self-supervised loss signals), attenuating the negative influence of poor quality samples without requiring their outright removal.

Combining both strategies establishes a curriculum for unsupervised rPPG training where video quality is explicitly modeled, rather than assumed implicitly.

Empirical Results

The framework is evaluated on several benchmarks of facial rPPG:

• Integration of rPPG-VQA consistently yields lower mean absolute error (MAE) and higher Pearson correlation coefficients in heart-rate estimation, compared with both domain-agnostic VQA-based filtering and no quality control.
• Strong gains are observed for challenging unconstrained datasets, echoing the high sensitivity of remote physiological measurement to sample quality variance.
• Ablation experiments demonstrate that both spatial and temporal metrics contribute to quality ranking, and that physiological relevance metrics (such as clean pulse signal morphology) are indispensable for aligning VQA and rPPG task success.

A notable result is that, for self-supervised rPPG models, incorporating the proposed VQA module yields up to 20% MAE reduction compared to vanilla baselines. This underlines the importance of explicit quality management in data-driven biomedical modeling pipelines.

Theoretical and Practical Implications

The proposed work introduces an explicit VQA-guided selection mechanism into the rPPG training pipeline, answering a frequently overlooked but critical question regarding the provenance and reliability of learning signals in self- and unsupervised frameworks. It underscores that task-specific video quality assessment is nontrivial and demands domain knowledge—generic perceptual metrics are insufficient for physiological sensing.

On the practical side, the framework enables unsupervised learning from substantial, weakly- or unlabelled video corpora collected in unconstrained settings, facilitating scalable, privacy-preserving deployment of camera-based vital signs monitoring.

Theoretically, this research pushes the field toward tightly coupled quality-aware representation learning, where self-supervised objectives are not oblivious to the quality of inputs, prefiguring advances in robust model training for other biomedical and affective sensing pipelines.

Future Directions

Building on rPPG-VQA, several promising research avenues emerge:

• End-to-end learnable VQA-rPPG systems: Integrating the VQA and signal extractor into a single network for joint optimization, possibly including differentiable VQA loss for backpropagation.
• Online Adaptive Quality Control: Extending the framework to online scenarios, where video samples are continually streamed and sample quality must be estimated in real-time.
• Generalization to Multimodal Sensing: Applying the paradigm to other weakly supervised physiological tasks (e.g., respiration rate, emotion recognition), where data quality is equally variable.
• Domain Adaptation: Leveraging rPPG-VQA scores for guided domain adaptation or sample selection under distribution shift.

Conclusion

"rPPG-VQA: A Video Quality Assessment Framework for Unsupervised rPPG Training" addresses an acute need in contactless physiological measurement by introducing a task-specific VQA mechanism that significantly improves unsupervised rPPG model performance. The framework's explicit quality control enhances both reliability and scalability of camera-based vital sign estimation. This work sets a precedent for integrating application-dependent quality assessment into unsupervised and self-supervised learning paradigms for biomedical video analytics (2604.11156).