HOT: Harmonic-Constrained Optimal Transport for Remote Photoplethysmography Domain Adaptation

Abstract: Remote photoplethysmography (rPPG) enables non-contact physiological measurement from facial videos; however, its practical deployment is often hindered by substantial performance degradation under domain shift. While recent deep learning-based rPPG methods have achieved strong performance on individual datasets, they frequently overfit to appearance-related factors, such as illumination, camera characteristics, and color response, that vary significantly across domains. To address this limitation, we introduce frequency domain adaptation (FDA) as a principled strategy for modeling appearance variation in rPPG. By transferring low-frequency spectral components that encode domain-dependent appearance characteristics, FDA encourages rPPG models to learn invariance to appearance variations while retaining cardiac-induced signals. To further support physiologically consistent alignment under such appearance variation, we propose Harmonic-Constrained Optimal Transport (HOT), which leverages the harmonic property of cardiac signals to guide alignment between original and FDA-transferred representations. Extensive cross-dataset experiments demonstrate that the proposed FDA and HOT framework effectively enhances the robustness and generalization of rPPG models across diverse datasets.

• The paper introduces HOT, which integrates frequency domain appearance adaptation with harmonic-constrained optimal transport to counteract domain shifts in rPPG signals.
• The methodology preserves key physiological cues by selectively replacing low-frequency amplitudes while retaining phase information to maintain temporal structure.
• Extensive experiments show significant improvements in metrics like MAE and Pearson-r, demonstrating HOT’s backbone-agnostic advantage in cross-domain generalization.

Harmonic-Constrained Optimal Transport for rPPG Domain Adaptation

Introduction

This paper presents a principled framework for addressing the domain shift problem in remote photoplethysmography (rPPG), a modality enabling non-contact physiological signal estimation from facial videos. Domain shift, arising from variations in illumination, camera characteristics, and subject appearance, degrades the performance of deep rPPG models when deployed outside their training domain. The authors propose a novel framework—Harmonic-Constrained Optimal Transport (HOT)—that combines frequency domain adaptation (FDA) and physiologically informed alignment mechanisms to address this challenge. The resulting methodology demonstrates both architecture-independent robustness and strong cross-domain generalization.

Figure 1: The proposed method synthesizes source data with target-domain appearance during training using unlabeled target samples, enabling appearance-invariant learning and superior cross-domain generalization.

Related Work and Problem Formulation

Traditional rPPG pipelines for physiological measurement via consumer cameras have evolved from signal processing chains reliant on handcrafted priors to end-to-end deep learning models that leverage spatio-temporal attention and transformer modules. Despite state-of-the-art results in within-domain scenarios, most deep approaches remain vulnerable to dataset shift. Feature-level or architectural regularization techniques only partially mitigate this phenomenon, and few have introduced explicit data-level or optimal transport-based adaptation. Existing domain generalization methods in vision tasks using FDA have not been systematically translated to the unique requirements of rPPG, where minute color fluctuations encode cardiac signals and appearance manipulations can compromise physiological integrity.

Methodology

The HOT framework introduces a comprehensive pipeline for rPPG cross-domain adaptation composed of two major contributions: frequency domain appearance adaptation (FDA) and harmonic-constrained optimal transport (HOT).

Frequency Domain Adaptation (FDA)

FDA replaces low-frequency amplitude components in the Fourier domain of source images with those from reference target-domain frames, while strictly preserving the phase. This preserves the underlying physiological signal while enforcing target-style appearance statistics during training. The method operates at a channel-wise, per-frame level, and the parameter $\beta$ controls the spatial extent of low-frequency replacement.

Figure 3: Low-frequency amplitude replacement illustrated, where target-domain appearance statistics are imposed in the Fourier domain while retaining phase and, hence, temporal structure.

Carefully selecting $\beta$ is critical: values that are too large can introduce detrimental appearance distortion, while values that are too small limit adaptation. As shown empirically, $\beta=0.05$ offers optimal tradeoff between adaptation and preservation of rPPG-relevant cues.

Figure 2: Ablation of low-frequency replacement ratio $\beta$ on performance. An optimal $\beta$ achieves strong domain transfer without disrupting rPPG signal structure.

Harmonic-Constrained Optimal Transport (HOT)

To robustly align source and FDA-augmented feature sequences, HOT introduces an optimal transport plan penalized by both geometric (cosine feature dissimilarity) and local harmonic inconsistency. Harmonic descriptors, derived from the cyclic temporal neighborhood of each token and computed via Hann-windowed DFT, encode the relative energy of dominant and second harmonics within a physiological frequency band (e.g., 0.7–4.0 Hz). The transport cost thus becomes

$$C_{ij} = (1 - \cos(z_i, \tilde{z}_j)) + \lambda_h | r_i - \tilde{r}_j |$$

with $\lambda_h$ controlling the balance between geometric and spectral regularization.

Figure 5: The overall architecture of HOT. Source and FDA-augmented video streams are processed by a shared backbone, with output tokens aligned by HOT.

The transport plan is solved with entropic regularization (Sinkhorn algorithm, 40 iterations), enabling efficient and stable learning.

Experimental Results

Setup

Evaluation is conducted using state-of-the-art rPPG models—covering both convolutional and transformer-based backbones—on the PURE, UBFC-rPPG, and MMPD datasets. Models are trained with supervised loss and HOT regularization, using 30% of the target set (unlabeled) for FDA reference and an 80/20 split on the source for training/validation.

Main Findings

In all protocols, HOT produces substantial improvements in mean absolute error (MAE), MAPE, RMSE, and Pearson-$r$ correlation. Importantly, performance gains are most pronounced for cross-dataset transfer to MMPD, which presents severe domain shift due to complex illumination and camera diversity.

Figure 4: Cross-dataset evaluation outcomes: integrating HOT yields reduced bias and tighter prediction agreement as shown in Bland-Altman plots and a higher density of predictions around the identity line.

Consistent improvements are demonstrated across network classes (CNN and transformer). The regularization effect of HOT produces representation geometry that is more robust to appearance shift, while the harmonic constraint enforces physiological plausibility in the learned alignment.

Ablation and Analyses

Progressive integration of geometric OT and harmonic regularization shows that only the addition of the harmonic constraint yields consistent cross-domain improvement, validating the importance of spectral structure in cardiac-signal alignment.

Figure 7: Performance of PhysNet as components of the objective are integrated, showing HOT’s distinct contribution beyond standard OT.

Sensitivity analysis reveals that moderate harmonic penalty ($\lambda_h=0.3$) and 40 Sinkhorn iterations strike a favorable efficiency-accuracy tradeoff; excess regularization or computation yields diminishing returns.

Figure 6: MAE heatmap across harmonic penalty and Sinkhorn iteration configurations, with optimal setting marked.

HOT increases training compute (~11% slower, ~2× peak memory) but not inference cost, since regularization is only imposed in training.

Implications and Future Directions

The presented FDA+HOT framework demonstrates a systematic and backbone-agnostic solution for rPPG cross-domain adaptation. By explicitly modeling appearance variation at the Fourier amplitude level and leveraging domain-specific physiological harmonics in the alignment process, the method advances the generalization and deployment viability of camera-based rPPG models, particularly where labeled data from the deployment domain is unavailable.

From a practical standpoint, HOT can be integrated with contemporary deep rPPG architectures in medical, affective sensing, and general HCI pipelines with minimal adaptation. The requirement of access to unlabeled target samples for FDA remains a limitation. Theoretically, this work highlights the benefit of incorporating temporal-spectral domain priors—absent from generic vision domain adaptation approaches—in physiological signal analysis.

Further research directions include extending HOT to unsupervised and semi-supervised settings, reducing dependency on target reference data through generative or simulation-based adaptation, and scaling validation to unconstrained, real-world environments with wider inter-subject variability.

Conclusion

This work introduces an appearance-invariant, physiologically guided domain adaptation framework for remote photoplethysmography by coupling frequency domain adaptation and harmonic-regularized optimal transport. HOT consistently improves cross-domain generalization of state-of-the-art models with only minor training overhead. The approach offers both theoretical insight and practical utility for robust non-contact physiological monitoring under real-world deployment conditions (2604.01675).