RPM-Distill: Physiology-guided Adaptive Cross-modal Distillation for Robust Remote Physiological Measurement

Abstract: Video-based remote physiological measurement (RPM) is highly accessible but remains fragile under varying illumination, skin tones, and motion. Radio frequency (RF) radar is largely invariant to illumination and appearance, providing complementary cardio-respiratory micro-motion cues; however, requiring radar at inference is often impractical due to its limited ubiquity and deployment overhead. We propose RPM-Distill, a physiology-guided cross-modal distillation framework that leverages synchronized radar only during training while retaining video-only inference. Our key observation is that although RGB and RF waveforms differ in sensing physics and time-domain morphology, they share similar latent periodic rhythm in the frequency domain. We thus distill physiology-structured spectral evidence to improve robustness, via losses that (i) anchor the fundamental peak, (ii) match the off-peak background distribution, and (iii) preserve spectral morphology and sharpness. To avoid negative transfer under sample-level teacher quality and alignment uncertainty, a spectral policy network predicts sample-level distillation gates and component weights from the student--teacher spectral relation map, learned with a meta bilevel objective on a small labeled validation split. Through extensive experiments in challenging conditions and cross-dataset settings, RPM-Distill brings 81\% MAE and 21\% correlation improvement over unimodal baselines. Code is at https://github.com/WJULYW/RPM-Distill.

• The paper introduces a physiology-structured spectral distillation framework that uses RF radar guidance during training to improve video-based RPM.
• It employs a spectral policy network with bilevel meta-learning to adaptively weight frequency-domain losses and prevent negative cross-modal transfer.
• Extensive experiments demonstrate reduced MAE, enhanced Pearson correlation, and strong generalization across diverse datasets and conditions.

RPM-Distill: Adaptive Cross-Modal Knowledge Distillation for Robust Video-Based Remote Physiological Measurement

Introduction and Motivation

Video-based remote physiological measurement (RPM), particularly via remote photoplethysmography (rPPG), offers significant non-contact accessibility for vital sign monitoring but is hindered by motion artifacts, diverse illumination, and skin tone heterogeneity. In contrast, radio frequency (RF) radar sensing is invariant to optical conditions and provides robust cardio-respiratory cues. However, radar deployment at scale is impractical due to limited ubiquity. Previous approaches that fuse video and RF require both modalities at inference, impeding broad adoption. Naïve feature or temporal alignment across modalities encounters negative transfer due to their fundamental physical and morphological disparities, especially under mismatched or corrupt conditions.

This work introduces RPM-Distill (2606.28089), a physiology-structured cross-modal distillation framework, where RF radar supervision is leveraged strictly during training to enhance video-only inference. The core insight is that while the modalities differ in time-domain morphology, both reflect the same latent physiological periodicity in the frequency domain, enabling meaningful cross-modal transfer at the spectral level.

Figure 1: rPPG and RF capture cardio-physiology via disparate sensing physics and waveform morphologies, but share matched fundamental frequency structure in the frequency domain; RF guidance is distilled to the video model during training exclusively.

Methodology

Physiology-Structured Spectral Distillation

RPM-Distill eschews feature-level transfer in favor of frequency-domain knowledge transfer, exploiting the shared periodic physiological rhythm observable in both RF and RGB in the spectral domain. The distillation objective is decomposed into three complementary components:

1. Fundamental Peak Anchoring: Enforces agreement around the teacher's dominant spectral peak, mitigating peak mislocalization failures in video-based RPM.
2. Off-Peak Background Matching: Matches the spectral "noise floor" beyond the fundamental peak, suppressing broadband motion and appearance-induced artifacts.
3. Spectral Morphology Consistency: Enforces match in spectral shape, including centroid alignment and sharpness (entropy), addressing rhythm smearing and non-physiological broadening.

These spectral constraints are implemented via differentiable Gaussian masks and softmax-normalized distributions over the physiological frequency band, with explicit handling of peak vicinity and off-peak energy.

Adaptive Spectral Policy Network and Bilevel Meta-Learning

To prevent negative transfer caused by unreliable or misaligned RF supervision, RPM-Distill introduces a spectral policy network, which dynamically predicts per-sample distillation gates and component weights. This policy network operates on concatenated student–teacher spectral maps and encompasses a 1D CNN encoder with a matrix decomposition-based decoder for parsimonious quality token extraction. The network outputs both a global gating scalar and a simplex of component weights per sample.

Policy parameters are meta-optimized via bilevel optimization: a virtual student update with the current policy is evaluated on a held-out supervised validation set, and policy parameters are updated to maximize supervised generalization. This closed-loop process ties the adaptive distillation directly to physiological waveform fidelity and robustness metrics.

Figure 2: RPM-Distill overview. Training-time RF serves as the fixed teacher; the spectral policy network dynamically gates the three spectral distillation losses and is meta-optimized using a bilevel scheme anchored by a supervised validation split.

Experimental Results

RPM-Distill is extensively validated on several multi-modal datasets (EquiPleth, PhysDrive), as well as RGB-only datasets (PURE, MMPD). Experimental protocols include cross-dataset generalization, label-scarce training, distributional shifts in lighting and motion conditions, and robustness to synthetic inter-modal misalignment.

Key Numerical Results:

• Cross-dataset Performance: On EquiPleth, RPM-Distill brings the mean absolute error (MAE) down to 1.57 bpm (beating the closest multimodal fusion by 21% and KD baseline by 36%) and raises Pearson's $r$ to 0.94.
• PhysDrive (challenging real-world conditions): RPM-Distill cuts MAE by 81% and improves correlation by 21% over the best RGB-only baseline.
• Label Efficiency: RPM-Distill reaches competitive performance with as little as 40% labeled data; results are consistently strong even with partial supervision, reflecting its strong utilization of unlabeled paired data and teacher signal.
• Ablations: Dropping any of the three spectral objectives, adaptive weights, gate, or the bilevel meta-policy consistently degrades robustness and overall accuracy, underscoring the necessity of each architectural component.

The method further demonstrates clear resilience to substantial synthetic temporal misalignment between video and radar at training time, with the learned distillation gate effectively shutting off unreliable or deleterious supervision.

Analysis and Discussion

Strong Claims and Observed Phenomena:

• Distinct Disentanglement of Modality-Specific Failure Modes: The frequency-domain distillation objective effectively targets dominant video RPM failure mechanisms, including spectral peak suppression, background pollution, and rhythm smearing, as supported by both ablation metrics and qualitative waveform recoveries.
• Sample-Adaptive Distillation: The spectral policy network exhibits scenario-dependent gating and weighting, confirmed quasi-statistically by distributional shifts in learned gate/weight values under harder vs. easier conditions.
• Out-of-Domain Robustness: The student model distilled via RPM-Distill not only outperforms video-only baselines in-sample but generalizes to standalone RGB datasets without RF availability, indicating its learning of modality-agnostic rhythmic priors absent in previous KD or fusion paradigms.

Implications and Future AI Directions:

This physiology-aware cross-modal distillation framework introduces a new operationalization for leveraging privileged modalities (LUPI) in settings with severe domain or physics mismatch. It forgoes restrictive multimodal dependencies at deployment, enabling scalable real-world RPM adoption. The modular spectral distillation approach—anchoring transfer to shared latent physiological rhythms, not raw signals—holds promise for other biomedical and cross-sensor tasks with heterogeneous but commensurable underlying processes.

Extensions could include adaptation to richer physiological signals (e.g., multi-task rhythm and arrhythmia detection, multi-vital inference), unsupervised or foundation-model-based cross-modality teachers, or more general forms of sample-adaptive policy learning. Handling severe desynchronization, missing radar segments, or more extreme label scarcity likewise remain open technical challenges.

Qualitative Visualizations

Figure 1: rPPG and RF each manifest distinct time-domain signatures but share a common frequency-domain heartbeat structure, justifying frequency-domain distillation.

Figure 2: Architecture of RPM-Distill, including spectral relation map construction, spectral policy network, and the meta-optimization loop.

Conclusion

RPM-Distill presents a rigorously validated, theoretically sound framework for adaptive cross-modal distillation, exploiting shared frequency-domain physiological content while dynamically mitigating inter-modal unreliability and negative transfer. It achieves state-of-the-art accuracy and robustness, especially in domain-shift and low-SNR scenarios, and generalizes across datasets and conditions without requiring multimodal sensors in operation. The spectral policy network and bilevel meta-objective emerge as essential design elements, setting a foundation for broader cross-modal transfer solutions in AI-driven signal processing and multi-sensor data fusion.