BTS-rPPG: Orthogonal Butterfly Temporal Shifting for Remote Photoplethysmography

Abstract: Remote photoplethysmography (rPPG) enables contactless physiological sensing from facial videos by analyzing subtle appearance variations induced by blood circulation. However, modeling the temporal dynamics of these signals remains challenging, as many deep learning methods rely on temporal shifting or convolutional operators that aggregate information primarily from neighboring frames, resulting in predominantly local temporal modeling and limited temporal receptive fields. To address this limitation, we propose BTS-rPPG, a temporal modeling framework based on Orthogonal Butterfly Temporal Shifting (BTS). Inspired by the butterfly communication pattern in the Fast Fourier Transform (FFT), BTS establishes structured frame interactions via an XOR-based butterfly pairing schedule, progressively expanding the temporal receptive field and enabling efficient propagation of information across distant frames. Furthermore, we introduce an orthogonal feature transfer mechanism (OFT) that filters the source feature with respect to the target context before temporal shifting, retaining only the orthogonal component for cross-frame transmission. This reduces redundant feature propagation and encourages complementary temporal interaction. Extensive experiments on multiple benchmark datasets demonstrate that BTS-rPPG improves long-range temporal modeling of physiological dynamics and consistently outperforms existing temporal modeling strategies for rPPG estimation.

• The paper proposes a novel BTS module integrated into a Swin Transformer backbone to exponentially expand temporal receptive fields.
• It utilizes Orthogonal Feature Transfer to ensure non-redundant feature exchange, significantly reducing error metrics on standard rPPG benchmarks.
• The approach achieves state-of-the-art accuracy with real-time inference and robust cross-dataset generalization in remote photoplethysmography.

BTS-rPPG: Orthogonal Butterfly Temporal Shifting for Remote Photoplethysmography

Introduction

Remote photoplethysmography (rPPG) addresses the challenge of contactless physiological measurement by analyzing subtle facial appearance changes induced by blood circulation. Despite advances in deep learning-based approaches for rPPG, capturing long-range temporal dependencies in physiological signals remains problematic. Contemporary models predominantly perform temporal aggregation across adjacent frames using shift or convolutional operators, leading to restricted temporal receptive fields and suboptimal modeling of periodic cardiac dynamics. The "BTS-rPPG: Orthogonal Butterfly Temporal Shifting for Remote Photoplethysmography" (2604.01679) paper tackles this by proposing an Orthogonal Butterfly Temporal Shifting (BTS) framework that integrates a sophisticated, multi-stage hierarchical temporal communication mechanism, inspired by the butterfly network structure of the Fast Fourier Transform (FFT), coupled with an Orthogonal Feature Transfer (OFT) strategy to achieve efficient and complementary temporal feature exchange.

Methodology

The BTS-rPPG framework comprises three principal components: an input representation module, a BTS-enhanced Swin Transformer backbone, and a predictor head.

Input Representation

Each input consists of facial video sequences, which are augmented with normalized difference frames (NDFs) to emphasize temporal appearance variations. For each frame $t$, the NDF is computed and concatenated channel-wise with the original RGB data, resulting in a $6$-channel tensor. This composite representation is embedded using a 3D convolutional layer before being passed into the Transformer backbone.

BTS-Enhanced Swin Transformer

The backbone is based on the Swin Transformer, modified by the insertion of the BTS module at each stage. Tokens are partitioned into windows, and at each BTS stage, temporal communication is executed according to a butterfly pairing schedule: frame $t$ communicates with $\pi_\ell(t) = t \oplus 2^{\ell-1}$, where $\oplus$ denotes bitwise XOR. This exponentially enlarges the receptive field with network depth, permitting efficient interaction from local to global temporal ranges.

Figure 1: The BTS-rPPG butterfly-inspired pairing mechanism enables hierarchical temporal mixing, in contrast to the strictly local communication of prior shift and convolution approaches.

Orthogonal Feature Transfer (OFT)

Naive shift-based approaches risk propagating redundant, correlated features across frames. OFT mitigates this by projecting the transferable feature fold onto the orthogonal complement of the target context, so only the linearly independent (novel) information is transferred. This projection utilizes a learned context matrix, ensuring only features that add unique, non-redundant temporal information are communicated between frames.

Figure 2: BTS-rPPG overview, depicting the input encoding, BTS-enabled Swin Transformer, and predictor head. Subfigure II details OFT, while Subfigure III illustrates BTS’s hierarchical communication.

Predictive Head and Loss

After multi-stage temporal-spatial encoding, BTS-rPPG applies global average pooling and a linear projection to regress per-frame rPPG signals. The learning objective is a differentiable negative Pearson correlation loss, which emphasizes the fidelity of the predicted waveform to the ground-truth signal.

Experimental Evaluation

Benchmarks and Protocol

BTS-rPPG is evaluated on three public rPPG datasets: PURE, UBFC-rPPG, and MMPD. These benchmarks are characterized by varying acquisition scenarios, covering controlled, appearance-diverse, and cross-domain settings, testing both the accuracy and generalization properties of the model.

Quantitative Results

On intra-dataset protocols, BTS-rPPG establishes state-of-the-art performance over competitive baselines (including PhysNet, EfficientPhys, RhythmFormer, and PhysFormer), achieving the lowest MAE, MAPE, and RMSE, and highest Pearson correlation scores across all tested datasets. Notably, for UBFC-rPPG, it achieves a relative error reduction over the next best method by more than 60% across all metrics, and a correlation coefficient ($r$) of $1.00$ on both PURE and UBFC-rPPG, indicating almost perfect waveform recovery. On the domain-shifted MMPD, the model retains robust performance, confirming superior generalization.

Cross-Dataset Generalization

Extensive cross-dataset tests (e.g., training on PURE/UBFC, testing on the other datasets) further validate BTS-rPPG’s domain robustness. In all these cases, it outperforms or matches the strongest previous methods, indicating that the architecture captures physiologically meaningful representations, rather than overfitting to spurious dataset-specific correlations.

Computational Efficiency

Despite the addition of butterfly-based temporal communication, BTS-rPPG supports real-time inference (2.05 kfps on Nvidia T4), with fewer parameters and lower computational demands compared to other recent Transformer-based methods (e.g., PhysFormer, LSTS, and Reperio-rPPG), yielding a favorable trade-off between accuracy and efficiency.

Ablation and Analysis

Ablation studies on intra-dataset MMPD clarify the independent and joint contributions of OFT and BTS. OFT alone improves feature transfer by mitigating redundancy, while the butterfly communication schedule (BTS) dramatically increases temporal modeling capacity. Combined, they produce the maximal reduction in all error metrics.

Figure 3: Ablation analysis demonstrates the additive benefits of OFT and BTS modules over the TCS baseline.

The OFT transfer fold ratio ($C_s/d$) is optimized at $1/4$, balancing the need for sufficient cross-frame interaction and retention of target-specific details. Excessively narrow or wide transfer folds compromise performance.

Figure 4: MAE and RMSE as functions of the OFT fold ratio, affirming optimal trade-off at a $1/4$ ratio on PURE and UBFC-rPPG.

Analyses varying the temporal communication schedule confirm that the butterfly (coarse-to-fine) order, not merely the inclusion of long-range links, is key. Local-only or uniform schedules underperform; reverse-butterfly (fine-to-coarse) also lags, corroborating the importance of hierarchically staged temporal expansion.

Theoretical and Practical Implications

The hierarchical butterfly temporal mixing aligns with both the short-term local continuity and long-range periodicity characteristic of rPPG signals. The combination with OFT advances the efficiency of deep temporal communication, reducing redundancy typically unresolved in prior shift or convolutional methods. Practically, these advantages translate into improved tracking accuracy in both controlled and real-world mobile environments, with clear implications for health monitoring and affective computing.

Theoretically, while the effectiveness of BTS is empirically substantiated, its precise relationship with the frequency-domain structure of physiological signals and the spectral biases inherent to butterfly networks remains an open subject. Future work could further connect the BTS temporal communication to underlying harmonic analysis principles, such as leveraging the exponential family mixing inherent to the FFT.

Conclusion

BTS-rPPG advances temporal modeling for contactless photoplethysmography by integrating hierarchical, butterfly-inspired multi-stage pairing and orthogonal feature transfer. This design yields consistent improvements over established local and global temporal aggregation schemes, both in accuracy and generalization, with no loss of efficiency. The results imply that structured, complementary temporal interactions are essential for robust physiological signal recovery and open avenues for further spectral theory-informed temporal modeling in video-based biosensing.

Reference:

"BTS-rPPG: Orthogonal Butterfly Temporal Shifting for Remote Photoplethysmography" (2604.01679)