PPG-GPT: Transformer Models for PPG Signals

• PPG-GPT is a family of foundation models using a decoder-only transformer architecture to model, generate, and analyze photoplethysmogram signals.
• It leverages self-supervised pretraining on massive, noisy datasets to drive accurate downstream tasks such as arrhythmia detection and heart rate estimation.
• Extensions include cross-modal digital twin synthesis, real-time predictive analytics, and bias-aware fine-tuning for equitable cardiovascular monitoring.

PPG-GPT (Photoplethysmogram Generative Pre-trained Transformer) refers to a family of foundation models based on the GPT (decoder-only transformer) architecture, pre-trained and adapted specifically for the modeling, generation, and analysis of PPG (photoplethysmography) signals. The PPG-GPT line of research provides a general-purpose, modality-agnostic digital twin for pulse waveforms, embracing scalable self-supervised learning, generative pretraining, and transfer to a wide spectrum of downstream physiological monitoring tasks. This class of models is notable for their scalability—ranging up to billions of parameters—their ability to generalize across settings (ICU, wearable, smartphone), and their capacity to drive both predictive and generative tasks relating to cardiovascular health from raw, noisy biosignals (Chen et al., 11 Mar 2025, Kataria et al., 12 Feb 2025, Panchumarthi et al., 20 Sep 2025, Saha et al., 3 Feb 2025, Filho et al., 26 Sep 2025).

1. Architectural Foundations and Pretraining Paradigm

PPG-GPT is implemented as an autoregressive, decoder-only transformer operating on segmented PPG waveforms. Pretraining is conducted in a self-supervised manner on massive corpora—for example, the UCSF ICU waveform repository, featuring 200 million 30 s single-channel PPG segments resampled to 40 Hz (Chen et al., 11 Mar 2025, Kataria et al., 12 Feb 2025, Panchumarthi et al., 20 Sep 2025). The model input consists of non-overlapping patches (e.g., 1 s/40-sample or 30 s/1200-sample), each embedded via a linear projection into a higher-dimensional space $E_t = W^{\mathrm{patch}} x_t + b^{\mathrm{patch}}$. A learnable or rotary positional encoding is added to retain within-sequence order.

The transformer core stacks $L$ identical blocks, each comprising multi-head self-attention (typically $h=4$–$16$ heads), a feed-forward subnetwork (with $$d_{\mathrm{ff}} \approx 4 \times d_{\mathrm{model}}$$), and residual/normalization structure. In the canonical 19M-parameter configuration: $L=6$, $d_{\mathrm{model}}=256$, $d_{\mathrm{ff}}=1024$, $h=4$ (Panchumarthi et al., 20 Sep 2025).

Pretraining objectives are generative—optimizing for one-step-ahead patch prediction via a logit-Laplace loss: for $x \in (0,1)$, $$f(x;\mu,b) = (1/(2 b x (1 - x))) \exp(-|\mathrm{logit}(x) - \mu|/b)$$, with $L_{\textrm{Laplace}}$ defined as the negative log-likelihood over the sequence (Chen et al., 11 Mar 2025, Panchumarthi et al., 20 Sep 2025). This objective encourages uncertainty-aware, amplitude-faithful modeling of continuous-valued PPG.

The architecture is adapted for temporal tasks, supporting test-time personalization, in-context domain adaptation, and multi-modal extensions. For non-sequence environments, convolutional backbones (e.g., ResNet-26 as in Pulse-PPG (Saha et al., 3 Feb 2025)) and motif cross-attention modules may substitute pure transformers but are compatible with the same transfer paradigm.

2. Fine-Tuning and Mixed-Objective Adaptation

Transfer to labelled tasks (e.g., heart rate regression, arrhythmia detection) is achieved by attaching a prediction head (e.g., linear or Gated MLP) to the pooled sequence representation, with downstream-specific objective $L_o$ (cross-entropy for classification, MSE/MAE for regression). Fine-tuning incorporates a composite loss: $$L = L_o(y, \hat{y}) + \lambda \cdot L_m(X, \hat{X})$$, where $L_m$ retains PPG structural fidelity through continued generative loss (Chen et al., 11 Mar 2025, Panchumarthi et al., 20 Sep 2025).

Best practices include low learning rates ($1e{-5}$) for the transformer backbone and modest rates ($2e{-4}$) for task heads or aggregators (Kataria et al., 12 Feb 2025). Aggregator modules (BLSTM with attention, Mamba, xLSTM) are employed in settings requiring long-range temporal pooling over hours of data (Kataria et al., 12 Feb 2025).

Empirical results indicate fine-tuned PPG-GPT achieves state-of-the-art metrics on ICU cardiac arrest prediction (AUROC≈0.82 at 1 hour pre-event) (Kataria et al., 12 Feb 2025), atrial fibrillation detection (F1=0.847), heart/respiration/blood pressure estimation (MAE competitive with leading models), and robust signal denoising (MAE <0.1 for up to 40% masked signals) (Chen et al., 11 Mar 2025).

3. Fairness, Domain Adaptation, and Debiasing

Although PPG-GPT substantially reduces average error rates after fine-tuning (up to 80%), naïve fine-tuning can inadvertently amplify demographic fairness gaps—e.g., increasing the absolute difference in MAE between male and female subgroups (Panchumarthi et al., 20 Sep 2025). To address this, FairTune introduces bias-aware fine-tuning with three mitigation strategies:

• Inverse-Frequency Weighting (IF): Samples are weighted by the inverse demographic frequency. This consistently reduces MAE and demographic gap by 10–30% and 30–50%, respectively.
• Group Distributionally Robust Optimization (GroupDRO): Minimizes the maximum subgroup loss. Effectiveness is domain dependent and may incur increased MAE under distribution shift.
• Adversarial Debiasing (ADV): Incorporates a gradient reversal adversary to suppress demographic information in the embeddings. Effectiveness is variable and tuning sensitive.

Representation analyses (Silhouette scores, MMD) show that IF most efficiently collapses gender clusters in the penultimate embedding space, yielding more physiologically grounded features without impairing accuracy (Panchumarthi et al., 20 Sep 2025).

4. Extensions: Multimodal, Artifact-Resilient, and Digital Twin Synthesis

The PPG-GPT paradigm generalizes beyond classical contact sensing. In "Radio-PPG," a generative framework synthesizes PPG digital twins from non-contact 6G/WiFi OFDM radar, paralleling the GPT pipeline: high-dimensional context (OFDM channel features), deep nonlinear mapping (MLP/U-Net cascade or transformer), and efficient decoding (inverse DCT or autoregressive sequence) (Filho et al., 26 Sep 2025). Proposed PPG-GPT extensions for cross-modality pretraining include:

• Input embeddings adapted for radar/camera/wearable channels, with positional encoding.
• Causal self-attention spanning time and channel features.
• Pretraining objectives such as autoregressive or masked sequence modeling on tokenized PPG amplitude.
• Fine-tuning adapters for personalized physiological signatures.

A plausible implication is that such architectures enable non-contact, universal digital twin generation for cardiovascular health across varied environments and demographics.

Artifact mitigation (e.g., pressure-induced distortion in wrist PPG) is addressed in frameworks like CP-PPG, whose loss functions penalize deviation from key morphological fiducials (systolic/diastolic peaks, notches) and apply peak-count constraints. Incorporating such morphology-aware or adversarial losses and multi-site curation can further endow PPG-GPT models with robust denoising and cross-modal generalization (Hung et al., 3 Apr 2025).

5. Training Data, Field-to-Lab Transfer, and Limitations

PPG-GPT and related foundation models are trained on massive datasets. ICU-based models utilize over 200 million clinical waveform segments (2.2 million hours). Field-trained models such as Pulse-PPG aggregate uncurated wearable data spanning 120 participants and 21 billion samples over 100 days, capturing motion/artifact variability absent from standardized clinical datasets (Saha et al., 3 Feb 2025).

Results demonstrate that models pretrained on field data generalize better to lab and clinical tasks compared to models trained on clean clinical data, particularly when subject to real-world noise and physiological variation. This suggests that embracing variability and artifact-rich corpora during pretraining leads to more robust and adaptable representations.

Limitations include sensitivity to distribution shifts, model size and memory overhead, and under-representation of certain populations or use-cases (e.g., sleep disturbance, pediatric/geriatric signals). Direct comparisons with encoder-only transformer architectures and patch-free sequence modeling (e.g., Mamba) remain open research questions (Chen et al., 11 Mar 2025).

6. Downstream Applications and Performance Benchmarks

PPG-GPT and derivative foundation models support a diverse set of downstream tasks:

Task Domain	State-of-the-Art Metric	PPG-GPT Performance
Atrial fibrillation	F1 (Stanford dataset)	0.847 (1B param), exceeding DeepBeat/SiamQuality
Heart rate estimation	MAE (WESAD/DaLiA/IEEE)	4.98/4.77/1.98 bpm (1B param)
Respiration rate	MAE (BIDMC 5-fold CV)	0.93 bpm (1B param)
Blood pressure	MAE (PulseDB SBP/DBP)	8.12/6.78 mmHg (1B param)
Cardiac arrest (ICU)	AUROC (24h, 1h horizon)	0.791–0.821 (345M param, fine-tuned)

These models are deployable for both real-time predictive analytics and generative tasks (e.g., waveform denoising, missing data imputation, digital twin simulation) (Chen et al., 11 Mar 2025, Kataria et al., 12 Feb 2025, Filho et al., 26 Sep 2025). The foundation model approach allows for rapid adaptation to new tasks by shallow fine-tuning or lightweight adapters, facilitating both generalization and personalization.

7. Implementation Practices and Outlook

Canonical PPG-GPT implementations utilize highly regularized Transformers (layernorm, dropout, weight decay), learned or rotary positional encoding, and composite loss objectives balancing task accuracy with waveform fidelity (Chen et al., 11 Mar 2025, Panchumarthi et al., 20 Sep 2025). Field-deployable models emphasize robust pre-training on noisy, artifact-rich, and demographically varied corpora (Saha et al., 3 Feb 2025).

For deployment, explicit fairness monitoring, lightweight bias mitigation (inverse-frequency weighting), and domain-tailored fine-tuning are recommended to avoid performance and equity degradation (Panchumarthi et al., 20 Sep 2025). Future directions include incorporating additional sensing modalities (accelerometer, radar), extending the transformer backbone to patch-free recurrent and state-space models, and converging clinical and wearable source domains to optimize cross-domain generalization.

PPG-GPT thus establishes the foundation for a new class of generalist biosignal models, supporting both universal and personalized physiological monitoring, robust to artifacts and distribution shifts, and adaptable to a heterogeneous landscape of biomedical applications.