PPG-Distill: Efficient PPG Signal Distillation

• The paper presents a novel distillation framework that explicitly transfers both local waveform details and long-range rhythm structure beyond standard KD.
• It introduces patch-level morphology and rhythm distillation to preserve aligned signal features critical for accurate heart rate and AF detection.
• Empirical results show improvements up to 21.8% in F1 score with 7× faster inference and 19× fewer parameters on wearable device benchmarks.

PPG-Distill is a knowledge distillation framework for efficient photoplethysmography signal analysis that transfers both global and local knowledge from large PPG foundation models to lightweight students through prediction-, feature-, and patch-level distillation. Its central claim is that standard output- and feature-level distillation is insufficient for PPG because PPG carries both local waveform morphology and longer-range rhythm structure, and that these properties should be distilled explicitly. The framework is evaluated on heart rate estimation on DaLiA and atrial fibrillation detection on StanfordAF, where it improves the lightweight GPT-PPG-1m student by up to 21.8% while achieving up to 7× faster inference and about 19× fewer parameters than GPT-PPG-19m (Ni et al., 23 Sep 2025).

1. Conceptual scope and motivation

PPG-Distill is situated within the problem of compressing large PPG foundation models for deployment on resource-limited wearable and edge devices. The framework is motivated by the observation that models such as GPT-PPG-19m and PaPaGei can learn strong general PPG representations but remain difficult to deploy under memory and latency constraints. The stated objective is therefore not merely model compression in the abstract, but efficient PPG analysis on wearables while preserving the signal characteristics that matter most for downstream physiological tasks (Ni et al., 23 Sep 2025).

The framework distinguishes two kinds of signal structure as especially important. The first is local waveform morphology, understood as short-window waveform shapes associated with cardiovascular events. The second is longer-range rhythm or temporal structure, including periodicity, beat-to-beat timing regularity, and autonomic dynamics. This motivates a distillation design that extends beyond conventional prediction matching and whole-signal feature matching. In the paper’s formulation, PPG-Distill transfers prediction-level knowledge, feature-level knowledge, and patch-level knowledge, with patch-level transfer divided into morphology distillation and rhythm distillation (Ni et al., 23 Sep 2025).

A further architectural implication is that patch structure is intrinsic to the method. For that reason, PPG-Distill is applied to GPT-PPG-1m rather than to the MLP baseline, because the MLP does not patchify the signal and therefore cannot support the proposed patch-level objectives. This suggests that, within this framework, compressibility is constrained not only by parameter count but also by whether the student retains a patchified sequence representation compatible with local and relational supervision (Ni et al., 23 Sep 2025).

2. Signal representation and teacher–student configuration

The framework models the raw PPG input as a one-dimensional signal

$X \in \mathbb{R}^{L},$

with regression targets represented as $Y \in \mathbb{R}^{1}$ and classification targets as $Y \in \mathbb{R}^{C}$. The input is divided into non-overlapping patches of length $P$, producing

$X_p \in \mathbb{R}^{P \times N}, \qquad N = L/P.$

The implementation specifies a patch size of $P=40$, and the paper’s figure illustrating StanfordAF examples explicitly depicts this patchification scheme (Ni et al., 23 Sep 2025).

Two PPG foundation models serve as teachers: GPT-PPG-19m and PaPaGei. The principal student is GPT-PPG-1m, while an MLP baseline is included for comparison under standard KD only. At the patch level, the student and teacher produce hidden representations

$$H_s^p \in \mathbb{R}^{N \times d_s}, \qquad H_t^p \in \mathbb{R}^{N \times d_t},$$

and at the whole-signal level they produce predictions $\hat{Y}_s$, $\hat{Y}_t$ and internal features $H_s$, $Y \in \mathbb{R}^{1}$0 (Ni et al., 23 Sep 2025).

The teacher is frozen during student training. Distillation therefore follows the conventional asymmetric teacher–student pattern, but the framework’s novelty lies in the granularity of transferred information. Position-wise patch correspondence is assumed: student patch $Y \in \mathbb{R}^{1}$1 aligns to teacher patch $Y \in \mathbb{R}^{1}$2. This is essential for the morphology loss and also anchors the later rhythm constraint, which acts on inter-patch relations rather than on individual patch identities alone (Ni et al., 23 Sep 2025).

3. Distillation objectives

The paper first presents a standard distillation objective,

$Y \in \mathbb{R}^{1}$3

where $Y \in \mathbb{R}^{1}$4 is the supervised task loss, $Y \in \mathbb{R}^{1}$5 is prediction-level distillation, and $Y \in \mathbb{R}^{1}$6 is feature-level distillation. It specifies MAE for regression and cross-entropy for classification as examples of $Y \in \mathbb{R}^{1}$7, but it does not provide explicit formulas for $Y \in \mathbb{R}^{1}$8, $Y \in \mathbb{R}^{1}$9, or $Y \in \mathbb{R}^{C}$0 (Ni et al., 23 Sep 2025).

The first PPG-specific extension is morphology distillation, which is defined over aligned patches. Because teacher and student patch dimensions may differ, a learnable adapter

$Y \in \mathbb{R}^{C}$1

is introduced, and the teacher patch representation is projected as

$Y \in \mathbb{R}^{C}$2

Patch vectors are then $Y \in \mathbb{R}^{C}$3-normalized row-wise, and the similarity matrix is

$Y \in \mathbb{R}^{C}$4

with temperature $Y \in \mathbb{R}^{C}$5. The morphology loss is an InfoNCE-style objective,

$Y \in \mathbb{R}^{C}$6

Its operational meaning is one-to-one patch alignment: student patch $Y \in \mathbb{R}^{C}$7 should align with teacher patch $Y \in \mathbb{R}^{C}$8 and not collapse onto other teacher patches. The paper explicitly states that “morphology” here denotes data-driven local waveform representations within patches rather than handcrafted clinical descriptors (Ni et al., 23 Sep 2025).

The second extension is rhythm distillation, which preserves inter-patch temporal structure by matching pairwise patch-distance matrices. The teacher and student distance matrices are defined as

$Y \in \mathbb{R}^{C}$9

and the loss is

$P$0

This loss acts on normalized relational geometry across all distinct patch pairs, and the paper interprets it as preserving beat-to-beat periodicity, timing regularity, and other inter-patch temporal structures. It also notes that the exact normalization producing $P$1 and the exact form of $P$2 are not fully specified in the text (Ni et al., 23 Sep 2025).

The final objective is

$P$3

which combines supervised task loss, standard KD, and the two patch-level losses. The implementation sets $P$4, and the appendix states that $P$5, $P$6, and $P$7 are searched within $P$8, although the sensitivity analysis later reports that $P$9 gave the best MAE in the reported DaLiA experiment (Ni et al., 23 Sep 2025).

4. Training procedure and evaluation setting

The evaluation follows GPT-PPG on two downstream tasks. Heart rate estimation is performed on DaLiA as a regression problem and reported with MSE and MAE. Atrial fibrillation detection is performed on StanfordAF as a classification problem and reported with Accuracy and F1. Teacher models are trained using their default configurations as reported in their original papers, while distillation experiments in this work are conducted on these two benchmarks (Ni et al., 23 Sep 2025).

Training uses PyTorch on one NVIDIA L40S GPU. The optimizer is Adam with an initial learning rate of $X_p \in \mathbb{R}^{P \times N}, \qquad N = L/P.$0 and a warmup-plus-cosine-annealing schedule, with $X_p \in \mathbb{R}^{P \times N}, \qquad N = L/P.$1, $X_p \in \mathbb{R}^{P \times N}, \qquad N = L/P.$2, and warmup ratio 25%. The batch size is 64, early stopping patience is 20 epochs, and the teacher remains frozen throughout distillation. These details indicate that the framework is implemented as downstream task adaptation rather than as standalone pretraining of a new foundation model (Ni et al., 23 Sep 2025).

The training logic also clarifies what the framework does not attempt. It does not report intermediate-layer matching across many depths, and the feature-level distillation appears to use a single whole-signal representation rather than a deeply supervised layer-by-layer alignment scheme. Likewise, the paper provides limited preprocessing detail beyond patchification; it does not specify window length $X_p \in \mathbb{R}^{P \times N}, \qquad N = L/P.$3, sampling rate, raw-signal normalization, filtering, or augmentation methods for the base PPG inputs in the manuscript excerpt summarized here (Ni et al., 23 Sep 2025).

5. Empirical performance and efficiency

On DaLiA with GPT-PPG-19m as teacher, the undistilled GPT-PPG-1m student achieves MSE 255.07 and MAE 10.08, standard KD improves this to MSE 234.16 and MAE 9.44, and PPG-Distill further improves it to MSE 215.36 and MAE 8.34. Relative to the undistilled GPT-PPG-1m baseline, that corresponds to +15.57% in MSE and +17.32% in MAE. The distilled student also surpasses the GPT-PPG-19m teacher itself on this benchmark, where the teacher reports MSE 221.78 and MAE 8.82 (Ni et al., 23 Sep 2025).

With PaPaGei as teacher on DaLiA, the same pattern holds. GPT-PPG-1m starts at MSE 255.07 and MAE 10.08, standard KD improves to 220.26 and 8.38, and PPG-Distill reaches MSE 202.31 and MAE 7.90. The relative gains over the undistilled student are +20.68% in MSE and +21.62% in MAE. The paper’s prose summarizes this as “up to a +13.7% relative MSE improvement on DaLiA,” but the tabulated values contain larger relative gains under some teacher settings; the table is therefore the more precise source (Ni et al., 23 Sep 2025).

On StanfordAF with GPT-PPG-19m as teacher, the undistilled GPT-PPG-1m student records Accuracy 0.81 and F1 0.64, standard KD records 0.82 and 0.65, and PPG-Distill records 0.87 and 0.77. With PaPaGei as teacher, the corresponding results are 0.81 and 0.64 for the baseline, 0.83 and 0.67 for KD, and 0.88 and 0.77 for PPG-Distill. The headline “up to 21.8%” refers to the relative F1 improvement on StanfordAF: +21.80% for GPT-PPG-19m teacher and +21.35% for PaPaGei teacher (Ni et al., 23 Sep 2025).

The efficiency results are central to the framework’s practical claim. On DaLiA, GPT-PPG-19m has 19,018,417 parameters and throughput 128.06 Batch/s, while GPT-PPG-1m has 1,017,197 parameters and throughput 291.50 Batch/s. On StanfordAF, GPT-PPG-19m has 19,034,290 parameters and throughput 39.19 Batch/s, while GPT-PPG-1m has 1,021,690 parameters and throughput 290.00 Batch/s. These figures support the statements that GPT-PPG-1m uses roughly 19× fewer parameters than GPT-PPG-19m and is up to about 7.4× faster on StanfordAF (Ni et al., 23 Sep 2025).

6. Interpretation, related directions, and limitations

PPG-Distill is a model-compression framework, but its technical identity is more specific than standard output-logit KD. The paper’s ablation summary indicates that $X_p \in \mathbb{R}^{P \times N}, \qquad N = L/P.$4 strongly influences performance, with small values helping and large values degrading performance; $X_p \in \mathbb{R}^{P \times N}, \qquad N = L/P.$5 is relatively stable; and $X_p \in \mathbb{R}^{P \times N}, \qquad N = L/P.$6 is non-monotonic, with $X_p \in \mathbb{R}^{P \times N}, \qquad N = L/P.$7 yielding the best reported DaLiA MAE in the sensitivity study. The paper also states that an ablation study exists for the importance of patch-level transfer as a whole, but the provided content does not report exact isolated performance drops for removing morphology distillation alone or rhythm distillation alone (Ni et al., 23 Sep 2025).

In the broader PPG literature, PPG-Distill occupies a distinct position. “Tri-Spectral PPG” learns to fuse green, red, and infrared reflective PPG into a single recovered waveform for downstream use, which is a form of waveform-level signal distillation but not foundation-model compression (Meier et al., 2024). “CAP” uses patient-level EHR supervision to align PPG representations with clinical semantics through cross-modal contrastive learning, which is closer to semantic distillation into a universal encoder than to lightweight student deployment (He et al., 13 Jun 2026). “Pulse-PPG” introduces a field-trained open PPG foundation model and does not study distillation directly, but its results suggest that a field-trained teacher can be highly transferable across wearable and clinical tasks (Saha et al., 3 Feb 2025).

The limitations stated for PPG-Distill are correspondingly specific. The conclusion emphasizes the need to extend the framework to more tasks and datasets, perform deeper analysis, and explore more diverse teacher models. The manuscript excerpt summarized here also does not report cross-device transfer, explicit robustness studies under severe noise or motion, or cross-domain evaluations beyond DaLiA and StanfordAF. A plausible implication is that PPG-Distill establishes patch-level morphology and rhythm transfer as a useful compression principle for PPG foundation models, but leaves open the question of how broadly that principle generalizes across sensing conditions, devices, and physiological tasks (Ni et al., 23 Sep 2025).