---
title: 'PPG-Distill: Efficient PPG Signal Distillation'
url: https://www.emergentmind.com/topics/ppg-distill
type: topic
---

# PPG-Distill: Efficient PPG Signal Distillation

PPG-Distill is a knowledge distillation framework for efficient photoplethysmography signal analysis that transfers both global and local knowledge from large PPG foundation models to lightweight students through prediction-, feature-, and patch-level distillation. Its central claim is that standard output- and feature-level distillation is insufficient for PPG because PPG carries both local waveform morphology and longer-range rhythm structure, and that these properties should be distilled explicitly. The framework is evaluated on heart rate estimation on DaLiA and atrial fibrillation detection on StanfordAF, where it improves the lightweight GPT-PPG-1m student by up to 21.8% while achieving up to 7× faster inference and about 19× fewer parameters than GPT-PPG-19m [2509.19215].

## 1. Conceptual scope and motivation

PPG-Distill is situated within the problem of compressing large PPG foundation models for deployment on resource-limited wearable and edge devices. The framework is motivated by the observation that models such as GPT-PPG-19m and PaPaGei can learn strong general PPG representations but remain difficult to deploy under memory and latency constraints. The stated objective is therefore not merely model compression in the abstract, but efficient PPG analysis on wearables while preserving the signal characteristics that matter most for downstream physiological tasks [2509.19215].

The framework distinguishes two kinds of signal structure as especially important. The first is local waveform morphology, understood as short-window waveform shapes associated with cardiovascular events. The second is longer-range rhythm or temporal structure, including periodicity, beat-to-beat timing regularity, and autonomic dynamics. This motivates a distillation design that extends beyond conventional prediction matching and whole-signal feature matching. In the paper’s formulation, PPG-Distill transfers prediction-level knowledge, feature-level knowledge, and patch-level knowledge, with patch-level transfer divided into morphology distillation and rhythm distillation [2509.19215].

A further architectural implication is that patch structure is intrinsic to the method. For that reason, PPG-Distill is applied to GPT-PPG-1m rather than to the MLP baseline, because the MLP does not patchify the signal and therefore cannot support the proposed patch-level objectives. This suggests that, within this framework, compressibility is constrained not only by parameter count but also by whether the student retains a patchified sequence representation compatible with local and relational supervision [2509.19215].

## 2. Signal representation and teacher–student configuration

The framework models the raw PPG input as a one-dimensional signal
$$
X \in \mathbb{R}^{L},
$$
with regression targets represented as $Y \in \mathbb{R}^{1}$ and classification targets as $Y \in \mathbb{R}^{C}$. The input is divided into non-overlapping patches of length $P$, producing
$$
X_p \in \mathbb{R}^{P \times N}, \qquad N = L/P.
$$
The implementation specifies a patch size of $P=40$, and the paper’s figure illustrating StanfordAF examples explicitly depicts this patchification scheme [2509.19215].

Two PPG foundation models serve as teachers: GPT-PPG-19m and PaPaGei. The principal student is GPT-PPG-1m, while an MLP baseline is included for comparison under standard KD only. At the patch level, the student and teacher produce hidden representations
$$
H_s^p \in \mathbb{R}^{N \times d_s}, \qquad H_t^p \in \mathbb{R}^{N \times d_t},
$$
and at the whole-signal level they produce predictions $\hat{Y}_s$, $\hat{Y}_t$ and internal features $H_s$, $H_t$ [2509.19215].

The teacher is frozen during student training. Distillation therefore follows the conventional asymmetric teacher–student pattern, but the framework’s novelty lies in the granularity of transferred information. Position-wise patch correspondence is assumed: student patch $i$ aligns to teacher patch $i$. This is essential for the morphology loss and also anchors the later rhythm constraint, which acts on inter-patch relations rather than on individual patch identities alone [2509.19215].

## 3. Distillation objectives

The paper first presents a standard distillation objective,
$$
\min_{\theta_s} \mathcal{L}_{sup}(Y, \hat{Y}_s) + \mathcal{L}_{KD}^Y(\hat{Y}_t, \hat{Y}_s) + \mathcal{L}_{KD}^H(H_t, H_s),
$$
where $\mathcal{L}_{sup}$ is the supervised task loss, $\mathcal{L}_{KD}^Y$ is prediction-level distillation, and $\mathcal{L}_{KD}^H$ is feature-level distillation. It specifies MAE for regression and cross-entropy for classification as examples of $\mathcal{L}_{sup}$, but it does not provide explicit formulas for $\mathcal{L}_{sup}$, $\mathcal{L}_{KD}^Y$, or $\mathcal{L}_{KD}^H$ [2509.19215].

The first PPG-specific extension is morphology distillation, which is defined over aligned patches. Because teacher and student patch dimensions may differ, a learnable adapter
$$
A \in \mathbb{R}^{d_t \times d_s}
$$
is introduced, and the teacher patch representation is projected as
$$
\tilde H_t^p = H_t^p A.
$$
Patch vectors are then $\ell_2$-normalized row-wise, and the similarity matrix is
$$
Z=\frac{\hat H_s^p(\hat H_t^p)^\top}{\tau}\in\mathbb{R}^{N\times N},
$$
with temperature $\tau$. The morphology loss is an InfoNCE-style objective,
$$
\mathcal{L}_{mor} =\frac{1}{N}\sum_{i=1}^{N} \left(-\log\frac{\exp(Z_{ii})}{\sum_{j=1}^{N}\exp(Z_{ij})}\right).
$$
Its operational meaning is one-to-one patch alignment: student patch $i$ should align with teacher patch $i$ and not collapse onto other teacher patches. The paper explicitly states that “morphology” here denotes data-driven local waveform representations within patches rather than handcrafted clinical descriptors [2509.19215].

The second extension is rhythm distillation, which preserves inter-patch temporal structure by matching pairwise patch-distance matrices. The teacher and student distance matrices are defined as
$$
[D_t]_{ij}=\big\| \phi(H_{t,i}^p)-\phi(H_{t,j}^p)\big\|_2, \qquad [D_s]_{ij}=\big\| H_{s,i}^p-H_{s,j}^p\big\|_2,
$$
and the loss is
$$
\mathcal{L}_{rhy} =\frac{1}{N(N-1)}\sum_{i\neq j} \operatorname{smoothL1}\!\left( [\tilde D_s]_{ij},\ [\tilde D_t]_{ij}\right).
$$
This loss acts on normalized relational geometry across all distinct patch pairs, and the paper interprets it as preserving beat-to-beat periodicity, timing regularity, and other inter-patch temporal structures. It also notes that the exact normalization producing $\tilde D_s,\tilde D_t$ and the exact form of $\phi(\cdot)$ are not fully specified in the text [2509.19215].

The final objective is
$$
\mathcal{L} = \mathcal{L}_{sup} + \alpha \mathcal{L}_{KD}^{Y} + \beta \mathcal{L}_{KD}^{H} + \gamma (\mathcal{L}_{mor} + \mathcal{L}_{rhy}),
$$
which combines supervised task loss, standard KD, and the two patch-level losses. The implementation sets $\tau=2$, and the appendix states that $\alpha$, $\beta$, and $\gamma$ are searched within $\{0.1, 0.5\}$, although the sensitivity analysis later reports that $\gamma=1$ gave the best MAE in the reported DaLiA experiment [2509.19215].

## 4. Training procedure and evaluation setting

The evaluation follows GPT-PPG on two downstream tasks. Heart rate estimation is performed on DaLiA as a regression problem and reported with MSE and MAE. Atrial fibrillation detection is performed on StanfordAF as a classification problem and reported with Accuracy and F1. Teacher models are trained using their default configurations as reported in their original papers, while distillation experiments in this work are conducted on these two benchmarks [2509.19215].

Training uses PyTorch on one NVIDIA L40S GPU. The optimizer is Adam with an initial learning rate of $1\times 10^{-5}$ and a warmup-plus-cosine-annealing schedule, with $\text{lr}_{\max}=1\times 10^{-3}$, $\eta_{\min}=1\times 10^{-6}$, and warmup ratio 25%. The batch size is 64, early stopping patience is 20 epochs, and the teacher remains frozen throughout distillation. These details indicate that the framework is implemented as downstream task adaptation rather than as standalone pretraining of a new foundation model [2509.19215].

The training logic also clarifies what the framework does not attempt. It does not report intermediate-layer matching across many depths, and the feature-level distillation appears to use a single whole-signal representation rather than a deeply supervised layer-by-layer alignment scheme. Likewise, the paper provides limited preprocessing detail beyond patchification; it does not specify window length $L$, sampling rate, raw-signal normalization, filtering, or augmentation methods for the base PPG inputs in the manuscript excerpt summarized here [2509.19215].

## 5. Empirical performance and efficiency

On DaLiA with GPT-PPG-19m as teacher, the undistilled GPT-PPG-1m student achieves MSE 255.07 and MAE 10.08, standard KD improves this to MSE 234.16 and MAE 9.44, and PPG-Distill further improves it to MSE 215.36 and MAE 8.34. Relative to the undistilled GPT-PPG-1m baseline, that corresponds to +15.57% in MSE and +17.32% in MAE. The distilled student also surpasses the GPT-PPG-19m teacher itself on this benchmark, where the teacher reports MSE 221.78 and MAE 8.82 [2509.19215].

With PaPaGei as teacher on DaLiA, the same pattern holds. GPT-PPG-1m starts at MSE 255.07 and MAE 10.08, standard KD improves to 220.26 and 8.38, and PPG-Distill reaches MSE 202.31 and MAE 7.90. The relative gains over the undistilled student are +20.68% in MSE and +21.62% in MAE. The paper’s prose summarizes this as “up to a +13.7% relative MSE improvement on DaLiA,” but the tabulated values contain larger relative gains under some teacher settings; the table is therefore the more precise source [2509.19215].

On StanfordAF with GPT-PPG-19m as teacher, the undistilled GPT-PPG-1m student records Accuracy 0.81 and F1 0.64, standard KD records 0.82 and 0.65, and PPG-Distill records 0.87 and 0.77. With PaPaGei as teacher, the corresponding results are 0.81 and 0.64 for the baseline, 0.83 and 0.67 for KD, and 0.88 and 0.77 for PPG-Distill. The headline “up to 21.8%” refers to the relative F1 improvement on StanfordAF: +21.80% for GPT-PPG-19m teacher and +21.35% for PaPaGei teacher [2509.19215].

The efficiency results are central to the framework’s practical claim. On DaLiA, GPT-PPG-19m has 19,018,417 parameters and throughput 128.06 Batch/s, while GPT-PPG-1m has 1,017,197 parameters and throughput 291.50 Batch/s. On StanfordAF, GPT-PPG-19m has 19,034,290 parameters and throughput 39.19 Batch/s, while GPT-PPG-1m has 1,021,690 parameters and throughput 290.00 Batch/s. These figures support the statements that GPT-PPG-1m uses roughly 19× fewer parameters than GPT-PPG-19m and is up to about 7.4× faster on StanfordAF [2509.19215].

## 6. Interpretation, related directions, and limitations

PPG-Distill is a model-compression framework, but its technical identity is more specific than standard output-logit KD. The paper’s ablation summary indicates that $\alpha$ strongly influences performance, with small values helping and large values degrading performance; $\beta$ is relatively stable; and $\gamma$ is non-monotonic, with $\gamma=1$ yielding the best reported DaLiA MAE in the sensitivity study. The paper also states that an ablation study exists for the importance of patch-level transfer as a whole, but the provided content does not report exact isolated performance drops for removing morphology distillation alone or rhythm distillation alone [2509.19215].

In the broader PPG literature, PPG-Distill occupies a distinct position. “Tri-Spectral PPG” learns to fuse green, red, and infrared reflective PPG into a single recovered waveform for downstream use, which is a form of waveform-level signal distillation but not foundation-model compression [2412.17549]. “CAP” uses patient-level EHR supervision to align PPG representations with clinical semantics through cross-modal contrastive learning, which is closer to semantic distillation into a universal encoder than to lightweight student deployment [2606.15284]. “Pulse-PPG” introduces a field-trained open PPG foundation model and does not study distillation directly, but its results suggest that a field-trained teacher can be highly transferable across wearable and clinical tasks [2502.01108].

The limitations stated for PPG-Distill are correspondingly specific. The conclusion emphasizes the need to extend the framework to more tasks and datasets, perform deeper analysis, and explore more diverse teacher models. The manuscript excerpt summarized here also does not report cross-device transfer, explicit robustness studies under severe noise or motion, or cross-domain evaluations beyond DaLiA and StanfordAF. A plausible implication is that PPG-Distill establishes patch-level morphology and rhythm transfer as a useful compression principle for PPG foundation models, but leaves open the question of how broadly that principle generalizes across sensing conditions, devices, and physiological tasks [2509.19215].

Source: https://www.emergentmind.com/topics/ppg-distill