---
title: 'PKDMark: Lightweight Deep Speech Watermarking'
url: https://www.emergentmind.com/topics/pkdmark
type: topic
---

# PKDMark: Lightweight Deep Speech Watermarking

Searching arXiv for the target paper and closely related speech watermarking baselines.
PKDMark is a lightweight deep learning-based speech watermarking method for speech synthesis that uses progressive knowledge distillation (PKD) to combine the robustness typically associated with deep learning-based watermarking and the efficiency more commonly associated with DSP-based methods. It is proposed as a response to unauthorized voice cloning, which is described as creating significant privacy and security risks, and it is designed for source tracing and misuse prevention. The method follows a two-stage strategy: training a high-performance teacher model with an invertible neural network (INN)-based architecture, then transferring its capabilities to a compact student model through progressive knowledge distillation. The reported outcome is a 93.6% reduction in computational costs while maintaining high robust performance and imperceptibility, including an average detection F1 score of 99.6% and a PESQ of 4.30 in advanced distortions [2509.19812].

## 1. Position within speech watermarking

PKDMark is situated within a taxonomy in which current watermarking technologies are divided into two broad categories: DSP-based methods and deep learning-based methods. In that framing, DSP-based methods are efficient but vulnerable to attacks, whereas deep learning-based methods provide robust protection at significantly higher computational cost. PKDMark is explicitly presented as a lightweight deep learning-based method intended to improve computational efficiency while enhancing robustness [2509.19812].

The method is specifically targeted at speech synthesis and voice cloning defenses. Its stated application setting is real-time speech synthesis, and its reported efficiency figures are tied to that goal. A central claim of the work is therefore not only that watermarking can be made robust, but that it can be made sufficiently efficient for deployment in synthesis pipelines without abandoning a learned watermarking formulation.

A common misunderstanding in this area is that efficiency and deep learning-based robustness are mutually exclusive design choices. PKDMark does not remove that trade-off in a universal sense, but it is presented as an instance in which a compact distilled model preserves the robust behavior of a larger model to a substantial degree. This suggests that model compression, if aligned with the watermarking objective rather than treated as a post hoc simplification step, can alter the conventional efficiency-robustness boundary.

## 2. End-to-end embedding and extraction pipeline

PKDMark operates in an encoder–attack–decoder framework. The input host waveform $x(t)$ is first converted by STFT into a complex spectrogram $X \in \mathbb{C}^{T \times F}$. A $K$-bit binary message $m \in \{0,1\}^K$ is passed through an embedding layer and a two-layer MLP, averaged to a real vector $h \in \mathbb{R}^F$, repeated along the time axis, and cast into the complex domain using the same vector for both real and imaginary parts:

$$
M = \mathrm{Repeat}_T(h + j \cdot h),
$$

where $\mathrm{Repeat}_T(\cdot)$ replicates the $F$-dimensional vector $h$ a total of $T$ times [2509.19812].

The encoder stage concatenates $X$ and $M$ channel-wise in the complex domain and feeds the result through a stack of invertible-network blocks, yielding the watermarked spectrogram $X_w$. Inverse STFT (ISTFT) then produces the watermarked waveform $x_w(t)$. To enforce robustness, the waveform is passed through an attack layer comprising a mixture of 14 distortions: nine “basic” distortions and five “advanced” distortions. The listed distortions are ND, Gaussian noise, median filter, MP3@64 kbps, low-pass, downsample, quantize, amplitude scale, echo, slow speed, fast speed, reverb with $\mathrm{RT60}=200\,\mathrm{ms}$, and $\pm 10\%$ pitch.

On the decoding side, the attacked waveform $x_a(t)$ is mapped back to the STFT domain, $\hat{Y}=\mathrm{STFT}(x_a)$, and passed through the same INN in reverse. A reduction over time and complex channels yields a feature vector, after which a two-layer MLP reconstructs the estimated message $\hat{h}(m)$. During teacher training only, an adversarial discriminator $d(\cdot)$ attempts to distinguish $x_w(t)$ from the original waveform $x(t)$, with the stated purpose of improving imperceptibility.

## 3. Teacher model architecture and optimization

The teacher encoder and decoder share parameters in an invertible neural network with $N=8$ invertible blocks. Each block follows a coupling-layer design described as being in the style of standard normalizing flows and WavMark, and internally uses a 5-layer ResNet with 32 channels per layer. The ResNets implementing the scale and translation transforms operate on each frequency/time slice in the complex domain while treating real and imaginary components as two channels [2509.19812].

A single INN block is abstracted, for real inputs $u$ split into $(u_1,u_2)$, as

$$
y_1 = u_1
$$

$$
y_2 = u_2 \odot \exp(s(u_1)) + t(u_1)
$$

$$
(u_1',u_2') = (y_1,y_2),
$$

and the inverse uses

$$
y_1,u_2 = (y_2-t(y_1)) \odot \exp(-s(y_1)).
$$

Here $s(\cdot)$ and $t(\cdot)$ are the small ResNets used inside the coupling transform.

Teacher training is end-to-end and multi-objective. The perceptual loss is a multi-scale STFT magnitude loss:

$$
L_{\mathrm{per}} = \sum_{i \in \{5 \ldots 11\}} \alpha_i \left\| \mathrm{STFT}_i[x(t)] - \mathrm{STFT}_i[x_w(t)] \right\|_2,
$$

with window sizes $2^i$ and hop $2^{i/4}$. The decoding loss is

$$
L_{\mathrm{dec}} = \| m - \hat{h}(m) \|_2 + \alpha \cdot \| M - \hat{h}(M) \|_2,
$$

where the first term is MSE on recovered bits and the second aligns the teacher’s internal feature maps. The adversarial terms are

$$
L_{\mathrm{dis}} = -[\log d(x) + \log(1-d(x_w))]
$$

for the discriminator and

$$
L_{\mathrm{adv}} = -\log(1-d(x_w))
$$

for the generator. The total teacher loss is

$$
L_{\mathrm{total}} = \lambda_{\mathrm{per}} L_{\mathrm{per}} + \lambda_{\mathrm{dec}} L_{\mathrm{dec}} + \lambda_{\mathrm{adv}} L_{\mathrm{adv}}.
$$

The architecture is therefore not only invertible but jointly optimized for transparency, recovery fidelity, and adversarial imperceptibility.

## 4. Progressive knowledge distillation and the compact student

The defining mechanism of PKDMark is progressive knowledge distillation rather than direct output supervision. The work states that direct KD (DKD), in which a small student is simply supervised by the teacher’s outputs, was found insufficient. Instead, the method introduces a mixing factor $\lambda(n)$ that grows from $0$ to $1$ over training steps $n$, thereby gradually shifting responsibility from the frozen teacher to the student [2509.19812].

At training step $n$, the student encoder produces $x_w^S(t)$, the frozen teacher encoder produces $x_w^T(t)$, and the mixed waveform is

$$
x_{\mathrm{com}}(t) = \lambda(n)\cdot x_w^S(t) + [1-\lambda(n)]\cdot x_w^T(t).
$$

This combined signal is fed into the frozen teacher decoder and discriminator. The training dynamics are described in three phases. In the early stage, when $\lambda \approx 0$, gradients flow almost entirely through the teacher branch, providing stable, strong-robustness guidance. In the mid stage, the student must reconcile its own representations with teacher outputs. In the late stage, when $\lambda=1$, the student is fully responsible for embedding and is finetuned with the teacher decoder.

The student loss mirrors the teacher formulation but uses teacher outputs as the reference for perceptual alignment and teacher-side components for decoding and adversarial evaluation:

$$
L_{\mathrm{per}}^S = \sum_i \alpha_i \left\| \mathrm{STFT}_i[x_w^S(t)] - \mathrm{STFT}_i[x_w^T(t)] \right\|_2
$$

$$
L_{\mathrm{dec}}^S = \| m - \hat{h}_S(m) \|_2
$$

$$
L_{\mathrm{adv}}^S = -\log(1-d(x_w^S))
$$

$$
L_{\mathrm{total}}^S = \lambda_{\mathrm{per}} L_{\mathrm{per}}^S + \lambda_{\mathrm{dec}} L_{\mathrm{dec}}^S + \lambda_{\mathrm{adv}} L_{\mathrm{adv}}^S.
$$

The compact student uses a shallower and narrower INN while retaining the same ResNet depth.

| Component | Teacher | Student |
|---|---:|---:|
| INN depth | 8 blocks | 2 blocks |
| Channels per ResNet | 32 | 16 |
| ResNet depth | 5 layers | 5 layers |
| GFLOPS | 36 | 2.3 |
| NVIDIA T4 RTF | 0.039 s/speech | 0.006 s/speech |
| Intel i9-13900 CPU RTF | 0.155 | 0.020 |

These values are reported as a 93.6% reduction in GFLOPS, approximately 84.7% faster execution on NVIDIA T4, and approximately 87.1% faster execution on Intel i9-13900 CPU. A recurrent misconception is that distillation in watermarking is merely a compression step applied after training; in PKDMark, distillation is itself part of the robustness-preserving training design.

## 5. Experimental protocol and reported performance

The experiments use 1,000 hours of 24 kHz audio drawn from four sources: 400 hours of CommonVoice (wild), 300 hours of LibriTTS (clean), 200 hours of ASVspoof21 DF, and 100 hours of high-quality TTS (Azure). The split is 900 hours for training, 50 hours for validation, and 50 hours for testing [2509.19812].

Imperceptibility is reported through PESQ, CMOS, and SNR. The teacher obtains PESQ 4.33, while the student obtains PESQ 4.30. The student’s CMOS relative to original audio is $-0.04$, described as nearly inaudible. The SNR values are 45.75 dB for the teacher and 45.54 dB for the student. Within the paper’s framing, these values support the claim that the distilled student remains transparent despite the large reduction in compute.

Robustness is evaluated at 16 bps capacity using average BER across all distortions. The reported BER values are approximately 8.58% for WavMark-32 bps, approximately 2.43% for WavMark-16 bps, approximately 18.66% for AudioSeal, approximately 0.56% for the teacher, and approximately 0.51% for the PKDMark student. AudioSeal is specifically noted as failing on advanced attacks. For the detection task, using an 8-bit sync code and measuring F1 score over all distortions, the reported value is 99.6%.

The discussion summarizes these outcomes by stating that PKDMark retains sub-1% BER under both basic and advanced distortions, matches or exceeds the full-size teacher, and far outperforms DSP or earlier deep methods. It also states that the method is suitable for on-device, real-time TTS embedding because the CPU real-time factor is at or below 0.02.

## 6. Interpretation, scope, and relation to broader deployment questions

PKDMark is best understood as an attempt to make robust learned speech watermarking operational in real-time synthesis settings rather than as a general statement about all watermarking regimes. Its principal empirical claims are tied to a particular evaluation configuration: 24 kHz audio, a 1,000-hour composite dataset, 14 distortions, 16 bps capacity for BER evaluation, and an 8-bit sync code for the detection task [2509.19812].

This qualification is important because one possible overreading of the reported results would be to treat them as proving that compact models are inherently more robust than larger ones. The paper does not make that general claim. Instead, it reports that the distilled student achieves approximately 0.51% BER, slightly better than the teacher’s approximately 0.56%, within the stated evaluation setup. A plausible implication is that the PKD schedule does more than compress the model: it can regularize the student toward the teacher’s robustness profile while preserving imperceptibility.

The work also frames its contribution as extending beyond a single architecture. It states that the same PKD framework generalizes to larger or different INN architectures for post-hoc watermarking beyond speech synthesis. That statement places PKDMark at the intersection of robust audio watermarking, invertible modeling, and deployment-oriented model compression. In that sense, PKDMark is not only a watermarking algorithm but also a specific recipe for transferring an INN-based watermarking teacher into a compact student without relying on the simpler direct KD strategy that the paper found insufficient.

Source: https://www.emergentmind.com/topics/pkdmark