Knowledge-Preserving Watermark Learning

• KPWL is a watermarking strategy that preserves model knowledge by freezing key layers and using teacher-student adaptation to mitigate domain shifts.
• The method employs symmetric knowledge distillation and L2-SP regularization to retain clean-domain performance while introducing detectable watermarks.
• Empirical results in audio anti-spoofing show KPWL reduces Equal Error Rates on watermarked data compared to baselines, though challenges remain with unseen watermark shifts.

Knowledge-Preserving Watermark Learning (KPWL) denotes watermarking or watermark-aware adaptation strategies that introduce a verifiable ownership signal while preserving the protected system’s original utility, representational capacity, or clean-domain performance. In the supplied literature, the term appears both as a general description of watermark learning that retains model knowledge and as the explicit name of a framework for adapting audio anti-spoofing models to watermark-induced domain shifts without sacrificing original-domain spoofing detection capability (Zhang et al., 25 Sep 2025).

1. Scope and problem setting

KPWL arises where watermarking and utility retention are in direct tension. In audio anti-spoofing, the central observation is that widely used audio watermarking introduces structured perturbations that constitute a significant but previously overlooked domain shift, and that watermarking consistently degrades anti-spoofing performance, with higher watermark density correlating with higher Equal Error Rates (EERs) (Zhang et al., 25 Sep 2025). In self-supervised pre-trained encoders, the analogous difficulty is that the lack of a priori knowledge of downstream tasks makes it non-trivial to protect the intellectual property of the pre-trained encoder by applying conventional watermarking methods (Zhang et al., 2022). SSL encoder watermarking work further frames the problem in terms of fidelity, effectiveness, and robustness for feature extractors that will later be reused across unknown downstream tasks (Cong et al., 2022).

A broader reading of the literature suggests that KPWL is not restricted to one modality. In LLM watermarking, preservation is expressed as keeping normal utility, task performance, fluency, or knowledge transfer efficiency while making outputs or model behavior detectably watermarked (Xu et al., 2024, Pan et al., 17 Feb 2025). In image watermarking, preservation can instead refer to retaining the watermark message under generative purification by learning a Minimal Sufficient Statistic of the message rather than overfitting to fragile cover details (He et al., 25 Feb 2026). This suggests a common design objective: encode a watermark signal in a subspace or behavior that remains verifiable without materially compromising the protected model’s primary function.

2. The named KPWL framework for audio anti-spoofing

In the anti-spoofing setting, KPWL is a two-phase adaptation strategy. The model is first pretrained in a supervised manner using only clean, unwatermarked data. Adaptation is then performed on watermarked data while freezing the SSL front-end and the final classifier layer, and only updating the intermediate backend layers (Zhang et al., 25 Sep 2025).

The paper gives three explicit motivations for this update pattern. It is intended to preserve the low-level feature extractor, preserve the decision boundary for spoof detection, and enable domain adaptation with minimal parameter drift. To prevent catastrophic forgetting of clean-domain knowledge during adaptation, KPWL combines teacher–student adaptation, symmetric knowledge distillation, and L2-SP anchoring. The complete training objective is

$$\mathcal{L} = \mathcal{L}_{\mathrm{task}} + \beta \mathcal{L}_{\mathrm{KD}} + \mu \mathcal{L}_{\mathrm{L2\text{-}SP}}$$

where $\mathcal{L}_{\mathrm{task}}$ is class-weighted supervised cross-entropy loss for the current domain,

$$\mathcal{L}_{\mathrm{KD}} = D_{\mathrm{KL}}(p_T \| p_S) + D_{\mathrm{KL}}(p_S \| p_T),$$

and

$$\mathcal{L}_{\mathrm{L2\text{-}SP}} = \sum_{i \in \mathcal{I}_{\mathrm{trainable}}} \| w_i - w_i^{(0)} \|_2^2.$$

Within this formulation, symmetric knowledge distillation constrains the adapted student model to stay close to the frozen teacher model’s predictions, while L2-SP regularization keeps trainable parameters close to their pre-adaptation values. The intended result is adaptation to watermark perturbations without loss of clean-domain spoofing detection capability (Zhang et al., 25 Sep 2025).

3. Training protocol and evaluation regime

The reported implementation uses audio resampled to 16kHz and normalized to a fixed length. RawBoost augmentation, including colored noise and random SNRs, is applied for robustness. Baseline pretraining uses Adam with learning rate $1 \times 10^{-7}$, cross-entropy loss, and up to 50 epochs. KPWL adaptation uses Adam with learning rate $5 \times 10^{-7}$, only intermediate layers trainable, and 2 epochs. The loss-balancing hyperparameters are $\beta = 0.3$ and $\mu = 10^{-4}$ (Zhang et al., 25 Sep 2025).

Phase	Trainable components	Reported settings
Baseline pretraining	Full anti-spoofing model	Adam, $1 \times 10^{-7}$, up to 50 epochs
KPWL adaptation	Only intermediate backend layers	Adam, $5 \times 10^{-7}$, 2 epochs

Evaluation is organized around Equal Error Rate (EER) and uses clean and watermarked versions of LA21 (ASVspoof 2021 LA), DF21 (ASVspoof 2021 DF), and ITW (In-The-Wild). Two regimes are distinguished. In Watermark-Spoofing Seen Evaluation, the watermark types used in training and testing match. In Watermark-Spoofing Unseen Evaluation, testing uses different watermarking methods. This separation is important because it distinguishes adaptation to known watermark distributions from robustness to novel watermark-induced shifts (Zhang et al., 25 Sep 2025).

4. Empirical behavior under watermark-induced domain shift

The central empirical finding is that watermarking increases EER for all evaluated anti-spoofing models, and that directly training on watermarked data can improve robustness to watermarked samples while degrading clean-domain performance. KPWL is proposed precisely to avoid that trade-off by combining domain adaptation with knowledge preservation (Zhang et al., 25 Sep 2025).

The reported comparison at 75% watermarking is:

Model	LA21 (75%)	ITW (75%)	DF21 (75%)
Baseline	3.68	8.46	2.23
Watermarked	3.28	9.03	2.13
KPWL	3.21	7.92	2.04

For clean LA21, the same comparison reports 3.02 for the baseline, 3.17 for the watermarked model, and 3.06 for KPWL. These values are used in the paper to support two claims: KPWL consistently yields lower EER on watermarked data than the baseline, and it maintains nearly original performance on clean audio (Zhang et al., 25 Sep 2025).

The paper also reports an important limitation. Under Watermark-Spoofing Unseen Evaluation, all models, including KPWL, exhibited higher EERs, and the baseline sometimes outperformed adapted variants. The stated interpretation is that KPWL is effective on known watermark types but may still struggle under truly novel watermark-induced shifts. A plausible implication is that KPWL, in this instantiation, is best understood as controlled domain adaptation rather than a complete solution to watermark distribution shift.

5. Related knowledge-preserving watermark strategies across domains

The broader literature shows several distinct implementations of the same preservation objective.

Method	Preservation mechanism	Verification or robustness signal
AWEncoder	Contrastive loss plus watermark loss	White-box and black-box verification
SSLGuard	Embedding matching and shadow training	Watermark Rate (WR)
RL LLM watermarking	KL regularization in PPO co-training	AUC, FPR@90/99, logPPL
FunctionMarker	Minimal dataset injection of watermarked knowledge	Black-box extraction from responses
WaterVIB	Variational Information Bottleneck	Zero-shot resilience against AIGC purification

For pre-trained encoders in contrastive learning, AWEncoder generates a universal adversarial perturbation so that perturbed samples cluster around the embedding of a randomly selected key image, then fine-tunes the encoder with a joint loss $\mathcal{L}_{\mathrm{task}}$0. The contrastive term preserves representational capacity, and the method supports both white-box and black-box ownership verification. The reported SimCLR/ImageNet downstream accuracy changes from 74.5% to 71.1% after watermarking, while robustness remains reliable under retraining or pruning up to 80% parameters (Zhang et al., 2022).

SSLGuard, described as the first watermarking scheme for pre-trained encoders, is a data-free, black-box watermarking scheme for SSL image encoders. Its key mechanisms are a key-tuple, correlation and uncorrelated losses, embedding matching loss, and a shadow encoder that simulates model stealing. Knowledge preservation is explicit in the embedding matching objective, and downstream accuracy drops are reported as negligible, commonly <1%, with example changes of 0.783 to 0.781 on STL-10 and 0.766 to 0.765 on CIFAR-10. Watermark Rate is close to 1 for the watermarked encoder and 0 for unrelated encoders (Cong et al., 2022).

In LLMs, one line of work co-trains a watermarked LLM and a paired detector through reinforcement learning. The detector acts as a learned reward model, while KL-divergence regularization penalizes deviation from the original LLM distribution. The reported result is AUC ~0.999, FPR@90 ≈ 0%, and logPPL indistinguishable from non-watermarked LLM, with low extra cost if combined with alignment because the additional overhead is only training an extra reward model (Xu et al., 2024). Another line uses knowledge injection rather than output-distribution shaping. FunctionMarker embeds watermarks into selected knowledge, specifically customizable mathematical functions whose coefficients encode the watermark, then injects the watermarked knowledge into the training data. The watermark ratio can be as low as 0.1–0.5%, extraction accuracy is reported as close to 100%, and at a ratio of 0.3% watermark extraction accuracy reached ≥90%, with almost no effect on model loss during fine-tuning (Li et al., 2023).

In image watermarking against regeneration-based AIGC attacks, WaterVIB reformulates the encoder as an information sieve via the Variational Information Bottleneck, forcing it to learn a Minimal Sufficient Statistic of the message. The stated goal is to retain only the essential signal invariant to regeneration. Reported results include BER reductions such as SD-Inpainting 0.35 to 0.03, SDXL-Refiner 0.30 to 0.03, and a Gradient Interference Ratio reduction from 0.4285 to 0.1167 (He et al., 25 Feb 2026).

6. Misconceptions, attack surface, and open questions

A common misconception is that training directly on watermarked data is sufficient for watermark robustness. In the anti-spoofing study, direct training on watermarked data sometimes improves robustness to watermarked samples but degrades clean-domain performance. KPWL is motivated precisely by the failure of that naive approach to maintain generalization across both domains (Zhang et al., 25 Sep 2025).

A second misconception is that watermark inheritance alone can robustly prevent unauthorized knowledge distillation. Work on LLM watermark radioactivity reports that student models trained on outputs of watermarked teachers do inherit strong watermark signals in the absence of attack, but also shows that targeted paraphrasing (TP) and inference-time watermark neutralization (WN) thoroughly eliminate inherited watermarks. WN does so while maintaining knowledge transfer efficiency and low computational overhead, with performance drop <5% and often marginal improvement, and multi-source knowledge distillation weakens watermark detectability even without explicit attack (Pan et al., 17 Feb 2025).

These results define a practical boundary for KPWL. Preservation alone does not guarantee security against adaptive attackers, and robustness on seen watermark types does not imply robustness to unseen watermark-induced shifts. The literature therefore points toward two simultaneous requirements: preserve the protected model’s original competence, and ensure that the watermark remains detectable under paraphrasing, regeneration, pruning, fine-tuning, or distributional change. The anti-spoofing KPWL framework establishes a benchmark for watermark-resilient anti-spoofing systems, but the handling of completely unseen watermark types remains an open research problem (Zhang et al., 25 Sep 2025).