Watermark-Spoofing Dataset

• Watermark-Spoofing dataset is a benchmark that augments audio anti-spoofing corpora with watermarks to simulate controlled domain shifts.
• It evaluates the effects of both handcrafted and DNN-based watermarking methods on spoof detectors using standardized preprocessing and split designs.
• Results show that increased watermark density degrades detection performance, emphasizing the need for watermark-resilient anti-spoofing strategies.

Searching arXiv for the specified dataset paper and closely related watermark-spoofing benchmark papers to ground the article. The Watermark-Spoofing dataset is a watermark-augmented benchmark for audio anti-spoofing in which established spoof-detection corpora are systematically transformed with audio watermarks in order to measure how watermark-induced perturbations alter bonafide/spoof discrimination (Zhang et al., 25 Sep 2025). It was introduced as part of the first study on the impact of audio watermarking on audio anti-spoofing countermeasures, with the explicit aims of quantifying the effect of both handcrafted and neural watermarking methods on state-of-the-art spoofing detectors, providing dedicated training and held-out evaluation corpora under controlled watermark-density conditions, and serving as a public benchmark for watermark-resilient anti-spoofing research (Zhang et al., 25 Sep 2025).

1. Research setting and intended use

Modern audio anti-spoofing systems are typically trained on clean or conventionally augmented speech, whereas real deployments may involve invisible watermarks inserted for copyright or provenance purposes. In the dataset description, such watermarks are treated as structured perturbations that can constitute a previously unstudied domain shift, and the benchmark is designed to make that shift measurable under controlled conditions (Zhang et al., 25 Sep 2025).

The benchmark has three stated objectives. First, it evaluates how a variety of handcrafted and neural-network watermarking methods affect spoof detectors. Second, it provides a training corpus and held-out evaluation corpora with explicit watermark ratios. Third, it establishes a public benchmark for developing robust, watermark-resilient anti-spoofing countermeasures. A central empirical claim associated with the benchmark is that watermarking consistently degrades anti-spoofing performance, with higher watermark density correlating with higher Equal Error Rates (EERs), and that naïve training on watermarked data only sometimes helps and usually hurts clean-data performance (Zhang et al., 25 Sep 2025).

This design places the dataset at the intersection of watermarking, speech deepfake detection, and domain adaptation. A plausible implication is that the dataset is not merely an augmentation pack, but a controlled stress test for assessing whether anti-spoofing models rely on brittle signal cues that are destabilized by watermark overlays.

2. Source corpora and waveform preprocessing

The dataset is built by augmenting four pre-existing anti-spoofing corpora: ASVspoof 2019 Logical Access training, ASVspoof 2021 Logical Access evaluation, ASVspoof 2021 DeepFake evaluation, and In-the-Wild evaluation (Zhang et al., 25 Sep 2025).

Corpus	Role in benchmark	Approximate composition
ASVspoof 2019 LA19	Training source	$\approx 25{,}260$ utterances: $2{,}580$ bonafide + $22{,}680$ spoof
ASVspoof 2021 LA21	Evaluation source	$\approx 30{,}750$ utterances: $7{,}359$ bonafide + $23{,}391$ spoof
ASVspoof 2021 DF21	Evaluation source	$\approx 12{,}880$ utterances: $2{,}582$ bonafide + $10{,}298$ spoof
In-the-Wild	Evaluation source	$\approx 1{,}800$ utterances, balanced bonafide/spoof

All waveforms undergo a common preprocessing pipeline. They are resampled to 16 kHz, normalized or pad/truncated to 64,600 samples, corresponding to approximately 4.03 seconds, and processed with RawBoost colored-noise augmentation with random SNR as in Tak et al., ICASSP ’22. The final output format is single-channel mono WAV (Zhang et al., 25 Sep 2025).

This preprocessing standardizes temporal extent and sampling conditions across corpora that were originally collected for different evaluation protocols. The resulting benchmark isolates watermark effects from confounds due to heterogeneous duration and channel configurations. Because the bonafide/spoof labels are inherited from the source corpora, the watermarking layer becomes an additional nuisance variable rather than a replacement label space.

3. Watermarking schemes and density control

The benchmark description states that the dataset uses nine distinct watermarking schemes, divided into six handcrafted methods and three DNN-based methods. The handcrafted methods are Blind SVD watermarking, improved phase-coding steganography, LSB amplitude steganography, Frequency Singular Value Coefficient modulation, Patchwork multilayer modulation, and Lifting Wavelet Transform. The DNN-based enumeration lists RobustDNN, WavMark, Timbre, and AudioSeal (Zhang et al., 25 Sep 2025).

This enumeration creates an internal discrepancy: the text states “nine distinct watermarking schemes” and “three DNN-based,” but it then lists four DNN-based methods. This suggests a mismatch between the stated total and the explicit method list rather than a substantive ambiguity about the methods actually named (Zhang et al., 25 Sep 2025).

The dataset defines a nominal watermark density

$2{,}580$0

In practice, however, the benchmark controls the proportion of utterances in each split that carry any watermark, while the exact bit-rate varies by method and is chosen to be imperceptible, with distortion below 1% (Zhang et al., 25 Sep 2025). The dataset-level watermark ratios are $2{,}580$1.

A further design choice is the treatment of mixed-watermark splits. In such splits, DNN and handcrafted methods are present in equal proportion, and within each group all methods are equally likely (Zhang et al., 25 Sep 2025). This makes the benchmark suitable for evaluating method-specific sensitivity as well as robustness to heterogeneous watermark families. It also means that “watermarked” is not a single nuisance source but a mixture distribution spanning transform-domain, steganographic, and learned watermarking procedures.

4. Dataset organization, splits, and metadata

The training portion, termed Watermark-Spoofing Training, is derived from LA19 train. It contains 25,260 utterances in total, consisting of 2,580 bonafide and 22,680 spoof utterances. Exactly 50% are watermarked, approximately 12,630 utterances, and 50% are clean. Within the watermarked half, 50% are DNN-based and 50% are handcrafted, and the split is class-balanced across bonafide and spoof (Zhang et al., 25 Sep 2025).

The evaluation portion, termed Watermark-Spoofing Seen Evaluation, releases three parallel variants for each of LA21, DF21, and ITW: 75% watermarked plus 25% clean, 50% watermarked plus 50% clean, and 25% watermarked plus 75% clean. All retain the original bonafide/spoof proportions of the underlying corpora (Zhang et al., 25 Sep 2025). For LA21, whose size is approximately 30,750 utterances, the example breakdowns are 23,063 watermarked and 7,688 clean for the 75% variant, 15,375 watermarked and 15,375 clean for the 50% variant, and 7,688 watermarked and 23,063 clean for the 25% variant (Zhang et al., 25 Sep 2025).

The released files are 16 kHz single-channel WAV with 16-bit PCM encoding. The directory structure includes train_LA19/clean, train_LA19/watermarked/handcrafted, train_LA19/watermarked/dnn_based, and evaluation directories such as eval_LA21_75, eval_DF21_50, and eval_ITW_25. Filenames follow the pattern <origSet>_<uttID>_<label>_<wmGroup>_<wmMethod>.wav, and accompanying CSV or TSV metadata contain the columns {filename, bonafide/spoof, watermarked:0/1, wm_group:{dnn,hand}, wm_method, split} (Zhang et al., 25 Sep 2025).

These structural choices make the benchmark amenable to controlled ablations. Because watermark status, watermark group, and watermark method are explicit metadata fields, one can evaluate robustness at several granularities: clean versus watermarked, handcrafted versus DNN-based, or method-specific performance. The directory layout also supports exact regeneration of splits rather than mere download-and-use evaluation.

5. Evaluation protocol and observed behavior

The primary evaluation metric is Equal Error Rate, defined by the threshold $2{,}580$2 satisfying

$2{,}580$3

The benchmark reports EER under varying watermark ratios and watermark families (Zhang et al., 25 Sep 2025).

An excerpted result on In-the-Wild using XLSR+SLS trained on clean LA19 shows the effect of single watermark methods. For AudioSeal, EER rises from 7.32 at 0% watermarking to 7.35 at 25%, 7.40 at 50%, and 7.46 at 75%, corresponding to $2{,}580$4. For Timbre, the values are 7.32, 7.53, 7.93, and 8.18, with $2{,}580$5. For WavMark, they are 7.32, 8.23, 9.06, and 9.90, with $2{,}580$6. For RobustDNN, they are 7.32, 8.06, 8.65, and 9.06, with $2{,}580$7 (Zhang et al., 25 Sep 2025).

The mixed-watermark setting extends this pattern: for XLSR-AASIST, XLSR-SLS, and XLSR-Nes2Net-X on LA21, DF21, and ITW, the reported EERs all increase monotonically as watermark ratio increases (Zhang et al., 25 Sep 2025). The benchmark’s key observation is therefore that watermarking acts as a hard domain shift. A common misconception is that watermarking is orthogonal to spoof detection because it is intended for copyright protection rather than speech manipulation; the benchmark directly contradicts that assumption by showing systematic performance degradation under watermark augmentation (Zhang et al., 25 Sep 2025).

The paper further introduces the Knowledge-Preserving Watermark Learning framework, described as a way to adapt models to watermark-induced shifts while preserving original-domain spoofing detection capability (Zhang et al., 25 Sep 2025). Within the logic of the dataset, this positions the benchmark not only as a diagnostic instrument but also as an evaluation bed for domain-adaptation methods that must retain clean-condition competence.

6. Position within watermark-spoofing research

The Watermark-Spoofing dataset belongs to a broader family of spoofing-oriented watermark benchmarks that now span audio, video, text, and dataset ownership verification. In video watermarking, VideoMarkBench evaluates watermark removal and watermark forgery across 1,350 AI-generated videos, four watermarking methods, seven aggregation strategies, and twelve perturbation types under white-box, black-box, and no-box threat models (Jiang et al., 27 May 2025). In video detection rather than watermark verification, RobustSora constructs an Authentic-Spoofed subset by adding extracted Sora 2 watermarks onto authentic clips and shows that many detectors incur 7–8 percentage-point drops in $2{,}580$8 under such spoofed overlays (Wang et al., 11 Dec 2025).

In language watermarking, one benchmark uses C4-realnewslike and LFQA anchor texts together with paraphrase, sentiment reversal, latter-half sentiment reversal, and hate-speech insertion to evaluate robustness and security against spoofing (An et al., 9 Apr 2025). A separate line of work constructs genuine and spoofed corpora from C4 prompts under KGW SumHash watermarking, retaining only sequences with detector score $2{,}580$9 and then applying statistical tests to distinguish spoofed from genuinely watermarked text (Gloaguen et al., 2024). Another study uses a compact human–watermarked paraphrase dataset built from C4-RealNewslike, Qwen3-8B, and a watermarked Llama3.1-8B-Instruct model to train a black-box reinforcement-learning spoofer with only 100 training pairs per scheme (Huang et al., 13 Apr 2026).

Related concerns also appear in public dataset protection. Backdoor watermarking forgery work shows that forged triggers can yield virtually identical or stronger statistical significance than original watermarks in copyright verification tests, indicating that ownership verification results are insufficient on their own to determine infringement (Li et al., 2024).

Seen against this cross-modal background, the Watermark-Spoofing dataset occupies a distinct role. It does not benchmark an attack that forges an anti-spoofing watermark detector, nor does it test removal of embedded audio watermarks. Instead, it treats watermarking as an external perturbation source that shifts the input distribution faced by bonafide/spoof classifiers. This suggests that spoof resilience and watermark resilience should be studied jointly rather than as separate robustness problems. The benchmark’s public scripts, metadata, Dockerfile, and regeneration pipeline reinforce that role by making watermark-conditioned anti-spoofing evaluation reproducible at the corpus, split, and method level (Zhang et al., 25 Sep 2025).