AudioWMD: Robust Audio Watermark Detection

• AudioWMD is a two-stage detector that uses a base classifier and a meta-classifier to assess confidence stability under small audio perturbations, ensuring method-agnostic watermark detection.
• It aggregates multiple log-mel spectrogram score responses to compute key stability metrics that differentiate natural variations from adversarial attacks.
• Evaluated on the VoxWatermark benchmark, AudioWMD demonstrates improved AUROC and accuracy over single-query baselines in diverse, cross-lingual, and cross-method scenarios.

Searching arXiv for the specified paper and the WMD baseline. {"query":"au: Pan audio watermark detection WMD ECCV 2024 arXiv", "max_results": 5} {"query":"VoxWatermark AudioWMD (Sedaghati et al., 13 Jun 2026)", "max_results": 5} AudioWMD is a two-stage, multi-query “stability” detector introduced alongside the VoxWatermark benchmark for audio watermark detection under perturbations. It is designed to decide whether an audio clip $x \in \mathbb{R}^T$ carries any imperceptible watermark ($y=1$) or not ($y=0$), while remaining agnostic to the specific embedding algorithm. Within the VoxWatermark study, AudioWMD serves as a robust baseline detector for large-scale, multi-method, cross-distribution settings, motivated by the need for verifiable source attribution and copyright accountability as speech generation systems are deployed in open environments (Sedaghati et al., 13 Jun 2026).

1. Problem formulation and benchmark context

The detector is situated in a benchmark setting intended to address a stated gap in prior work: the lack of a unified benchmark that systematically compares different watermark injection methods under realistic distribution shifts. VoxWatermark addresses this by applying 10 watermarking methods, comprising 4 neural and 6 traditional methods, with unified injection and annotation on multilingual, multi-source corpora, and by introducing no-box, black-box, and white-box perturbations to simulate real recording and transmission conditions (Sedaghati et al., 13 Jun 2026).

Within this benchmark, AudioWMD is defined by a detection objective rather than by identification of a particular watermark family. Given an audio clip, the task is binary detection of watermark presence, not recovery of payload content and not attribution to a specific embedding scheme. This design choice is explicit in the formulation that the detector should operate “agnostic to the specific embedding algorithm.”

A central premise of the benchmark is that injection-method diversity and distribution shifts affect detection stability. AudioWMD is proposed precisely to exploit stability as a discriminative signal. This suggests a shift from single-instance confidence estimation toward analysis of local behavior under controlled perturbation, especially when watermark families, languages, and accents vary across training and deployment.

2. Two-stage architecture

AudioWMD consists of two stages: a base detector $f_\theta$ and a meta-classifier $g_\psi$. The first stage maps a single audio input to a real-valued confidence score, and the second stage examines the stability of that score under a small set of random audio transformations before making the final binary decision (Sedaghati et al., 13 Jun 2026).

For Stage I, the input representation is a 16 kHz audio signal converted to an 80-channel log-mel spectrogram $\mathbf{X} \in \mathbb{R}^{80 \times L}$. The backbone is described as a standard convolutional encoder, exemplified by a ResNet-style CNN. Training uses binary cross-entropy on single examples:

$$\mathcal{L}_{\mathrm{base}}(\theta) = -\frac{1}{N}\sum_{i=1}^N\Bigl[y_i\log\sigma(f_\theta(x_i)) + (1-y_i)\log(1-\sigma(f_\theta(x_i)))\Bigr].$$

After training, the base detector outputs

$s(x)=\sigma(f_\theta(x)) \in [0,1].$

Stage II does not trust a single score $s(x)$ in isolation. Instead, it probes local decision stability under a set of random, benign audio transforms $\mathcal{T}=\{T_1,\dots,T_{K-1}\}$, such as tiny time shifts, pitch jitter, and low-level noise. It forms $y=1$0 queries:

$y=1$1

and collects the corresponding scores $y=1$2.

From these scores, AudioWMD computes a 5-dimensional meta feature $y=1$3:

$y=1$4

A lightweight logistic-regression head then predicts the final label:

$y=1$5

The implementation logic is therefore explicitly hierarchical: single-query scoring first, query-statistics aggregation second. A plausible implication is that the architecture separates representation learning from robustness inference, allowing the second stage to exploit score geometry without requiring per-watermark redesign.

3. Stability-based detection principle

The distinguishing principle of AudioWMD is stability-aware detection. Rather than treating classification confidence as sufficient evidence, it evaluates whether confidence remains locally consistent under small benign transformations. The method’s core defense is stated directly: natural perturbations cause only small score-flips under $y=1$6, whereas targeted attacks produce inconsistent outputs across the $y=1$7 queries; the meta-features amplify these inconsistencies, yielding stronger separation in the meta-classifier (Sedaghati et al., 13 Jun 2026).

The five statistics play distinct roles in this stability analysis. The mean $y=1$8 summarizes average confidence across probes. The standard deviation $y=1$9 and range $y=0$0 quantify dispersion. The fraction $y=0$1 records how often the score exceeds the decision threshold $y=0$2. The flip rate $y=0$3 measures disagreement with the original query’s sign relative to that threshold. Collectively, these statistics convert local score behavior into a low-dimensional descriptor of detector reliability.

This approach is explicitly method-agnostic. AudioWMD makes no assumption about how the watermark was embedded, and there is no per-method submodule. Because the same $y=0$4-statistics apply regardless of watermark type or language, the detector is intended to fuse evidence across heterogeneous embedding schemes and speech domains. This suggests that the method treats watermark detection as a problem of invariant decision behavior rather than signature matching.

4. Multi-method training and cross-distribution evaluation

During training, the base detector sees a mixture of watermarked examples from six distinct schemes—four traditional and two neural—and clean speech from multilingual corpora. At test time, it is evaluated on unseen watermark algorithms, including Patchwork, Echo Hiding, and WavMark, and under distribution shifts including cross-lingual evaluation over 25 languages and cross-accent evaluation on VCTK (Sedaghati et al., 13 Jun 2026).

The dataset configuration used for AudioWMD is summarized below.

Split	Content	Size
Training/validation (in-domain)	LibriSpeech, Common Voice (English & Mandarin), AISHELL-1; 6 seen schemes: LSB, QIM, DSSS, AudioSeal, Timbre, Phase Coding	61,200 clips (30.6 K clean + 30.6 K watermarked)
Validation	10% stratified split	Not separately enumerated beyond the 10% split
Out-of-domain Test 1	Common Voice, 25 non-EN/ZH languages; unseen schemes: Patchwork, Echo, WavMark	2 K clips
Out-of-domain Test 2	VCTK accent diversity; same unseen schemes	2 K clips

The training procedure is therefore supervised on a finite set of seen watermark schemes, while the evaluation protocol explicitly probes generalization across both embedding-method shift and speech-distribution shift. The paper states that the same statistics are used regardless of language or embedding mechanism. A plausible implication is that the benchmark is designed to test universal detection behavior rather than specialization to a closed-world watermark inventory.

5. Perturbation taxonomy and robustness strategy

VoxWatermark simulates three real-world attack classes: no-box, black-box, and white-box perturbations. AudioWMD’s robustness claims are framed relative to this taxonomy (Sedaghati et al., 13 Jun 2026).

In the no-box setting, the benchmark uses 17 generic distortions, including time stretch $y=0$5, Gaussian noise (SNR 5–40 dB), background noise (SNR 5–35 dB), echo, low/high/band-pass filters, bit-depth reduction, neural/lossy codecs (EnCodec, Opus), dynamic range compression, and phase/frequency jitter. These are applied without any knowledge of the detector.

In the black-box setting, the attacks include HopSkipJumpAttack (HSJ) on raw waveforms and on log-mel spectrograms, as well as Square Attack with block-wise spectrogram perturbations. The adversary may query the detector’s confidence scores to mount removal attacks on watermarked audio or forgery attacks on clean audio.

In the white-box setting, for neural watermarks such as Timbre and WavMark, the perturbation is optimized by directly minimizing or maximizing

$y=0$6

with respect to an additive perturbation $y=0$7, subject to an SNR-based fidelity constraint. For traditional schemes such as QIM and echo hiding, differentiable surrogates are used with the same gradient-based optimization.

The paper identifies a specific failure mode: AudioWMD can be weaker on spectrogram HSJ, which exposes a limitation of purely waveform-level transforms. This is a narrowly stated criticism rather than a general rejection of the method. It indicates that the benign probe set $y=0$8 is itself part of the threat model; if $y=0$9 lacks spectral randomization, mel-domain attacks may circumvent the stability heuristic.

6. Experimental results, comparison, and limitations

The evaluation uses Accuracy, AUROC, Precision, Recall, and F1 score, with Accuracy defined as

$f_\theta$0

The reported results show a consistent gap between AudioWMD and the single-query baseline “WMD” in several settings (Sedaghati et al., 13 Jun 2026).

Setting	AudioWMD	WMD
In-domain validation	AUROC = 0.883, Acc = 84%	0.720 / 67%
Cross-lingual Test 1	AUROC = 0.638, Acc = 53%	0.571 / 55%
Cross-accent Test 2	AUROC = 0.632, Acc = 58%	0.579 / 56%
No-box robustness	modest AUROC gains of $f_\theta$1 over WMD	Lower by the stated margin
White-box attacks	AUROC $f_\theta$2, F1 $f_\theta$3	$f_\theta$4, F1 $f_\theta$5

The paper further characterizes WMD, identified as “Pan et al. ECCV 2024,” as a single-query ConvNeXt-V2 detector trained with an asymmetric loss and operating on one spectrogram. By contrast, AudioWMD is described as stability-aware, cross-method in its feature design, improved in OOD performance, and stronger under white-box attack. The stated summary improvements are $f_\theta$6 AUROC under unseen languages and accents, and $f_\theta$7 absolute AUROC versus WMD under white-box attack.

The limitations are explicitly enumerated. No-box detection still hovers near chance under strong codec/noise combinations, indicating that further feature engineering or adaptive transforms are needed. Black-box spectrogram attackers, specifically HSJ on the mel domain, can defeat AudioWMD if the probe set lacks spectral randomization. Training remains supervised on seen watermark schemes, and the text notes that self-supervised or one-class approaches might further improve unseen-method robustness. Proposed future directions include richer multi-modal probe sets combining waveform and spectrogram views, adaptive thresholding under distribution shift, end-to-end adversarial training of the two stages, and real-time optimization for deployment.

Taken together, these points place AudioWMD within a specific research trajectory: universal, robust, and scalable audio-watermark detection under heterogeneous watermark methods and realistic perturbation regimes. The open-sourcing of VoxWatermark and AudioWMD is presented as a mechanism to catalyze that line of research (Sedaghati et al., 13 Jun 2026).