Audio Pirates: Black-box Audio Watermark Removal via Diffusion Priors

Abstract: With the rise of AI-generated audio, watermarking has become widely used for detecting misuse and protecting intellectual property. However, adversaries may try to remove these watermarks, making it critical to evaluate how well watermarking schemes withstand removal attacks. Existing attacks are often impractical: they either noticeably degrade perceptual quality or require access to the watermarking scheme. We propose DiffErase, a black-box watermark removal attack that assumes no knowledge of the target watermarking scheme while maintaining perceptual quality. DiffErase perturbs watermarked audio to an intermediate diffusion noise level and regenerates it using a pretrained denoising model, effectively suppressing watermark signals. Theoretical analysis and extensive experiments demonstrate that inaudible audio watermarks are highly vulnerable: across multiple audio domains, DiffErase consistently removes watermarks while preserving perceptual quality. These findings highlight the need for future audio watermarking designs to consider diffusion-based threats. Code and demos are available at https://differase.github.io/DiffErase/.

• The paper's primary contribution is DiffErase, a method that leverages pretrained diffusion models to remove audio watermarks under black-box conditions.
• The method applies a two-stage process with forward noising and reverse denoising to project watermarked audio onto the natural manifold, effectively reducing watermark residues.
• Experiments across multiple audio domains demonstrate that DiffErase consistently lowers watermark detection rates while maintaining high perceptual fidelity and quality metrics.

Black-box Audio Watermark Removal via Diffusion Priors: An Expert Review

Introduction and Motivation

The prevalence of AI-generated audio has necessitated robust watermarking as a core mechanism for detecting content provenance and safeguarding intellectual property. Audio watermarking schemes embed imperceptible signals, enabling downstream verification of generative origins. However, real-world watermarking is inherently adversarial—malicious actors may explicitly target these embedded signatures for removal to facilitate unauthorized redistribution or avoid accountability. While existing removal attacks either depend on signal transformations (sacrificing perceptual quality), require system-level access, or rely on detector queries, this work critically broadens the threat model by investigating watermark removal under pure black-box assumptions.

The central contribution is DiffErase, a black-box audio watermark removal attack based on diffusion priors. DiffErase assumes no knowledge of the underlying watermarking system and no access to detection outputs, leveraging only a pretrained generative diffusion model and the watermarked audio input. The attack framework moves beyond classical signal-level manipulations by projecting the watermarked signal toward the data manifold: it systematically applies forward noising followed by reverse diffusion, effectively contracting off-manifold watermark residues and restoring perceptual fidelity. Theoretical analysis is furnished, formalizing the mechanism by which diffusion trajectories attenuate watermark perturbations. Large-scale experiments across multiple domains and watermarking schemes robustly support the approach.

Methodology: DiffErase Framework

DiffErase is constructed as a two-stage pipeline:

1. Forward Noising: The watermarked audio is mapped to a mel-spectrogram and diffused to an intermediate noise level $t^*$ using a forward diffusion process. $t^*$ is a key hyperparameter that governs the trade-off between removal aggressiveness and perceptual quality.
2. Reverse Denoising: Starting from the noisy representation, a pretrained diffusion model is used to reverse the process, guiding the sample toward the distribution of genuine, unaltered audio. A neural vocoder reconstructs the waveform.

   Figure 1: Overview of DiffErase. Watermarked audio is converted to a mel-spectrogram via STFT, perturbed to an intermediate noise level $t^*$ via forward noising, and then denoised back to $t=0$. A vocoder reconstructs the attacked audio, which evades watermark detection. The noise level $t^*$ controls the trade-off between removal strength and perceptual fidelity.

This process is executed in both the mel-spectrogram space and, alternatively, in a learned latent space (using a VAE encoder/decoder), yielding two instantiations: DiffErase-mel and DiffErase-latent, which offer distinct trade-offs with respect to computational efficiency and information bottlenecking.

The theoretical justification is built on a manifold-plus-perturbation model: watermarked audio is considered as lying off the manifold of natural audio by a small, structured perturbation. The diffusion process, and specifically the learned reverse dynamics, contract these off-manifold deviations at each step (with provable exponential decay dependent on $t^*$), resulting in an audio reconstruction with vanishing watermark residue.

Evaluation Protocol and Results

Experiments are conducted across three core audio domains—speech, music, and environmental sounds—using five advanced neural watermarking schemes: AudioSeal, TimbreWM, WavMark, Perth, and SilentCipher. Baseline attacks include classic signal-level and codec-level attacks, as well as adaptive adversarial methods requiring detector queries.

Key findings include:

• Signal-level attacks (e.g., pitch shifting, filtering, additive noise) require severe degradation to disable watermark detection, yielding poor perceptual metrics / subjective MUSHRA scores.
• Codec-based attacks (MP3, EnCodec) generally do not disable watermark detection in modern schemes, maintaining high fidelity but offering limited removal efficacy.
• Adaptive adversarial attacks (e.g., Square Attack) can reduce detectability but at the cost of artifacts and are contingent on non-black-box access.

DiffErase-mel and DiffErase-latent, in contrast, consistently reduce watermark detector $\mathrm{TPR}@1\%\,\mathrm{FPR}$ to zero across all watermarking schemes—except for Perth, a system which embeds unusually strong perturbations and shows partial resistance in non-speech domains—but retain high perceptual quality (objective ViSQOL $>$ 3.9, MUSHRA $>$ 86, and SQUIM-MOS $>$ 4.2).

Figure 2: Effect of noise level $t^*$0 for DiffErase-mel. Trade-off between audio quality (ViSQOL, left axis) and watermark removal (FNR $t^*$1, right axis), evaluated on Perth. Increasing $t^*$2 strengthens watermark removal at the expense of perceptual quality.

The attack can be tuned by varying $t^*$3, with higher diffusion strengths achieving robust removal but incrementally sacrificing fidelity. These empirical findings align closely with the theoretical bounds on residue contraction through the denoising trajectory.

Analysis and Visualization

The $t^*$4 norm analysis reveals that Perth's robustness derives from embedding perturbations 4–10$t^*$5 stronger than competing watermarks, at a measurable cost to audio quality.

Figure 3: The $t^*$6 distance between clean and watermarked audio across five watermarking methods on three domains. Perth embeds substantially stronger perturbations.

Spectrogram visualizations further elucidate the result: characteristic watermark-induced periodic patterns and frequency-domain artifacts—evident across multiple schemes and domains—are visibly suppressed or eliminated by DiffErase, despite retention of salient acoustic content.

Figure 4: Spectrogram visualization. Top: original audio. Middle: watermarked audio. Bottom: after DiffErase-mel ($t^*$7). Watermark patterns (middle row) are attenuated while acoustic content is preserved.

Alternative attack representations were considered. Waveform-level diffusion (DiffWave) and linear spectrogram methods (WavePurifier) achieved some watermark suppression but with notable drawbacks: DiffWave over-smooths temporal and harmonic details, and WavePurifier introduces artifacts due to inconsistent magnitude/phase reconstruction. DiffErase's design—denoising in mel-spectrogram space, coupled with a state-of-the-art vocoder—best balances removability and perceptual transparency.

Figure 5: Mel-spectrogram visualization across different attack representations. DiffWave (c) over-smooths harmonic details. WavePurifier (d) introduces audible artifacts. DiffErase (e) preserves the spectral structure while removing watermark perturbations.

Figure 6: Waveform visualization across different attack representations. Red waveforms show reconstructed audio, and gray shows input watermarked audio. DiffErase (d) closely matches the original temporal envelope.

Theoretical Implications and Future Trajectories

The manifold-based formulation and associated diffusion contraction bounds of DiffErase have several implications for both attack and defense in neural audio watermarking:

• Diffusion-based black-box removal fundamentally challenges watermarking-by-perturbation: Any scheme that injects low-magnitude, structured, but imperceptible modifications is susceptible to removal by generative priors that project perceptually plausible signals onto the data manifold.
• Stronger watermarking entails an inevitable perceptual trade-off: Enhancing robustness by increasing perturbation magnitude, as performed by Perth, leaves artifacts and sacrifices fidelity, fundamentally limiting the usefulness of watermarking for high-quality generative audio.
• Robust watermark defense may require new paradigms: Future defenses must anticipate generative model-based removal, e.g., by embedding non-removable semantic cues, leveraging cryptographically secure signatures resilient to "in-manifold" projection, or directly watermarking the generative model weights rather than the output waveform.

Practical deployment protocols must incorporate diffusion-based removal attacks in robustness evaluations. Forensic tools based on generative priors are already capable enough to undermine watermarking mechanisms that do not anticipate this threat model.

Conclusion

DiffErase provides a rigorous blueprint for black-box removal of neural audio watermarks. The use of diffusion priors not only achieves state-of-the-art removal performance without significant loss in perceptual fidelity, but also exposes intrinsic weaknesses in the current design philosophy of audio watermarking systems. As generative modeling continues its rapid advance, watermarking strategies must evolve to account for—and withstand—the manifold projection capacity of modern priors. The broader implication is an arms race between watermark embedding and generative removal, positioning robust detection and red-teaming as critical requirements for effective content provenance in the age of synthetic media.