DualMark: Dual-Provenance Audio Watermarking

• DualMark is a dual-provenance framework that embeds distinct binary signatures for both model identity and data origin in generated audio.
• It employs a dual watermark embedding module with a consistency loss to ensure reliable decoding under common audio manipulations.
• Benchmark evaluations demonstrate high detection accuracy and robustness to noise, compression, and resampling in diverse audio scenarios.

Searching arXiv for the named paper and closely related watermarking/provenance work in audio generation. Searching arXiv for "DualMark generated audio provenance watermarking". Searching arXiv for exact identifier (Yang et al., 21 Aug 2025) and related terms. DualMark is a dual-provenance watermarking framework for audio generative models that embeds and reliably decodes two distinct attribution signatures—one for the model identity and another for the training data origin—from a single generated audio sample. It was introduced to address a specific limitation of prior audio watermarking techniques: existing methods support model-level attribution but do not trace the datasets that influenced the generator, leaving unresolved provenance questions in settings involving copyright, licensing restrictions, and accountability. DualMark operates during training in Mel-spectrogram space, couples watermark injection with a consistency objective, and evaluates robustness through the Dual Attribution Benchmark (DAB), a benchmark tailored to joint model–data attribution (Yang et al., 21 Aug 2025).

1. Problem formulation and design objectives

DualMark formalizes dual attribution as the recovery of both generator identity and data origin from an audio signal $s$ synthesized by model $\mathcal{G}_i$ trained using data from $\mathcal{C}_j$. The paper defines the mapping

$$\Psi: s \mapsto (\hat{\mathcal{G}}, \hat{\mathcal{C}})$$

with similarity-based inference

$$\Psi(s) = \left( \arg\max_{\mathcal{G} \in \mathbb{G}} \kappa(f_{\mathcal{G}}(s), \mathcal{G}), \arg\max_{\mathcal{C} \in \mathbb{C}} \kappa(f_{\mathcal{C}}(s), \mathcal{C}) \right),$$

where $f_{\mathcal{G}}(s)$ and $f_{\mathcal{C}}(s)$ are the predictions for model- and data-level attribution, and $\kappa$ is a similarity function against candidate identifiers.

The motivation is tied to a distinction between model provenance and data provenance. Existing watermarking methods for audio generation focus on identifying whether audio was produced by a particular model, but they do not encode dataset provenance. This missing data attribution weakens copyright protection for data providers and limits accountability when outputs may derive from specific training sources carrying licensing restrictions. The paper also distinguishes post-hoc external watermarks, which are vulnerable to pruning and do not reliably survive common audio edits, from intrinsic parameter-based watermarks, which resist pruning but still do not encode dataset-level provenance.

The stated design goals are high attribution accuracy for both signatures; robustness to pruning of plugins, lossy compression, resampling, and additive noise; imperceptibility and minimal impact on perceived audio quality; low training and inference overhead; and scalability to multiple models and data sources. In this formulation, dual provenance is not an auxiliary annotation layer but a property learned into the generative pipeline itself.

2. Generative backbone and Dual Watermark Embedding

DualMark is trained end-to-end on a latent diffusion backbone. AudioLDM serves as the backbone latent diffusion model. Given a Mel-spectrogram $S \in \mathbb{R}^{T \times F}$, a VAE encoder $f_{\mathrm{enc}}$ maps to a latent $\mathcal{G}_i$0. The paper writes the forward diffusion as

$\mathcal{G}_i$1

and trains the denoising network $\mathcal{G}_i$2 with latent diffusion loss

$\mathcal{G}_i$3

The watermark payload is explicitly dual. DualMark encodes a binary dual watermark vector

$\mathcal{G}_i$4

where $\mathcal{G}_i$5 identifies the generation model, $\mathcal{G}_i$6 identifies the data source, and $\mathcal{G}_i$7 is the total payload length. In the reported experiments, each identity uses a 7-bit binary code, so $\mathcal{G}_i$8 and $\mathcal{G}_i$9 bits per sample.

The central embedding mechanism is the Dual Watermark Embedding (DWE) module. DWE injects $\mathcal{C}_j$0 into Mel-spectrograms via a pretrained watermark encoder $\mathcal{C}_j$1 derived from RoSteALS and an adapter autoencoder $\mathcal{C}_j$2 that maps Mel-spectrograms to and from a learned hidden space: $\mathcal{C}_j$3 Here $\mathcal{C}_j$4 is the watermarked Mel-spectrogram used in training. The encoder $\mathcal{C}_j$5 produces an additive watermark perturbation in the hidden space, and the adapter autoencoder is fixed during fine-tuning to avoid collapsing fidelity. The framework does not impose an explicit orthogonality constraint between model and data signatures; instead, it relies on the consistency objective to keep both signatures reliably decodable and disentangled in practice.

This design is notable because the watermark is embedded during model training rather than appended as an external plugin. As a result, attribution is tied to the generative model’s internal spectrogram-space behavior rather than to a separable post-processing component.

3. Consistency loss, training procedure, and attribution pipeline

DualMark couples generative training with a watermark preservation objective. At training time, a frozen pretrained watermark decoder $\mathcal{C}_j$6 predicts the dual watermark vector from generated spectrograms, $\mathcal{C}_j$7, and the system enforces bitwise consistency with binary cross-entropy: $\mathcal{C}_j$8 The total objective is

$\mathcal{C}_j$9

with $$\Psi: s \mapsto (\hat{\mathcal{G}}, \hat{\mathcal{C}})$$0 in the main experiments.

Only the latent diffusion UNet is updated; $$\Psi: s \mapsto (\hat{\mathcal{G}}, \hat{\mathcal{C}})$$1, $$\Psi: s \mapsto (\hat{\mathcal{G}}, \hat{\mathcal{C}})$$2, the VAE, and the adapter autoencoder remain frozen. The training setup uses AudioLDM-S-Full and AudioLDM-M-Full, the GTZAN music genre dataset with 1,000 audio signals across 10 genres at 16 kHz and 64 mel bins, and prompts of the form “This is a [genre] music piece.” Each genre is treated as a distinct data origin. Fine-tuning runs for 30 epochs on a single NVIDIA RTX 3090 with Adam, learning rate $$\Psi: s \mapsto (\hat{\mathcal{G}}, \hat{\mathcal{C}})$$3, batch size 2, and linear warm-up over 2,000 steps. A sensitivity sweep selected $$\Psi: s \mapsto (\hat{\mathcal{G}}, \hat{\mathcal{C}})$$4 as the best trade-off between fidelity and watermark strength, and 200 DDIM steps yielded the best empirical attribution.

At inference, generated audio is converted to a Mel-spectrogram $$\Psi: s \mapsto (\hat{\mathcal{G}}, \hat{\mathcal{C}})$$5 and decoded by $$\Psi: s \mapsto (\hat{\mathcal{G}}, \hat{\mathcal{C}})$$6 to obtain $$\Psi: s \mapsto (\hat{\mathcal{G}}, \hat{\mathcal{C}})$$7. The recovered vector is split as $$\Psi: s \mapsto (\hat{\mathcal{G}}, \hat{\mathcal{C}})$$8 and matched against a predefined codebook of model and data identities. The paper formalizes attribution with a similarity-based decision rule, but does not specify exact thresholds or statistical decision rules; in practice, bitwise decoding and codebook matching suffice, and evaluation uses per-genre one-vs-rest ROC/AUC and F1/Recall metrics.

The ablation on $$\Psi: s \mapsto (\hat{\mathcal{G}}, \hat{\mathcal{C}})$$9 is structurally important. Without $$\Psi(s) = \left( \arg\max_{\mathcal{G} \in \mathbb{G}} \kappa(f_{\mathcal{G}}(s), \mathcal{G}), \arg\max_{\mathcal{C} \in \mathbb{C}} \kappa(f_{\mathcal{C}}(s), \mathcal{C}) \right),$$0, model attribution degenerates to chance, with Det. Acc $$\Psi(s) = \left( \arg\max_{\mathcal{G} \in \mathbb{G}} \kappa(f_{\mathcal{G}}(s), \mathcal{G}), \arg\max_{\mathcal{C} \in \mathbb{C}} \kappa(f_{\mathcal{C}}(s), \mathcal{C}) \right),$$1, F1 $$\Psi(s) = \left( \arg\max_{\mathcal{G} \in \mathbb{G}} \kappa(f_{\mathcal{G}}(s), \mathcal{G}), \arg\max_{\mathcal{C} \in \mathbb{C}} \kappa(f_{\mathcal{C}}(s), \mathcal{C}) \right),$$2, and Recall $$\Psi(s) = \left( \arg\max_{\mathcal{G} \in \mathbb{G}} \kappa(f_{\mathcal{G}}(s), \mathcal{G}), \arg\max_{\mathcal{C} \in \mathbb{C}} \kappa(f_{\mathcal{C}}(s), \mathcal{C}) \right),$$3, while data attribution falls near chance, with AUC $$\Psi(s) = \left( \arg\max_{\mathcal{G} \in \mathbb{G}} \kappa(f_{\mathcal{G}}(s), \mathcal{G}), \arg\max_{\mathcal{C} \in \mathbb{C}} \kappa(f_{\mathcal{C}}(s), \mathcal{C}) \right),$$4, F1 $$\Psi(s) = \left( \arg\max_{\mathcal{G} \in \mathbb{G}} \kappa(f_{\mathcal{G}}(s), \mathcal{G}), \arg\max_{\mathcal{C} \in \mathbb{C}} \kappa(f_{\mathcal{C}}(s), \mathcal{C}) \right),$$5, and Recall $$\Psi(s) = \left( \arg\max_{\mathcal{G} \in \mathbb{G}} \kappa(f_{\mathcal{G}}(s), \mathcal{G}), \arg\max_{\mathcal{C} \in \mathbb{C}} \kappa(f_{\mathcal{C}}(s), \mathcal{C}) \right),$$6. With $$\Psi(s) = \left( \arg\max_{\mathcal{G} \in \mathbb{G}} \kappa(f_{\mathcal{G}}(s), \mathcal{G}), \arg\max_{\mathcal{C} \in \mathbb{C}} \kappa(f_{\mathcal{C}}(s), \mathcal{C}) \right),$$7, DualMark attains the reported strong performance. Feature visualization by t-SNE further shows that $$\Psi(s) = \left( \arg\max_{\mathcal{G} \in \mathbb{G}} \kappa(f_{\mathcal{G}}(s), \mathcal{G}), \arg\max_{\mathcal{C} \in \mathbb{C}} \kappa(f_{\mathcal{C}}(s), \mathcal{C}) \right),$$8 induces structure: samples from the same data origin cluster and different origins separate, enabling reliable data watermark decoding (Yang et al., 21 Aug 2025).

4. Dual Attribution Benchmark and empirical results

The Dual Attribution Benchmark evaluates joint model–data attribution under realistic conditions. It covers text-to-audio generation on GTZAN, treats each genre as a distinct data origin, and uses an evaluation set of 100 watermarked and 100 non-watermarked generated audios per genre, for 2,000 audios total. The benchmark reports model-level detection accuracy, Recall, and F1; and data-level one-vs-rest AUC, Recall, and F1. It includes pruning, additive white noise, additive pink noise, resampling, and AAC lossy compression as attack conditions (Yang et al., 21 Aug 2025).

The main reported results are as follows.

Setting	Model-level attribution	Data-level attribution
AudioLDM-S-Full	97.10% Det. Acc, 97.01% F1, 94.20% Recall	91.51% AUC, 93.68% F1, 83.60% Recall
AudioLDM-M-Full	92.65% Det. Acc, 92.07% F1, 85.30% Recall	87.55% AUC, 90.83% F1, 75.78% Recall

Under DAB attacks on AudioLDM-S-Full, the clean-audio baseline remains the strongest condition, but robustness is sustained across common manipulations. For white noise, model F1 is 87.64% and data AUC is 79.50%. For pink noise, model F1 is 92.01% and data AUC is 84.75%. For resampling, model F1 is 94.20% and data AUC is 90.75%. For AAC compression, model F1 is 96.27% and data AUC is 91.25%. These figures support the paper’s claim that the training-time spectrogram watermark survives operations that commonly degrade audio watermark signals.

The comparison with prior audio watermarking methods is asymmetric across tasks. AudioSeal and Timbre provide only model-level attribution, and under clean conditions their model attribution is comparable to DualMark. Under pruning, however, they collapse, with detection accuracy near 50% and F1 near 0%, while DualMark is unaffected by plugin pruning because its watermarking is embedded during training and decoded intrinsically from the spectrogram. For data provenance, the comparison is categorical rather than incremental: only DualMark supports dataset-origin attribution in the reported setup.

The paper also reports that attribution is stronger before waveform reconstruction, with model-level metrics reaching 100% before the vocoder stage. This indicates that vocoder reconstruction can slightly degrade watermark integrity, although post-vocoder performance remains high.

5. Robustness profile, perceptual quality, and deployment characteristics

The threat model centers on black-box audio manipulations and on pruning or removal of external watermarking components. DualMark resists additive noise, resampling, AAC compression, and pruning because the watermark is learned in the generative pipeline on Mel-spectrograms and decoded intrinsically by $$\Psi(s) = \left( \arg\max_{\mathcal{G} \in \mathbb{G}} \kappa(f_{\mathcal{G}}(s), \mathcal{G}), \arg\max_{\mathcal{C} \in \mathbb{C}} \kappa(f_{\mathcal{C}}(s), \mathcal{C}) \right),$$9; no external plugin needs to be preserved. White-box attacks, particularly parameter tampering, are not explicitly evaluated.

Imperceptibility is assessed by subjective listening tests with 16 raters, using overall listening quality (OVL) and prompt relevance (REL). DualMark’s OVL and REL scores are close to the original backbones and comparable to competing watermarks, indicating minimal perceptual degradation. Spectrogram visualizations are reported to show close match to originals. The paper therefore treats fidelity degradation as limited but nonzero, and explicitly notes that overemphasizing $f_{\mathcal{G}}(s)$0 with large $f_{\mathcal{G}}(s)$1 hurts fidelity and robustness.

From a systems standpoint, the training overhead is bounded to fine-tuning the UNet for 30 epochs with batch size 2 on an RTX 3090, while all auxiliary components remain frozen. Inference overhead consists of a single Mel-spectrogram extraction and a forward pass of $f_{\mathcal{G}}(s)$2, described as negligible relative to generation. The payload capacity is $f_{\mathcal{G}}(s)$3 bits per sample; with $f_{\mathcal{G}}(s)$4, the implementation encodes 14 bits, comprising 7 model bits and 7 data bits. The framework is designed for latent diffusion on Mel-spectrograms, including AudioLDM, and is presented as portable to other audio diffusion backbones that operate in spectrogram space. Autoregressive and GAN backbones are not evaluated, and the method assumes access to Mel-spectrograms during training and inference.

The paper’s stated limitations are correspondingly specific: vocoder reconstruction slightly weakens watermarks; increasing $f_{\mathcal{G}}(s)$5 beyond the selected operating point degrades fidelity and robustness; attribution depends on Mel-spectrogram extraction; and current experiments focus on GTZAN genres and AudioLDM. A plausible implication is that scaling to larger identity spaces would require stronger supervision or error-correction, since the paper notes that increasing $f_{\mathcal{G}}(s)$6 may require both, while not including error-correcting codes in the current system.

6. Position within watermarking research and provenance governance

DualMark is situated against two prior families of watermarking methods. External plugin-based watermarks, including post- or pre-processing schemes, are easy to deploy but vulnerable to pruning or removal and offer only model-level attribution. Intrinsic parameter-based watermarks improve pruning robustness but still do not embed training data provenance. DualMark’s novelty lies in combining training-time embedding, spectrogram-space operation, a dual signature $f_{\mathcal{G}}(s)$7, and a consistency loss that preserves decodability through generation and common signal operations. It also introduces DAB as a benchmark explicitly tailored to evaluate joint model–data attribution robustness (Yang et al., 21 Aug 2025).

A useful comparison arises with recent diffusion-model watermarking outside audio. “GaussMarker: Robust Dual-Domain Watermark for Diffusion Models” embeds a multi-bit spatial watermark and a zero-bit frequency watermark into the initial latent Gaussian noise of image diffusion models, uses a model-independent Gaussian Noise Restorer, and fuses detection scores for robustness under image distortions and advanced attacks. The paper explicitly notes that it does not use the name “DualMark,” that it targets images rather than audio, and that it is not a parameter watermark: the watermark lives in generation noise rather than in model parameters (Li et al., 13 Jun 2025). This contrast clarifies DualMark’s particular contribution: not merely dual watermarking, but dual provenance in audio generation, with explicit recovery of both model identity and dataset origin from a single output.

The broader significance claimed for DualMark is legal and governance-oriented rather than purely forensic. Dual attribution strengthens protection for both model creators and data providers, enabling accountability and compliance, including auditing whether outputs derive from licensed datasets. The framework embeds only abstract binary identifiers, and the paper emphasizes that governance should define policies for codebook management and disclosure. In that sense, DualMark is presented not as a complete solution to provenance in audio generation, but as a technical basis for accountable audio generative models with stronger copyright protection and responsibility tracing capabilities.