WaterVIB: Learning Minimal Sufficient Watermark Representations via Variational Information Bottleneck

Abstract: Robust watermarking is critical for intellectual property protection, whereas existing methods face a severe vulnerability against regeneration-based AIGC attacks. We identify that existing methods fail because they entangle the watermark with high-frequency cover texture, which is susceptible to being rewritten during generative purification. To address this, we propose WaterVIB, a theoretically grounded framework that reformulates the encoder as an information sieve via the Variational Information Bottleneck. Instead of overfitting to fragile cover details, our approach forces the model to learn a Minimal Sufficient Statistic of the message. This effectively filters out redundant cover nuances prone to generative shifts, retaining only the essential signal invariant to regeneration. We theoretically prove that optimizing this bottleneck is a necessary condition for robustness against distribution-shifting attacks. Extensive experiments demonstrate that WaterVIB significantly outperforms state-of-the-art methods, achieving superior zero-shot resilience against unknown diffusion-based editing.

• The paper introduces a stochastic Variational Information Bottleneck that learns a minimal sufficient watermark representation, reducing dependence on fragile cover-image textures while preserving message information.
• WaterVIB improves EditGuard’s localized-edit BER from 0.26‰ to 0.07‰, cuts gradient interference by 73%, and nearly eliminates resize failures without sacrificing visual quality.
• The method generalizes across standard and generative attacks, but performance remains uneven across diffusion models and depends on carefully tuning backbone-specific bottleneck parameters.

Motivation and problem statement

Deep watermarking schemes such as HiDDeN, TrustMark, and EditGuard achieve strong robustness against classical signal-processing distortions through end-to-end encoder–decoder training with differentiable noise layers. The authors of WaterVIB argue that this paradigm fails under a qualitatively different threat: generative purification, in which diffusion-based models (e.g., SDXL, DiffPure) project a watermarked image back onto the natural image manifold, rewriting high-frequency texture while preserving perceptual quality. Because conventional encoders satisfy the invisibility constraint by hiding message bits in exactly these high-frequency textures — exploiting the reduced sensitivity of human vision in complex regions — the watermark becomes statistically entangled with cover details that generative models preferentially regenerate. WaterVIB addresses this by reformulating the encoder as an information sieve that learns a Minimal Sufficient Statistic (MSS) of the message via the Variational Information Bottleneck (VIB), explicitly minimizing mutual information between the latent representation and fragile cover content (2602.21508).

Empirical diagnosis of failure

The paper grounds its hypothesis in two quantitative analyses on the EditGuard backbone over 500 COCO validation images attacked with SDXL 1.0 image-to-image purification.

Spectral and spatial alignment. The watermark signal concentrates 75.65% of its energy in mid-high frequency bands; the AIGC distortion targets the same bands (43.85% energy), whereas natural images carry only 13% of their energy there. Spatially, the purification distortion correlates with the cover image at Pearson correlation $\rho \approx 0.60$, and the watermark residual itself at $\rho \approx 0.45$, versus essentially zero for random noise ($-0.0003$). This establishes that both the watermark and the attack live in the intersection of the high-frequency subspace and the texture-dependent subspace — the attack is content-dependent rather than additive noise.

Gradient counter-optimization. Via a first-order Taylor expansion of the decoding loss around the watermarked image, the authors show that attack-induced loss increase is approximated by the inner product $$\langle s_{atk}, \nabla_x \mathcal{L}(x_{wm}) \rangle$$. Empirically, the watermark signal projects negatively onto the loss gradient ($-0.063$), while the AIGC distortion projects positively ($+0.027$), yielding an interference ratio of approximately 42.9%. Generative purification therefore behaves as an adversarial update that cancels nearly half of the embedding optimization effort. A supplementary analysis further shows the AIGC distortion is negatively correlated with the cover image (PCC $\approx -0.34$), indicating the generator actively subtracts high-frequency "perceptual noise" — a content-adaptive erasure mechanism distinct from passive degradation.

Theoretical framework

The proposed remedy is to learn a representation $Z$ satisfying sufficiency, $I(Z; M) = I(X; M)$, and minimality, $I(Z; X) \le I(\tilde{Z}; X)$ for all sufficient alternatives $\rho \approx 0.45$0. Under the assumptions of discrete domains and deterministic decoding maps, the appendix proves bidirectional equivalence between this information-theoretic characterization and the classical Lehmann–Scheffé definition of minimal sufficiency: sufficiency corresponds to preservation of $\rho \approx 0.45$1, and minimality follows from the Data Processing Inequality applied to the Markov chain induced by minimal sufficiency, with the converse established via invertibility from information equality. Relaxing strict sufficiency to an $\rho \approx 0.45$2-approximate constraint and applying Lagrange duality yields the standard IB objective:

$\rho \approx 0.45$3

The appendix additionally proves strict convexity and monotonicity of the Minimal Rate Curve, establishing a bijective correspondence between the multiplier $\rho \approx 0.45$4 and the tolerance $\rho \approx 0.45$5 via KKT analysis — so tuning $\rho \approx 0.45$6 is equivalent to traversing the Pareto frontier of the $\rho \approx 0.45$7-MSS problem. One caveat worth noting: the equivalence results hold for discrete alphabets and deterministic statistics, which is an idealization relative to continuous pixel spaces and stochastic encoders used in practice.

Method

WaterVIB inserts a stochastic bottleneck layer between the backbone feature extractor and the message decoder. Deterministic features $\rho \approx 0.45$8 are mapped by parallel MLPs (or CNNs for spatial features) to distributional parameters $\rho \approx 0.45$9 and $-0.0003$0, and the latent is sampled as $-0.0003$1 with $-0.0003$2, using the reparameterization trick with a variance-scaling factor $-0.0003$3 for training stability. Training minimizes BCE reconstruction loss plus $-0.0003$4 times the KL divergence to a fixed Gaussian prior; inference uses the deterministic map $-0.0003$5. The module is plug-and-play: for EditGuard it is a CNN-based bottleneck after the bit decoder (channel reduction $-0.0003$6, $-0.0003$7, $-0.0003$8); for HiDDeN it is an MLP bottleneck with $-0.0003$9 ($$\langle s_{atk}, \nabla_x \mathcal{L}(x_{wm}) \rangle$$0, $$\langle s_{atk}, \nabla_x \mathcal{L}(x_{wm}) \rangle$$1).

Results

Zero-shot robustness to generative editing. On AGE-Set localized editing, integrating WaterVIB into EditGuard reduces average BER from 0.26‰ to 0.07‰ (73% relative reduction), with over 90% error reduction under SD-Inpainting and SDXL-Refiner. Under global purification, where baseline BER ranges from roughly 26% to 61%, WaterVIB achieves up to 67% relative improvement under pixel-space DDPM and about 44–49% under SDXL-based purification. Residual weakness under SD-v1.5 (BER still above 35%) is attributed to that model's low reconstruction fidelity (PSNR ≈ 15 dB); notably, ControlNet-Inpainting shows no improvement ($$\langle s_{atk}, \nabla_x \mathcal{L}(x_{wm}) \rangle$$2), indicating the defense does not uniformly dominate across all generators.

Standard distortions. Against Gaussian, Poisson, and JPEG noise, EditGuard-VIB reduces BER from 3.21‰ to 0.08‰, outperforming TrustMark (6.60‰) and WM-A (140.20‰) while maintaining comparable PSNR (40.3 dB vs. 40.4 dB baseline). Per-distortion analysis reveals a striking result: the baseline collapses under Resize (81.75% BER), which WaterVIB eliminates almost entirely (0.01%), suggesting the bottleneck enforces invariance to grid resampling. On HiDDeN, WaterVIB improves combined average robustness by 38%, including a 61% reduction under PGD adversarial attack (76.0% → 30.0% BER), without any adversarial training.

Mechanistic verification. t-SNE visualization shows baseline decoder embeddings drift severely under purification while WaterVIB representations remain anchored to clean-sample clusters. The Gradient Interference Ratio drops from 0.4285 to 0.1167 (73% reduction), and orthogonality analysis shows near-zero cosine similarity between watermark and attack directions ($$\langle s_{atk}, \nabla_x \mathcal{L}(x_{wm}) \rangle$$3), suppressing effective destructive interference from 15.7% to 3.0% of the watermark magnitude despite a ~12 dB energy disparity favoring the attack. Residual maps confirm the qualitative shift: baseline residuals cluster on sharp edges, whereas VIB residuals diffuse more uniformly across textured regions.

Generalization and ablation. Validation-to-training loss ratios indicate WaterVIB substantially narrows the generalization gap attributable to texture overfitting, and the KL term stabilizes at ≈ 0.0005 after 20 epochs. The $$\langle s_{atk}, \nabla_x \mathcal{L}(x_{wm}) \rangle$$4 ablation exhibits a U-shaped curve on HiDDeN: BER falls from 20.93% ($$\langle s_{atk}, \nabla_x \mathcal{L}(x_{wm}) \rangle$$5) to a minimum of 11.59% at $$\langle s_{atk}, \nabla_x \mathcal{L}(x_{wm}) \rangle$$6, then rises sharply under over-compression — confirming that excessive bottleneck capacity discards essential watermark information.

Extensions. In the appendix, WaterVIB integrated into NeRF-Signature improves rendering PSNR from 57.17 dB to 58.99 dB on Blender and reduces LPIPS by 40% at zero BER, while preserving ~100% bit accuracy under 2D distortions. A detection-based defense against re-embedding attacks exploits higher decoder logit confidence: under noise, false-negative detection drops from 38.50% (baseline) to 1.98%.

Limitations and open questions

Several caveats are acknowledged or evident. First, the MSS equivalence theory assumes discrete domains and deterministic statistics, an idealization not strictly satisfied by continuous images and stochastic encoders. Second, gains are uneven across generators: ControlNet-Inpainting yields no improvement locally, and SD-v1.5 purification leaves BER above 35%, so the framework's protection degrades as reconstruction fidelity of the attacking generator decreases. Third, hyperparameters $$\langle s_{atk}, \nabla_x \mathcal{L}(x_{wm}) \rangle$$7 and $$\langle s_{atk}, \nabla_x \mathcal{L}(x_{wm}) \rangle$$8 are backbone-specific and tuned empirically; the U-shaped sensitivity implies performance depends on locating the compression sweet spot, and no automatic selection procedure is provided. Fourth, global purification BERs remain far from zero even with WaterVIB, leaving open whether information-theoretic disentanglement alone can survive aggressive semantic regeneration, and how the approach scales to text-conditioned edits that intentionally alter semantics rather than textures.

Conclusion

WaterVIB reframes deep watermarking as learning a minimal sufficient statistic of the embedded message, proving within its framework that IB optimization is a necessary condition for robustness against distribution-shifting generative attacks. The empirical evidence — including a 42.9% gradient interference ratio in the baseline reduced by 73%, near-total elimination of resize vulnerability, and consistent zero-shot gains across diffusion-based purifiers — supports the central claim that texture entanglement, not merely attack strength, drives watermark failure under generative purification. The work positions theoretically grounded representation learning as an alternative to heuristic noise-layer augmentation, while leaving open the practical limits of the approach against high-fidelity semantic regeneration.