Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
97 tokens/sec
GPT-4o
53 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Tree-Ring Watermarks: Fingerprints for Diffusion Images that are Invisible and Robust (2305.20030v3)

Published 31 May 2023 in cs.LG, cs.CR, and cs.CV

Abstract: Watermarking the outputs of generative models is a crucial technique for tracing copyright and preventing potential harm from AI-generated content. In this paper, we introduce a novel technique called Tree-Ring Watermarking that robustly fingerprints diffusion model outputs. Unlike existing methods that perform post-hoc modifications to images after sampling, Tree-Ring Watermarking subtly influences the entire sampling process, resulting in a model fingerprint that is invisible to humans. The watermark embeds a pattern into the initial noise vector used for sampling. These patterns are structured in Fourier space so that they are invariant to convolutions, crops, dilations, flips, and rotations. After image generation, the watermark signal is detected by inverting the diffusion process to retrieve the noise vector, which is then checked for the embedded signal. We demonstrate that this technique can be easily applied to arbitrary diffusion models, including text-conditioned Stable Diffusion, as a plug-in with negligible loss in FID. Our watermark is semantically hidden in the image space and is far more robust than watermarking alternatives that are currently deployed. Code is available at https://github.com/YuxinWenRick/tree-ring-watermark.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (43)
  1. Ali Al-Haj. Combined DWT-DCT Digital Image Watermarking. Journal of Computer Science, 3(9):740–746, September 2007. ISSN 15493636. doi: 10.3844/jcssp.2007.740.746. URL http://www.thescipub.com/abstract/?doi=jcssp.2007.740.746.
  2. Certified Neural Network Watermarks with Randomized Smoothing. In Proceedings of the 39th International Conference on Machine Learning, pages 1450–1465. PMLR, June 2022. URL https://proceedings.mlr.press/v162/bansal22a.html.
  3. On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, FAccT ’21, pages 610–623, New York, NY, USA, March 2021. Association for Computing Machinery. ISBN 978-1-4503-8309-7. doi: 10.1145/3442188.3445922. URL https://doi.org/10.1145/3442188.3445922.
  4. Francis Morgan Boland. Watermarking digital images for copyright protection. 1996. URL http://www.tara.tcd.ie/handle/2262/19682.
  5. SVD-based digital image watermarking scheme. Pattern Recognition Letters, 26(10):1577–1586, July 2005. ISSN 0167-8655. doi: 10.1016/j.patrec.2005.01.004. URL https://www.sciencedirect.com/science/article/pii/S0167865505000140.
  6. Reproducible scaling laws for contrastive language-image learning. ArXiv, abs/2212.07143, 2022.
  7. Secure spread spectrum watermarking for images, audio and video. Proceedings of 3rd IEEE International Conference on Image Processing, 3:243–246, 1996. doi: 10.1109/ICIP.1996.560429. URL http://ieeexplore.ieee.org/document/560429/.
  8. Digital Watermarking and Steganography. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 2 edition, 2007. ISBN 9780080555805.
  9. Imagenet: A large-scale hierarchical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition, pages 248–255, 2009. doi: 10.1109/CVPR.2009.5206848.
  10. Diffusion Models Beat GANs on Image Synthesis. arxiv:2105.05233[cs, stat], June 2021. doi: 10.48550/arXiv.2105.05233. URL http://arxiv.org/abs/2105.05233.
  11. Supervised GAN Watermarking for Intellectual Property Protection. arxiv:2209.03466[cs], September 2022. doi: 10.48550/arXiv.2209.03466. URL http://arxiv.org/abs/2209.03466.
  12. The Stable Signature: Rooting Watermarks in Latent Diffusion Models. arxiv:2303.15435[cs], March 2023. doi: 10.48550/arXiv.2303.15435. URL http://arxiv.org/abs/2303.15435.
  13. Paul Glasserman. Monte Carlo Methods in Financial Engineering, volume 53 of Stochastic Modelling and Applied Probability. Springer, New York, NY, 2003. ISBN 978-1-4419-1822-2 978-0-387-21617-1. doi: 10.1007/978-0-387-21617-1. URL http://link.springer.com/10.1007/978-0-387-21617-1.
  14. Generative adversarial nets. In Z. Ghahramani, M. Welling, C. Cortes, N. Lawrence, and K.Q. Weinberger, editors, Advances in Neural Information Processing Systems, volume 27. Curran Associates, Inc., 2014. URL https://proceedings.neurips.cc/paper_files/paper/2014/file/5ca3e9b122f61f8f06494c97b1afccf3-Paper.pdf.
  15. The Ethical Need for Watermarks in Machine-Generated Language. arxiv:2209.03118[cs], September 2022. doi: 10.48550/arXiv.2209.03118. URL http://arxiv.org/abs/2209.03118.
  16. Generating steganographic images via adversarial training. In Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017. URL https://papers.nips.cc/paper_files/paper/2017/hash/fe2d010308a6b3799a3d9c728ee74244-Abstract.html.
  17. Gans trained by a two time-scale update rule converge to a local nash equilibrium. In NIPS, 2017.
  18. Denoising Diffusion Probabilistic Models. In Advances in Neural Information Processing Systems, volume 33, pages 6840–6851. Curran Associates, Inc., 2020. URL https://proceedings.neurips.cc/paper/2020/hash/4c5bcfec8584af0d967f1ab10179ca4b-Abstract.html.
  19. A Watermark for Large Language Models. arxiv:2301.10226[cs], January 2023. doi: 10.48550/arXiv.2301.10226. URL http://arxiv.org/abs/2301.10226.
  20. Martin Kutter and Fabien A. P. Petitcolas. Fair benchmark for image watermarking systems. In Security and Watermarking of Multimedia Contents, volume 3657, pages 226–239. SPIE, April 1999. doi: 10.1117/12.344672. URL https://www.spiedigitallibrary.org/conference-proceedings-of-spie/3657/0000/Fair-benchmark-for-image-watermarking-systems/10.1117/12.344672.full.
  21. Watermarking digital image and video data. A state-of-the-art overview. IEEE Signal Processing Magazine, 17(5):20–46, September 2000. ISSN 1558-0792. doi: 10.1109/79.879337.
  22. Microsoft coco: Common objects in context. In European Conference on Computer Vision, 2014.
  23. An Optimized Image Watermarking Method Based on HD and SVD in DWT Domain. IEEE Access, 7:80849–80860, 2019. ISSN 2169-3536. doi: 10.1109/ACCESS.2019.2915596.
  24. Improved Denoising Diffusion Probabilistic Models. arxiv:2102.09672[cs, stat], February 2021. doi: 10.48550/arXiv.2102.09672. URL http://arxiv.org/abs/2102.09672.
  25. J.J.K. O’Ruanaidh and T. Pun. Rotation, scale and translation invariant digital image watermarking. In Proceedings of International Conference on Image Processing, volume 1, pages 536–539 vol.1, October 1997. doi: 10.1109/ICIP.1997.647968.
  26. P. B. Patnaik. The Non-Central X𝑋Xitalic_X2- and F-Distribution and their Applications. Biometrika, 36(1/2):202–232, 1949. ISSN 0006-3444. doi: 10.2307/2332542. URL https://www.jstor.org/stable/2332542.
  27. I. Pitas. A method for watermark casting on digital image. IEEE Transactions on Circuits and Systems for Video Technology, 8(6):775–780, October 1998. ISSN 1558-2205. doi: 10.1109/76.728421.
  28. Learning transferable visual models from natural language supervision. In International Conference on Machine Learning, 2021.
  29. High-Resolution Image Synthesis With Latent Diffusion Models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10684–10695, 2022. URL https://openaccess.thecvf.com/content/CVPR2022/html/Rombach_High-Resolution_Image_Synthesis_With_Latent_Diffusion_Models_CVPR_2022_paper.html.
  30. A robust image fingerprinting system using the Radon transform. Signal Processing: Image Communication, 19(4):325–339, April 2004. ISSN 0923-5965. doi: 10.1016/j.image.2003.12.001. URL https://www.sciencedirect.com/science/article/pii/S0923596503001541.
  31. V. Solachidis and L. Pitas. Circularly symmetric watermark embedding in 2-D DFT domain. IEEE Transactions on Image Processing, 10(11):1741–1753, November 2001. ISSN 1941-0042. doi: 10.1109/83.967401.
  32. Generative Modeling by Estimating Gradients of the Data Distribution. arXiv:1907.05600 [cs, stat], October 2019. URL http://arxiv.org/abs/1907.05600.
  33. Improved Techniques for Training Score-Based Generative Models. arXiv:2006.09011 [cs, stat], June 2020. URL http://arxiv.org/abs/2006.09011.
  34. Embedding Watermarks into Deep Neural Networks. In Proceedings of the 2017 ACM on International Conference on Multimedia Retrieval, pages 269–277, Bucharest Romania, June 2017. ACM. ISBN 978-1-4503-4701-3. doi: 10.1145/3078971.3078974. URL https://dl.acm.org/doi/10.1145/3078971.3078974.
  35. EDICT: Exact Diffusion Inversion via Coupled Transformations. arxiv:2211.12446[cs], December 2022. doi: 10.48550/arXiv.2211.12446. URL http://arxiv.org/abs/2211.12446.
  36. A comprehensive survey on robust image watermarking. Neurocomputing, 488:226–247, June 2022. ISSN 0925-2312. doi: 10.1016/j.neucom.2022.02.083. URL https://www.sciencedirect.com/science/article/pii/S0925231222002533.
  37. Artificial Fingerprinting for Generative Models: Rooting Deepfake Attribution in Training Data. arxiv:2007.08457[cs], March 2022. doi: 10.48550/arXiv.2007.08457. URL http://arxiv.org/abs/2007.08457.
  38. Securing deep generative models with universal adversarial signature. arXiv preprint arXiv:2305.16310, 2023.
  39. Protecting intellectual property of deep neural networks with watermarking. In ACM Symposium on Information, Computer and Communications Security. Association for Computing Machinery, Inc., May 2018. ISBN 978-1-4503-5576-6. doi: 10.1145/3196494.3196550. URL https://research.ibm.com/publications/protecting-intellectual-property-of-deep-neural-networks-with-watermarking.
  40. On the Robustness of Diffusion Inversion in Image Manipulation. In ICLR 2023 Workshop on Trustworthy and Reliable Large-Scale Machine Learning Models, April 2023. URL https://openreview.net/forum?id=fr8kurMWJIP.
  41. Robust Invisible Video Watermarking with Attention. arxiv:1909.01285[cs], September 2019. doi: 10.48550/arXiv.1909.01285. URL http://arxiv.org/abs/1909.01285.
  42. A Recipe for Watermarking Diffusion Models. arxiv:2303.10137[cs], March 2023. doi: 10.48550/arXiv.2303.10137. URL http://arxiv.org/abs/2303.10137.
  43. HiDDeN: Hiding Data with Deep Networks. In Proceedings of the European Conference on Computer Vision (ECCV), pages 657–672, 2018. URL https://openaccess.thecvf.com/content_ECCV_2018/html/Jiren_Zhu_HiDDeN_Hiding_Data_ECCV_2018_paper.html.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (4)
  1. Yuxin Wen (33 papers)
  2. John Kirchenbauer (21 papers)
  3. Jonas Geiping (73 papers)
  4. Tom Goldstein (226 papers)
Citations (68)
View on Semantic Scholar

Summary

  • The paper introduces a watermarking technique that embeds an imperceptible signal at the diffusion model's initial noise stage.
  • It employs Fourier space embedding to resist transformations like rotations, crops, and compression while preserving image quality.
  • Experiments on Stable Diffusion and ImageNet models demonstrate high detection efficacy and minimal impact on generative performance.

Overview of Tree-Ring Watermarking for Diffusion Images

The paper introduces "Tree-Ring Watermarks," a novel watermarking approach designed for diffusion models that generate images. The technique marks a departure from existing post-hoc image watermarking methods by integrating an imperceptible watermark during the image generation process. This method not only ensures the watermark's invisibility to human observers but also maintains its robustness under various image manipulations.

Methodology and Key Techniques

The core concept of Tree-Ring Watermarking involves embedding a watermark at the initiation of the diffusion process, specifically into the initial noise vector. By injecting a structured pattern in the Fourier space of this noise vector, the watermark achieves invariance to typical image alterations such as crops, rotations, and color adjustments. Detection of the watermark entails inverting the diffusion process to derive the original noise vector, which is analyzed for the watermark signal.

Key techniques central to the watermarking approach include:

  • Fourier Space Embedding: The watermark is embedded in the low-frequency modes of the Fourier transform of the initial noise. This choice leverages the Fourier properties that preserve watermark integrity under spatial transformations.
  • No Training Required: The method is designed to be incorporated into existing diffusion model APIs without additional training, thus enabling straightforward integration.
  • Model Owner Verification: Only parties with access to the generative model can verify the presence of the watermark, enhancing protection against unauthorized usage.

Experimental Evaluation

In evaluating the methodology, the authors focus on two models: Stable Diffusion and an ImageNet-based diffusion model. Their experiments cover benign and adversarial settings, demonstrating strong performance in both. Key metrics include:

  • Detection Efficacy: The approach records high AUC values and true positive rates at low false positive rates in clean conditions. Even under adversarial settings, the watermark remains detectably robust.
  • Minimal Impact on Image Quality: Tree-Ring Watermarking exhibits negligible effects on image quality, as substantiated by metrics such as the Frechet Inception Distance (FID) and CLIP score. This ensures the generative capabilities of the models remain uncompromised.
  • Robustness Against Transformations: The watermark demonstrates resilience to a suite of common modifications, such as rotations, JPEG compression, and Gaussian noise, underscoring its practicality in diverse application contexts.

Implications and Future Prospects

The proposed Tree-Ring Watermarking approach offers significant implications for the secure, traceable use of diffusion models in content creation. By integrating watermarking directly into the generation process, it mitigates risks associated with post-generation image manipulation and enhances accountability in the provenance of AI-generated images.

Future research could expand this methodology to accommodate alternative sampling techniques beyond DDIM, and explore the potential for assigning unique keys to various entities within the generative framework. Moreover, advancements in inverse diffusion processes may further strengthen watermarking fidelity and application.

This watermarking method positions itself as a critical tool in addressing ethical and copyright concerns associated with AI-generated content, ensuring alignment with broader societal standards for transparency and authenticity in digital media.

File Document Download Save Streamline Icon: https://streamlinehq.com PDF File Document Download Save Streamline Icon: https://streamlinehq.com Markdown
Github Logo Streamline Icon: https://streamlinehq.com

GitHub

  1. GitHub - YuxinWenRick/tree-ring-watermark (307 stars)
Youtube Logo Streamline Icon: https://streamlinehq.com

YouTube