Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
102 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

CGI-DM: Digital Copyright Authentication for Diffusion Models via Contrasting Gradient Inversion (2403.11162v1)

Published 17 Mar 2024 in cs.CV, cs.AI, cs.CR, cs.CY, and cs.LG

Abstract: Diffusion Models (DMs) have evolved into advanced image generation tools, especially for few-shot generation where a pretrained model is fine-tuned on a small set of images to capture a specific style or object. Despite their success, concerns exist about potential copyright violations stemming from the use of unauthorized data in this process. In response, we present Contrasting Gradient Inversion for Diffusion Models (CGI-DM), a novel method featuring vivid visual representations for digital copyright authentication. Our approach involves removing partial information of an image and recovering missing details by exploiting conceptual differences between the pretrained and fine-tuned models. We formulate the differences as KL divergence between latent variables of the two models when given the same input image, which can be maximized through Monte Carlo sampling and Projected Gradient Descent (PGD). The similarity between original and recovered images serves as a strong indicator of potential infringements. Extensive experiments on the WikiArt and Dreambooth datasets demonstrate the high accuracy of CGI-DM in digital copyright authentication, surpassing alternative validation techniques. Code implementation is available at https://github.com/Nicholas0228/Revelio.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (40)
  1. Extracting Training Data from Diffusion Models. In USENIX Security, 2023.
  2. Emerging Properties in Self-supervised Vision Transformers. In ICCV, 2021.
  3. Knowledge-enriched Distributional Model Inversion Attacks. In ICCV, 2021.
  4. Randaugment: Practical Automated Data Augmentation with a Reduced Search Space. In CVPR, 2020.
  5. DiffusionShield: A Watermark for Copyright Protection against Generative Diffusion Models. arXiv preprint arXiv:2306.04642, 2023.
  6. Andrew Deck. AI-Generated Art Sparks Furious Backlash from Japan’s Anime Community. https://restofworld.org/2022/ai-backlash-anime-artists/, 2022.
  7. Improved Regularization of Convolutional Neural Networks with Cutout. arXiv preprint arXiv:1708.04552, 2017.
  8. Are Diffusion Models Vulnerable to Membership Inference Attacks? arXiv preprint arXiv:2302.01316, 2023.
  9. Privacy in Pharmacogenetics: An End-to-End Case Study of Personalized Warfarin dosing. In USENIX Security, 2014.
  10. Do Gradient Inversion Attacks make Federated Learning Unsafe? IEEE Transactions on Medical Imaging, 42(7):2044–2056, 2023.
  11. Clipscore: A Reference-free Evaluation Metric for Image Captioning. arXiv preprint arXiv:2104.08718, 2021.
  12. Denoising Diffusion Probabilistic Models. In NeurIPS, 2020.
  13. Lora: Low-rank Adaptation of Large Language Models. arXiv preprint arXiv:2106.09685, 2021.
  14. Imagic: Text-Based Real Image Editing With Diffusion Models. arXiv preprint arXiv:2210.09276, 2022.
  15. Diffusionclip: Text-guided Diffusion Models for Robust Image Manipulation. In CVPR, 2022.
  16. An Efficient Membership Inference Attack for the Diffusion Model by Proximal Initialization. arXiv preprint arXiv:2305.18355, 2023.
  17. Adversarial Example Does Good: Preventing Painting Imitation from Diffusion Models via Adversarial Examples. In ICML, 2023.
  18. Towards Deep Learning Models Resistant to Adversarial Attacks. In ICLR, 2018.
  19. Deborah MT. How AI Art Can Free Artists, Not Replace Them. https://medium.com/thesequence/how-ai-art-can-free-artists-not-replace-them-a23a5cb0461e, 2022.
  20. Re-thinking Model Inversion Attacks Against Deep Neural Networks. In CVPR, 2023.
  21. K. Nichol. Painter by Numbers, WikiArt. https://www.kaggle.com/c/painter-by-numbers, 2016.
  22. White-box Membership Inference Attacks against Diffusion Models. arXiv preprint arXiv:2308.06405, 2023.
  23. High-Resolution Image Synthesis with Latent Diffusion Models. In CVPR, 2022.
  24. Dreambooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation. In CVPR, 2023.
  25. Glaze: Protecting artists from style mimicry by text-to-image models. arXiv preprint arXiv:2302.04222, 2023.
  26. Membership Inference Attacks against Machine Learning Models. In S&P, 2017.
  27. Understanding and Mitigating Copying in Diffusion Models. arXiv preprint arXiv:2305.20086, 2023.
  28. Denoising Diffusion Implicit Models. In ICLR, 2020.
  29. Score-Based Generative Modeling Through Stochastic Differential Equations. arXiv preprint arXiv:2011.13456, 2020.
  30. Anti-DreamBooth: Protecting Users from Personalized Text-to-image Synthesis. In ICCV, 2023.
  31. James Vincent. The Scary Truth About AI Copyright Is Nobody Knows What Will Happen Next. https://www.theverge.com/23444685/generative-ai-copyright-infringement-legal-fair-use-training-data, 2022.
  32. Security and Privacy on Generative Data in AIGC: A Survey. arXiv preprint arXiv:2309.09435, 2023.
  33. Zero-Shot Image Restoration Using Denoising Diffusion Null-Space Model. In ICLR, 2022.
  34. Tree-Ring Watermarks: Fingerprints for Diffusion Images that are Invisible and Robust. arXiv preprint arXiv:2305.20030, 2023.
  35. Diffusion Probabilistic Modeling for Video Generation. arXiv preprint arXiv:2203.09481, 2022.
  36. AltDiffusion: A Multilingual Text-to-Image Diffusion Model. arXiv preprint arXiv:2308.09991, 2023.
  37. See Through Gradients: Image Batch Recovery via Gradinversion. In CVPR, 2021.
  38. The Secret Revealer: Generative Model-inversion Attacks against Deep Neural Networks. In CVPR, 2020.
  39. Generative Autoencoders as Watermark Attackers: Analyses of Vulnerabilities and Threats. arXiv preprint arXiv:2306.01953, 2023.
  40. A Recipe for Watermarking Diffusion Models. arXiv preprint arXiv:2303.10137, 2023.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (7)
  1. Xiaoyu Wu (43 papers)
  2. Yang Hua (43 papers)
  3. Chumeng Liang (10 papers)
  4. Jiaru Zhang (8 papers)
  5. Hao Wang (1120 papers)
  6. Tao Song (50 papers)
  7. Haibing Guan (24 papers)
Citations (4)
View on Semantic Scholar
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets