Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
119 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

A Dataset and Benchmark for Copyright Infringement Unlearning from Text-to-Image Diffusion Models (2403.12052v3)

Published 4 Jan 2024 in cs.CV

Abstract: Copyright law confers upon creators the exclusive rights to reproduce, distribute, and monetize their creative works. However, recent progress in text-to-image generation has introduced formidable challenges to copyright enforcement. These technologies enable the unauthorized learning and replication of copyrighted content, artistic creations, and likenesses, leading to the proliferation of unregulated content. Notably, models like stable diffusion, which excel in text-to-image synthesis, heighten the risk of copyright infringement and unauthorized distribution.Machine unlearning, which seeks to eradicate the influence of specific data or concepts from machine learning models, emerges as a promising solution by eliminating the \enquote{copyright memories} ingrained in diffusion models. Yet, the absence of comprehensive large-scale datasets and standardized benchmarks for evaluating the efficacy of unlearning techniques in the copyright protection scenarios impedes the development of more effective unlearning methods. To address this gap, we introduce a novel pipeline that harmonizes CLIP, ChatGPT, and diffusion models to curate a dataset. This dataset encompasses anchor images, associated prompts, and images synthesized by text-to-image models. Additionally, we have developed a mixed metric based on semantic and style information, validated through both human and artist assessments, to gauge the effectiveness of unlearning approaches. Our dataset, benchmark library, and evaluation metrics will be made publicly available to foster future research and practical applications (https://rmpku.github.io/CPDM-page/, website / http://149.104.22.83/unlearning.tar.gz, dataset).

Definition Search Book Streamline Icon: https://streamlinehq.com
References (44)
  1. Blended diffusion for text-driven editing of natural images. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2022.
  2. ediff-i: Text-to-image diffusion models with an ensemble of expert denoisers, 2023.
  3. Machine unlearning. In 2021 IEEE Symposium on Security and Privacy (SP), pages 141–159. IEEE, 2021.
  4. Extracting training data from diffusion models. arXiv preprint arXiv:2301.13188, 2023.
  5. When machine unlearning jeopardizes privacy. In Proceedings of the 2021 ACM SIGSAC conference on computer and communications security, pages 896–911, 2021.
  6. Diffedit: Diffusion-based semantic image editing with mask guidance. In The Eleventh International Conference on Learning Representations, 2023.
  7. The fréchet distance between multivariate normal distributions. Journal of multivariate analysis, 12(3):450–455, 1982.
  8. The lottery ticket hypothesis: Finding sparse, trainable neural networks. International Conference On Learning Representations, 2018.
  9. Erasing concepts from diffusion models. arXiv preprint arXiv:2303.07345, 2023.
  10. Implicit diffusion models for continuous super-resolution, 2023.
  11. A neural algorithm of artistic style, 2015.
  12. Making ai forget you: Data deletion in machine learning. Advances in neural information processing systems, 32, 2019.
  13. Pair-diffusion: Object-level image editing with structure-and-appearance paired diffusion models, 2023.
  14. Generative adversarial networks. Communications of the ACM, 63(11):139–144, 2020.
  15. Amnesiac machine learning. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 11516–11524, 2021.
  16. Certified data removal from machine learning models. arXiv preprint arXiv:1911.03030, 2019.
  17. Deep compression: Compressing deep neural network with pruning, trained quantization and huffman coding. arXiv: Computer Vision and Pattern Recognition, 2015a.
  18. Learning both weights and connections for efficient neural networks. NEURIPS, 2015b.
  19. Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in neural information processing systems, 30, 2017.
  20. Gans trained by a two time-scale update rule converge to a local nash equilibrium, 2018.
  21. Denoising diffusion probabilistic models, 2020.
  22. Unlearnable examples: Making personal data unexploitable. arXiv preprint arXiv:2101.04898, 2021.
  23. Recognizing image style. In Proceedings of the British Machine Vision Conference. BMVA Press, 2014.
  24. Imagic: Text-based real image editing with diffusion models, 2023.
  25. Srdiff: Single image super-resolution with diffusion probabilistic models, 2021.
  26. The artbench dataset: Benchmarking generative models with artworks. arXiv preprint arXiv:2206.11404, 2022.
  27. Microsoft coco: Common objects in context. In Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13, pages 740–755. Springer, 2014.
  28. Rethinking the value of network pruning. International Conference On Learning Representations, 2018.
  29. A survey of machine unlearning. arXiv preprint arXiv:2209.02299, 2022.
  30. Glide: Towards photorealistic image generation and editing with text-guided diffusion models, 2022.
  31. Learning transferable visual models from natural language supervision, 2021.
  32. Hierarchical text-conditional image generation with clip latents, 2022.
  33. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10684–10695, 2022a.
  34. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2022b.
  35. Photorealistic text-to-image diffusion models with deep language understanding, 2022.
  36. Laion-5b: An open large-scale dataset for training next generation image-text models. arXiv preprint arXiv:2210.08402, 2022.
  37. Image information and visual quality. IEEE Transactions on image processing, 15(2):430–444, 2006.
  38. Diffusion art or digital forgery? investigating data replication in diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6048–6058, 2023.
  39. Rethinking the inception architecture for computer vision, 2015.
  40. Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing, 13(4):600–612, 2004.
  41. Diffusion model for generative image denoising, 2023.
  42. Forget-me-not: Learning to forget in text-to-image diffusion models. arXiv preprint arXiv:2303.17591, 2023a.
  43. A review on machine unlearning. SN Computer Science, 4(4):337, 2023b.
  44. The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 586–595, 2018.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (14)
  1. Rui Ma (112 papers)
  2. Qiang Zhou (124 papers)
  3. Bangjun Xiao (2 papers)
  4. Yizhu Jin (4 papers)
  5. Daquan Zhou (47 papers)
  6. Xiuyu Li (24 papers)
  7. Aishani Singh (2 papers)
  8. Yi Qu (6 papers)
  9. Kurt Keutzer (200 papers)
  10. Xiaodong Xie (23 papers)
  11. Jingtong Hu (51 papers)
  12. Zhen Dong (87 papers)
  13. Shanghang Zhang (173 papers)
  14. Shiji Zhou (13 papers)
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now