Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
169 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Generative Models are Self-Watermarked: Declaring Model Authentication through Re-Generation (2402.16889v1)

Published 23 Feb 2024 in cs.LG, cs.AI, and cs.CR

Abstract: As machine- and AI-generated content proliferates, protecting the intellectual property of generative models has become imperative, yet verifying data ownership poses formidable challenges, particularly in cases of unauthorized reuse of generated data. The challenge of verifying data ownership is further amplified by using Machine Learning as a Service (MLaaS), which often functions as a black-box system. Our work is dedicated to detecting data reuse from even an individual sample. Traditionally, watermarking has been leveraged to detect AI-generated content. However, unlike watermarking techniques that embed additional information as triggers into models or generated content, potentially compromising output quality, our approach identifies latent fingerprints inherently present within the outputs through re-generation. We propose an explainable verification procedure that attributes data ownership through re-generation, and further amplifies these fingerprints in the generative models through iterative data re-generation. This methodology is theoretically grounded and demonstrates viability and robustness using recent advanced text and image generative models. Our methodology is significant as it goes beyond protecting the intellectual property of APIs and addresses important issues such as the spread of misinformation and academic misconduct. It provides a useful tool to ensure the integrity of sources and authorship, expanding its application in different scenarios where authenticity and ownership verification are essential.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (70)
  1. Mesonet: a compact facial video forgery detection network. In 2018 IEEE international workshop on information forensics and security (WIFS), pp.  1–7. IEEE, 2018.
  2. Redmark: Framework for residual diffusion watermarking based on deep networks. Expert Systems with Applications, 146:113157, 2020.
  3. Banach, S. Sur les opérations dans les ensembles abstraits et leur application aux équations intégrales. Fundamenta mathematicae, 3(1):133–181, 1922.
  4. The malicious use of artificial intelligence: Forecasting, prevention, and mitigation. arXiv preprint arXiv:1802.07228, 2018.
  5. Emerging properties in self-supervised vision transformers. In Proceedings of the IEEE/CVF international conference on computer vision, pp.  9650–9660, 2021.
  6. Practical Linguistic Steganography using Contextual Synonym Substitution and a Novel Vertex Coding Method. Computational Linguistics, 40(2):403–448, 06 2014. ISSN 0891-2017. doi: 10.1162/COLI_a_00176. URL https://doi.org/10.1162/COLI_a_00176.
  7. Ciesielski, K. On stefan banach and some of his results. Banach Journal of Mathematical Analysis, 1(1):1–10, 2007.
  8. Digital Watermarking and Steganography. The Morgan Kaufmann Series in Multimedia Information and Systems, 2 edition, 2008.
  9. Watch your up-convolution: Cnn based generative deep neural networks are failing to reproduce spectral distributions. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp.  7890–7899, 2020.
  10. Beyond english-centric multilingual machine translation. The Journal of Machine Learning Research, 22(1):4839–4886, 2021.
  11. Watermarking images in self-supervised latent spaces. In ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp.  3054–3058, 2022. doi: 10.1109/ICASSP43922.2022.9746058.
  12. The stable signature: Rooting watermarks in latent diffusion models. arXiv preprint arXiv:2303.15435, 2023.
  13. Gaspari, F. Look who’s translating. impersonations, Chinese whispers and fun with machine translation on the Internet. In Hansen, V. and Maegaard, B. (eds.), Proceedings of the 11th Annual Conference of the European Association for Machine Translation, Oslo, Norway, June 19–20 2006. European Association for Machine Translation. URL https://aclanthology.org/2006.eamt-1.19.
  14. Fixed Point Theory, volume 14. Springer, 2003.
  15. Deepfake video detection using recurrent neural networks. In 2018 15th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), pp.  1–6, 2018. doi: 10.1109/AVSS.2018.8639163.
  16. Harris, D. A. Deepfakes: False pornography is here and the law cannot protect you. Duke law and technology review, 17:99–127, 2019. URL https://api.semanticscholar.org/CorpusID:158596286.
  17. Hazell, J. Large language models can be used to effectively scale spear phishing campaigns. arXiv preprint arXiv:2305.06972, 2023.
  18. Protecting intellectual property of language generation apis with lexical watermark. Proceedings of the AAAI Conference on Artificial Intelligence, 36(10):10758–10766, Jun. 2022a. doi: 10.1609/aaai.v36i10.21321. URL https://ojs.aaai.org/index.php/AAAI/article/view/21321.
  19. CATER: Intellectual property protection on text generation APIs via conditional watermarks. In Oh, A. H., Agarwal, A., Belgrave, D., and Cho, K. (eds.), Advances in Neural Information Processing Systems, 2022b. URL https://openreview.net/forum?id=L7P3IvsoUXY.
  20. Hernandez, J. That panicky call from a relative? it could be a thief using a voice clone, ftc warns. NPR, 2023. URL https://www.npr.org/2023/03/22/1165448073/voice-clones-ai-scams-ftc.
  21. Image quality metrics: Psnr vs. ssim. In 2010 20th international conference on pattern recognition, pp.  2366–2369. IEEE, 2010.
  22. Automatic detection of generated text is easiest when humans are fooled. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp.  1808–1822, Online, July 2020. Association for Computational Linguistics. doi: 10.18653/v1/2020.acl-main.164. URL https://aclanthology.org/2020.acl-main.164.
  23. Mistral 7b. arXiv preprint arXiv:2310.06825, 2023a.
  24. Evading watermark based detection of ai-generated content. In Proceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security, CCS ’23, pp.  1168–1181, New York, NY, USA, 2023b. Association for Computing Machinery. ISBN 9798400700507. doi: 10.1145/3576915.3623189. URL https://doi.org/10.1145/3576915.3623189.
  25. Information hiding techniques for steganography and digital watermaking. EDPACS, 28(6):1–2, 2000. doi: 10.1201/1079/43263.28.6.20001201/30373.5. URL https://doi.org/10.1201/1079/43263.28.6.20001201/30373.5.
  26. On architectural compression of text-to-image diffusion models. arXiv preprint arXiv:2305.15798, 2023.
  27. A watermark for large language models. International Conference on Machine Learning, 2023.
  28. Robust distortion-free watermarks for language models. arXiv preprint arXiv:2307.15593, 2023.
  29. Who wrote this code? watermarking for code generation. arXiv preprint arXiv:2305.15060, 2023.
  30. Lin, C.-Y. ROUGE: A package for automatic evaluation of summaries. In Text Summarization Branches Out, pp.  74–81, Barcelona, Spain, July 2004. Association for Computational Linguistics. URL https://aclanthology.org/W04-1013.
  31. Microsoft coco: Common objects in context. In Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13, pp.  740–755. Springer, 2014.
  32. Global texture enhancement for fake face detection in the wild. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp.  8060–8069, 2020.
  33. Chatgpt and a new academic reality: Artificial intelligence-written research papers and the ethics of the large language models in scholarly publishing. Journal of the Association for Information Science and Technology, 74(5):570–581, 2023.
  34. Data contamination: From memorization to exploitation. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pp.  157–165, 2022.
  35. Do gans leave artificial fingerprints? In 2019 IEEE Conference on Multimedia Information Processing and Retrieval (MIPR), pp.  506–511, 2019. doi: 10.1109/MIPR.2019.00103.
  36. Use of llms for illicit purposes: Threats, prevention measures, and vulnerabilities. arXiv preprint arXiv:2308.12833, 2023.
  37. Pantserev, K. A. The Malicious Use of AI-Based Deepfake Technology as the New Threat to Psychological Security and Political Stability, pp.  37–55. Springer International Publishing, Cham, 2020. ISBN 978-3-030-35746-7. doi: 10.1007/978-3-030-35746-7_3. URL https://doi.org/10.1007/978-3-030-35746-7_3.
  38. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pp.  311–318, Philadelphia, Pennsylvania, USA, July 2002. Association for Computational Linguistics. doi: 10.3115/1073083.1073135. URL https://aclanthology.org/P02-1040.
  39. Sdxl: Improving latent diffusion models for high-resolution image synthesis. arXiv preprint arXiv:2307.01952, 2023.
  40. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp.  8748–8763. PMLR, 2021.
  41. Hierarchical text-conditional image generation with clip latents.
  42. Zero-shot text-to-image generation. In International Conference on Machine Learning, pp.  8821–8831. PMLR, 2021.
  43. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp.  10684–10695, June 2022.
  44. Ethics and creativity in computer vision. arXiv preprint arXiv:2112.03111, 2021.
  45. Laion-5b: An open large-scale dataset for training next generation image-text models. Advances in Neural Information Processing Systems, 35:25278–25294, 2022.
  46. Sjouwerman, S. How ai is changing social engineering forever. Forbes, 2023. URL https://www.forbes.com/sites/forbestechcouncil/2023/05/26/how-ai-is-changing-social-engineering-forever/?sh=2031e2c0321b.
  47. Release strategies and the social impacts of language models. CoRR, abs/1908.09203, 2019. URL http://arxiv.org/abs/1908.09203.
  48. Multilingual translation from denoising pre-training. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pp.  3450–3466, Online, August 2021. Association for Computational Linguistics. doi: 10.18653/v1/2021.findings-acl.304. URL https://aclanthology.org/2021.findings-acl.304.
  49. The hiding virtues of ambiguity: Quantifiably resilient watermarking of natural language text through synonym substitutions. In Proceedings of the 8th Workshop on Multimedia and Security, MM&Sec ’06, pp.  164–174, New York, NY, USA, 2006. Association for Computing Machinery. ISBN 1595934936. doi: 10.1145/1161366.1161397. URL https://doi.org/10.1145/1161366.1161397.
  50. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288, 2023.
  51. Watermarking the outputs of structured prediction with an application in statistical machine translation. In Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing, pp.  1363–1372, Edinburgh, Scotland, UK., July 2011. Association for Computational Linguistics. URL https://aclanthology.org/D11-1126.
  52. Verma, P. They thought loved ones were calling for help. it was an ai scam. The Washington Post, 2023. URL https://www.washingtonpost.com/technology/2023/03/05/ai-voice-scam/.
  53. Diffusers: State-of-the-art diffusion models. https://github.com/huggingface/diffusers, 2022.
  54. Imitation attacks and defenses for black-box machine translation systems. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp.  5531–5546, Online, November 2020. Association for Computational Linguistics. doi: 10.18653/v1/2020.emnlp-main.446. URL https://aclanthology.org/2020.emnlp-main.446.
  55. Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing, 13(4):600–612, 2004.
  56. Diffusiondb: A large-scale prompt gallery dataset for text-to-image generative models. arXiv preprint arXiv:2210.14896, 2022.
  57. Security challenges in natural language processing models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing: Tutorial Abstracts, pp.  7–12, 2023.
  58. Student surpasses teacher: Imitation attack for black-box NLP APIs. In Proceedings of the 29th International Conference on Computational Linguistics, pp.  2849–2860, Gyeongju, Republic of Korea, October 2022. International Committee on Computational Linguistics. URL https://aclanthology.org/2022.coling-1.251.
  59. Provably secure generative steganography based on autoregressive model. In Yoo, C. D., Shi, Y.-Q., Kim, H. J., Piva, A., and Kim, G. (eds.), Digital Forensics and Watermarking, pp.  55–68, Cham, 2019. Springer International Publishing. ISBN 978-3-030-11389-6.
  60. Robust multi-bit natural language watermarking through invariant features. In Rogers, A., Boyd-Graber, J., and Okazaki, N. (eds.), Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp.  2092–2115, Toronto, Canada, July 2023. Association for Computational Linguistics. doi: 10.18653/v1/2023.acl-long.117. URL https://aclanthology.org/2023.acl-long.117.
  61. A siamese cnn for image steganalysis. IEEE Transactions on Information Forensics and Security, 16:291–306, 2020.
  62. Attributing fake images to gans: Learning and analyzing gan fingerprints. In Proceedings of the IEEE/CVF international conference on computer vision, pp.  7556–7566, 2019.
  63. A proposed secure multiple watermarking technique based on dwt, dct and svd for application in medicine. Multimedia tools and applications, 77:4863–4882, 2018.
  64. Zhang, R. richzhang/perceptualsimilarity, Sep 2023. URL https://github.com/richzhang/PerceptualSimilarity.
  65. The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp.  586–595, 2018.
  66. Bertscore: Evaluating text generation with bert. In International Conference on Learning Representations, 2020. URL https://openreview.net/forum?id=SkeHuCVFDr.
  67. Detecting and simulating artifacts in gan fake images. In 2019 IEEE International Workshop on Information Forensics and Security (WIFS), pp.  1–6, 2019. doi: 10.1109/WIFS47025.2019.9035107.
  68. Protecting language generation models via invisible watermarking. arXiv preprint arXiv:2302.03162, 2023a.
  69. A recipe for watermarking diffusion models. arXiv preprint arXiv:2303.10137, 2023b.
  70. Hidden: Hiding data with deep networks. In Proceedings of the European conference on computer vision (ECCV), pp.  657–672, 2018.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now