Papers
Topics
Authors
Recent
Search
2000 character limit reached

Watermarking Makes Language Models Radioactive

Published 22 Feb 2024 in cs.CR, cs.AI, cs.CL, and cs.LG | (2402.14904v2)

Abstract: We investigate the radioactivity of text generated by LLMs (LLM), i.e. whether it is possible to detect that such synthetic input was used to train a subsequent LLM. Current methods like membership inference or active IP protection either work only in settings where the suspected text is known or do not provide reliable statistical guarantees. We discover that, on the contrary, it is possible to reliably determine if a LLM was trained on synthetic data if that data is output by a watermarked LLM. Our new methods, specialized for radioactivity, detects with a provable confidence weak residuals of the watermark signal in the fine-tuned LLM. We link the radioactivity contamination level to the following properties: the watermark robustness, its proportion in the training set, and the fine-tuning process. For instance, if the suspect model is open-weight, we demonstrate that training on watermarked instructions can be detected with high confidence ($p$-value $< 10{-5}$) even when as little as $5\%$ of training text is watermarked.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (61)
  1. Watermarking GPT outputs, 2023. https://scottaaronson.blog/?m=202302.
  2. The secret sharer: Evaluating and testing unintended memorization in neural networks, 2019.
  3. Extracting training data from large language models, 2021.
  4. Membership inference attacks from first principles. In 2022 IEEE Symposium on Security and Privacy (SP), pages 1897–1914. IEEE, 2022.
  5. Mark Chen et al. Evaluating large language models trained on code. arXiv, 2021.
  6. Undetectable watermarks for language models. Cryptology ePrint Archive, 2023.
  7. Scaling instruction-finetuned language models. arXiv preprint arXiv:2210.11416, 2022.
  8. Karl Cobbe et al. Training verifiers to solve math word problems. arXiv, 2021.
  9. Digital watermarking and steganography. Morgan kaufmann, 2007.
  10. Machine generated text: A comprehensive survey of threat models and detection methods. arXiv preprint arXiv:2210.07321, 2022.
  11. Qlora: Efficient finetuning of quantized llms. arXiv preprint arXiv:2305.14314, 2023.
  12. Hierarchical neural story generation. arXiv preprint arXiv:1805.04833, 2018.
  13. Three bricks to consolidate watermarks for large language models. 2023 IEEE International Workshop on Information Forensics and Security (WIFS), 2023.
  14. Team Gemini. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805, 2023.
  15. Time travel in llms: Tracing data contamination in large language models. arXiv preprint arXiv:2308.08493, 2023.
  16. On the learnability of watermarks for language models. arXiv preprint arXiv:2312.04469, 2023.
  17. Textbooks are all you need. arXiv preprint arXiv:2306.11644, 2023.
  18. Protecting intellectual property of language generation apis with lexical watermark. In AAAI, 2022a.
  19. CATER: Intellectual property protection on text generation APIs via conditional watermarks. In NeurIPS, 2022b.
  20. Measuring massive multitask language understanding. arXiv preprint arXiv:2009.03300, 2020.
  21. Membership inference attacks on sequence-to-sequence models: Is my data in your machine translation system? Transactions of the Association for Computational Linguistics, 8:49–63, 2020.
  22. The curious case of neural text degeneration. arXiv preprint arXiv:1904.09751, 2019.
  23. Unnatural instructions: Tuning language models with (almost) no human labor. arXiv preprint arXiv:2212.09689, 2022.
  24. Towards optimal statistical watermarking, 2023.
  25. Triviaqa: A large scale distantly supervised challenge dataset for reading comprehension. arXiv, 2017.
  26. Adam: A method for stochastic optimization, 2017.
  27. A watermark for large language models. arXiv preprint arXiv:2301.10226, 2023a.
  28. On the reliability of watermarks for large language models, 2023b.
  29. Robust distortion-free watermarks for language models. arXiv preprint arXiv:2307.15593, 2023.
  30. Tom Kwiatkowski et al. Natural questions: a benchmark for question answering research. Trans. of the ACL, 7, 2019.
  31. Protecting intellectual property of large language model-based code generation apis via watermarks. In Proceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security, pages 2336–2350, 2023.
  32. SGDR: Stochastic gradient descent with warm restarts. In International Conference on Learning Representations, 2017.
  33. Membership inference on word embedding and beyond. arXiv preprint arXiv:2106.11384, 2021.
  34. Frank J Massey. The kolmogorov-smirnov test for goodness of fit. Journal of the American Statistical Association, 46(253):68–78, 1951.
  35. Orca: Progressive learning from complex explanation traces of gpt-4. arXiv preprint arXiv:2306.02707, 2023.
  36. Comprehensive privacy analysis of deep learning: Passive and active white-box inference attacks against centralized and federated learning. In 2019 IEEE symposium on security and privacy (SP), pages 739–753. IEEE, 2019.
  37. Scalable extraction of training data from (production) language models, 2023.
  38. OpenAI. Gpt-4 technical report. arXiv, 2023.
  39. Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems, 35:27730–27744, 2022.
  40. Instruction tuning with gpt-4. arXiv preprint arXiv:2304.03277, 2023a.
  41. Are you copying my model? protecting the copyright of large language models for eaas via backdoor watermark. arXiv preprint arXiv:2305.10036, 2023b.
  42. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9, 2019.
  43. Greg Roelofs. zlib: A massively spiffy yet delicately unobtrusive compression library. http://www. zlib. net/, 2017.
  44. D\\\backslash\’ej\\\backslash\a vu: an empirical evaluation of the memorization properties of convnets. arXiv preprint arXiv:1809.06396, 2018.
  45. White-box vs black-box: Bayes optimal strategies for membership inference. In International Conference on Machine Learning, pages 5558–5567. PMLR, 2019.
  46. Radioactive data: tracing through training. In International Conference on Machine Learning, pages 8326–8335. PMLR, 2020.
  47. Detecting pretraining data from large language models. arXiv preprint arXiv:2310.16789, 2023.
  48. Membership inference attacks against machine learning models, 2017a.
  49. Membership inference attacks against machine learning models. In 2017 IEEE symposium on security and privacy (SP), pages 3–18. IEEE, 2017b.
  50. Stanford Alpaca: An instruction-following LLaMA model, 2023.
  51. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971, 2023a.
  52. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288, 2023b.
  53. Artificial artificial artificial intelligence: Crowd workers widely use large language models for text production tasks, 2023.
  54. Imitation attacks and defenses for black-box machine translation systems. In EMNLP, 2020.
  55. Self-instruct: Aligning language model with self generated instructions. arXiv preprint arXiv:2212.10560, 2022.
  56. On the importance of difficulty calibration in membership inference attacks. arXiv preprint arXiv:2111.08440, 2021.
  57. Finetuned language models are zero-shot learners. In International Conference on Learning Representations, 2022.
  58. Taxonomy of risks posed by language models. In 2022 ACM Conference on Fairness, Accountability, and Transparency, pages 214–229, 2022.
  59. Baize: An open-source chat model with parameter-efficient tuning on self-chat data. arXiv preprint arXiv:2304.01196, 2023.
  60. Privacy risk in machine learning: Analyzing the connection to overfitting. In 2018 IEEE 31st computer security foundations symposium (CSF), pages 268–282. IEEE, 2018.
  61. Protecting language generation models via invisible watermarking. arXiv preprint arXiv:2302.03162, 2023.
Citations (4)
View on Semantic Scholar

Summary

  • The paper reveals that watermarked text leaves detectable traces in language models—termed 'radioactivity'—when used during fine-tuning.
  • It extends membership inference attacks by comparing traditional methods with new techniques designed to detect watermarked inputs.
  • The approach offers a novel data governance tool, enabling precise auditing of training data origins and addressing copyright concerns.

Exploring LLM Radioactivity through Watermarking

Introduction

Recent advancements in LLM (LM) development have significantly focused on the fine-tuning process to enhance performance and generalization capabilities. This paper investigates the concept of "radioactivity" in LLMs, particularly in the context of using watermarked text for fine-tuning. "Radioactivity" in this sense refers to the detectable traces left in a model when trained on watermarked data. This study extends the conventional framework for Membership Inference Attacks (MIA) by examining the distinguishability and implications of training on watermarked text. The research contributes to understanding the effects of embedding watermarks in LLM-generated texts, highlighting the potential of watermarked text to serve not only as a method for content authentication but also as a means to detect and analyze the lineage of model training data.

Radioactivity of Watermarked Text

Fine-tuning LLMs often involves training on synthetic data generated by pretrained models. This process raises questions about copyright, data ownership, and the provenance of model-generated content. Watermarking, a method of embedding identifiable information into content, is explored here as a tool to trace whether specific content has been used in the fine-tuning of LLMs. The study highlights how watermarked text, when used in fine-tuning, introduces a specific form of "radioactivity" detectable in subsequent model outputs.

Detection Methods and Implications

The paper presents methods for detecting this radioactivity, comparing conventional MIA techniques against those designed for identifying watermarked texts. Remarkably, the detection of watermarked text can be achieved with high confidence even when a minimal proportion (as low as 5%) of the training data is watermarked. These findings underline the sensitivity of LLMs to watermarked inputs and the potential of watermarking as a technique for auditing and tracking the use of specific data within model training pipelines.

Theoretical and Practical Contributions

From a theoretical standpoint, the concept of radioactivity in LLMs broadens our understanding of how data characteristics transfer and persist through the training process. Practically, it introduces watermarking as a powerful tool for data governance, specifically in the management and tracking of data usage across model training endeavors. The research proposes a systemic way to approach model provenance issues, catering to the growing concerns around data privacy, content authenticity, and copyright in generative AI.

Future Directions

This paper opens several avenues for further research. Future studies could explore the resistance of different watermarking techniques to various forms of model fine-tuning and adaptation, examining how the robustness of watermarks might be optimized to ensure persistent radioactivity. Additionally, investigating the ethical and legal implications of deploying watermarked data in model training could provide valuable insights into the responsible use of generative AI technologies. Lastly, extending the concept of radioactivity to other domains such as images, audio, and video content may uncover new methodologies for content tracing and copyright enforcement in multimodal LLMs.

Conclusion

The investigation into the radioactivity of watermarked texts in LLMs offers a novel perspective on the traceability and accountability of data used in model training processes. By leveraging the inherent radioactivity of watermarked data, researchers and practitioners can gain deeper insights into the lifecycle of training data, facilitating more transparent, responsible, and ethical AI development practices.

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Collections

Sign up for free to add this paper to one or more collections.

Sign Up

Tweets

Sign up for free to view the 12 tweets with 119 likes about this paper.

Sign Up for Free

HackerNews

  1. Watermarking Makes Language Models Radioactive (2 points, 0 comments)