Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
97 tokens/sec
GPT-4o
53 tokens/sec
Gemini 2.5 Pro Pro
44 tokens/sec
o3 Pro
5 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

DeepEclipse: How to Break White-Box DNN-Watermarking Schemes (2403.03590v1)

Published 6 Mar 2024 in cs.CR and cs.LG

Abstract: Deep Learning (DL) models have become crucial in digital transformation, thus raising concerns about their intellectual property rights. Different watermarking techniques have been developed to protect Deep Neural Networks (DNNs) from IP infringement, creating a competitive field for DNN watermarking and removal methods. The predominant watermarking schemes use white-box techniques, which involve modifying weights by adding a unique signature to specific DNN layers. On the other hand, existing attacks on white-box watermarking usually require knowledge of the specific deployed watermarking scheme or access to the underlying data for further training and fine-tuning. We propose DeepEclipse, a novel and unified framework designed to remove white-box watermarks. We present obfuscation techniques that significantly differ from the existing white-box watermarking removal schemes. DeepEclipse can evade watermark detection without prior knowledge of the underlying watermarking scheme, additional data, or training and fine-tuning. Our evaluation reveals that DeepEclipse excels in breaking multiple white-box watermarking schemes, reducing watermark detection to random guessing while maintaining a similar model accuracy as the original one. Our framework showcases a promising solution to address the ongoing DNN watermark protection and removal challenges.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (81)
  1. Turning your weakness into a strength: Watermarking deep neural networks by backdooring. In 27th USENIX Security Symposium, pages 1615–1631, 2018.
  2. Abien Fred Agarap. Deep learning using rectified linear units (relu). arXiv preprint arXiv:1803.08375, 2018.
  3. Neural network laundering: Removing black-box backdoor watermarks from deep neural networks, 2020.
  4. Waffle: Watermarking in federated learning, 2021.
  5. Certified neural network watermarks with randomized smoothing, 2022.
  6. Neunac: A novel fragile watermarking algorithm for integrity protection of neural networks. Information Sciences, 576:228–241, 2021.
  7. John S. Bridle. Probabilistic interpretation of feedforward classification network outputs, with relationships to statistical pattern recognition. In Neurocomputing, pages 227–236, Berlin, Heidelberg, 1990. Springer Berlin Heidelberg.
  8. Cosine model watermarking against ensemble distillation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36.9, pages 9512–9520, 2022.
  9. Deepattest: An end-to-end attestation framework for deep neural networks. In Proceedings of the 46th International Symposium on Computer Architecture, pages 487–498, 2019.
  10. Deepmarks: A secure fingerprinting framework for digital rights management of deep learning models. In Proceedings of the 2019 on International Conference on Multimedia Retrieval, pages 105–113, 2019.
  11. Blackmarks: Blackbox multibit watermarking for deep neural networks. arXiv preprint arXiv:1904.00344, 2019.
  12. Specmark: A spectral watermarking framework for ip protection of speech recognition systems. In INTERSPEECH, pages 2312–2316, 2020.
  13. You are caught stealing my winning lottery ticket! making a lottery ticket claim its ownership. Advances in neural information processing systems, 34:1780–1791, 2021.
  14. Sslguard: A watermarking scheme for self-supervised learning pre-trained encoders, 2022.
  15. Copycat cnn: Stealing knowledge by persuading confession with random non-labeled data. In 2018 International Joint Conference on Neural Networks (IJCNN), pages 1–8. IEEE, 2018.
  16. Deepsigns: An end-to-end watermarking framework for ownership protection of deep neural networks. In Proceedings of the Twenty-Fourth International Conference on Architectural Support for Programming Languages and Operating Systems, pages 485–497, 2019.
  17. Bert: Pre-training of deep bidirectional transformers for language understanding, 2019.
  18. Deepip: Deep neural network intellectual property protection with passports. IEEE Transactions on Pattern Analysis & Machine Intelligence, 01:1–1, 2021.
  19. Le Feng and Xinpeng Zhang. Watermarking neural network with compensation mechanism. In Knowledge Science, Engineering and Management: 13th International Conference, KSEM 2020, Hangzhou, China, August 28–30, 2020, Proceedings, Part II 13, pages 363–375. Springer, 2020.
  20. Reversible watermarking in deep convolutional neural networks for integrity authentication. In Proceedings of the 28th ACM International Conference on Multimedia, pages 2273–2280, 2020.
  21. Watermarking deep neural networks for embedded systems. In 2018 IEEE/ACM International Conference on Computer-Aided Design (ICCAD), pages 1–8. IEEE, 2018.
  22. Deep residual learning for image recognition, 2015.
  23. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016.
  24. Towards security threats of deep learning systems: A survey. IEEE Transactions on Software Engineering, 48(5):1743–1770, 2020.
  25. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531, 2015.
  26. Quantized neural networks: Training neural networks with low precision weights and activations, 2016.
  27. Entangled watermarks as a defense against model extraction. In 30th USENIX Security Symposium (USENIX Security 21), pages 1937–1954, 2021.
  28. Subnetwork-lossless robust watermarking for hostile theft attacks in deep transfer learning models. IEEE Transactions on Dependable and Secure Computing, pages 1–16, 2022.
  29. A watermark for large language models. arXiv preprint arXiv:2301.10226, 2023.
  30. Learning Multiple Layers of Features from Tiny Images, 2009.
  31. White-box watermarking scheme for fully-connected layers in fine-tuning model. In Proceedings of the 2021 ACM Workshop on Information Hiding and Multimedia Security, pages 165–170, 2021.
  32. Identification for deep neural network: Simply adjusting few weights! In 2022 IEEE 38th International Conference on Data Engineering (ICDE), pages 1328–1341. IEEE, 2022.
  33. Deepauth: A dnn authentication framework by model-unique and fragile signature embedding. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36.9, pages 9595–9603, 2022.
  34. Adversarial frontier stitching for remote neural network watermarking. Neural Computing and Applications, 32:9233–9244, 2020.
  35. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11), 1998.
  36. Fedipr: Ownership verification for federated deep neural network models, 2022.
  37. Untargeted backdoor watermark: Towards harmless and stealthy dataset copyright protection. Advances in Neural Information Processing Systems, 35:13238–13250, 2022.
  38. Move: Effective and harmless ownership verification via embedded external features. arXiv preprint arXiv:2208.02820, 2022.
  39. Defending against model stealing via verifying embedded external features. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36.2, pages 1464–1472, 2022.
  40. How to prove your model belongs to you: A blind-watermark based framework to protect intellectual property of dnn. In Proceedings of the 35th Annual Computer Security Applications Conference, pages 126–137, 2019.
  41. Protect, show, attend and tell: Empowering image captioning models with ownership protection. Pattern Recognition, 122:108285, 2022.
  42. Watermarking deep neural networks with greedy residuals. In ICML, pages 6978–6988, 2021.
  43. Fine-pruning: Defending against backdooring attacks on deep neural networks, 2018.
  44. Yes we can: Watermarking machine learning models beyond classification. In 2021 IEEE 34th Computer Security Foundations Symposium (CSF), pages 1–14, 2021.
  45. Sok: How robust is image classification deep neural network watermarking? In 2022 IEEE Symposium on Security and Privacy (SP), pages 787–804. IEEE, 2022.
  46. Towards deep learning models resistant to adversarial attacks, 2019.
  47. Piracy-resistant dnn watermarking by block-wise image transformation with secret key. In Proceedings of the 2021 ACM Workshop on Information Hiding and Multimedia Security, pages 159–164, 2021.
  48. Aime: watermarking ai models by leveraging errors. In 2022 Design, Automation & Test in Europe Conference & Exhibition (DATE), pages 304–309. IEEE, 2022.
  49. Matrix analysis and applied linear algebra. SIAM, 2023.
  50. Deeptextmark: Deep learning based text watermarking for detection of large language model generated text. arXiv preprint arXiv:2305.05773, 2023.
  51. Robust watermarking of neural network with exponential weighting, 2019.
  52. Protecting intellectual property of generative adversarial networks from ambiguity attack, 2021.
  53. R OpenAI. Gpt-4 technical report. arXiv, pages 2303–08774, 2023.
  54. Pytorch: An imperative style, high-performance deep learning library, 2019.
  55. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9, 2019.
  56. Exploring the limits of transfer learning with a unified text-to-text transformer, 2023.
  57. Protecting intellectual property with reliable availability of learning models in ai-based cybersecurity services. IEEE Transactions on Dependable and Secure Computing, pages 1–18, 2022.
  58. Radioactive data: tracing through training. In International Conference on Machine Learning, pages 8326–8335. PMLR, 2020.
  59. On the robustness of backdoor-based watermarking in deep neural networks. In Proceedings of the 2021 ACM Workshop on Information Hiding and Multimedia Security, IH&MMSec ’21, page 177–188, New York, NY, USA, 2021. Association for Computing Machinery.
  60. On the robustness of backdoor-based watermarking in deep neural networks. In Proceedings of the 2021 ACM Workshop on Information Hiding and Multimedia Security, pages 177–188, 2021.
  61. Fedtracker: Furnishing ownership verification and traceability for federated learning model, 2023.
  62. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556, 2014.
  63. Deep intellectual property: A survey. arXiv preprint arXiv:2304.14613, 2023.
  64. Rethinking the inception architecture for computer vision. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 2818–2826, 2016.
  65. Stealing machine learning models via prediction {{\{{APIs}}\}}. In 25th USENIX security symposium (USENIX Security 16), pages 601–618, 2016.
  66. Embedding watermarks into deep neural networks. In Proceedings of the 2017 ACM on International Conference on Multimedia Retrieval. ACM, jun 2017.
  67. Foundations of signal processing. Cambridge University Press, 2014.
  68. Neural cleanse: Identifying and mitigating backdoor attacks in neural networks. In 2019 IEEE Symposium on Security and Privacy (SP), pages 707–723. IEEE, 2019.
  69. Non-transferable learning: A new approach for model ownership verification and applicability authorization, 2022.
  70. Riga: Covert and robust white-box watermarking of deep neural networks. In Proceedings of the Web Conference 2021, pages 993–1004, 2021.
  71. Cits-mew: Multi-party entangled watermark in cooperative intelligent transportation system. IEEE Transactions on Intelligent Transportation Systems, 24(3):3528–3540, 2023.
  72. When nas meets watermarking: ownership verification of dnn models via cache side channels. arXiv e-prints, pages arXiv–2102, 2021.
  73. Deepmark: Embedding watermarks into deep neural network using pruning. In 2021 IEEE 33rd International Conference on Tools with Artificial Intelligence (ICTAI), pages 169–175. IEEE, 2021.
  74. Rethinking white-box watermarks on deep learning models under neural structural obfuscation. In 32th USENIX security symposium (USENIX Security 23), 2023.
  75. Robust watermarking for deep neural networks via bi-level optimization. In 2021 IEEE/CVF International Conference on Computer Vision (ICCV), pages 14821–14830, 2021.
  76. Protecting intellectual property of deep neural networks with watermarking. In Proceedings of the 2018 on Asia conference on computer and communications security, pages 159–172, 2018.
  77. Passport-aware normalization for deep model protection. Advances in Neural Information Processing Systems, 33:22619–22628, 2020.
  78. Structural watermarking to deep neural networks via network channel pruning. In 2021 IEEE International Workshop on Information Forensics and Security (WIFS), pages 1–6. IEEE, 2021.
  79. Protecting ip of deep neural networks with watermarking: A new label helps. In Advances in Knowledge Discovery and Data Mining: 24th Pacific-Asia Conference, PAKDD 2020, Singapore, May 11–14, 2020, Proceedings, Part II 24, pages 462–474. Springer, 2020.
  80. To prune, or not to prune: exploring the efficacy of pruning for model compression, 2017.
  81. Secure neural network watermarking protocol against forging attack. EURASIP Journal on Image and Video Processing, 2020:1–12, 2020.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (4)
  1. Alessandro Pegoraro (5 papers)
  2. Carlotta Segna (2 papers)
  3. Kavita Kumari (12 papers)
  4. Ahmad-Reza Sadeghi (66 papers)

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now