Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
126 tokens/sec
GPT-4o
47 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

PreGIP: Watermarking the Pretraining of Graph Neural Networks for Deep Intellectual Property Protection (2402.04435v1)

Published 6 Feb 2024 in cs.LG and cs.AI

Abstract: Pretraining on Graph Neural Networks (GNNs) has shown great power in facilitating various downstream tasks. As pretraining generally requires huge amount of data and computational resources, the pretrained GNNs are high-value Intellectual Properties (IP) of the legitimate owner. However, adversaries may illegally copy and deploy the pretrained GNN models for their downstream tasks. Though initial efforts have been made to watermark GNN classifiers for IP protection, these methods require the target classification task for watermarking, and thus are not applicable to self-supervised pretraining of GNN models. Hence, in this work, we propose a novel framework named PreGIP to watermark the pretraining of GNN encoder for IP protection while maintain the high-quality of the embedding space. PreGIP incorporates a task-free watermarking loss to watermark the embedding space of pretrained GNN encoder. A finetuning-resistant watermark injection is further deployed. Theoretical analysis and extensive experiments show the effectiveness of {\method} in IP protection and maintaining high-performance for downstream tasks.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (40)
  1. Turning your weakness into a strength: Watermarking deep neural networks by backdooring. In 27th USENIX Security Symposium (USENIX Security 18), pp. 1615–1631, 2018.
  2. Certified neural network watermarks with randomized smoothing. In Proceedings of the 39th International Conference on Machine Learning, pp.  1450–1465, 2022.
  3. Certified adversarial robustness via randomized smoothing. In international conference on machine learning, pp. 1310–1320. PMLR, 2019.
  4. Sslguard: A watermarking scheme for self-supervised learning pre-trained encoders. In Proceedings of the 2022 ACM SIGSAC Conference on Computer and Communications Security, pp.  579–593, 2022.
  5. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805, 2018.
  6. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929, 2020.
  7. On random graphs i. Publ. math. debrecen, 6(290-297):18, 1959.
  8. A survey of graph neural networks for recommender systems: Challenges, methods, and directions. ACM Transactions on Recommender Systems, 1(1):1–51, 2023.
  9. Graphmae: Self-supervised masked graph autoencoders. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp.  594–604, 2022.
  10. Strategies for pre-training graph neural networks. In International Conference on Learning Representations, 2020.
  11. Semi-supervised classification with graph convolutional networks. In ICLR, 2017.
  12. Spherical message passing for 3d molecular graphs. In International Conference on Learning Representations, 2022.
  13. Attending to graph transformers. arXiv preprint arXiv:2302.04181, 2023.
  14. On first-order meta-learning algorithms. arXiv preprint arXiv:1803.02999, 2018.
  15. Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748, 2018.
  16. Graph invariant kernels. In Proceedings of the twenty-fourth international joint conference on artificial intelligence, volume 2015, pp.  3756–3762, 2015.
  17. Gcc: Graph contrastive coding for graph neural network pre-training. In Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining, pp.  1150–1160, 2020.
  18. Deepsigns: A generic watermarking framework for ip protection of deep learning models. arXiv preprint arXiv:1804.00750, 2018.
  19. Understanding isomorphism bias in graph data sets, 2020.
  20. Zinc 15 – ligand discovery for everyone. Journal of Chemical Information and Modeling, 55(11):2324–2337, 2015. doi: 10.1021/acs.jcim.5b00559. PMID: 26479676.
  21. All in one: Multi-task prompting for graph neural networks. 2023a.
  22. Deep intellectual property: A survey. arXiv preprint arXiv:2304.14613, 2023b.
  23. Embedding watermarks into deep neural networks. In Proceedings of the 2017 ACM on international conference on multimedia retrieval, pp.  269–277, 2017.
  24. Lipschitz regularity of deep neural networks: analysis and efficient estimation. Advances in Neural Information Processing Systems, 31, 2018.
  25. Moleculenet: a benchmark for molecular machine learning. Chemical science, 9(2):513–530, 2018.
  26. Simgrace: A simple framework for graph contrastive learning without data augmentation. In Proceedings of the ACM Web Conference 2022, pp. 1070–1079, 2022.
  27. Watermarking graph neural networks based on backdoor attacks. In 2023 IEEE 8th European Symposium on Security and Privacy (EuroS&P), pp.  1179–1197. IEEE, 2023a.
  28. How powerful are graph neural networks? arXiv preprint arXiv:1810.00826, 2018.
  29. Graph and geometry generative modeling for drug discovery. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp.  5833–5834, 2023b.
  30. Robust watermarking for deep neural networks via bi-level optimization. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp.  14841–14850, 2021.
  31. Do transformers really perform badly for graph representation? Advances in Neural Information Processing Systems, 34:28877–28888, 2021.
  32. Graph convolutional neural networks for web-scale recommender systems. In SIGKDD, pp.  974–983, 2018.
  33. Graph contrastive learning with augmentations. Advances in neural information processing systems, 33:5812–5823, 2020.
  34. Graph contrastive backdoor attacks. In International Conference on Machine Learning, pp. 40888–40910. PMLR, 2023.
  35. Protecting intellectual property of deep neural networks with watermarking. In Proceedings of the 2018 on Asia conference on computer and communications security, pp.  159–172, 2018.
  36. Graph-bert: Only attention is needed for learning graph representations. arXiv preprint arXiv:2001.05140, 2020.
  37. Protein representation learning by geometric structure pretraining. arXiv preprint arXiv:2203.06125, 2022.
  38. Semi-supervised graph-to-graph translation. In CIKM, pp.  1863–1872, 2020.
  39. Watermarking graph neural networks by random graphs. In 2021 9th International Symposium on Digital Forensics and Security (ISDFS), pp.  1–6. IEEE, 2021.
  40. An Empirical Study of Graph Contrastive Learning. arXiv.org, September 2021.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets