Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
125 tokens/sec
GPT-4o
53 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Certifiably Robust Graph Contrastive Learning (2310.03312v1)

Published 5 Oct 2023 in cs.CR and cs.LG

Abstract: Graph Contrastive Learning (GCL) has emerged as a popular unsupervised graph representation learning method. However, it has been shown that GCL is vulnerable to adversarial attacks on both the graph structure and node attributes. Although empirical approaches have been proposed to enhance the robustness of GCL, the certifiable robustness of GCL is still remain unexplored. In this paper, we develop the first certifiably robust framework in GCL. Specifically, we first propose a unified criteria to evaluate and certify the robustness of GCL. We then introduce a novel technique, RES (Randomized Edgedrop Smoothing), to ensure certifiable robustness for any GCL model, and this certified robustness can be provably preserved in downstream tasks. Furthermore, an effective training method is proposed for robust GCL. Extensive experiments on real-world datasets demonstrate the effectiveness of our proposed method in providing effective certifiable robustness and enhancing the robustness of any GCL model. The source code of RES is available at https://github.com/ventr1c/RES-GCL.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (64)
  1. Inductive representation learning on large graphs. Advances in Neural Information Processing Systems, 30, 2017.
  2. A semi-supervised graph attentive network for financial fraud detection. In IEEE International Conference on Data Mining, pages 598–607, 2019.
  3. Molecular contrastive learning of representations via graph neural networks. Nature Machine Intelligence, 4(3):279–287, 2022.
  4. Semi-supervised classification with graph convolutional networks. In International Conference on Learning Representations, 2017.
  5. How powerful are graph neural networks? In International Conference on Learning Representations, 2018.
  6. Learning how to propagate messages in graph neural networks. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, pages 1894–1903, 2021.
  7. Deep Graph Contrastive Representation Learning. In ICML Workshop on Graph Representation Learning and Beyond, 2020.
  8. Infograph: Unsupervised and semi-supervised graph-level representation learning via mutual information maximization. In International Conference on Learning Representations, 2020.
  9. Graph contrastive learning with augmentations. In Advances in Neural Information Processing Systems, pages 5812–5823, 2020.
  10. Adversarial attack on graph structured data. International Conference on Machine Learning, 2018.
  11. Adversarial attacks on neural networks for graph data. In the ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 2847–2856, 2018.
  12. Topology attack and defense for graph neural networks: An optimization perspective. In IJCAI, pages 3961–3967, 2019.
  13. Evasion attacks to graph neural networks via influence function. arXiv preprint arXiv:2009.00203, 2020.
  14. Towards robust graph contrastive learning. arXiv preprint arXiv:2102.13085, 2021.
  15. Adversarial graph contrastive learning with information regularization. In Proceedings of The Web Conference, pages 1362–1371, 2022.
  16. Unsupervised graph poisoning attack via contrastive loss back-propagation. In Proceedings of The Web Conference, pages 1322–1330, 2022.
  17. Towards robust graph neural networks for noisy graphs with sparse labels. In WSDM, pages 181–191, 2022.
  18. A unified framework of graph information bottleneck for robustness and membership privacy. arXiv preprint arXiv:2306.08604, 2023.
  19. Understanding and improving graph injection attack by promoting unnoticeability. In International Conference on Learning Representations, 2022.
  20. Unnoticeable backdoor attacks on graph neural networks. In Proceedings of the ACM Web Conference 2023, pages 2263–2273, 2023.
  21. Training with more confidence: Mitigating injected and natural backdoors during training. Advances in Neural Information Processing Systems, 35:36396–36410, 2022.
  22. Rethinking the reverse-engineering of trojan triggers. Advances in Neural Information Processing Systems, 35:9738–9753, 2022.
  23. Certifiable robustness and robust training for graph convolutional networks. In the ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 246–256, 2019.
  24. Certified robustness of graph neural networks against adversarial structural perturbation. In the ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, page 1645–1653, 2021.
  25. Certified robustness of graph classification against topology attack with randomized smoothing. In GLOBECOM 2020-2020 IEEE Global Communications Conference, pages 1–6. IEEE, 2020.
  26. Certified robustness of graph convolution networks for graph classification under topological attacks. In Advances in Neural Information Processing Systems, pages 8463–8474, 2020.
  27. Graph contrastive learning with adaptive augmentation. In Proceedings of the Web Conference 2021, pages 2069–2080, 2021.
  28. Certifiable robustness to graph perturbations. Advances in Neural Information Processing Systems, 32, 2019.
  29. Efficient robustness certificates for discrete data: Sparsity-aware randomized smoothing for graphs, images and more. In International Conference on Machine Learning, pages 1003–1013, 2020.
  30. Deep graph infomax. International Conference on Learning Representations, 2(3):4, 2019.
  31. Contrastive multi-view representation learning on graphs. In International Conference on Machine Learning, pages 4116–4126, 2020.
  32. Simple unsupervised graph representation learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 7797–7805, 2022.
  33. Decoupled self-supervised learning for graphs. Advances in Neural Information Processing Systems, 35:620–634, 2022.
  34. Towards self-explainable graph neural network. In Proceedings of the 30th ACM International Conference on Information & Knowledge Management, pages 302–311, 2021.
  35. Graph contrastive backdoor attacks. In Proceedings of the 40th International Conference on Machine Learning, 2023.
  36. A comprehensive survey on trustworthy graph neural networks: Privacy, robustness, fairness, and explainability. arXiv preprint arXiv:2204.08570, 2022.
  37. Certifiable robustness of graph convolutional networks under structure perturbations. In Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining, pages 1656–1665, 2020.
  38. A theoretical analysis of contrastive unsupervised representation learning. In International Conference on Learning Representations, pages 5628–5637, 2019.
  39. Robustness verification for contrastive learning. In International Conference on Machine Learning, pages 22865–22883, 2022.
  40. Label-invariant augmentation for semi-supervised graph classification. In Advances in Neural Information Processing Systems.
  41. Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748, 2018.
  42. Turning strengths into weaknesses: A certified robustness inspired attack framework against graph neural networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023.
  43. An introduction to statistical modeling of extreme values, volume 208. Springer, 2001.
  44. Certified robustness to adversarial examples with differential privacy. In 2019 IEEE Symposium on Security and Privacy (SP), pages 656–672. IEEE, 2019.
  45. Certified adversarial robustness via randomized smoothing. In International Conference on Machine Learning, volume 97, pages 1310–1320, 2019.
  46. Collective classification in network data. AI magazine, 29(3):93–93, 2008.
  47. Pitfalls of graph neural network evaluation. arXiv preprint arXiv:1811.05868, 2018.
  48. Open graph benchmark: Datasets for machine learning on graphs. In Advances in Neural Information Processing Systems, volume 33, pages 22118–22133, 2020.
  49. Tudataset: A collection of benchmark datasets for learning with graphs. In ICML 2020 Workshop on Graph Representation Learning and Beyond (GRL+ 2020), 2020.
  50. Robustness of graph neural networks at scale. Advances in Neural Information Processing Systems, 34:7637–7649, 2021.
  51. Large-scale representation learning on graphs via bootstrapping. In International Conference on Learning Representations, 2022.
  52. Node2vec: Scalable feature learning for networks. In the ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 2016.
  53. Variational graph auto-encoders. NIPS Workshop on Bayesian Deep Learning, 2016.
  54. A simple framework for contrastive learning of visual representations. In International Conference on Machine Learning, 2020.
  55. Ralph Linsker. Self-organization in a perceptual network. Computer, 21(3):105–117, 1988.
  56. Investigating the role of negatives in contrastive representation learning. arXiv preprint arXiv:2106.09943, 2021.
  57. Adversarial graph augmentation to improve graph contrastive learning. In Advances in Neural Information Processing Systems, 2021.
  58. Moleculenet: a benchmark for molecular machine learning. Chemical science, 9(2):513–530, 2018.
  59. Tdgia: Effective injection attacks on graph neural networks. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, pages 2461–2471, 2021.
  60. Bootstrap your own latent-a new approach to self-supervised learning. In Advances in neural information processing systems, pages 21271–21284, 2020.
  61. Learning deep representations by mutual information estimation and maximization. In International Conference on Learning Representations, 2019.
  62. Adversarial examples for graph data: Deep insights into attack and defense. In IJCAI, pages 4816–4823, 7 2019.
  63. An Empirical Study of Graph Contrastive Learning. arXiv.org, September 2021.
  64. Let invariant rationale discovery inspire graph contrastive learning. In Proceedings of the 39th International Conference on Machine Learning, 2022.
Citations (2)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now