Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
110 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
44 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

On the Adversarial Robustness of Graph Contrastive Learning Methods (2311.17853v2)

Published 29 Nov 2023 in cs.LG

Abstract: Contrastive learning (CL) has emerged as a powerful framework for learning representations of images and text in a self-supervised manner while enhancing model robustness against adversarial attacks. More recently, researchers have extended the principles of contrastive learning to graph-structured data, giving birth to the field of graph contrastive learning (GCL). However, whether GCL methods can deliver the same advantages in adversarial robustness as their counterparts in the image and text domains remains an open question. In this paper, we introduce a comprehensive robustness evaluation protocol tailored to assess the robustness of GCL models. We subject these models to adaptive adversarial attacks targeting the graph structure, specifically in the evasion scenario. We evaluate node and graph classification tasks using diverse real-world datasets and attack strategies. With our work, we aim to offer insights into the robustness of GCL methods and hope to open avenues for potential future research directions.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (59)
  1. Learning Representations by Maximizing Mutual Information Across Views. In Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019.
  2. Adversarial Attacks on Node Embeddings via Graph Poisoning. In Proceedings of the 36th International Conference on Machine Learning, pages 695–704. PMLR, May 2019.
  3. Geometric Deep Learning: Going beyond Euclidean data. IEEE Signal Processing Magazine, 34(4):18–42, July 2017.
  4. Towards Evaluating the Robustness of Neural Networks. In 2017 IEEE Symposium on Security and Privacy (SP), pages 39–57. IEEE Computer Society, May 2017.
  5. Adversarial Attacks and Defences: A Survey. arXiv preprint arXiv:1810.00069, September 2018.
  6. A Simple Framework for Contrastive Learning of Visual Representations. In Proceedings of the 37th International Conference on Machine Learning, pages 1597–1607. PMLR, November 2020.
  7. Adversarial Attack on Graph Structured Data. In Proceedings of the 35th International Conference on Machine Learning, pages 1115–1124. PMLR, July 2018.
  8. Asymptotic evaluation of certain markov process expectations for large time. Communications on Pure and Applied Mathematics, 28(1):1–47, 1975.
  9. Robustness of Graph Neural Networks at Scale. In Advances in Neural Information Processing Systems, volume 34, pages 7637–7649. Curran Associates, Inc., 2021.
  10. Neural Message Passing for Quantum Chemistry. In Proceedings of the 34th International Conference on Machine Learning, pages 1263–1272. PMLR, July 2017.
  11. Explaining and Harnessing Adversarial Examples. arXiv preprint arXiv:1412.6572, December 2014.
  12. Adversarial Training for Graph Neural Networks: Pitfalls, Solutions, and New Directions. In Advances in Neural Information Processing Systems, volume 36, pages 8954–8968. Curran Associates, Inc., 2023a.
  13. Revisiting Robustness in Graph Machine Learning. In International Conference on Learning Representations, 2023b.
  14. Stephan Günnemann. Graph Neural Networks: Adversarial Robustness. In Lingfei Wu, Peng Cui, Jian Pei, and Liang Zhao, editors, Graph Neural Networks: Foundations, Frontiers, and Applications, pages 149–176. Springer Nature, Singapore, 2022.
  15. Noise-contrastive estimation: A new estimation principle for unnormalized statistical models. In Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, volume 9 of Proceedings of Machine Learning Research, pages 297–304. PMLR, May 2010.
  16. Dimensionality Reduction by Learning an Invariant Mapping. In 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’06), volume 2, pages 1735–1742, June 2006.
  17. Using self-supervised learning can improve model robustness and uncertainty. In Advances in Neural Information Processing Systems, volume 32, pages 15663–15674. Curran Associates Inc., December 2019.
  18. Learning deep representations by mutual information estimation and maximization. In International Conference on Learning Representations, 2019.
  19. Open Graph Benchmark: Datasets for Machine Learning on Graphs. In Advances in Neural Information Processing Systems, volume 33, pages 22118–22133. Curran Associates, Inc., 2020.
  20. A Survey on Contrastive Self-Supervised Learning. Technologies, 9(1):2, March 2021.
  21. Categorical Reparameterization with Gumbel-Softmax. In International Conference on Learning Representations, 2017.
  22. Towards Robust Graph Contrastive Learning. arXiv preprint arXiv:2102.13085, February 2021.
  23. Contrastive Self-Supervised Learning: A Survey on Different Architectures. In 2022 2nd International Conference on Artificial Intelligence (ICAI), pages 1–6, March 2022.
  24. Adversarial Self-Supervised Contrastive Learning. In Advances in Neural Information Processing Systems, volume 33, pages 2983–2994. Curran Associates, Inc., 2020.
  25. Adam: A Method for Stochastic Optimization. arXiv preprint arXiv:1412.6980, December 2014.
  26. Semi-Supervised Classification with Graph Convolutional Networks. In International Conference on Learning Representations, November 2016.
  27. On Information and Sufficiency. The Annals of Mathematical Statistics, 22(1):79 – 86, 1951.
  28. Disentangled Graph Contrastive Learning With Independence Promotion. IEEE Transactions on Knowledge and Data Engineering, pages 1–14, 2022.
  29. Towards Effective and Robust Graph Contrastive Learning with Graph Autoencoding. IEEE Transactions on Knowledge and Data Engineering, pages 1–14, 2023.
  30. Self-Supervised Learning: Generative or Contrastive. IEEE Transactions on Knowledge and Data Engineering, 35(1):857–876, June 2021.
  31. The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables. In International Conference on Learning Representations, 2017.
  32. Towards Deep Learning Models Resistant to Adversarial Attacks. In International Conference on Learning Representations, February 2018.
  33. TUDataset: A collection of benchmark datasets for learning with graphs. arXiv preprint arXiv:2007.08663, July 2020.
  34. Are Defenses for Graph Neural Networks Robust? In Advances in Neural Information Processing Systems, volume 35, pages 8954–8968. Curran Associates, Inc., December 2022.
  35. Query-driven Active Surveying for Collective Classification. In Workshop on Mining and Learning with Graphs, International Conference on Machine Learning, volume 8, 2012.
  36. F-GAN: Training Generative Neural Samplers using Variational Divergence Minimization. In Advances in Neural Information Processing Systems, volume 29. Curran Associates, Inc., 2016.
  37. Representation Learning with Contrastive Predictive Coding. arXiv preprint arXiv:1807.03748, July 2018.
  38. The Graph Neural Network Model. IEEE Transactions on Neural Networks, 20(1):61–80, January 2009.
  39. Collective Classification in Network Data. AI Magazine, 29(3):93–93, September 2008.
  40. How Robust is Unsupervised Representation Learning to Distribution Shift? arXiv preprint arXiv:2206.08871, December 2022.
  41. Kihyuk Sohn. Improved Deep Metric Learning with Multi-class N-pair Loss Objective. In Advances in Neural Information Processing Systems, volume 29. Curran Associates, Inc., 2016.
  42. InfoGraph: Unsupervised and Semi-supervised Graph-Level Representation Learning via Mutual Information Maximization. In International Conference on Learning Representations, September 2019.
  43. Adversarial Graph Augmentation to Improve Graph Contrastive Learning. In Advances in Neural Information Processing Systems, volume 34, pages 15920–15933. Curran Associates, Inc., 2021.
  44. Deep learning and the information bottleneck principle. In 2015 IEEE Information Theory Workshop (ITW), pages 1–5, April 2015.
  45. The information bottleneck method. arXiv preprint arXiv:physics/0004057, April 2000.
  46. Graph Attention Networks. In International Conference on Learning Representations, February 2018.
  47. Deep Graph Infomax. In International Conference on Learning Representations, 2019.
  48. Augmentation-Free Graph Contrastive Learning with Performance Guarantee. arXiv preprint arXiv:2204.04874, June 2022.
  49. Unsupervised Feature Learning via Non-parametric Instance Discrimination. In 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3733–3742, June 2018.
  50. Self-Supervised Learning of Graph Neural Networks: A Unified Review. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(2):2412–2429, April 2022.
  51. Topology Attack and Defense for Graph Neural Networks: An Optimization Perspective. In Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, pages 3961–3967. International Joint Conferences on Artificial Intelligence Organization, August 2019.
  52. How Powerful are Graph Neural Networks? In International Conference on Learning Representations, September 2018.
  53. Investigating Why Contrastive Learning Benefits Robustness against Label Noise. In Proceedings of the 39th International Conference on Machine Learning, pages 24851–24871. PMLR, June 2022.
  54. Revisiting Semi-Supervised Learning with Graph Embeddings. In Proceedings of The 33rd International Conference on Machine Learning, pages 40–48. PMLR, June 2016.
  55. Graph contrastive learning with augmentations. In Advances in Neural Information Processing Systems, volume 33, 2020.
  56. Deep Graph Contrastive Representation Learning. arXiv preprint arXiv:2006.04131, July 2020.
  57. Graph Contrastive Learning with Adaptive Augmentation. In Proceedings of the Web Conference 2021, WWW ’21, pages 2069–2080, New York, NY, USA, April 2021. Association for Computing Machinery.
  58. Adversarial Attacks on Graph Neural Networks via Meta Learning. In International Conference on Learning Representations, May 2019.
  59. Adversarial Attacks on Neural Networks for Graph Data. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD ’18, pages 2847–2856, New York, NY, USA, July 2018. Association for Computing Machinery.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (6)
  1. Filippo Guerranti (1 paper)
  2. Zinuo Yi (1 paper)
  3. Anna Starovoit (1 paper)
  4. Rafiq Kamel (1 paper)
  5. Simon Geisler (24 papers)
  6. Stephan Günnemann (169 papers)
Citations (2)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now