Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
36 tokens/sec
GPT-4o
12 tokens/sec
Gemini 2.5 Pro Pro
37 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
4 tokens/sec
DeepSeek R1 via Azure Pro
33 tokens/sec
2000 character limit reached

Adversarial Robustness in Graph Neural Networks: A Hamiltonian Approach (2310.06396v1)

Published 10 Oct 2023 in cs.LG

Abstract: Graph neural networks (GNNs) are vulnerable to adversarial perturbations, including those that affect both node features and graph topology. This paper investigates GNNs derived from diverse neural flows, concentrating on their connection to various stability notions such as BIBO stability, Lyapunov stability, structural stability, and conservative stability. We argue that Lyapunov stability, despite its common use, does not necessarily ensure adversarial robustness. Inspired by physics principles, we advocate for the use of conservative Hamiltonian neural flows to construct GNNs that are robust to adversarial attacks. The adversarial robustness of different neural flow GNNs is empirically compared on several benchmark datasets under a variety of adversarial attacks. Extensive numerical experiments demonstrate that GNNs leveraging conservative Hamiltonian flows with Lyapunov stability substantially improve robustness against adversarial perturbations. The implementation code of experiments is available at https://github.com/zknus/NeurIPS-2023-HANG-Robustness.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (74)
  1. X. Yue, Z. Wang, J. Huang, S. Parthasarathy, S. Moosavinasab, Y. Huang, S. M. Lin, W. Zhang, P. Zhang, and H. Sun, “Graph embedding on biomedical networks: methods, applications and evaluations,” Bioinformatics, vol. 36, no. 4, pp. 1241–1251, 2019.
  2. H. Ashoor, X. Chen, W. Rosikiewicz, J. Wang, A. Cheng, P. Wang, Y. Ruan, and S. Li, “Graph embedding and unsupervised learning predict genomic sub-compartments from hic chromatin interaction data,” Nat. Commun., vol. 11, 2020.
  3. T. N. Kipf and M. Welling, “Semi-supervised classification with graph convolutional networks,” in Proc. Int. Conf. Learn. Representations, 2017.
  4. Z. Zhang, P. Cui, and W. Zhu, “Deep learning on graphs: A survey,” IEEE Trans. Knowl. Data Eng., vol. 34, no. 1, pp. 249–270, Jan 2022.
  5. Z. Wu, S. Pan, F. Chen, G. Long, C. Zhang, and P. S. Yu, “A comprehensive survey on graph neural networks,” IEEE Trans. Neural Netw. Learn. Syst., vol. 32, no. 1, pp. 4–24, 2021.
  6. P. Veličković, G. Cucurull, A. Casanova, A. Romero, P. Lio, and Y. Bengio, “Graph attention networks,” in Proc. Int. Conf. Learn. Representations, 2018, pp. 1–12.
  7. F. Ji, S. H. Lee, H. Meng, K. Zhao, J. Yang, and W. P. Tay, “Leveraging label non-uniformity for node classification in graph neural networks,” in Proc. Int. Conf. Mach. Learn., vol. 202, Jul. 2023, pp. 14 869–14 885.
  8. S. H. Lee, F. Ji, and W. P. Tay, “SGAT: Simplicial graph attention network,” in Proc. Inter. Joint Conf. Artificial Intell., Jul. 2022.
  9. D. Zügner, A. Akbarnejad, and S. Günnemann, “Adversarial attacks on neural networks for graph data,” in Proc. Int. Conf. Knowl. Discovery Data Mining, 2018.
  10. J. Wang, M. Luo, F. Suya, J. Li, Z. Yang, and Q. Zheng, “Scalable attack on graph data by injecting vicious nodes,” Data Mining Knowl. Discovery, pp. 1 – 27, 2020.
  11. X. Zou, Q. Zheng, Y. Dong, X. Guan, E. Kharlamov, J. Lu, and J. Tang, “Tdgia: Effective injection attacks on graph neural networks,” in Proc. Int. Conf. Knowl. Discovery Data Mining, 2021, p. 2461–2471.
  12. H. Hussain, M. Cao, S. Sikdar, D. Helic, E. Lex, M. Strohmaier, and R. Kern, “Adversarial inter-group link injection degrades the fairness of graph neural networks,” in Prof. Int. Conf. Data Mining, 2022, pp. 975–980.
  13. J. Chen, Y. Wu, X. Xu, Y. Chen, H. Zheng, and Q. Xuan, “Fast gradient attack on network embedding,” arXiv preprint arXiv:1809.02797, 2018.
  14. M. Waniek, T. P. Michalak, M. J. Wooldridge, and T. Rahwan, “Hiding individuals and communities in a social network,” Nature Human Behaviour, vol. 2, no. 1, pp. 139–147, 2018.
  15. J. Du, S. Zhang, G. Wu, J. M. F. Moura, and S. Kar, “Topology adaptive graph convolutional networks,” ArXiv, vol. abs/1710.10370, 2017.
  16. D. Zügner and S. Günnemann, “Adversarial attacks on graph neural networks via meta learning,” in Proc. Int. Conf. Learn. Representations, 2019.
  17. R. T. Chen, Y. Rubanova, J. Bettencourt, and D. Duvenaud, “Neural ordinary differential equations,” in Advances Neural Inf. Process. Syst., 2018.
  18. H. Yan, J. Du, V. Y. Tan, and J. Feng, “On robustness of neural ordinary differential equations,” in Advances Neural Inf. Process. Syst., 2018, pp. 1–13.
  19. Q. Kang, Y. Song, Q. Ding, and W. P. Tay, “Stable neural ODE with Lyapunov-stable equilibrium points for defending against adversarial attacks,” in Advances Neural Inf. Process. Syst., 2021.
  20. I. D. J. Rodriguez, A. Ames, and Y. Yue, “Lyanet: A lyapunov framework for training neural odes,” in Proc. Int. Conf. Mach. Learn., 2022, pp. 18 687–18 703.
  21. R. She, Q. Kang, S. Wang, Y.-R. Yáng, K. Zhao, Y. Song, and W. P. Tay, “Robustmat: Neural diffusion for street landmark patch matching under challenging environments,” IEEE Trans. Image Process., 2023.
  22. S. Wang, Q. Kang, R. She, W. P. Tay, A. Hartmannsgruber, and D. N. Navarro, “RobustLoc: Robust camera pose regression in challenging driving environments,” in Proc. AAAI Conference on Artificial Intelligence, Feb. 2023.
  23. R. She, Q. Kang, S. Wang, W. P. Tay, Y. L. Guan, D. N. Navarro, and A. Hartmannsgruber, “Image patch-matching with graph-based learning in street scenes,” IEEE Trans. Image Process., vol. 32, pp. 3465–3480, 2023.
  24. K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proc. IEEE Int. Conf. Comput. Vision, 2016, pp. 770–778.
  25. Y. D. Zhong, B. Dey, and A. Chakraborty, “Symplectic ode-net: Learning hamiltonian dynamics with control,” in Proc. Int. Conf. Learn. Representations, 2020.
  26. Y. Chen, T. Matsubara, and T. Yaguchi, “Neural symplectic form: Learning hamiltonian equations on general coordinate systems,” in Advances Neural Inf. Process. Syst., 2021.
  27. Y. Huang, Y. Yu, H. Zhang, Y. Ma, and Y. Yao, “Adversarial robustness of stabilized neural ode might be from obfuscated gradients,” in Proc. Math. Sci. Mach. Learn. Conf., J. Bruna, J. Hesthaven, and L. Zdeborova, Eds., 2022, pp. 497–515.
  28. B. P. Chamberlain, J. Rowbottom, M. Goronova, S. Webb, E. Rossi, and M. M. Bronstein, “Grand: Graph neural diffusion,” in Proc. Int. Conf. Mach. Learn., 2021.
  29. M. Thorpe, T. M. Nguyen, H. Xia, T. Strohmer, A. Bertozzi, S. Osher, and B. Wang, “Grand++: Graph neural diffusion with a source term,” in Proc. Int. Conf. Learn. Representations, 2021.
  30. B. P. Chamberlain, J. Rowbottom, D. Eynard, F. Di Giovanni, D. Xiaowen, and M. M. Bronstein, “Beltrami flow and neural diffusion on graphs,” in Advances Neural Inf. Process. Syst., 2021.
  31. Y. Song, Q. Kang, S. Wang, K. Zhao, and W. P. Tay, “On the robustness of graph neural diffusion to topology perturbations,” in Advances Neural Inf. Process. Syst., New Orleans, USA, Nov. 2022.
  32. K. Zhao, Q. Kang, Y. Song, R. She, S. Wang, and W. P. Tay, “Graph neural convection-diffusion with heterophily,” in Proc. Inter. Joint Conf. Artificial Intell., Aug. 2023.
  33. Q. Kang, K. Zhao, Y. Song, S. Wang, and W. P. Tay, “Node embedding from neural Hamiltonian orbits in graph neural networks,” in Proc. Int. Conf. Mach. Learn., vol. 202, Jul. 2023, pp. 15 786–15 808.
  34. T. K. Rusch, B. P. Chamberlain, J. Rowbottom, S. Mishra, and M. M. Bronstein, “Graph-coupled oscillator networks,” in Proc. Int. Conf. Mach. Learn., 2022.
  35. M. E. Sander, P. Ablin, M. Blondel, and G. Peyré, “Sinkformers: Transformers with doubly stochastic attention,” in International Conference on Artificial Intelligence and Statistics.   PMLR, 2022, pp. 3515–3530.
  36. W. Curtin and H. Scher, “Mechanics modeling using a spring network,” J. Mater. Research, vol. 5, no. 3, pp. 554–562, 1990.
  37. P. Veličković, G. Cucurull, A. Casanova, A. Romero, P. Liò, and Y. Bengio, “Graph attention networks,” in Proc. Int. Conf. Learn. Representations, 2018, pp. 1–12.
  38. W. L. Hamilton, R. Ying, and J. Leskovec, “Inductive representation learning on large graphs,” in Advances Neural Inf. Process. Syst., 2017.
  39. Y. Chen, H. Yang, Y. Zhang, K. Ma, T. Liu, B. Han, and J. Cheng, “Understanding and improving graph injection attack by promoting unnoticeability,” in Proc. Int. Conf. Learn. Representations, 2022.
  40. A. Mądry, A. Makelov, L. Schmidt, D. Tsipras, and A. Vladu, “Towards deep learning models resistant to adversarial attacks,” in Proc. Int. Conf. Learn. Representations, 2018.
  41. Z. Yang, W. Cohen, and R. Salakhudinov, “Revisiting semi-supervised learning with graph embeddings,” in Proc. Int. Conf. Mach. Learn., 2016, pp. 40–48.
  42. O. Shchur, M. Mumme, A. Bojchevski, and S. Günnemann, “Pitfalls of graph neural network evaluation,” Relational Representation Learning Workshop, Advances Neural Inf. Process. Syst., 2018.
  43. W. Hu, M. Fey, M. Zitnik, Y. Dong, H. Ren, B. Liu, M. Catasta, and J. Leskovec, “Open graph benchmark: Datasets for machine learning on graphs,” Advances Neural Inf. Process. Syst., vol. 33, pp. 22 118–22 133, 2020.
  44. Q. Zheng, X. Zou, Y. Dong, Y. Cen, D. Yin, J. Xu, Y. Yang, and J. Tang, “Graph robustness benchmark: Benchmarking the adversarial robustness of graph machine learning,” Advances Neural Inf. Process. Syst. Track Datasets Benchmarks, 2021.
  45. W. Jin, Y. Ma, X. Liu, X. Tang, S. Wang, and J. Tang, “Graph structure learning for robust graph neural networks,” in Proc. Int. Conf. Knowl. Discovery Data Mining, 2020, pp. 66–74.
  46. Y. Li, W. Jin, H. Xu, and J. Tang, “Deeprobust: A pytorch library for adversarial attacks and defenses,” arXiv preprint arXiv:2005.06149, 2020.
  47. D. Zhu, Z. Zhang, P. Cui, and W. Zhu, “Robust graph convolutional networks against adversarial attacks,” in Proc. Int. Conf. Knowl. Discovery Data Mining, 2019, pp. 1399–1407.
  48. N. Entezari, S. A. Al-Sayouri, A. Darvishzadeh, and E. E. Papalexakis, “All you need is low (rank): Defending against adversarial attacks on graphs,” in Proc. Int. Conf. Web Search Data Mining, 2020, p. 169–177.
  49. L. A. Adamic and N. Glance, “The political blogosphere and the 2004 us election: divided they blog,” in Proc. Int. Workshop Link Discovery, 2005, pp. 36–43.
  50. A. Madry, A. Makelov, L. Schmidt, D. Tsipras, and A. Vladu, “Towards deep learning models resistant to adversarial attacks,” arXiv preprint arXiv:1706.06083, 2017.
  51. X. Zhang and M. Zitnik, “Gnnguard: Defending graph neural networks against adversarial attacks,” Advances Neural Inf. Process. Syst., vol. 33, pp. 9263–9275, 2020.
  52. OpenAI, “Chatgpt-4,” 2022, available at: https://www.openai.com (Accessed: 26 September 2023).
  53. Y. Ma, S. Wang, T. Derr, L. Wu, and J. Tang, “Graph adversarial attack via rewiring,” in Proc. Int. Conf. Knowl. Discovery Data Mining, 2021, p. 1161–1169.
  54. Y. Sun, S. Wang, X. Tang, T.-Y. Hsieh, and V. Honavar, “Adversarial attacks on graph neural networks via node injections: A hierarchical reinforcement learning approach,” in Proc. Web Conf., 2020, p. 673–683.
  55. X. Wan, H. Kenlay, B. Ru, A. Blaas, M. A. Osborne, and X. Dong, “Adversarial attacks on graph classification via bayesian optimisation,” arXiv preprint arXiv:2111.02842, 2021.
  56. S. Geisler, T. Schmidt, H. Şirin, D. Zügner, A. Bojchevski, and S. Günnemann, “Robustness of graph neural networks at scale,” Advances Neural Inf. Process. Syst., vol. 34, pp. 7637–7649, 2021.
  57. J. Ma, J. Deng, and Q. Mei, “Adversarial attack on graph neural networks as an influence maximization problem,” in Prof. Int. Conf. Web Search Data Mining, 2022, pp. 675–685.
  58. B. Finkelshtein, C. Baskin, E. Zheltonozhskii, and U. Alon, “Single-node attacks for fooling graph neural networks,” Neurocomputing, vol. 513, pp. 1–12, 2022.
  59. D. Zhu, Z. Zhang, P. Cui, and W. Zhu, “Robust graph convolutional networks against adversarial attacks,” in Proc. Int. Conf. Knowl. Discovery Data Mining, 2019, p. 1399–1407.
  60. W. Feng, J. Zhang, Y. Dong, Y. Han, H. Luan, Q. Xu, Q. Yang, E. Kharlamov, and J. Tang, “Graph random neural networks for semi-supervised learning on graphs,” in Proc. Advances Neural Inf. Process. Syst., 2020.
  61. W. Jin, Y. Ma, X. Liu, X. Tang, S. Wang, and J. Tang, “Graph structure learning for robust graph neural networks,” in Proc. Int. Conf. Knowl. Discovery Data Mining, 2020, p. 66–74.
  62. X. Gao, W. Hu, and Z. Guo, “Exploring structure-adaptive graph learning for robust semi-supervised classification,” in 2020 IEEE International Conference on Multimedia and Expo (ICME).   IEEE, 2020, pp. 1–6.
  63. T. Zhao, Y. Liu, L. Neves, O. Woodford, M. Jiang, and N. Shah, “Data augmentation for graph neural networks,” in Proc. AAAI Conference on Artificial Intelligence, vol. 35, no. 12, 2021, pp. 11 015–11 023.
  64. K. Li, Y. Liu, X. Ao, J. Chi, J. Feng, H. Yang, and Q. He, “Reliable representations make a stronger defender: Unsupervised structure refinement for robust gnn,” in Proc. Int. Conf. Knowl. Discovery Data Mining, 2022, pp. 925–935.
  65. B. Runwal, S. Kumar et al., “Robust graph neural networks using weighted graph laplacian,” arXiv preprint arXiv:2208.01853, 2022.
  66. X. Zhang and M. Zitnik, “Gnnguard: Defending graph neural networks against adversarial attacks,” in Proc. Advances Neural Inf. Process. Syst., 2020.
  67. S. Greydanus, M. Dzamba, and J. Yosinski, “Hamiltonian neural networks,” in Advances Neural Inf. Process. Syst., 2019.
  68. Z. Chen, J. Zhang, M. Arjovsky, and L. Bottou, “Symplectic recurrent neural networks,” in Proc. Int. Conf. Learn. Representations, 2020.
  69. A. Choudhary, J. F. Lindner, E. G. Holliday, S. T. Miller, S. Sinha, and W. L. Ditto, “Physics-enhanced neural networks learn order and chaos,” Physical Review E, vol. 101, no. 6, p. 062207, 2020.
  70. E. Haber and L. Ruthotto, “Stable architectures for deep neural networks,” Inverse Problems, vol. 34, no. 1, pp. 1–23, Dec. 2017.
  71. J. Li, J. Peng, L. Chen, Z. Zheng, T. Liang, and Q. Ling, “Spectral adversarial training for robust graph neural network,” IEEE Transactions on Knowledge and Data Engineering, 2022.
  72. C. Deng, X. Li, Z. Feng, and Z. Zhang, “Garnet: Reduced-rank topology learning for robust and scalable graph neural networks,” in Learning on Graphs Conference.   PMLR, 2022, pp. 3–1.
  73. V. Nikiforov, “Revisiting schur’s bound on the largest singular value,” arXiv preprint math/0702722, 2007.
  74. H. K. Khalil, “Nonlinear control,” 2015, available at: https://www.egr.msu.edu/~khalil/NonlinearControl/Slides-Short/Lecture_2.pdf (Accessed: 26 September 2023).
Citations (16)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
Dice Question Streamline Icon: https://streamlinehq.com

Follow-up Questions

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now