Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
175 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Robust Node Representation Learning via Graph Variational Diffusion Networks (2312.10903v1)

Published 18 Dec 2023 in cs.LG and cs.AI

Abstract: Node representation learning by using Graph Neural Networks (GNNs) has been widely explored. However, in recent years, compelling evidence has revealed that GNN-based node representation learning can be substantially deteriorated by delicately-crafted perturbations in a graph structure. To learn robust node representation in the presence of perturbations, various works have been proposed to safeguard GNNs. Within these existing works, Bayesian label transition has been proven to be more effective, but this method is extensively reliant on a well-built prior distribution. The variational inference could address this limitation by sampling the latent node embedding from a Gaussian prior distribution. Besides, leveraging the Gaussian distribution (noise) in hidden layers is an appealing strategy to strengthen the robustness of GNNs. However, our experiments indicate that such a strategy can cause over-smoothing issues during node aggregation. In this work, we propose the Graph Variational Diffusion Network (GVDN), a new node encoder that effectively manipulates Gaussian noise to safeguard robustness on perturbed graphs while alleviating over-smoothing issues through two mechanisms: Gaussian diffusion and node embedding propagation. Thanks to these two mechanisms, our model can generate robust node embeddings for recovery. Specifically, we design a retraining mechanism using the generated node embedding to recover the performance of node classifications in the presence of perturbations. The experiments verify the effectiveness of our proposed model across six public datasets.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (59)
  1. Beyond low-frequency information in graph convolutional networks. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, 3950–3957.
  2. Friend or Faux: Graph-Based Early Detection of Fake Accounts on Social Networks. In Proceedings of The Web Conference 2020, 1287–1297.
  3. Spectral networks and locally connected networks on graphs. arXiv preprint arXiv:1312.6203.
  4. Not All Low-Pass Filters are Robust in Graph Convolutional Networks. Advances in Neural Information Processing Systems, 34.
  5. Semi-supervised User Profiling with Heterogeneous Graph Attention Networks. In IJCAI, volume 19, 2116–2122.
  6. Adversarial attack on graph structured data. In International conference on machine learning, 1115–1124. PMLR.
  7. Convolutional neural networks on graphs with fast localized spectral filtering. In Advances in neural information processing systems, 3844–3852.
  8. All you need is low (rank) defending against adversarial attacks on graphs. In Proceedings of the 13th International Conference on Web Search and Data Mining, 169–177.
  9. Community-based question answering via heterogeneous social network learning. In Proceedings of the AAAI conference on artificial intelligence, volume 30.
  10. Graph Random Neural Networks for Semi-Supervised Learning on Graphs. Advances in Neural Information Processing Systems, 33.
  11. Dropout as a bayesian approximation: Representing model uncertainty in deep learning. In international conference on machine learning, 1050–1059. PMLR.
  12. Simple gnn regularisation for 3d molecular property prediction and beyond. In International conference on learning representations.
  13. Label propagation via teaching-to-learn and learning-to-teach. IEEE transactions on neural networks and learning systems, 28(6): 1452–1465.
  14. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572.
  15. Inductive representation learning on large graphs. In Advances in neural information processing systems, 1024–1034.
  16. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems, 33: 6840–6851.
  17. Combining label propagation and simple models out-performs graph neural networks. arXiv preprint arXiv:2010.13993.
  18. Label propagation for deep semi-supervised learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 5070–5079.
  19. Node similarity preserving graph convolutional networks. In Proceedings of the 14th ACM International Conference on Web Search and Data Mining.
  20. Adversarial Attacks and Defenses on Graphs: A Review, A Tool and Empirical Studies. arXiv preprint arXiv:2003.00653.
  21. Graph structure learning for robust graph neural networks. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 66–74.
  22. Variational diffusion models. Advances in neural information processing systems, 34: 21696–21707.
  23. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114.
  24. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907.
  25. Variational graph auto-encoders. arXiv preprint arXiv:1611.07308.
  26. Variational diffusion autoencoders with random walk sampling. In European Conference on Computer Vision, 362–378. Springer.
  27. Source Localization of Graph Diffusion via Variational Autoencoders for Graph Inverse Problems. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 1010–1020.
  28. Elastic graph neural networks. In International Conference on Machine Learning. PMLR.
  29. Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv:1706.06083.
  30. Revisiting graph neural networks: All we have is low-pass filters. arXiv preprint arXiv:1905.09550.
  31. Adversarially regularized graph autoencoder for graph embedding. arXiv preprint arXiv:1802.04407.
  32. Diffusion variational autoencoders. arXiv preprint arXiv:1901.08991.
  33. Dropedge: Towards deep graph convolutional networks on node classification. arXiv preprint arXiv:1907.10903.
  34. Collective classification in network data. AI magazine.
  35. Pitfalls of Graph Neural Network Evaluation. Relational Representation Learning Workshop, NeurIPS 2018.
  36. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502.
  37. Adversarial attack and defense on graph data: A survey. arXiv preprint arXiv:1812.10528.
  38. Recurrent knowledge graph embedding for effective recommendation. In Proceedings of the 12th ACM conference on recommender systems, 297–305.
  39. Fiedler regularization: Learning neural networks with graph sparsity. In International Conference on Machine Learning, 9346–9355. PMLR.
  40. Balanced label propagation for partitioning massive graphs. In Proceedings of the sixth ACM international conference on Web search and data mining, 507–516.
  41. A simple introduction to Markov Chain Monte–Carlo sampling. Psychonomic bulletin & review, 25(1): 143–154.
  42. Graph Attention Networks. In International Conference on Learning Representations.
  43. Label propagation through linear neighborhoods. IEEE Transactions on Knowledge and Data Engineering, 20(1): 55–67.
  44. Unifying graph convolutional neural networks and label propagation. arXiv preprint arXiv:2002.06755.
  45. Simplifying graph convolutional networks. arXiv preprint arXiv:1902.07153.
  46. Adversarial examples on graph data: Deep insights into attack and defense. arXiv preprint arXiv:1903.01610.
  47. Explicit semantic ranking for academic search via knowledge graph embedding. In Proceedings of the 26th international conference on world wide web, 1271–1279.
  48. Sparse graph attention networks. IEEE Transactions on Knowledge and Data Engineering.
  49. Graph convolutional neural networks for web-scale recommender systems. In Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining, 974–983.
  50. Graphsaint: Graph sampling based inductive learning method. arXiv preprint arXiv:1907.04931.
  51. Variational graph neural networks for road traffic prediction in intelligent transportation systems. IEEE Transactions on Industrial Informatics, 17(4): 2802–2812.
  52. Robust graph convolutional networks against adversarial attacks. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining.
  53. Beyond homophily in graph neural networks: Current limitations and effective designs. Advances in Neural Information Processing Systems, 33: 7793–7804.
  54. Learning from labeled and unlabeled data with label propagation.
  55. Defending Graph Convolutional Networks against Dynamic Graph Perturbations via Bayesian Self-Supervision. Proceedings of the AAAI Conference on Artificial Intelligence, 36(4): 4405–4413.
  56. Deperturbation of Online Social Networks via Bayesian Label Transition. In Proceedings of the 2022 SIAM International Conference on Data Mining (SDM), 603–611. SIAM.
  57. Robust Node Classification on Graphs: Jointly from Bayesian Label Transition and Topology-based Label Propagation. In Proceedings of the 31st ACM International Conference on Information & Knowledge Management, 2795–2805.
  58. How Does Bayesian Noisy Self-Supervision Defend Graph Convolutional Networks? Neural Processing Letters, 1–22.
  59. Adversarial attacks on neural networks for graph data. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 2847–2856.
Citations (7)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets