Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
119 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

A Unified Framework of Graph Information Bottleneck for Robustness and Membership Privacy (2306.08604v1)

Published 14 Jun 2023 in cs.LG, cs.AI, and cs.CR

Abstract: Graph Neural Networks (GNNs) have achieved great success in modeling graph-structured data. However, recent works show that GNNs are vulnerable to adversarial attacks which can fool the GNN model to make desired predictions of the attacker. In addition, training data of GNNs can be leaked under membership inference attacks. This largely hinders the adoption of GNNs in high-stake domains such as e-commerce, finance and bioinformatics. Though investigations have been made in conducting robust predictions and protecting membership privacy, they generally fail to simultaneously consider the robustness and membership privacy. Therefore, in this work, we study a novel problem of developing robust and membership privacy-preserving GNNs. Our analysis shows that Information Bottleneck (IB) can help filter out noisy information and regularize the predictions on labeled samples, which can benefit robustness and membership privacy. However, structural noises and lack of labels in node classification challenge the deployment of IB on graph-structured data. To mitigate these issues, we propose a novel graph information bottleneck framework that can alleviate structural noises with neighbor bottleneck. Pseudo labels are also incorporated in the optimization to minimize the gap between the predictions on the labeled set and unlabeled set for membership privacy. Extensive experiments on real-world datasets demonstrate that our method can give robust predictions and simultaneously preserve membership privacy.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (57)
  1. Deep learning with differential privacy. In CCS. 308–318.
  2. Deep variational information bottleneck. arXiv preprint arXiv:1612.00410 (2016).
  3. Molecular generative Graph Neural Networks for Drug Discovery. Neurocomputing 450 (2021), 242–252.
  4. Differentially private empirical risk minimization. JMLR 12, 3 (2011).
  5. Understanding structural vulnerability in graph convolutional networks. arXiv preprint arXiv:2108.06280 (2021).
  6. Label-only membership inference attacks. In ICML. PMLR, 1964–1974.
  7. NRGNN: Learning a Label Noise-Resistant Graph Neural Network on Sparsely and Noisily Labeled Graphs. arXiv preprint arXiv:2106.04714 (2021).
  8. Towards robust graph neural networks for noisy graphs with sparse labels. In WSDM. 181–191.
  9. Unnoticeable Backdoor Attacks on Graph Neural Networks. In WWW. 2263–2273.
  10. Enyan Dai and Suhang Wang. 2021a. Say No to the Discrimination: Learning Fair Graph Neural Networks with Limited Sensitive Attribute Information. In WSDM. 680–688.
  11. Enyan Dai and Suhang Wang. 2021b. Towards self-explainable graph neural network. In Proceedings of the 30th ACM International Conference on Information & Knowledge Management. 302–311.
  12. A Comprehensive Survey on Trustworthy Graph Neural Networks: Privacy, Robustness, Fairness, and Explainability. arXiv preprint arXiv:2204.08570 (2022).
  13. Label-Wise Graph Convolutional Network for Heterophilic Graphs. In Learning on Graphs Conference. https://openreview.net/forum?id=HRmby7yVVuF
  14. Adversarial attack on graph structured data. ICML (2018).
  15. All You Need Is Low (Rank) Defending Against Adversarial Attacks on Graphs. In WSDM. 169–177.
  16. Robustness of graph neural networks at scale. NeurIPS 34 (2021), 7637–7649.
  17. Inductive representation learning on large graphs. In NeurIPS. 1024–1034.
  18. Logan: Membership inference attacks against generative models. arXiv preprint arXiv:1705.07663 (2017).
  19. Node-level membership inference attacks against graph neural networks. arXiv preprint arXiv:2102.05429 (2021).
  20. Learning deep representations by mutual information estimation and maximization. arXiv preprint arXiv:1808.06670 (2018).
  21. ZINC: a free tool to discover chemistry for biology. Journal of chemical information and modeling 52, 7 (2012), 1757–1768.
  22. Categorical reparameterization with gumbel-softmax. arXiv preprint arXiv:1611.01144 (2016).
  23. Graph structure learning for robust graph neural networks. In SIGKDD. 66–74.
  24. Distilling robust and non-robust features in adversarial examples by information bottleneck. NeurIPS 34 (2021), 17148–17159.
  25. Thomas N Kipf and Max Welling. 2016. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907 (2016).
  26. Dong-Hyun Lee et al. 2013. Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks. In Workshop on challenges in representation learning, ICML, Vol. 3. 896.
  27. Reliable Representations Make A Stronger Defender: Unsupervised Structure Refinement for Robust GNN. In SIGKDD. 925–935.
  28. Elastic graph neural networks. In ICML. PMLR, 6837–6849.
  29. Haohui Lu and Shahadat Uddin. 2021. A weighted patient network-based framework for predicting chronic diseases using graph neural networks. Scientific reports 11, 1 (2021), 22607.
  30. Interpretable and generalizable graph learning via stochastic attention mechanism. In ICML. PMLR, 15524–15543.
  31. Machine learning with membership privacy using adversarial regularization. In CCS. 634–646.
  32. Membership inference attack on graph neural networks. In TPS-ISA. IEEE, 11–20.
  33. Semi-supervised knowledge transfer for deep learning from private training data. arXiv preprint arXiv:1610.05755 (2016).
  34. Gcc: Graph contrastive coding for graph neural network pre-training. In SIGKDD. 1150–1160.
  35. Ml-leaks: Model and data independent membership inference attacks and defenses on machine learning models. arXiv preprint arXiv:1806.01246 (2018).
  36. Reza Shokri and Vitaly Shmatikov. 2015. Privacy-preserving deep learning. In CCS. 1310–1321.
  37. Membership inference attacks against machine learning models. In 2017 IEEE symposium on security and privacy (SP). IEEE, 3–18.
  38. Graph structure learning with variational information bottleneck. In AAAI, Vol. 36. 4165–4174.
  39. Transferring Robustness for Graph Neural Network Against Poisoning Attacks. In WSDM. 600–608.
  40. The information bottleneck method. arXiv preprint physics/0004057 (2000).
  41. Graph attention networks. ICLR (2018).
  42. A Semi-supervised Graph Attentive Network for Financial Fraud Detection. In ICDM. IEEE, 598–607.
  43. Revisiting Hilbert-Schmidt Information Bottleneck for Adversarial Robustness. NeurIPS 34 (2021), 586–597.
  44. Adapting membership inference attacks to gnn for graph classification: Approaches and implications. In ICDM. IEEE, 1421–1426.
  45. Simplifying graph convolutional networks. In ICML. PMLR, 6861–6871.
  46. Adversarial examples on graph data: Deep insights into attack and defense. arXiv preprint arXiv:1903.01610 (2019).
  47. Graph information bottleneck. NeurIPS 33 (2020), 20437–20448.
  48. Topology attack and defense for graph neural networks: An optimization perspective. arXiv preprint arXiv:1906.04214 (2019).
  49. Graph convolutional neural networks for web-scale recommender systems. In SIGKDD. 974–983.
  50. Graph information bottleneck for subgraph recognition. arXiv preprint arXiv:2010.05563 (2020).
  51. GraphSAINT: Graph Sampling Based Inductive Learning Method. In ICLR.
  52. Xiang Zhang and Marinka Zitnik. 2020. GNNGuard: Defending Graph Neural Networks against Adversarial Attacks. In NeurIPS, Vol. 33. 9263–9275.
  53. Semi-Supervised Graph-to-Graph Translation. In CIKM. 1863–1872.
  54. Robust graph convolutional networks against adversarial attacks. In SIGKDD. 1399–1407.
  55. Tdgia: Effective injection attacks on graph neural networks. In SIGKDD, pages=2461–2471, year=2021.
  56. Adversarial attacks on neural networks for graph data. In SIGKDD. 2847–2856.
  57. Daniel Zügner and Stephan Günnemann. 2019. Adversarial Attacks on Graph Neural Networks via Meta Learning. In ICLR.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (8)
  1. Enyan Dai (32 papers)
  2. Limeng Cui (19 papers)
  3. Zhengyang Wang (48 papers)
  4. Xianfeng Tang (62 papers)
  5. Yinghan Wang (7 papers)
  6. Monica Cheng (3 papers)
  7. Bing Yin (56 papers)
  8. Suhang Wang (118 papers)
Citations (12)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now