Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
139 tokens/sec
GPT-4o
47 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Federated Graph Condensation with Information Bottleneck Principles (2405.03911v4)

Published 7 May 2024 in cs.LG, cs.AI, cs.CR, and cs.DC

Abstract: Graph condensation (GC), which reduces the size of a large-scale graph by synthesizing a small-scale condensed graph as its substitution, has benefited various graph learning tasks. However, existing GC methods rely on centralized data storage, which is unfeasible for real-world decentralized data distribution, and overlook data holders' privacy-preserving requirements. To bridge this gap, we propose and study the novel problem of federated graph condensation (FGC) for graph neural networks (GNNs). Specifically, we first propose a general framework for FGC, where we decouple the typical gradient matching process for GC into client-side gradient calculation and server-side gradient matching, integrating knowledge from multiple clients' subgraphs into one smaller condensed graph. Nevertheless, our empirical studies show that under the federated setting, the condensed graph will consistently leak data membership privacy, i.e., the condensed graph during federated training can be utilized to steal training data under the membership inference attack (MIA). To tackle this issue, we innovatively incorporate information bottleneck principles into the FGC, which only needs to extract partial node features in one local pre-training step and utilize the features during federated training. Theoretical and experimental analyses demonstrate that our framework consistently protects membership privacy during training. Meanwhile, it can achieve comparable and even superior performance against existing centralized GC and federated graph learning (FGL) methods.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (54)
  1. Deep Variational Information Bottleneck. In ICLR (Poster). OpenReview.net.
  2. Personalized Subgraph Federated Learning. In ICML (Proceedings of Machine Learning Research, Vol. 202). PMLR, 1396–1415.
  3. Specformer: Spectral Graph Neural Networks Meet Transformers. In ICLR. OpenReview.net.
  4. Beyond Low-frequency Information in Graph Convolutional Networks. In AAAI. AAAI Press, 3950–3957.
  5. The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks. In USENIX Security Symposium. USENIX Association, 267–284.
  6. Dataset Distillation by Matching Training Trajectories. In CVPR. IEEE, 10708–10717.
  7. Poster: Membership Inference Attacks via Contrastive Learning. In CCS. ACM, 3555–3557.
  8. A Unified Framework of Graph Information Bottleneck for Robustness and Membership Privacy. In KDD. ACM, 368–379.
  9. Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering. In NIPS. 3837–3845.
  10. Privacy for Free: How does Dataset Condensation Help Privacy?. In ICML (Proceedings of Machine Learning Research, Vol. 162). PMLR, 5378–5396.
  11. Metapath-guided Heterogeneous Graph Neural Network for Intent Recommendation. In KDD. ACM, 2478–2486.
  12. Protein Interface Prediction using Graph Convolutional Networks. In NIPS. 6530–6539.
  13. A Survey on Heterogeneous Federated Learning. CoRR abs/2210.04505 (2022).
  14. Shion Guha and Stephen B. Wicker. 2015. Do Birds of a Feather Watch Each Other?: Homophily and Social Surveillance in Location Based Social Networks. In CSCW. ACM, 1010–1020.
  15. Inductive Representation Learning on Large Graphs. In NIPS. 1024–1034.
  16. Node-Level Membership Inference Attacks Against Graph Neural Networks. CoRR abs/2102.05429 (2021).
  17. Condensing Graphs via One-Step Gradient Matching. In KDD. ACM, 720–730.
  18. Graph Condensation for Graph Neural Networks. In ICLR. OpenReview.net.
  19. Label-Only Model Inversion Attacks via Boundary Repulsion. In CVPR. IEEE, 15025–15033.
  20. Thomas N. Kipf and Max Welling. 2017. Semi-Supervised Classification with Graph Convolutional Networks. In ICLR (Poster). OpenReview.net.
  21. Predict then Propagate: Graph Neural Networks meet Personalized PageRank. In ICLR (Poster). OpenReview.net.
  22. Query-Efficient Black-Box Attack by Active Learning. In ICDM. IEEE Computer Society, 1200–1205.
  23. Zhizhong Li and Derek Hoiem. 2016. Learning Without Forgetting. In ECCV (4) (Lecture Notes in Computer Science, Vol. 9908). Springer, 614–629.
  24. Graph Condensation via Receptive Field Distribution Matching. CoRR abs/2206.13697 (2022).
  25. Communication-Efficient Learning of Deep Networks from Decentralized Data. In AISTATS (Proceedings of Machine Learning Research, Vol. 54). PMLR, 1273–1282.
  26. Machine Learning with Membership Privacy using Adversarial Regularization. In CCS. ACM, 634–646.
  27. Comprehensive Privacy Analysis of Deep Learning: Passive and Active White-box Inference Attacks against Centralized and Federated Learning. In IEEE Symposium on Security and Privacy. IEEE, 739–753.
  28. Membership Inference Attack on Graph Neural Networks. In TPS-ISA. IEEE, 11–20.
  29. Vulnerability Intelligence Alignment via Masked Graph Attention Networks. In CCS. ACM, 2202–2216.
  30. ML-Leaks: Model and Data Independent Membership Inference Attacks and Defenses on Machine Learning Models. In NDSS. The Internet Society.
  31. Reza Shokri and Vitaly Shmatikov. 2015. Privacy-Preserving Deep Learning. In CCS. ACM, 1310–1321.
  32. Membership Inference Attacks Against Machine Learning Models. In 2017 IEEE Symposium on Security and Privacy, SP 2017, San Jose, CA, USA, May 22-26, 2017. IEEE Computer Society, 3–18.
  33. Federated Learning on Non-IID Graphs via Structural Knowledge Sharing. In AAAI. AAAI Press, 9953–9961.
  34. Graph Attention Networks. In ICLR (Poster). OpenReview.net.
  35. BNS-GCN: Efficient Full-Graph Training of Graph Convolutional Networks with Partition-Parallelism and Random Boundary Node Sampling. In MLSys. mlsys.org.
  36. GraphFL: A Federated Learning Framework for Semi-Supervised Node Classification on Graphs. In ICDM. IEEE, 498–507.
  37. Dataset Distillation. CoRR abs/1811.10959 (2018).
  38. Simplifying Graph Convolutional Networks. In ICML (Proceedings of Machine Learning Research, Vol. 97). PMLR, 6861–6871.
  39. Graph Information Bottleneck. In NeurIPS.
  40. Federated Graph Classification over Non-IID Graphs. In NeurIPS. 18839–18852.
  41. How Powerful are Graph Neural Networks?. In ICLR. OpenReview.net.
  42. Federated Learning - Privacy and Incentive. Lecture Notes in Computer Science, Vol. 12500. Springer.
  43. FedFed: Feature Distillation against Data Heterogeneity in Federated Learning. In NeurIPS.
  44. FedGCN: Convergence-Communication Tradeoffs in Federated Training of Graph Convolutional Networks. In NeurIPS.
  45. Federated Graph Learning - A Position Paper. CoRR abs/2105.11099 (2021).
  46. Subgraph Federated Learning with Missing Neighbor Generation. In NeurIPS. 6671–6682.
  47. BaLeNAS: Differentiable Architecture Search via the Bayesian Learning Rule. In CVPR. IEEE, 11861–11870.
  48. Membership Inference Attacks Against Recommender Systems. In CCS. ACM, 864–879.
  49. Bo Zhao and Hakan Bilen. 2021. Dataset Condensation with Differentiable Siamese Augmentation. In ICML (Proceedings of Machine Learning Research, Vol. 139). PMLR, 12674–12685.
  50. Bo Zhao and Hakan Bilen. 2023. Dataset Condensation with Distribution Matching. In WACV. IEEE, 6503–6512.
  51. Dataset Condensation with Gradient Matching. In ICLR. OpenReview.net.
  52. Structure-free Graph Condensation: From Large-scale Graphs to Condensed Graph-free Data. In NeurIPS.
  53. Interpreting and Unifying Graph Neural Networks with An Optimization Framework. In WWW. ACM / IW3C2, 1215–1226.
  54. Federated learning of molecular properties with graph neural networks in a heterogeneous setting. Patterns 3, 6 (2022), 100521.
Citations (2)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now