Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
167 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Deep Efficient Private Neighbor Generation for Subgraph Federated Learning (2401.04336v3)

Published 9 Jan 2024 in cs.LG, cs.AI, and cs.CR

Abstract: Behemoth graphs are often fragmented and separately stored by multiple data owners as distributed subgraphs in many realistic applications. Without harming data privacy, it is natural to consider the subgraph federated learning (subgraph FL) scenario, where each local client holds a subgraph of the entire global graph, to obtain globally generalized graph mining models. To overcome the unique challenge of incomplete information propagation on local subgraphs due to missing cross-subgraph neighbors, previous works resort to the augmentation of local neighborhoods through the joint FL of missing neighbor generators and GNNs. Yet their technical designs have profound limitations regarding the utility, efficiency, and privacy goals of FL. In this work, we propose FedDEP to comprehensively tackle these challenges in subgraph FL. FedDEP consists of a series of novel technical designs: (1) Deep neighbor generation through leveraging the GNN embeddings of potential missing neighbors; (2) Efficient pseudo-FL for neighbor generation through embedding prototyping; and (3) Privacy protection through noise-less edge-local-differential-privacy. We analyze the correctness and efficiency of FedDEP, and provide theoretical guarantees on its privacy. Empirical results on four real-world datasets justify the clear benefits of proposed techniques.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (53)
  1. Deep learning with differential privacy. In CCS, 2016.
  2. Publishing social network graph eigenspectrum with privacy guarantees. IEEE TNSE, 7:892–906, 2019.
  3. Personalized subgraph federated learning. In ICML, 2023.
  4. Privacy profiles and amplification by subsampling. JPC, 10(1), 2020.
  5. Fast unfolding of communities in large networks. JSTAT, 2008.
  6. Fedgraph: Federated graph learning with intelligent sampling. IEEE TPDS, 2021.
  7. Fede: Embedding knowledge graphs in federated setting. In IJCKG, 2020.
  8. Gram: graph-based attention model for healthcare representation learning. In WWW, 2017.
  9. Guaranteeing local differential privacy on ultra-low-power systems. In ISCA, 2018.
  10. Cynthia Dwork. Differential privacy. In ICALP, 2006.
  11. Inverting gradients-how easy is it to break privacy in federated learning? In NeurIPS, 2020.
  12. Dynamic activation of clients and parameters for federated learning over heterogeneous graphs. In ICDE, 2023.
  13. Inductive representation learning on large graphs. In NeurIPS, 2017.
  14. Lightgcn: Simplifying and powering graph convolution network for recommendation. In SIGIR, 2020.
  15. Differentially private triangle and 4-cycle counting in the shuffle model. In SIGSAC, 2022.
  16. The composition theorem for differential privacy. In ICML, 2015.
  17. Variational graph auto-encoders. In Workshop of NeurIPS, 2016.
  18. Semi-supervised classification with graph convolutional networks. In ICLR, 2017.
  19. Fednlp: A research platform for federated learning in natural language processing. In Findings of ACL: NAACL, 2021.
  20. Towards private learning on decentralized graphs with local differential privacy. IEEE TIFS, 17:2936–2946, 2022.
  21. Local augmentation for graph neural networks. In ICML, 2022.
  22. Fedvision: An online visual object detection platform powered by federated learning. In AAAI, 2020.
  23. Exponential random graph estimation under differential privacy. In KDD, 2014.
  24. Feature inference attack on model predictions in vertical federated learning. In ICDE, 2021.
  25. Query-driven active surveying for collective classification. In MLG workshop, 2012.
  26. Comprehensive privacy analysis of deep learning: Passive and active white-box inference attacks against centralized and federated learning. In S&P, 2019.
  27. Lumos: Heterogeneity-aware federated graph learning over decentralized devices. ICDE, 2023.
  28. Differentially private federated knowledge graphs embedding. In CIKM, 2021.
  29. Generating synthetic decentralized social graphs with local differential privacy. In SIGSAC, 2017.
  30. Dropedge: Towards deep graph convolutional networks on node classification. In ICLR, 2020.
  31. Collective classification in network data. AI magazine, 29(3):93–93, 2008.
  32. Pitfalls of graph neural network evaluation. arXiv preprint arXiv:1811.05868, 2018.
  33. Federated model distillation with noise-free differential privacy. In IJCAI, 2021.
  34. Fedproto: Federated prototype learning across heterogeneous clients. In AAAI, 2022.
  35. Graph auto-encoder via neighborhood wasserstein reconstruction. In ICLR, 2022.
  36. Paul Voigt and Axel Von dem Bussche. The eu general data protection regulation (gdpr). 2017.
  37. Traffic flow prediction via spatial temporal graph neural network. In WWW, 2020.
  38. Federatedscope-gnn: Towards a unified, comprehensive and efficient package for federated graph learning. In KDD, 2022.
  39. A federated graph neural network framework for privacy-preserving personalization. Nature Communications, 2022.
  40. Differentially private network data release via structural inference. In KDD, 2014.
  41. Federated graph classification over non-iid graphs. In NeurIPS, 2021.
  42. How powerful are graph neural networks? In ICLR, 2019.
  43. Secure deep graph generation with link differential privacy. In IJCAI, 2021.
  44. Federated machine learning: Concept and applications. TIST, 10(2):1–19, 2019.
  45. Fedgcn: Convergence and communication tradeoffs in federated training of graph convolutional networks. NeurIPS, 2023.
  46. Graph convolutional neural networks for web-scale recommender systems. In WWW, 2018.
  47. Efficient federated learning on knowledge graphs via privacy-preserving relation embedding aggregation. In Findings of EMNLP, 2022.
  48. Subgraph federated learning over heterogeneous graphs. In FedGraph-CIKM, 2022.
  49. Subgraph federated learning with missing neighbor generation. In NeurIPS, 2021.
  50. Fedgsl: Federated graph structure learning for local subgraph augmentation. In Big Data, 2022.
  51. Vertically federated graph neural network for privacy-preserving node classification. In IJCAI, 2021.
  52. Deep leakage from gradients. In NeurIPS, 2019.
  53. Transfer learning of graph neural networks with ego-graph information maximization. In NeurIPS, 2021.
Citations (4)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now