Bkd-FedGNN: A Benchmark for Classification Backdoor Attacks on Federated Graph Neural Network (2306.10351v1)
Abstract: Federated Graph Neural Network (FedGNN) has recently emerged as a rapidly growing research topic, as it integrates the strengths of graph neural networks and federated learning to enable advanced machine learning applications without direct access to sensitive data. Despite its advantages, the distributed nature of FedGNN introduces additional vulnerabilities, particularly backdoor attacks stemming from malicious participants. Although graph backdoor attacks have been explored, the compounded complexity introduced by the combination of GNNs and federated learning has hindered a comprehensive understanding of these attacks, as existing research lacks extensive benchmark coverage and in-depth analysis of critical factors. To address these limitations, we propose Bkd-FedGNN, a benchmark for backdoor attacks on FedGNN. Specifically, Bkd-FedGNN decomposes the graph backdoor attack into trigger generation and injection steps, and extending the attack to the node-level federated setting, resulting in a unified framework that covers both node-level and graph-level classification tasks. Moreover, we thoroughly investigate the impact of multiple critical factors in backdoor attacks on FedGNN. These factors are categorized into global-level and local-level factors, including data distribution, the number of malicious attackers, attack time, overlapping rate, trigger size, trigger type, trigger position, and poisoning rate. Finally, we conduct comprehensive evaluations on 13 benchmark datasets and 13 critical factors, comprising 1,725 experimental configurations for node-level and graph-level tasks from six domains. These experiments encompass over 8,000 individual tests, allowing us to provide a thorough evaluation and insightful observations that advance our understanding of backdoor attacks on FedGNN.The Bkd-FedGNN benchmark is publicly available at https://github.com/usail-hkust/BkdFedGCN.
- Beyond real-world benchmark datasets: An empirical study of node classification with gnns. In NeurIPS, 2022.
- BOND: benchmarking unsupervised outlier node detection on static attributed graphs. In NeurIPS, 2022.
- Multi-agent graph convolutional reinforcement learning for dynamic electric vehicle charging pricing. In Proceedings of the 28th ACM SIGKDD conference on knowledge discovery and data mining, pages 2471–2481, 2022.
- CGLB: benchmark tasks for continual graph learning. In NeurIPS, 2022.
- GOOD: A graph out-of-distribution benchmark. In NeurIPS, 2022.
- Nas-bench-graph: Benchmarking graph neural architecture search. In NeurIPS, 2022.
- A comprehensive study on large-scale graph training: Benchmarking and rethinking. In NeurIPS, 2022.
- Dgraph: A large-scale financial dataset for graph anomaly detection. In NeurIPS, 2022.
- Spreadgnn: Decentralized multi-task federated learning for graph neural networks on molecular data, 2021.
- Attack of the tails: Yes, you really can backdoor federated learning. In In NeurIPS, volume 33, pages 16070–16084, 2020.
- How to backdoor federated learning. In International Conference on Artificial Intelligence and Statistics, pages 2938–2948. PMLR, 2020.
- CRFL: certifiably robust federated learning against backdoor attacks. In Proceedings of the 38th International Conference on Machine Learning, ICML 2021, 18-24 July 2021, Virtual Event, volume 139 of Proceedings of Machine Learning Research, pages 11372–11382. PMLR, 2021.
- Learning to backdoor federated learning. arXiv preprint arXiv:2303.03320, 2023.
- Defending against backdoors in federated learning with robust learning rate. In Thirty-Fifth AAAI Conference on Artificial Intelligence, AAAI 2021, pages 9268–9276. AAAI Press, 2021.
- Graph-fraudster: Adversarial attacks on graph neural network-based vertical federated learning. IEEE Transactions on Computational Social Systems, 2022.
- More is better (mostly): On the backdoor attacks in federated graph neural networks. In Proceedings of the 38th Annual Computer Security Applications Conference, pages 684–698, 2022.
- Federated unlearning: How to efficiently erase a client in FL? CoRR, abs/2207.05521, 2022.
- Detrust-FL: Privacy-preserving federated learning in decentralized trust setting. In IEEE 15th International Conference on Cloud Computing, CLOUD 2022, Barcelona, Spain, July 10-16, 2022, pages 417–426. IEEE, 2022.
- Neural message passing for quantum chemistry. In Proceedings of the 34th International Conference on Machine Learning, ICML 2017, Sydney, NSW, Australia, 6-11 August 2017, volume 70 of Proceedings of Machine Learning Research, pages 1263–1272. PMLR, 2017.
- Deep graph learning: Foundations, advances and applications. In KDD ’20: The 26th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Virtual Event, CA, USA, August 23-27, 2020, pages 3555–3556. ACM, 2020.
- Semi-supervised classification with graph convolutional networks. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings. OpenReview.net, 2017.
- Graph attention networks. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings, 2018.
- Inductive representation learning on large graphs. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA, pages 1024–1034, 2017.
- Communication-efficient learning of deep networks from decentralized data. In Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, AISTATS 2017, 20-22 April 2017, Fort Lauderdale, FL, USA, volume 54 of Proceedings of Machine Learning Research, pages 1273–1282. PMLR, 2017.
- Backdoor attacks to graph neural networks. In Proceedings of the 26th ACM Symposium on Access Control Models and Technologies, pages 15–26, 2021.
- Federatedscope-gnn: Towards a unified, comprehensive and efficient package for federated graph learning. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD ’22, page 4110–4120, New York, NY, USA, 2022.
- Subgraph federated learning with missing neighbor generation. Advances in Neural Information Processing Systems, 34:6671–6682, 2021.
- Local model poisoning attacks to byzantine-robust federated learning. In 29th USENIX Security Symposium, USENIX Security 2020, August 12-14, 2020, pages 1605–1622. USENIX Association, 2020.
- Federated learning on non-iid data silos: An experimental study. In 2022 IEEE 38th International Conference on Data Engineering (ICDE), pages 965–978. IEEE, 2022.
- Collective dynamics of ‘small-world’networks. nature, 393(6684):440–442, 1998.
- Emergence of scaling in random networks. science, 286(5439):509–512, 1999.
- Generating random regular graphs quickly. Combinatorics, Probability and Computing, 8(4):377–396, 1999.
- Graph backdoor. In USENIX Security Symposium, pages 1523–1540, 2021.
- Unnoticeable backdoor attacks on graph neural networks. In Proceedings of the ACM Web Conference 2023, WWW ’23, page 2263–2273, New York, NY, USA, 2023.
- Edgar N Gilbert. Random graphs. The Annals of Mathematical Statistics, 30(4):1141–1144, 1959.
- Rethinking the trigger-injecting position in graph backdoor attack. arXiv preprint arXiv:2304.02277, 2023.
- Transferable graph backdoor attack. In Proceedings of the 25th International Symposium on Research in Attacks, Intrusions and Defenses, pages 321–332, 2022.
- Explainability-based backdoor attacks against graph neural networks. In Proceedings of the 3rd ACM Workshop on Wireless Security and Machine Learning, pages 31–36, 2021.
- Motif-backdoor: Rethinking the backdoor attack on graph neural networks via motifs. IEEE Transactions on Computational Social Systems, 2023.
- Data poisoning attacks against federated learning systems. In Computer Security–ESORICS 2020: 25th European Symposium on Research in Computer Security, ESORICS 2020, Guildford, UK, September 14–18, 2020, Proceedings, Part I 25, pages 480–501. Springer, 2020.
- Revisiting semi-supervised learning with graph embeddings. In Proceedings of the 33nd International Conference on Machine Learning, ICML 2016, New York City, NY, USA, June 19-24, 2016, volume 48 of JMLR Workshop and Conference Proceedings, pages 40–48. JMLR.org, 2016.
- Pitfalls of graph neural network evaluation. Relational Representation Learning Workshop, NeurIPS 2018, 2018.
- Image-based recommendations on styles and substitutes. In Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’15, page 43–52. Association for Computing Machinery, 2015.
- IAM graph database repository for graph based pattern recognition and machine learning. In Structural, Syntactic, and Statistical Pattern Recognition, Joint IAPR International Workshop, SSPR & SPR 2008, Orlando, USA, December 4-6, 2008. Proceedings, volume 5342 of Lecture Notes in Computer Science, pages 287–297. Springer, 2008.
- Comparison of descriptor spaces for chemical compound retrieval and classification. In Sixth International Conference on Data Mining (ICDM’06), pages 678–689, 2006.
- The network data repository with interactive graph analytics and visualization. Proceedings of the AAAI Conference on Artificial Intelligence, 29(1), Mar. 2015.
- Protein function prediction via graph kernels. In Proceedings Thirteenth International Conference on Intelligent Systems for Molecular Biology 2005, Detroit, MI, USA, 25-29 June 2005, pages 47–56, 2005.
- Democracy and dictatorship revisited. Public choice, pages 67–101, 2010.
- Understanding attention and generalization in graph neural networks. Advances in neural information processing systems, 32, 2019.