Equipping Federated Graph Neural Networks with Structure-aware Group Fairness (2310.12350v3)
Abstract: Graph Neural Networks (GNNs) have been widely used for various types of graph data processing and analytical tasks in different domains. Training GNNs over centralized graph data can be infeasible due to privacy concerns and regulatory restrictions. Thus, federated learning (FL) becomes a trending solution to address this challenge in a distributed learning paradigm. However, as GNNs may inherit historical bias from training data and lead to discriminatory predictions, the bias of local models can be easily propagated to the global model in distributed settings. This poses a new challenge in mitigating bias in federated GNNs. To address this challenge, we propose $\text{F}2$GNN, a Fair Federated Graph Neural Network, that enhances group fairness of federated GNNs. As bias can be sourced from both data and learning algorithms, $\text{F}2$GNN aims to mitigate both types of bias under federated settings. First, we provide theoretical insights on the connection between data bias in a training graph and statistical fairness metrics of the trained GNN models. Based on the theoretical analysis, we design $\text{F}2$GNN which contains two key components: a fairness-aware local model update scheme that enhances group fairness of the local models on the client side, and a fairness-weighted global model update scheme that takes both data bias and fairness metrics of local models into consideration in the aggregation process. We evaluate $\text{F}2$GNN empirically versus a number of baseline methods, and demonstrate that $\text{F}2$GNN outperforms these baselines in terms of both fairness and model accuracy.
- Fairness and Machine Learning: Limitations and Opportunities. fairmlbook.org, 2019.
- Structured graph convolutional networks with stochastic masks for recommender systems. In SIGIR ’21: The 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 614–623, 2021.
- Edits: Modeling and mitigating data bias for graph neural networks. In Proceedings of the ACM Web Conference, pages 1259–1269, 2022.
- Malicious transaction identification in digital currency via federated graph deep learning. In IEEE INFOCOM 2022 - IEEE Conference on Computer Communications Workshops, pages 1–6, 2022.
- Say no to the discrimination: Learning fair graph neural networks with limited sensitive attribute information. In WSDM ’21, The Fourteenth ACM International Conference on Web Search and Data Mining, pages 680–688, 2021.
- Fairness-aware agnostic federated learning. In Proceedings of the 2021 SIAM International Conference on Data Mining, pages 181–189, 2021.
- Congestion control and fairness for many-to-one routing in sensor networks. In Proceedings of the 2nd International Conference on Embedded Networked Sensor Systems, pages 148–161, 2004.
- Atilla Eryilmaz and R. Srikant. Joint congestion control, routing, and MAC for stability and fairness in wireless networks. IEEE J. Sel. Areas Commun., 24(8):1514–1524, 2006.
- Fairfed: Enabling group fairness in federated learning. CoRR, abs/2110.00857, 2021.
- Certifying and removing disparate impact. In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 259–268, 2015.
- Fast graph representation learning with PyTorch Geometric. In ICLR Workshop on Representation Learning on Graphs and Manifolds, 2019.
- Graph neural networks for social recommendation. In The World Wide Web Conference, pages 417–426, 2019.
- Debiasing knowledge graph embeddings. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 7332–7345, 2020.
- Deep fair models for complex data: Graphs labeling and explainable face recognition. Neurocomputing, 470:318–334, 2022.
- Kunihiko Fukushima. Visual feature extraction by a multilayered network of analog threshold elements. IEEE Transactions on Systems Science and Cybernetics, 5(4):322–333, 1969.
- Statistical methods in education and psychology. Allyn and Bacon Boston, 1996.
- node2vec: Scalable feature learning for networks. In Proceedings of the 22nd ACM SIGKDD international conference on Knowledge discovery and data mining, pages 855–864, 2016.
- Fedgraphnn: A federated learning benchmark system for graph neural networks. In ICLR Workshop on Distributed and Private Machine Learning, 2021.
- Adversarial inter-group link injection degrades the fairness of graph neural networks. arXiv preprint arXiv:2209.05957, 2022.
- Adversarial inter-group link injection degrades the fairness of graph neural networks. In IEEE International Conference on Data Mining, pages 975–980, 2022.
- Knowledge transfer for out-of-knowledge-base entities : A graph neural network approach. In Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, pages 1802–1808, 2017.
- Equality of opportunity in supervised learning. In Advances in Neural Information Processing Systems 29: Annual Conference on Neural Information Processing Systems, pages 3315–3323, 2016.
- Exploring network structure, dynamics, and function using networkx. In Proceedings of the 7th Python in Science Conference, pages 11 – 15, 2008.
- Provably fair federated learning via bounded group loss. CoRR, abs/2203.10190, 2022.
- Inductive representation learning on large graphs. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems, pages 1024–1034, 2017.
- FMP: toward fair graph message passing against topology bias. CoRR, abs/2202.04187, 2022.
- Fairness-aware classifier with prejudice remover regularizer. In Machine Learning and Knowledge Discovery in Databases - European Conference, volume 7524, pages 35–50, 2012.
- Adam: A method for stochastic optimization. In 3rd International Conference on Learning Representations, 2015.
- Fair node representation learning via adaptive data augmentation. arXiv preprint, abs/2201.08549, 2022.
- Semi-supervised classification with graph convolutional networks. In 5th International Conference on Learning Representations, Conference Track Proceedings, 2017.
- Ditto: Fair and robust federated learning through personalization. In Proceedings of the 38th International Conference on Machine Learning, volume 139, pages 6357–6368, 2021.
- Personalized federated learning towards communication efficiency, robustness and fairness. Advances in Neural Information Processing Systems, 2022.
- On the convergence of fedavg on non-iid data. In 8th International Conference on Learning Representations, 2020.
- Learning fair graph representations via automated data augmentations. In The Eleventh International Conference on Learning Representations, 2023.
- Automated data augmentations for graph classification. CoRR, abs/2202.13248, 2022.
- All of the fairness for edge prediction with optimal transport. In The 24th International Conference on Artificial Intelligence and Statistics, volume 130, pages 1774–1782, 2021.
- Fair resource allocation in federated learning. In 8th International Conference on Learning Representations, 2020.
- Spherical message passing for 3d molecular graphs. In The 10th International Conference on Learning Representations, 2022.
- Deep learning of high-order interactions for protein interface prediction. In KDD ’20: The 26th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 679–687, 2020.
- Communication-efficient learning of deep networks from decentralized data. In Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, pages 1273–1282, 2017.
- A survey on bias and fairness in machine learning. ACM Comput. Surv., 54(6):115:1–115:35, 2022.
- Bursting the filter bubble: Fairness-aware network link prediction. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pages 841–848, 2020.
- Bursting the filter bubble: Fairness-aware network link prediction. In The Thirty-Fourth AAAI Conference on Artificial Intelligence, pages 841–848, 2020.
- Frank Nielsen. On a variational definition for the jensen-shannon symmetrization of distances based on the information radius. Entropy, 23(4):464, 2021.
- Topology-aware graph signal sampling for pooling in graph neural networks. In 26th International Computer Conference, Computer Society of Iran, pages 1–7, 2021.
- Learning deep fair graph neural networks. In 28th European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning, pages 31–36, 2020.
- Domain-aware federated social bot detection with multi-relational graph neural networks. In International Joint Conference on Neural Networks, pages 1–8, 2022.
- Fairvfl: A fair vertical federated learning framework with contrastive adversarial learning. Advances in Neural Information Processing Systems, 35:7852–7865, 2022.
- Ensuring fairness in machine learning to advance health equity. Annals of internal medicine, 169(12):866–872, 2018.
- Andrew Michael Saxe. Deep linear neural networks: A theory of learning in the brain and mind. Stanford University, 2015.
- A framework for understanding unintended consequences of machine learning. CoRR, abs/1901.10002, 2019.
- Exact solutions to the nonlinear dynamics of learning in deep linear neural networks. In 2nd International Conference on Learning Representations, 2014.
- Fairdrop: Biased edge dropout for enhancing fairness in graph representation learning. IEEE Transactions on Artificial Intelligence, 3(3):344–354, 2021.
- Fairdrop: Biased edge dropout for enhancing fairness in graph representation learning. IEEE Trans. Artif. Intell., 3(3):344–354, 2022.
- Towards federated graph learning for collaborative financial crimes detection. CoRR, abs/1909.12946, 2019.
- Data analysis in public social networks. In International scientific conference and international workshop present day trends of innovations, volume 1, 2012.
- Mitigating group bias in federated learning: Beyond local fairness. arXiv preprint arXiv:2305.09931, 2023.
- Uncovering the structural fairness in graph contrastive learning. In Advances in Neural Information Processing Systems, 2022.
- Improving fairness in graph neural networks via mitigating sensitive attribute leakage. In KDD ’22: The 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 1938–1948, 2022.
- Deep graph library: A graph-centric, highly-performant package for graph neural networks. arXiv preprint arXiv:1909.01315, 2019.
- Federated graph classification over non-iid graphs. In Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems, pages 18839–18852, 2021.
- Graph contrastive learning automated. In Proceedings of the 38th International Conference on Machine Learning, volume 139, pages 12121–12132, 2021.
- Beyond parity: Fairness objectives for collaborative filtering. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems, pages 2921–2930, 2017.
- Fedgcn: Convergence and communication tradeoffs in federated training of graph convolutional networks. CoRR, abs/2201.12433, 2022.
- On explainability of graph neural networks via subgraph explorations. In Proceedings of the 38th International Conference on Machine Learning, volume 139, pages 12241–12252, 2021.
- Improving fairness via federated learning. CoRR, abs/2110.15545, 2021.
- Fair representation learning for heterogeneous information networks. In Proceedings of the Fifteenth International AAAI Conference on Web and Social Media, pages 877–887, 2021.
- Federated learning with non-iid data. arXiv preprint arXiv:1806.00582, 2018.
- Deep graph contrastive representation learning. CoRR, abs/2006.04131, 2020.
- ASFGNN: automated separated-federated graph neural network. Peer-to-Peer Netw. Appl., 14(3):1692–1704, 2021.
- Nan Cui (5 papers)
- Xiuling Wang (4 papers)
- Wendy Hui Wang (4 papers)
- Violet Chen (3 papers)
- Yue Ning (24 papers)