FairSample: Training Fair and Accurate Graph Convolutional Neural Networks Efficiently (2401.14702v1)
Abstract: Fairness in Graph Convolutional Neural Networks (GCNs) becomes a more and more important concern as GCNs are adopted in many crucial applications. Societal biases against sensitive groups may exist in many real world graphs. GCNs trained on those graphs may be vulnerable to being affected by such biases. In this paper, we adopt the well-known fairness notion of demographic parity and tackle the challenge of training fair and accurate GCNs efficiently. We present an in-depth analysis on how graph structure bias, node attribute bias, and model parameters may affect the demographic parity of GCNs. Our insights lead to FairSample, a framework that jointly mitigates the three types of biases. We employ two intuitive strategies to rectify graph structures. First, we inject edges across nodes that are in different sensitive groups but similar in node features. Second, to enhance model fairness and retain model quality, we develop a learnable neighbor sampling policy using reinforcement learning. To address the bias in node features and model parameters, FairSample is complemented by a regularization objective to optimize fairness.
- W. Huang, T. Zhang, Y. Rong, and J. Huang, “Adaptive Sampling Towards Fast Graph Representation Learning,” in NeurIPS, 2018.
- T. N. Kipf and M. Welling, “Semi-supervised classification with graph convolutional networks,” in ICLR, 2017.
- L. Liu, W. Zhang, J. Liu, W. Shi, and Y. Huang, “Learning multi-graph neural network for data-driven job skill prediction,” in IJCNN, 2021.
- L. Cui, H. Seo, M. Tabar, F. Ma, S. Wang, and D. Lee, “DETERRENT: knowledge guided graph attention network for detecting healthcare misinformation,” in KDD, 2020.
- X. He, K. Deng, X. Wang, Y. Li, Y. Zhang, and M. Wang, “Lightgcn: Simplifying and powering graph convolution network for recommendation,” in SIGIR, 2020.
- E. Dai and S. Wang, “Say no to the discrimination: Learning fair graph neural networks with limited sensitive attribute information,” in WSDM, 2021.
- C. Laclau, I. Redko, M. Choudhary, and C. Largeron, “All of the fairness for edge prediction with optimal transport,” in AISTATS, 2021.
- P. Li, Y. Wang, H. Zhao, P. Hong, and H. Liu, “On dyadic fairness: Exploring and mitigating bias in graph connections,” in ICLR, 2021.
- N. Mehrabi, F. Morstatter, N. Saxena, K. Lerman, and A. Galstyan, “A survey on bias and fairness in machine learning,” ACM Comput. Surv., vol. 54, no. 6, pp. 115:1–115:35, 2021.
- A. Agarwal, A. Beygelzimer, M. Dudík, J. Langford, and H. M. Wallach, “A reductions approach to fair classification,” in ICML, 2018.
- T. A. Rahman, B. Surma, M. Backes, and Y. Zhang, “Fairwalk: Towards fair graph embedding,” in IJCAI, 2019.
- S. Xu and T. Strohmer, “Fair data representation for machine learning at the pareto frontier,” CoRR, vol. abs/2201.00292, 2022.
- M. Thelwall, “Homophily in myspace,” Journal of the American Society for Information Science and Technology, vol. 60, no. 2, pp. 219–231, 2009.
- Y. Dong, N. Liu, B. Jalaian, and J. Li, “EDITS: modeling and mitigating data bias for graph neural networks,” in WWW, 2022.
- C. Agarwal, H. Lakkaraju, and M. Zitnik, “Towards a unified framework for fair and stable graph representation learning,” in UAI, 2021.
- A. K. Menon and R. C. Williamson, “The cost of fairness in binary classification,” in FAT, 2018.
- H. Zhao and G. J. Gordon, “Inherent tradeoffs in learning fair representations,” J. Mach. Learn. Res., vol. 23, pp. 57:1–57:26, 2022.
- A. J. Bose and W. L. Hamilton, “Compositional fairness constraints for graph embeddings,” in ICML, 2019.
- W. L. Hamilton, Z. Ying, and J. Leskovec, “Inductive representation learning on large graphs,” in NeurIPS, 2017.
- J. Chen, T. Ma, and C. Xiao, “Fastgcn: Fast learning with graph convolutional networks via importance sampling,” arXiv preprint arXiv:1801.10247, 2018.
- M. Yoon, T. Gervet, B. Shi, S. Niu, Q. He, and J. Yang, “Performance-adaptive sampling strategy towards fast and accurate graph neural networks,” in KDD, 2021.
- F. Kamiran, T. Calders, and M. Pechenizkiy, “Discrimination aware decision tree learning,” in ICDM, 2010.
- F. Kamiran and T. Calders, “Data preprocessing techniques for classification without discrimination,” Knowl. Inf. Syst., vol. 33, no. 1, pp. 1–33, 2011.
- B. L. Thanh, S. Ruggieri, and F. Turini, “k-nn as an implementation of situation testing for discrimination discovery and prevention,” in KDD, 2011.
- S. Hajian and J. Domingo-Ferrer, “A methodology for direct and indirect discrimination prevention in data mining,” IEEE Trans. Knowl. Data Eng., vol. 25, no. 7, pp. 1445–1459, 2013.
- N. Kilbertus, M. G. Rodriguez, B. Schölkopf, K. Muandet, and I. Valera, “Fair decisions despite imperfect predictions,” in AISTATS, 2020.
- M. Feldman, S. A. Friedler, J. Moeller, C. Scheidegger, and S. Venkatasubramanian, “Certifying and removing disparate impact,” in KDD, 2015.
- F. Kamiran, A. Karim, and X. Zhang, “Decision theory for discrimination-aware classification,” in ICDM, 2012.
- I. Valera, A. Singla, and M. G. Rodriguez, “Enhancing the accuracy and fairness of human decision making,” in NeurIPS 2018, 2018.
- M. Donini, L. Oneto, S. Ben-David, J. Shawe-Taylor, and M. Pontil, “Empirical risk minimization under fairness constraints,” in NeurIPS, 2018.
- P. Manisha and S. Gujar, “FNNC: achieving fairness through neural networks,” in IJCAI, 2020.
- L. Oneto, N. Navarin, and M. Donini, “Learning deep fair graph neural networks,” in ESANN, 2020, pp. 31–36.
- A. Grover and J. Leskovec, “node2vec: Scalable feature learning for networks,” in KDD, 2016.
- Ö. D. Kose and Y. Shen, “Fair node representation learning via adaptive data augmentation,” CoRR, 2022.
- J. Ma, R. Guo, M. Wan, L. Yang, A. Zhang, and J. Li, “Learning fair node representations with graph counterfactual fairness,” in WSDM. ACM, 2022.
- M. J. Kusner, J. R. Loftus, C. Russell, and R. Silva, “Counterfactual fairness,” in NeurIPS, 2017.
- Y. Dong, J. Kang, H. Tong, and J. Li, “Individual fairness for graph neural networks: A ranking based approach,” in KDD, 2021.
- J. Ma, J. Deng, and Q. Mei, “Subgroup generalization and fairness of graph neural networks,” in NeurIPS, 2021.
- X. Tang, H. Yao, Y. Sun, Y. Wang, J. Tang, C. Aggarwal, P. Mitra, and S. Wang, “Investigating and mitigating degree-related biases in graph convoltuional networks,” in CIKM, 2020.
- J. Kang, Y. Zhu, Y. Xia, J. Luo, and H. Tong, “Rawlsgcn: Towards rawlsian difference principle on graph convolutional network,” in WWW, 2022.
- X. Huang, Q. Song, Y. Li, and X. Hu, “Graph Recurrent Networks With Attributed Random Walks,” in KDD, 2019.
- A. Bojchevski, J. Klicpera, B. Perozzi, A. Kapoor, M. Blais, B. Rózemberczki, M. Lukasik, and S. Günnemann, “Scaling Graph Neural Networks with Approximate PageRank,” in KDD, 2020.
- Z. Liu, Z. Wu, Z. Zhang, J. Zhou, S. Yang, L. Song, and Y. Qi, “Bandit Samplers for Training Graph Neural Networks,” arXiv:2006.05806 [cs, stat], Jun. 2020.
- W. Cong, R. Forsati, M. Kandemir, and M. Mahdavi, “Minimal Variance Sampling with Provable Guarantees for Fast Training of Graph Neural Networks,” in KDD, 2020.
- Z. Jia, S. Lin, R. Ying, J. You, J. Leskovec, and A. Aiken, “Redundancy-free computation for graph neural networks,” in KDD, 2020.
- Z. Liu, K. Zhou, F. Yang, L. Li, R. Chen, and X. Hu, “EXACT: scalable graph neural networks training via extreme activation compression,” in ICLR, 2022.
- C. Dwork, M. Hardt, T. Pitassi, O. Reingold, and R. S. Zemel, “Fairness through awareness,” in Innovations in Theoretical Computer Science 2012, Cambridge, MA, USA, January 8-10, 2012, 2012.
- F. Wu, A. H. S. Jr., T. Zhang, C. Fifty, T. Yu, and K. Q. Weinberger, “Simplifying graph convolutional networks,” in ICML, 2019.
- L. Chen, L. Wu, R. Hong, K. Zhang, and M. Wang, “Revisiting graph based collaborative filtering: A linear residual graph convolutional network approach,” in AAAI, 2020.
- X. Wang, H. Jin, A. Zhang, X. He, T. Xu, and T. Chua, “Disentangled graph collaborative filtering,” in SIGIR, 2020.
- S. Wu, W. Zhang, F. Sun, and B. Cui, “Graph neural networks in recommender systems: A survey,” CoRR, vol. abs/2011.02260, 2020.
- B. Shin, H. Yang, and J. D. Choi, “The pupil has become the master: Teacher-student model-based word embedding distillation with ensemble learning,” in IJCAI, 2019.
- T. Miyato, T. Kataoka, M. Koyama, and Y. Yoshida, “Spectral normalization for generative adversarial networks,” in ICLR, 2018.
- F. du Pin Calmon, D. Wei, B. Vinzamuri, K. N. Ramamurthy, and K. R. Varshney, “Optimized pre-processing for discrimination prevention,” in NeurIPS, 2017.
- E. Dai, C. Aggarwal, and S. Wang, “NRGNN: learning a label noise resistant graph neural network on sparsely and noisily labeled graphs,” in KDD, 2021.
- T. Zhao, Y. Liu, L. Neves, O. J. Woodford, M. Jiang, and N. Shah, “Data augmentation for graph neural networks,” in EAAI, 2021.
- A. Iscen, G. Tolias, Y. Avrithis, and O. Chum, “Label propagation for deep semi-supervised learning,” in CVPR, 2019.
- L. Yen, M. Saerens, A. Mantrach, and M. Shimbo, “A family of dissimilarity measures between nodes generalizing both the shortest-path and the commute-time distances,” in KDD, 2008.
- M. Chen, Z. Wei, Z. Huang, B. Ding, and Y. Li, “Simple and deep graph convolutional networks,” in ICML, 2020.
- M. Liu, H. Gao, and S. Ji, “Towards deeper graph neural networks,” in KDD, 2020.
- V. François-Lavet, P. Henderson, R. Islam, M. G. Bellemare, and J. Pineau, “An introduction to deep reinforcement learning,” Found. Trends Mach. Learn., vol. 11, no. 3-4, pp. 219–354, 2018.
- A. Beutel, J. Chen, T. Doshi, H. Qian, A. Woodruff, C. Luu, P. Kreitmann, J. Bischof, and E. H. Chi, “Putting fairness principles into practice: Challenges, metrics, and improvements,” in AIES, 2019.
- L. Takac and M. Zabovsky, “Data analysis in public social networks,” in International scientific conference and international workshop present day trends of innovations, vol. 1, no. 6, 2012.
- P. Velickovic, G. Cucurull, A. Casanova, A. Romero, P. Liò, and Y. Bengio, “Graph attention networks,” in ICLR, 2018.
- P. Velickovic, W. Fedus, W. L. Hamilton, P. Liò, Y. Bengio, and R. D. Hjelm, “Deep graph infomax,” in ICLR, 2019.
- Y. Zhu, Y. Xu, F. Yu, Q. Liu, S. Wu, and L. Wang, “Graph contrastive learning with adaptive augmentation,” in WWW, J. Leskovec, M. Grobelnik, M. Najork, J. Tang, and L. Zia, Eds. ACM / IW3C2, 2021.
- “Fairgnn,” https://github.com/EnyanDai/FairGNN, 2022, accessed: 2022-11-22.
- “Dgi,” https://github.com/PetarV-/DGI, 2022, accessed: 2022-11-22.
- “Gca,” https://github.com/CRIPAC-DIG/GCA, 2022, accessed: 2022-11-22.
- “Fairadj,” https://github.com/brandeis-machine-learning/FairAdj, 2022, accessed: 2022-11-22.
- “Fcge,” https://github.com/brandeis-machine-learning/FairAdj/blob/main/src/adv.py, 2022, accessed: 2022-11-22.
- “Edits,” https://github.com/yushundong/EDITS, 2022, accessed: 2022-11-22.
- “Nifty,” https://github.com/chirag126/nifty, 2022, accessed: 2022-11-22.
- R. Islam, S. Pan, and J. R. Foulds, “Can we obtain fairness for free?” in AIES. ACM, 2021.
- L. Chu, L. Wang, Y. Dong, J. Pei, Z. Zhou, and Y. Zhang, “Fedfair: Training fair models in cross-silo federated learning,” arXiv preprint arXiv:2109.05662, 2021.
- P. Ngatchou, A. Zarei, and A. El-Sharkawi, “Pareto multi objective optimization,” in Proceedings of the 13th International Conference on, Intelligent Systems Application to Power Systems. IEEE, 2006.
- R. F. Woolson, “Wilcoxon signed-rank test,” Wiley encyclopedia of clinical trials, pp. 1–3, 2007.