MAPPING: Debiasing Graph Neural Networks for Fair Node Classification with Limited Sensitive Information Leakage (2401.12824v1)
Abstract: Despite remarkable success in diverse web-based applications, Graph Neural Networks(GNNs) inherit and further exacerbate historical discrimination and social stereotypes, which critically hinder their deployments in high-stake domains such as online clinical diagnosis, financial crediting, etc. However, current fairness research that primarily craft on i.i.d data, cannot be trivially replicated to non-i.i.d. graph structures with topological dependence among samples. Existing fair graph learning typically favors pairwise constraints to achieve fairness but fails to cast off dimensional limitations and generalize them into multiple sensitive attributes; besides, most studies focus on in-processing techniques to enforce and calibrate fairness, constructing a model-agnostic debiasing GNN framework at the pre-processing stage to prevent downstream misuses and improve training reliability is still largely under-explored. Furthermore, previous work on GNNs tend to enhance either fairness or privacy individually but few probe into their interplays. In this paper, we propose a novel model-agnostic debiasing framework named MAPPING (\underline{M}asking \underline{A}nd \underline{P}runing and Message-\underline{P}assing train\underline{ING}) for fair node classification, in which we adopt the distance covariance($dCov$)-based fairness constraints to simultaneously reduce feature and topology biases in arbitrary dimensions, and combine them with adversarial debiasing to confine the risks of attribute inference attacks. Experiments on real-world datasets with different GNN variants demonstrate the effectiveness and flexibility of MAPPING. Our results show that MAPPING can achieve better trade-offs between utility and fairness, and mitigate privacy risks of sensitive information leakage.
- Towards a unified framework for fair and stable graph representation learning. In Proceedings of the Thirty-Seventh Conference on Uncertainty in Artificial Intelligence (Proceedings of Machine Learning Research, Vol. 161). PMLR, 2114–2124. https://proceedings.mlr.press/v161/agarwal21b.html
- Optuna: A Next-Generation Hyperparameter Optimization Framework. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (Anchorage, AK, USA) (KDD ’19). Association for Computing Machinery, New York, NY, USA, 2623–2631. https://doi.org/10.1145/3292500.3330701
- Mutual Information Neural Estimation. In Proceedings of the 35th International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 80). PMLR, 531–540. https://proceedings.mlr.press/v80/belghazi18a.html
- Data Decisions and Theoretical Implications when Adversarially Learning Fair Representations. arXiv:1707.00075 [cs.LG]
- Avishek Joey Bose and William Hamilton. 2019. Compositional Fairness Constraints for Graph Embeddings. (2019).
- H. Chang and R. Shokri. 2021. On the Privacy Risks of Algorithmic Fairness. In 2021 IEEE European Symposium on Security and Privacy (EuroS&P). IEEE Computer Society, Los Alamitos, CA, USA, 292–303. https://doi.org/10.1109/EuroSP51992.2021.00028
- When Fairness Meets Privacy: Fair Classification with Semi-Private Sensitive Attributes. In Workshop on Trustworthy and Socially Responsible Machine Learning, NeurIPS 2022.
- A Fair Classifier Using Mutual Information. In 2020 IEEE International Symposium on Information Theory (ISIT) (Los Angeles, CA, USA). IEEE Press, 2521–2526. https://doi.org/10.1109/ISIT44484.2020.9174293
- Enyan Dai and Suhang Wang. 2021. Say No to the Discrimination: Learning Fair Graph Neural Networks with Limited Sensitive Attribute Information. In Proceedings of the 14th ACM International Conference on Web Search and Data Mining. 680–688.
- Enyan Dai and Suhang Wang. 2022. Learning Fair Graph Neural Networks with Limited and Private Sensitive Attribute Information. IEEE Transactions on Knowledge and Data Engineering (2022), 1–14. https://doi.org/10.1109/TKDE.2022.3197554
- An Empirical Analysis of Fairness Notions under Differential Privacy. arXiv:2302.02910 [cs.LG]
- Differentially private and fair classification via calibrated functional mechanism. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 34. 622–629.
- Edits: Modeling and mitigating data bias for graph neural networks. In Proceedings of the ACM Web Conference 2022. 1259–1269.
- Fairness in Graph Mining: A Survey. IEEE Transactions on Knowledge and Data Engineering 01 (apr 5555), 1–22. https://doi.org/10.1109/TKDE.2023.3265598
- Quantifying Privacy Leakage in Graph Embedding. In MobiQuitous 2020 - 17th EAI International Conference on Mobile and Ubiquitous Systems: Computing, Networking and Services. ACM. https://doi.org/10.1145/3448891.3448939
- Fairness through Awareness. In Proceedings of the 3rd Innovations in Theoretical Computer Science Conference (Cambridge, Massachusetts) (ITCS ’12). Association for Computing Machinery, New York, NY, USA, 214–226. https://doi.org/10.1145/2090236.2090255
- Calibrating Noise to Sensitivity in Private Data Analysis. In Theory of Cryptography. Springer Berlin Heidelberg, Berlin, Heidelberg, 265–284.
- Graph Neural Networks for Social Recommendation. In The World Wide Web Conference (San Francisco, CA, USA) (WWW ’19). Association for Computing Machinery, New York, NY, USA, 417–426. https://doi.org/10.1145/3308558.3313488
- Matthias Fey and Jan E. Lenssen. 2019. Fast Graph Representation Learning with PyTorch Geometric. In ICLR Workshop on Representation Learning on Graphs and Manifolds.
- Katarina Hamberg. 2008. Gender Bias in Medicine. Women’s Health 4, 3 (2008), 237–243. https://doi.org/10.2217/17455057.4.3.237 arXiv:https://doi.org/10.2217/17455057.4.3.237
- Inductive Representation Learning on Large Graphs. In NIPS.
- Equality of Opportunity in Supervised Learning. In Proceedings of the 30th International Conference on Neural Information Processing Systems (Barcelona, Spain) (NIPS’16). Curran Associates Inc., Red Hook, NY, USA, 3323–3331.
- Distance correlation application to gene co-expression network analysis. BMC Bioinformatics 23 (02 2022). https://doi.org/10.1186/s12859-022-04609-x
- Learning Privacy-Preserving Graph Convolutional Network with Partially Observed Sensitive Attributes. In Proceedings of the ACM Web Conference 2022. 3552–3561.
- Cheng Huang and Xiaoming Huo. 2022. A Statistically and Numerically Efficient Independence Test Based on Random Projections and Distance Covariance. Frontiers in Applied Mathematics and Statistics 7 (2022). https://doi.org/10.3389/fams.2021.779841
- Xiaoming Huo and Gábor J. Székely. 2016. Fast Computing for Distance Covariance. Technometrics 58, 4 (2016), 435–447. https://doi.org/10.1080/00401706.2015.1054435 arXiv:https://doi.org/10.1080/00401706.2015.1054435
- Bargav Jayaraman and David Evans. 2022. Are Attribute Inference Attacks Just Imputation?. In Proceedings of the 2022 ACM SIGSAC Conference on Computer and Communications Security (Los Angeles, CA, USA) (CCS ’22). Association for Computing Machinery, New York, NY, USA, 1569–1582. https://doi.org/10.1145/3548606.3560663
- FMP: Toward Fair Graph Message Passing against Topology Bias. arXiv:2202.04187 [cs.LG]
- Faisal Kamiran and Toon Calders. 2011. Data Pre-Processing Techniques for Classification without Discrimination. Knowledge and Information Systems 33 (10 2011). https://doi.org/10.1007/s10115-011-0463-8
- Diederik P. Kingma and Jimmy Ba. 2015. Adam: A Method for Stochastic Optimization. In 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings. http://arxiv.org/abs/1412.6980
- Adversarial Privacy-Preserving Graph Embedding Against Inference Attack. IEEE Internet of Things Journal 8 (2020), 6904–6915.
- Private Graph Data Release: A Survey. arXiv:2107.04245 [cs.CR]
- Information Obfuscation of Graph Neural Networks. In Proceedings of the 38th International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 139). PMLR, 6600–6610. http://proceedings.mlr.press/v139/liao21a.html
- The Variational Fair Autoencoder. arXiv:1511.00830 [stat.ML]
- Bursting the Filter Bubble: Fairness-Aware Network Link Prediction. Proceedings of the AAAI Conference on Artificial Intelligence 34, 01 (Apr. 2020), 841–848. https://doi.org/10.1609/aaai.v34i01.5429
- Learning Deep Fair Graph Neural Networks. In 28th European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning, ESANN 2020, Bruges, Belgium, October 2-4, 2020. 31–36. https://www.esann.org/sites/default/files/proceedings/2020/ES2020-75.pdf
- PyTorch: An Imperative Style, High-Performance Deep Learning Library. In Advances in Neural Information Processing Systems 32. Curran Associates, Inc., 8024–8035. http://papers.neurips.cc/paper/9015-pytorch-an-imperative-style-high-performance-deep-learning-library.pdf
- Fair Decision Making Using Privacy-Protected Data. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency (Barcelona, Spain) (FAT* ’20). Association for Computing Machinery, New York, NY, USA, 189–199. https://doi.org/10.1145/3351095.3372872
- Fairwalk: Towards Fair Graph Embedding. In Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI-19. International Joint Conferences on Artificial Intelligence Organization, 3289–3295. https://doi.org/10.24963/ijcai.2019/456
- FR-Train: A Mutual Information-Based Approach to Fair and Robust Training. In Proceedings of the 37th International Conference on Machine Learning (ICML’20). JMLR.org, Article 754, 11 pages.
- JENNIFER L. SKEEM and CHRISTOPHER T. LOWENKAMP. 2016. RISK, RACE, AND RECIDIVISM: PREDICTIVE BIAS AND DISPARATE IMPACT*. Criminology 54, 4 (2016), 680–712. https://doi.org/10.1111/1745-9125.12123 arXiv:https://onlinelibrary.wiley.com/doi/pdf/10.1111/1745-9125.12123
- Jiaming Song and Stefano Ermon. 2020. Understanding the Limitations of Variational Mutual Information Estimators. (2020).
- FairDrop: Biased Edge Dropout for Enhancing Fairness in Graph Representation Learning. IEEE Transactions on Artificial Intelligence (2021), 1–1. https://doi.org/10.1109/TAI.2021.3133818
- When OT meets MoM: Robust estimation of Wasserstein Distance. In Proceedings of The 24th International Conference on Artificial Intelligence and Statistics (Proceedings of Machine Learning Research, Vol. 130). PMLR, 136–144. https://proceedings.mlr.press/v130/staerman21a.html
- Harini Suresh and John Guttag. 2021. A Framework for Understanding Sources of Harm throughout the Machine Learning Life Cycle. In Equity and Access in Algorithms, Mechanisms, and Optimization. ACM. https://doi.org/10.1145/3465416.3483305
- Measuring and testing dependence by correlation of distances. The Annals of Statistics 35, 6 (2007), 2769 – 2794. https://doi.org/10.1214/009053607000000505
- Measuring and Testing Dependence by Correlation of Distances. The Annals of Statistics 35, 6 (2007), 2769–2794. http://www.jstor.org/stable/25464608
- Laurens van der Maaten and Geoffrey Hinton. 2008. Visualizing Data using t-SNE. Journal of Machine Learning Research 9, 86 (2008), 2579–2605. http://jmlr.org/papers/v9/vandermaaten08a.html
- C. Villani. 2003. Topics in Optimal Transportation Theory. 58 (01 2003). https://doi.org/10.1090/gsm/058
- Privacy-Preserving Representation Learning on Graphs: A Mutual Information Perspective. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (Virtual Event, Singapore) (KDD ’21). Association for Computing Machinery, New York, NY, USA, 1667–1676. https://doi.org/10.1145/3447548.3467273
- Repairing without Retraining: Avoiding Disparate Impact with Counterfactual Distributions. In Proceedings of the 36th International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 97). PMLR, 6618–6627. https://proceedings.mlr.press/v97/wang19l.html
- Improving Fairness in Graph Neural Networks via Mitigating Sensitive Attribute Leakage. In SIGKDD.
- Boris Weisfeiler and AA Lehman. 1968. A reduction of a graph to a canonical form and an algebra arising during this reduction.. In Nauchno-Technicheskaya Informatsia, Vol. 2(9). 12–16.
- A comprehensive survey on graph neural networks. IEEE Transactions on Neural Networks and Learning Systems (mar 2020). https://arxiv.org/pdf/1901.00596.pdf
- How Powerful are Graph Neural Networks?. In International Conference on Learning Representations. https://openreview.net/forum?id=ryGs6iA5Km
- WTAGRAPH: Web Tracking and Advertising Detection using Graph Neural Networks. In 2022 IEEE Symposium on Security and Privacy (SP). 1540–1557. https://doi.org/10.1109/SP46214.2022.9833670
- Fairness Beyond Disparate Treatment and Disparate Impact: Learning Classification without Disparate Mistreatment. In Proceedings of the 26th International Conference on World Wide Web (Perth, Australia) (WWW ’17). International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva, CHE, 1171–1180. https://doi.org/10.1145/3038912.3052660
- Mitigating Unwanted Biases with Adversarial Learning. In Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society (New Orleans, LA, USA) (AIES ’18). Association for Computing Machinery, New York, NY, USA, 335–340. https://doi.org/10.1145/3278721.3278779
- On the Interaction between Node Fairness and Edge Privacy in Graph Neural Networks. arXiv:2301.12951 [cs.LG]
- Towards Fair Classifiers Without Sensitive Attributes: Exploring Biases in Related Features. In Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining (Virtual Event, AZ, USA) (WSDM ’22). Association for Computing Machinery, New York, NY, USA, 1433–1442. https://doi.org/10.1145/3488560.3498493
- Huaisheng Zhu and Suhang Wang. 2022. Learning Fair Models without Sensitive Attributes: A Generative Approach. arXiv:2203.16413 [cs.LG]
- Ying Song (14 papers)
- Balaji Palanisamy (18 papers)