Graph Unlearning with Efficient Partial Retraining (2403.07353v2)
Abstract: Graph Neural Networks (GNNs) have achieved remarkable success in various real-world applications. However, GNNs may be trained on undesirable graph data, which can degrade their performance and reliability. To enable trained GNNs to efficiently unlearn unwanted data, a desirable solution is retraining-based graph unlearning, which partitions the training graph into subgraphs and trains sub-models on them, allowing fast unlearning through partial retraining. However, the graph partition process causes information loss in the training graph, resulting in the low model utility of sub-GNN models. In this paper, we propose GraphRevoker, a novel graph unlearning framework that better maintains the model utility of unlearnable GNNs. Specifically, we preserve the graph property with graph property-aware sharding and effectively aggregate the sub-GNN models for prediction with graph contrastive sub-model aggregation. We conduct extensive experiments to demonstrate the superiority of our proposed approach.
- Streaming graph partitioning: an experimental study. In VLDB.
- Machine unlearning. In IEEE S&P.
- Jonathan Brophy and Daniel Lowd. 2021. Machine unlearning for random forests. In ICML.
- Yinzhi Cao and Junfeng Yang. 2015. Towards making systems forget with machine unlearning. In IEEE S&P.
- Recommendation unlearning. In WWW.
- Knowledge-enhanced black-box attacks for recommendations. In KDD.
- Graph unlearning. In ACM CCS.
- GNNDelete: A General Unlearning Strategy for Graph Neural Networks. In ICLR.
- Graph trend filtering networks for recommendation. In SIGIR.
- Untargeted Black-box Attacks for Social Recommendations. arXiv preprint (2023).
- Jointly attacking graph neural network and its explanations. In ICDE.
- Predict then Propagate: Graph Neural Networks meet Personalized PageRank. In ICLR.
- Eternal sunshine of the spotless net: Selective forgetting in deep networks. In CVPR.
- Certified data removal from machine learning models. In ICML.
- Inductive representation learning on large graphs. In NeurIPS.
- George Karypis and Vipin Kumar. 1997. METIS: A software package for partitioning unstructured graphs, partitioning meshes, and computing fill-reducing orderings of sparse matrices. Technical Report (1997).
- Thomas N Kipf and Max Welling. 2016. Semi-Supervised Classification with Graph Convolutional Networks. In ICLR.
- Wenqing Lin. 2021. Large-scale network embedding in apache spark. In KDD.
- Graph adversarial attack via rewiring. In KDD.
- Benedek Rozemberczki and Rik Sarkar. 2020. Characteristic functions on graphs: Birds of a feather, from statistical descriptors to parametric models. In CIKM.
- Isabelle Stanton and Gabriel Kliot. 2012. Streaming graph partitioning for large distributed graphs. In KDD.
- On the necessity of auditable algorithmic definitions for machine unlearning. In USENIX Security.
- Graph Attention Networks. In ICLR.
- Fast graph condensation with structure-based neural tangent kernel. In WWW.
- Machine unlearning of features and labels. In NDSS.
- Disentangled contrastive learning for social recommendation. In CIKM.
- GIF: A General Graph Unlearning Strategy via Influence Function. In WWW.
- Distributed power-law graph computing: Theoretical and empirical analysis. NeurIPS.
- Representation learning on graphs with jumping knowledge networks. In ICML.
- ARCANE: An Efficient Architecture for Exact Machine Unlearning.. In IJCAI.
- Revisiting semi-supervised learning with graph embeddings. In ICML.
- GraphSAINT: Graph Sampling Based Inductive Learning Method. In ICLR.
- Graph edge partitioning via neighborhood heuristic. In KDD.
- Linear-Time Graph Neural Networks for Scalable Recommendations. In WWW.
- Graph contrastive learning with adaptive augmentation. In WWW.