Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
158 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Unlearning Graph Classifiers with Limited Data Resources (2211.03216v2)

Published 6 Nov 2022 in cs.LG and cs.CR

Abstract: As the demand for user privacy grows, controlled data removal (machine unlearning) is becoming an important feature of machine learning models for data-sensitive Web applications such as social networks and recommender systems. Nevertheless, at this point it is still largely unknown how to perform efficient machine unlearning of graph neural networks (GNNs); this is especially the case when the number of training samples is small, in which case unlearning can seriously compromise the performance of the model. To address this issue, we initiate the study of unlearning the Graph Scattering Transform (GST), a mathematical framework that is efficient, provably stable under feature or graph topology perturbations, and offers graph classification performance comparable to that of GNNs. Our main contribution is the first known nonlinear approximate graph unlearning method based on GSTs. Our second contribution is a theoretical analysis of the computational complexity of the proposed unlearning mechanism, which is hard to replicate for deep neural networks. Our third contribution are extensive simulation results which show that, compared to complete retraining of GNNs after each removal request, the new GST-based approach offers, on average, a 10.38x speed-up and leads to a 2.6% increase in test accuracy during unlearning of 90 out of 100 training graphs from the IMDB dataset (10% training ratio). Our implementation is available online at https://doi.org/10.5281/zenodo.7613150.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (61)
  1. Martin Anthony and Peter L Bartlett. 2009. Neural network learning: Theoretical foundations. cambridge university press.
  2. Graph neural networks with convolutional arma filters. IEEE Transactions on Pattern Analysis and Machine Intelligence (2021).
  3. Machine unlearning. In 2021 IEEE Symposium on Security and Privacy (SP). IEEE, 141–159.
  4. Joan Bruna and Stéphane Mallat. 2013. Invariant scattering convolution networks. IEEE transactions on pattern analysis and machine intelligence 35, 8 (2013), 1872–1886.
  5. Yinzhi Cao and Junfeng Yang. 2015. Towards making systems forget with machine unlearning. In 2015 IEEE Symposium on Security and Privacy. IEEE, 463–480.
  6. Differentially private empirical risk minimization. Journal of Machine Learning Research 12, 3 (2011).
  7. Graph unlearning. In Proceedings of the 2022 ACM SIGSAC Conference on Computer and Communications Security. 499–513.
  8. Certified Graph Unlearning. In NeurIPS 2022 Workshop: New Frontiers in Graph Learning.
  9. Efficient Model Updates for Approximate Unlearning of Graph-Structured Data. In International Conference on Learning Representations.
  10. Node-Level Differentially Private Graph Neural Networks. arXiv preprint arXiv:2111.15521 (2021).
  11. Li Deng. 2012. The mnist database of handwritten digit images for machine learning research. IEEE Signal Processing Magazine 29, 6 (2012), 141–142.
  12. Paul D Dobson and Andrew J Doig. 2003. Distinguishing enzyme structures from non-enzymes without alignments. Journal of molecular biology 330, 4 (2003), 771–783.
  13. Convolutional networks on graphs for learning molecular fingerprints. Advances in neural information processing systems 28 (2015).
  14. Benchmarking graph neural networks. arXiv preprint arXiv:2003.00982 (2020).
  15. Cynthia Dwork. 2011. Differential privacy. Encyclopedia of cryptography and security.
  16. Graph neural networks for social recommendation. In The world wide web conference. 417–426.
  17. One-shot learning of object categories. IEEE transactions on pattern analysis and machine intelligence 28, 4 (2006), 594–611.
  18. Matthias Fey and Jan Eric Lenssen. 2019. Fast graph representation learning with PyTorch Geometric. arXiv preprint arXiv:1903.02428 (2019).
  19. Model inversion attacks that exploit confidence information and basic countermeasures. In Proceedings of the 22nd ACM SIGSAC conference on computer and communications security. 1322–1333.
  20. Diffusion Scattering Transforms on Graphs. In International Conference on Learning Representations.
  21. Stability of graph scattering transforms. Advances in Neural Information Processing Systems 32 (2019).
  22. Geometric scattering for graph data analysis. In International Conference on Machine Learning. 2122–2131.
  23. Utilizing graph machine learning within drug discovery and development. Briefings in bioinformatics 22, 6 (2021), bbab159.
  24. Making ai forget you: Data deletion in machine learning. Advances in Neural Information Processing Systems 32 (2019).
  25. David F Gleich. 2015. PageRank beyond the Web. siam REVIEW 57, 3 (2015), 321–363.
  26. Eternal sunshine of the spotless net: Selective forgetting in deep networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 9304–9312.
  27. Ming Gu and Stanley C Eisenstat. 1995. Downdating the singular value decomposition. SIAM J. Matrix Anal. Appl. 16, 3 (1995), 793–810.
  28. Certified Data Removal from Machine Learning Models. In International Conference on Machine Learning. PMLR, 3832–3842.
  29. Inductive representation learning on large graphs. Advances in neural information processing systems 30 (2017).
  30. Wavelets on graphs via spectral graph theory. Applied and Computational Harmonic Analysis 30, 2 (2011), 129–150.
  31. Knowledge-aware coupled graph neural network for social recommendation. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 35. 4115–4122.
  32. Pruned graph scattering transforms. In International Conference on Learning Representations.
  33. Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014).
  34. Learning multiple layers of features from tiny images. (2009).
  35. Semi-supervised graph classification: A hierarchical graph perspective. In The World Wide Web Conference. 972–982.
  36. Skeleton graph scattering networks for 3d skeleton-based human motion prediction. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 854–864.
  37. Few-shot learning for new user recommendation in location-based social networks. In Proceedings of The Web Conference 2020. 2472–2478.
  38. Graph neural network for interpreting task-fmri biomarkers. In International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 485–493.
  39. Stéphane Mallat. 2012. Group invariant scattering. Communications on Pure and Applied Mathematics 65, 10 (2012), 1331–1398.
  40. Medgcn: Graph convolutional networks for multiple medical tasks. arXiv preprint arXiv:1904.00326 (2019).
  41. Scattering gcn: Overcoming oversmoothness in graph convolutional networks. Advances in Neural Information Processing Systems 33 (2020), 14498–14508.
  42. Identifying fake accounts on social networks based on graph analysis and classification algorithms. Security and Communication Networks 2018 (2018).
  43. Tudataset: A collection of benchmark datasets for learning with graphs. arXiv preprint arXiv:2007.08663 (2020).
  44. Differentially Private Graph Classification with GNNs. arXiv preprint arXiv:2202.02575 (2022).
  45. Spatio-Temporal Graph Scattering Transform. In International Conference on Learning Representations.
  46. Machine Unlearning of Federated Clusters. In International Conference on Learning Representations.
  47. GAP: Differentially Private Graph Neural Networks with Aggregation Perturbation. arXiv preprint arXiv:2203.00949 (2022).
  48. Aliaksei Sandryhaila and José MF Moura. 2013. Discrete signal processing on graphs: Graph Fourier transform. In 2013 IEEE International Conference on Acoustics, Speech and Signal Processing. IEEE, 6167–6170.
  49. Victor Garcia Satorras and Joan Bruna Estrach. 2018. Few-shot learning with graph neural networks. In International conference on learning representations.
  50. Remember what you want to forget: Algorithms for machine unlearning. Advances in Neural Information Processing Systems 34 (2021).
  51. The emerging field of signal processing on graphs: Extending high-dimensional data analysis to networks and other irregular domains. IEEE signal processing magazine 30, 3 (2013), 83–98.
  52. Spectrum-adapted tight graph wavelet and vertex-frequency frames. IEEE Transactions on Signal Processing 63, 16 (2015), 4223–4235.
  53. Algorithms that remember: model inversion attacks and data protection law. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences 376, 2133 (2018), 20180083.
  54. Generalizing from a few examples: A survey on few-shot learning. ACM computing surveys (csur) 53, 3 (2020), 1–34.
  55. Max Welling and Thomas N Kipf. 2017. Semi-supervised classification with graph convolutional networks. In International Conference on Learning Representations.
  56. Simplifying graph convolutional networks. In International conference on machine learning. PMLR, 6861–6871.
  57. Graph neural networks in recommender systems: a survey. ACM Computing Surveys (CSUR) (2020).
  58. How Powerful are Graph Neural Networks?. In International Conference on Learning Representations.
  59. Pinar Yanardag and SVN Vishwanathan. 2015. Deep graph kernels. In Proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining. 1365–1374.
  60. Graph convolutional neural networks for web-scale recommender systems. In Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining. 974–983.
  61. Dongmian Zou and Gilad Lerman. 2020. Graph convolutional neural networks via scattering. Applied and Computational Harmonic Analysis 49, 3 (2020), 1046–1074.
Citations (28)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now