Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
144 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Towards Independence Criterion in Machine Unlearning of Features and Labels (2403.08124v1)

Published 12 Mar 2024 in cs.LG, cs.AI, and cs.CR

Abstract: This work delves into the complexities of machine unlearning in the face of distributional shifts, particularly focusing on the challenges posed by non-uniform feature and label removal. With the advent of regulations like the GDPR emphasizing data privacy and the right to be forgotten, machine learning models face the daunting task of unlearning sensitive information without compromising their integrity or performance. Our research introduces a novel approach that leverages influence functions and principles of distributional independence to address these challenges. By proposing a comprehensive framework for machine unlearning, we aim to ensure privacy protection while maintaining model performance and adaptability across varying distributions. Our method not only facilitates efficient data removal but also dynamically adjusts the model to preserve its generalization capabilities. Through extensive experimentation, we demonstrate the efficacy of our approach in scenarios characterized by significant distributional shifts, making substantial contributions to the field of machine unlearning. This research paves the way for developing more resilient and adaptable unlearning techniques, ensuring models remain robust and accurate in the dynamic landscape of data privacy and machine learning.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (39)
  1. Minimax regret optimization for robust machine learning under distribution shift. In Conference on Learning Theory, pp.  2704–2729. PMLR, 2022.
  2. Machine unlearning. In 2021 IEEE Symposium on Security and Privacy (SP), pp. 141–159. IEEE, 2021.
  3. Towards making systems forget with machine unlearning. In 2015 IEEE symposium on security and privacy, pp. 463–480. IEEE, 2015.
  4. When machine unlearning jeopardizes privacy. In Proceedings of the 2021 ACM SIGSAC conference on computer and communications security, pp.  896–911, 2021.
  5. Graph unlearning. In Proceedings of the 2022 ACM SIGSAC Conference on Computer and Communications Security, pp.  499–513, 2022.
  6. Characterizations of an empirical influence function for detecting influential cases in regression. Technometrics, pp.  495–508, 1980.
  7. Differential privacy—a primer for the perplexed,”. Joint UNECE/Eurostat work session on statistical data confidentiality, 11, 2011.
  8. Adversarial robustness with non-uniform perturbations. Advances in Neural Information Processing Systems, 34:19147–19159, 2021.
  9. Fast machine unlearning without retraining through selective synaptic dampening. arXiv preprint arXiv:2308.07707, 2023.
  10. Making ai forget you: Data deletion in machine learning. Advances in neural information processing systems, 32, 2019.
  11. Giraud, C. Introduction to high-dimensional statistics. CRC Press, 2021.
  12. Eternal sunshine of the spotless net: Selective forgetting in deep networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.  9304–9312, 2020.
  13. The information bottleneck problem and its applications in machine learning. IEEE Journal on Selected Areas in Information Theory, 1(1):19–38, 2020.
  14. Measuring statistical dependence with hilbert-schmidt norms. In International conference on algorithmic learning theory, pp.  63–77. Springer, 2005.
  15. Certified data removal from machine learning models. arXiv preprint arXiv:1911.03030, 2019.
  16. Multilayer feedforward networks are universal approximators. Neural networks, 2(5):359–366, 1989.
  17. Approximate data deletion from machine learning models. In International Conference on Artificial Intelligence and Statistics, pp.  2008–2016. PMLR, 2021.
  18. Time-domain neural network receiver for nonlinear frequency division multiplexed systems. IEEE Photonics Technology Letters, 30(12):1079–1082, 2018.
  19. Multiple incremental decremental learning of support vector machines. IEEE Transactions on Neural Networks, 21(7):1048–1059, 2010.
  20. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907, 2016.
  21. Understanding black-box predictions via influence functions. In International conference on machine learning, pp. 1885–1894. PMLR, 2017.
  22. Langley, P. Crafting papers on machine learning. In Langley, P. (ed.), Proceedings of the 17th International Conference on Machine Learning (ICML 2000), pp.  1207–1216, Stanford, CA, 2000. Morgan Kaufmann.
  23. Word-level training of a handwritten word recognizer based on convolutional neural networks. In Proceedings of the 12th IAPR International Conference on Pattern Recognition, vol. 3-Conference C: Signal Processing (Cat. No. 94CH3440-5), volume 2, pp.  88–92. IEEE, 1994.
  24. LeCun, Y. The mnist database of handwritten digits. http://yann. lecun. com/exdb/mnist/, 1998.
  25. A critical examination of robustness and generalizability of machine learning prediction of materials properties. npj Computational Materials, 9(1):55, 2023.
  26. The hsic bottleneck: Deep learning without back-propagation. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pp.  5085–5092, 2020.
  27. A survey of machine unlearning. arXiv preprint arXiv:2209.02299, 2022.
  28. Regulation, P. Regulation (eu) 2016/679 of the european parliament and of the council. Regulation (eu), 679:2016, 2016.
  29. ” why should i trust you?” explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pp.  1135–1144, 2016.
  30. Hedgecut: Maintaining randomised trees for low-latency machine unlearning. In Proceedings of the 2021 International Conference on Management of Data, pp.  1545–1557, 2021.
  31. Remember what you want to forget: Algorithms for machine unlearning. Advances in Neural Information Processing Systems, 34:18075–18086, 2021.
  32. The information bottleneck method. arXiv preprint physics/0004057, 2000.
  33. Machine unlearning of features and labels. arXiv preprint arXiv:2108.11577, 2021.
  34. Gif: A general graph unlearning strategy via influence function. In Proceedings of the ACM Web Conference 2023, pp.  651–661, 2023.
  35. Deltagrad: Rapid retraining of machine learning models. In International Conference on Machine Learning, pp. 10355–10366. PMLR, 2020.
  36. How powerful are graph neural networks? arXiv preprint arXiv:1810.00826, 2018.
  37. Arcane: An efficient architecture for exact machine unlearning. In Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence, IJCAI-22, pp.  4006–4013, 2022.
  38. Riden: Neural-based uniform density histogram for distribution shift detection. In Proceedings of the Second International Conference on AI-ML Systems, pp.  1–9, 2022.
  39. Recommendation unlearning via influence function. arXiv preprint arXiv:2307.02147, 2023.
Citations (3)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now