Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
119 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Challenging Forgets: Unveiling the Worst-Case Forget Sets in Machine Unlearning (2403.07362v4)

Published 12 Mar 2024 in cs.LG, cs.AI, and cs.CV

Abstract: The trustworthy ML community is increasingly recognizing the crucial need for models capable of selectively 'unlearning' data points after training. This leads to the problem of machine unlearning (MU), aiming to eliminate the influence of chosen data points on model performance, while still maintaining the model's utility post-unlearning. Despite various MU methods for data influence erasure, evaluations have largely focused on random data forgetting, ignoring the vital inquiry into which subset should be chosen to truly gauge the authenticity of unlearning performance. To tackle this issue, we introduce a new evaluative angle for MU from an adversarial viewpoint. We propose identifying the data subset that presents the most significant challenge for influence erasure, i.e., pinpointing the worst-case forget set. Utilizing a bi-level optimization principle, we amplify unlearning challenges at the upper optimization level to emulate worst-case scenarios, while simultaneously engaging in standard training and unlearning at the lower level, achieving a balance between data influence erasure and model utility. Our proposal offers a worst-case evaluation of MU's resilience and effectiveness. Through extensive experiments across different datasets (including CIFAR-10, 100, CelebA, Tiny ImageNet, and ImageNet) and models (including both image classifiers and generative models), we expose critical pros and cons in existing (approximate) unlearning strategies. Our results illuminate the complex challenges of MU in practice, guiding the future development of more accurate and robust unlearning algorithms. The code is available at https://github.com/OPTML-Group/Unlearn-WorstCase.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (79)
  1. Y. Cao and J. Yang, “Towards making systems forget with machine unlearning,” in 2015 IEEE Symposium on Security and Privacy, pp. 463–480, IEEE, 2015.
  2. L. Bourtoule, V. Chandrasekaran, C. A. Choquette-Choo, H. Jia, A. Travers, B. Zhang, D. Lie, and N. Papernot, “Machine unlearning,” in 2021 IEEE Symposium on Security and Privacy (SP), pp. 141–159, IEEE, 2021.
  3. T. T. Nguyen, T. T. Huynh, P. L. Nguyen, A. W.-C. Liew, H. Yin, and Q. V. H. Nguyen, “A survey of machine unlearning,” arXiv preprint arXiv:2209.02299, 2022.
  4. S. Liu, Y. Yao, J. Jia, S. Casper, N. Baracaldo, P. Hase, X. Xu, Y. Yao, H. Li, K. R. Varshney, et al., “Rethinking machine unlearning for large language models,” arXiv preprint arXiv:2402.08787, 2024.
  5. C. J. Hoofnagle, B. van der Sloot, and F. Z. Borgesius, “The european union general data protection regulation: what it is and what it means,” Information & Communications Technology Law, vol. 28, no. 1, pp. 65–98, 2019.
  6. J. Rosen, “The right to be forgotten,” Stan. L. Rev. Online, vol. 64, p. 88, 2011.
  7. Y. Liu, M. Fan, C. Chen, X. Liu, Z. Ma, L. Wang, and J. Ma, “Backdoor defense with machine unlearning,” arXiv preprint arXiv:2201.09538, 2022.
  8. J. Jia, J. Liu, P. Ram, Y. Yao, G. Liu, Y. Liu, P. Sharma, and S. Liu, “Model sparsity can simplify machine unlearning,” Advances in neural information processing systems, vol. 36, 2023.
  9. A. Oesterling, J. Ma, F. P. Calmon, and H. Lakkaraju, “Fair machine unlearning: Data removal while mitigating disparities,” arXiv preprint arXiv:2307.14754, 2023.
  10. P. Sattigeri, S. Ghosh, I. Padhi, P. Dognin, and K. R. Varshney, “Fair infinitesimal jackknife: Mitigating the influence of biased training data points without refitting,” in Advances in Neural Information Processing Systems, 2022.
  11. R. Chen, J. Yang, H. Xiong, J. Bai, T. Hu, J. Hao, Y. Feng, J. T. Zhou, J. Wu, and Z. Liu, “Fast model debias with machine unlearning,” Advances in Neural Information Processing Systems, vol. 36, 2024.
  12. Y. Zhang, Y. Zhang, Y. Yao, J. Jia, J. Liu, X. Liu, and S. Liu, “Unlearncanvas: A stylized image dataset to benchmark machine unlearning for diffusion models,” arXiv preprint arXiv:2402.11846, 2024.
  13. A. Achille, M. Kearns, C. Klingenberg, and S. Soatto, “Ai model disgorgement: Methods and choices,” arXiv preprint arXiv:2304.03545, 2023.
  14. R. Eldan and M. Russinovich, “Who’s harry potter? approximate unlearning in llms,” 2023.
  15. R. Gandikota, J. Materzynska, J. Fiotto-Kaufman, and D. Bau, “Erasing concepts from diffusion models,” arXiv preprint arXiv:2303.07345, 2023.
  16. E. Zhang, K. Wang, X. Xu, Z. Wang, and H. Shi, “Forget-me-not: Learning to forget in text-to-image diffusion models,” arXiv preprint arXiv:2303.17591, 2023.
  17. C. Fan, J. Liu, Y. Zhang, D. Wei, E. Wong, and S. Liu, “Salun: Empowering machine unlearning via gradient-based weight saliency in both image classification and generation,” arXiv preprint arXiv:2310.12508, 2023.
  18. A. Warnecke, L. Pirch, C. Wressnegger, and K. Rieck, “Machine unlearning of features and labels,” arXiv preprint arXiv:2108.11577, 2021.
  19. Z. Izzo, M. A. Smart, K. Chaudhuri, and J. Zou, “Approximate data deletion from machine learning models,” in International Conference on Artificial Intelligence and Statistics, pp. 2008–2016, PMLR, 2021.
  20. A. Golatkar, A. Achille, and S. Soatto, “Eternal sunshine of the spotless net: Selective forgetting in deep networks,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9304–9312, 2020.
  21. A. Thudi, G. Deza, V. Chandrasekaran, and N. Papernot, “Unrolling sgd: Understanding factors influencing machine unlearning,” in 2022 IEEE 7th European Symposium on Security and Privacy (EuroS&P), pp. 303–319, IEEE, 2022.
  22. N. Kumari, B. Zhang, S.-Y. Wang, E. Shechtman, R. Zhang, and J.-Y. Zhu, “Ablating concepts in text-to-image diffusion models,” 2023.
  23. R. Gandikota, H. Orgad, Y. Belinkov, J. Materzyńska, and D. Bau, “Unified concept editing in diffusion models,” in Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 5111–5120, 2024.
  24. A. Heng and H. Soh, “Selective amnesia: A continual learning approach to forgetting in deep generative models,” 2023.
  25. J. Wang, S. Guo, X. Xie, and H. Qi, “Federated unlearning via class-discriminative pruning,” in Proceedings of the ACM Web Conference 2022, pp. 622–632, 2022.
  26. Y. Liu, L. Xu, X. Yuan, C. Wang, and B. Li, “The right to be forgotten in federated learning: An efficient realization with rapid retraining,” arXiv preprint arXiv:2203.07320, 2022.
  27. L. Wu, S. Guo, J. Wang, Z. Hong, J. Zhang, and Y. Ding, “Federated unlearning: Guarantee the right of clients to forget,” IEEE Network, vol. 36, no. 5, pp. 129–135, 2022.
  28. T. Che, Y. Zhou, Z. Zhang, L. Lyu, J. Liu, D. Yan, D. Dou, and J. Huan, “Fast federated machine unlearning with nonlinear functional theory,” 2023.
  29. M. Chen, Z. Zhang, T. Wang, M. Backes, M. Humbert, and Y. Zhang, “Graph unlearning,” in Proceedings of the 2022 ACM SIGSAC Conference on Computer and Communications Security, pp. 499–513, 2022.
  30. E. Chien, C. Pan, and O. Milenkovic, “Certified graph unlearning,” arXiv preprint arXiv:2206.09140, 2022.
  31. J. Cheng, G. Dasoulas, H. He, C. Agarwal, and M. Zitnik, “Gnndelete: A general strategy for unlearning in graph neural networks,” arXiv preprint arXiv:2302.13406, 2023.
  32. X. Wu, J. Li, M. Xu, W. Dong, S. Wu, C. Bian, and D. Xiong, “Depn: Detecting and editing privacy neurons in pretrained language models,” arXiv preprint arXiv:2310.20138, 2023.
  33. C. Yu, S. Jeoung, A. Kasi, P. Yu, and H. Ji, “Unlearning bias in language models by partitioning gradients,” in Findings of the Association for Computational Linguistics: ACL 2023, pp. 6032–6048, 2023.
  34. J. Zhang, S. Chen, J. Liu, and J. He, “Composing parameter-efficient modules with arithmetic operations,” arXiv preprint arXiv:2306.14870, 2023.
  35. Y. Yao, X. Xu, and Y. Liu, “Large language model unlearning,” arXiv preprint arXiv:2310.10683, 2023.
  36. M. Kurmanji, P. Triantafillou, and E. Triantafillou, “Towards unbounded machine unlearning,” arXiv preprint arXiv:2302.09880, 2023.
  37. M. Cotogni, J. Bonato, L. Sabetta, F. Pelosin, and A. Nicolosi, “Duck: Distance-based unlearning via centroid kinematics,” arXiv preprint arXiv:2312.02052, 2023.
  38. R. Shokri, M. Stronati, C. Song, and V. Shmatikov, “Membership inference attacks against machine learning models,” in 2017 IEEE symposium on security and privacy (SP), pp. 3–18, IEEE, 2017.
  39. N. Carlini, S. Chien, M. Nasr, S. Song, A. Terzis, and F. Tramer, “Membership inference attacks from first principles,” in 2022 IEEE Symposium on Security and Privacy (SP), pp. 1897–1914, IEEE, 2022.
  40. Y. Zhang, J. Jia, X. Chen, A. Chen, Y. Zhang, J. Liu, K. Ding, and S. Liu, “To generate or not? safety-driven unlearned diffusion models are still easy to generate unsafe images… for now,” arXiv preprint arXiv:2310.11868, 2023.
  41. A. Ginart, M. Guan, G. Valiant, and J. Y. Zou, “Making ai forget you: Data deletion in machine learning,” Advances in neural information processing systems, vol. 32, 2019.
  42. S. Neel, A. Roth, and S. Sharifi-Malvajerdi, “Descent-to-delete: Gradient-based methods for machine unlearning,” in Algorithmic Learning Theory, pp. 931–962, PMLR, 2021.
  43. E. Ullah, T. Mai, A. Rao, R. A. Rossi, and R. Arora, “Machine unlearning via algorithmic stability,” in Conference on Learning Theory, pp. 4126–4142, PMLR, 2021.
  44. A. Sekhari, J. Acharya, G. Kamath, and A. T. Suresh, “Remember what you want to forget: Algorithms for machine unlearning,” Advances in Neural Information Processing Systems, vol. 34, pp. 18075–18086, 2021.
  45. A. Thudi, H. Jia, I. Shumailov, and N. Papernot, “On the necessity of auditable algorithmic definitions for machine unlearning,” in 31st USENIX Security Symposium (USENIX Security 22), pp. 4007–4022, 2022.
  46. C. Dwork, K. Kenthapadi, F. McSherry, I. Mironov, and M. Naor, “Our data, ourselves: Privacy via distributed noise generation,” in Annual international conference on the theory and applications of cryptographic techniques, pp. 486–503, Springer, 2006.
  47. L. Graves, V. Nagisetty, and V. Ganesh, “Amnesiac machine learning,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, pp. 11516–11524, 2021.
  48. C. Guo, T. Goldstein, A. Hannun, and L. Van Der Maaten, “Certified data removal from machine learning models,” arXiv preprint arXiv:1911.03030, 2019.
  49. L. Song and P. Mittal, “Systematic evaluation of privacy risks of machine learning models,” in 30th USENIX Security Symposium (USENIX Security 21), pp. 2615–2632, 2021.
  50. A. Becker and T. Liebig, “Evaluating machine unlearning via epistemic uncertainty,” arXiv preprint arXiv:2208.10836, 2022.
  51. A. Schioppa, P. Zablotskaia, D. Vilar, and A. Sokolov, “Scaling up influence functions,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 36, pp. 8179–8186, 2022.
  52. M. Paul, S. Ganguli, and G. K. Dziugaite, “Deep learning on a data diet: Finding important examples early in training,” Advances in Neural Information Processing Systems, vol. 34, pp. 20596–20607, 2021.
  53. S. Yang, Z. Xie, H. Peng, M. Xu, M. Sun, and P. Li, “Dataset pruning: Reducing training data by examining generalization influence,” arXiv preprint arXiv:2205.09329, 2022.
  54. G. Pruthi, F. Liu, S. Kale, and M. Sundararajan, “Estimating training data influence by tracing gradient descent,” Advances in Neural Information Processing Systems, vol. 33, pp. 19920–19930, 2020.
  55. Y. Zhang, Y. Zhang, A. Chen, J. Liu, G. Liu, M. Hong, S. Chang, S. Liu, et al., “Selectivity drives productivity: Efficient dataset pruning for enhanced transfer learning,” Advances in Neural Information Processing Systems, vol. 36, 2024.
  56. J. Huggins, T. Campbell, and T. Broderick, “Coresets for scalable bayesian logistic regression,” Advances in neural information processing systems, vol. 29, 2016.
  57. X. Xia, J. Liu, J. Yu, X. Shen, B. Han, and T. Liu, “Moderate coreset: A universal method of data selection for real-world data-efficient deep learning,” in The Eleventh International Conference on Learning Representations, 2022.
  58. Z. Borsos, M. Mutny, and A. Krause, “Coresets via bilevel optimization for continual learning and streaming,” Advances in Neural Information Processing Systems, vol. 33, pp. 14879–14890, 2020.
  59. S. Kim, S. Bae, and S.-Y. Yun, “Coreset sampling from open-set for fine-grained self-supervised learning,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7537–7547, 2023.
  60. O. Sener and S. Savarese, “Active learning for convolutional neural networks: A core-set approach,” arXiv preprint arXiv:1708.00489, 2017.
  61. C. Coleman, C. Yeh, S. Mussmann, B. Mirzasoleiman, P. Bailis, P. Liang, J. Leskovec, and M. Zaharia, “Selection via proxy: Efficient data selection for deep learning,” arXiv preprint arXiv:1906.11829, 2019.
  62. Y. Zeng, M. Pan, H. Jahagirdar, M. Jin, L. Lyu, and R. Jia, “How to sift out a clean data subset in the presence of data poisoning?,” arXiv preprint arXiv:2210.06516, 2022.
  63. S. Goel, A. Prabhu, A. Sanyal, S.-N. Lim, P. Torr, and P. Kumaraguru, “Towards adversarial evaluations for inexact machine unlearning,” arXiv preprint arXiv:2201.06640, 2022.
  64. M. Chen, W. Gao, G. Liu, K. Peng, and C. Wang, “Boundary unlearning: Rapid forgetting of deep networks via shifting the decision boundary,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7766–7775, 2023.
  65. Y. Zhang, P. Khanduri, I. Tsaknakis, Y. Yao, M. Hong, and S. Liu, “An introduction to bi-level optimization: Foundations and applications in signal processing and machine learning,” arXiv preprint arXiv:2308.00788, 2023.
  66. Springer Science & Business Media, 2002.
  67. A. Shaban, C.-A. Cheng, N. Hatch, and B. Boots, “Truncated back-propagation for bilevel optimization,” in The 22nd International Conference on Artificial Intelligence and Statistics, pp. 1723–1732, PMLR, 2019.
  68. J. Bernstein, Y.-X. Wang, K. Azizzadenesheli, and A. Anandkumar, “signsgd: Compressed optimisation for non-convex problems,” in International Conference on Machine Learning, pp. 560–569, PMLR, 2018.
  69. S. P. Boyd and L. Vandenberghe, Convex optimization. Cambridge university press, 2004.
  70. A. Krizhevsky, G. Hinton, et al., “Learning multiple layers of features from tiny images,” 2009.
  71. K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770–778, 2016.
  72. Z. Liu, P. Luo, X. Wang, and X. Tang, “Deep learning face attributes in the wild,” in Proceedings of the IEEE international conference on computer vision, pp. 3730–3738, 2015.
  73. K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” arXiv preprint arXiv:1409.1556, 2014.
  74. J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, “Imagenet: A large-scale hierarchical image database,” in 2009 IEEE conference on computer vision and pattern recognition, pp. 248–255, Ieee, 2009.
  75. R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer, “High-resolution image synthesis with latent diffusion models,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10684–10695, 2022.
  76. B. Mirzasoleiman, J. Bilmes, and J. Leskovec, “Coresets for data-efficient training of machine learning models,” in International Conference on Machine Learning, pp. 6950–6960, PMLR, 2020.
  77. S. Sagawa, A. Raghunathan, P. W. Koh, and P. Liang, “An investigation of why overparameterization exacerbates spurious correlations,” in International Conference on Machine Learning, pp. 8346–8356, PMLR, 2020.
  78. K. Xu, H. Chen, S. Liu, P.-Y. Chen, T.-W. Weng, M. Hong, and X. Lin, “Topology attack and defense for graph neural networks: An optimization perspective,” arXiv preprint arXiv:1906.04214, 2019.
  79. Y. Le and X. Yang, “Tiny imagenet visual recognition challenge,” CS 231N, vol. 7, no. 7, p. 3, 2015.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (4)
  1. Chongyu Fan (9 papers)
  2. Jiancheng Liu (19 papers)
  3. Alfred Hero (67 papers)
  4. Sijia Liu (204 papers)
Citations (14)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now