Papers
Topics
Authors
Recent
Search
2000 character limit reached

Privacy-Preserving Federated Learning via Dataset Distillation

Published 25 Oct 2024 in cs.LG | (2410.19548v3)

Abstract: Federated Learning (FL) allows users to share knowledge instead of raw data to train a model with high accuracy. Unfortunately, during the training, users lose control over the knowledge shared, which causes serious data privacy issues. We hold that users are only willing and need to share the essential knowledge to the training task to obtain the FL model with high accuracy. However, existing efforts cannot help users minimize the shared knowledge according to the user intention in the FL training procedure. This work proposes FLiP, which aims to bring the principle of least privilege (PoLP) to FL training. The key design of FLiP is applying elaborate information reduction on the training data through a local-global dataset distillation design. We measure the privacy performance through attribute inference and membership inference attacks. Extensive experiments show that FLiP strikes a good balance between model accuracy and privacy protection.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (21)
  1. Q. Yang, Y. Liu, T. Chen, and Y. Tong, “Federated machine learning: Concept and applications,” ACM Transactions on Intelligent Systems and Technology (TIST), vol. 10, no. 2, pp. 1–19, 2019.
  2. L. Lyu and C. Chen, “A novel attribute reconstruction attack in federated learning,” arXiv preprint arXiv:2108.06910, 2021.
  3. C. Chen, L. Lyu, H. Yu, and G. Chen, “Practical attribute reconstruction attack against federated learning,” IEEE Transactions on Big Data, 2022.
  4. M. Abadi, A. Chu, I. Goodfellow, H. B. McMahan, I. Mironov, K. Talwar, and L. Zhang, “Deep learning with differential privacy,” in Proceedings of the 2016 ACM SIGSAC conference on computer and communications security, 2016, pp. 308–318.
  5. M. Seif, R. Tandon, and M. Li, “Wireless federated learning with local differential privacy,” in 2020 IEEE International Symposium on Information Theory (ISIT), 2020, pp. 2604–2609.
  6. Z. Hu, J. Yan, and Y.-J. A. Zhang, “Communication-learning co-design for differentially private over-the-air federated learning with device sampling,” IEEE Transactions on Wireless Communications, 2024.
  7. Y. Shi, Y. Yang, and Y. Wu, “Federated edge learning with differential privacy: An active reconfigurable intelligent surface approach,” IEEE Transactions on Wireless Communications, 2024.
  8. X. Yang, W. Huang, and M. Ye, “Dynamic personalized federated learning with adaptive differential privacy,” Advances in Neural Information Processing Systems, vol. 36, pp. 72 181–72 192, 2023.
  9. H. Wu, X. Tian, Y. Gong, X. Su, M. Li, and F. Xu, “Dapter: Preventing user data abuse in deep learning inference services,” in Proceedings of the Web Conference 2021, 2021, pp. 1017–1028.
  10. W. Gao, S. Guo, T. Zhang, H. Qiu, Y. Wen, and Y. Liu, “Privacy-preserving collaborative learning with automatic transformation search,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 114–123.
  11. B. Xin, W. Yang, Y. Geng, S. Chen, S. Wang, and L. Huang, “Private fl-gan: Differential privacy synthetic data generation based on federated learning,” in ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).   IEEE, 2020, pp. 2927–2931.
  12. J. Wu, M. Hayat, M. Zhou, and M. Harandi, “Concealing sensitive samples against gradient leakage in federated learning,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 38, no. 19, 2024, pp. 21 717–21 725.
  13. R. Yu, S. Liu, and X. Wang, “Dataset distillation: A comprehensive review,” arXiv preprint arXiv:2301.07014, 2023.
  14. G. Hinton, O. Vinyals, and J. Dean, “Distilling the knowledge in a neural network,” arXiv preprint arXiv:1503.02531, 2015.
  15. I. Sucholutsky and M. Schonlau, “Soft-label dataset distillation and text dataset distillation,” in 2021 International Joint Conference on Neural Networks (IJCNN), 2021, pp. 1–8.
  16. B. Zhao and H. Bilen, “Dataset condensation with distribution matching,” in Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 2023, pp. 6514–6523.
  17. Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, “Gradient-based learning applied to document recognition,” Proceedings of the IEEE, vol. 86, no. 11, pp. 2278–2324, 1998.
  18. A. Krizhevsky, G. Hinton et al., “Learning multiple layers of features from tiny images,” 2009.
  19. A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep convolutional neural networks,” Communications of the ACM, vol. 60, no. 6, pp. 84–90, 2017.
  20. B. Zhao and H. Bilen, “Dataset condensation with differentiable siamese augmentation,” in International Conference on Machine Learning, 2021, pp. 12 674–12 685.
  21. K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 770–778.

Summary

No one has generated a summary of this paper yet.

Sign Up to Summarize

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Generate Now

Collections

Sign up for free to add this paper to one or more collections.

Sign Up