Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
11 tokens/sec
GPT-4o
12 tokens/sec
Gemini 2.5 Pro Pro
40 tokens/sec
o3 Pro
5 tokens/sec
GPT-4.1 Pro
37 tokens/sec
DeepSeek R1 via Azure Pro
33 tokens/sec
2000 character limit reached

Logit Poisoning Attack in Distillation-based Federated Learning and its Countermeasures (2401.17746v1)

Published 31 Jan 2024 in cs.CR

Abstract: Distillation-based federated learning has emerged as a promising collaborative learning approach, where clients share the output logit vectors of a public dataset rather than their private model parameters. This practice reduces the risk of privacy invasion attacks and facilitates heterogeneous learning. The landscape of poisoning attacks within distillation-based federated learning is complex, with existing research employing traditional data poisoning strategies targeting the models' parameters. However, these attack schemes primarily have shortcomings rooted in their original designs, which target the model parameters rather than the logit vectors. Furthermore, they do not adequately consider the role of logit vectors in carrying information during the knowledge transfer process. This misalignment results in less efficiency in the context of distillation-based federated learning. Due to the limitations of existing methodologies, our research delves into the intrinsic properties of the logit vector, striving for a more nuanced understanding. We introduce a two-stage scheme for logit poisoning attacks, addressing previous shortcomings. Initially, we collect the local logits, generate the representative vectors, categorize the logit elements within the vector, and design a shuffling table to maximize information entropy. Then, we intentionally scale the shuffled logit vectors to enhance the magnitude of the target vectors. Concurrently, we propose an efficient defense algorithm to counter this new poisoning scheme by calculating the distance between estimated benign vectors and vectors uploaded by users. Through extensive experiments, our study illustrates the significant threat posed by the proposed logit poisoning attack and highlights the effectiveness of our defense algorithm.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (50)
  1. B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Arcas, “Communication-efficient learning of deep networks from decentralized data,” in Artificial intelligence and statistics.   PMLR, 2017, pp. 1273–1282.
  2. J. Park, S. Samarakoon, M. Bennis, and M. Debbah, “Wireless network intelligence at the edge,” Proceedings of the IEEE, vol. 107, no. 11, pp. 2204–2239, 2019.
  3. W. Y. B. Lim, N. C. Luong, D. T. Hoang, Y. Jiao, Y.-C. Liang, Q. Yang, D. Niyato, and C. Miao, “Federated learning in mobile edge networks: A comprehensive survey,” IEEE Communications Surveys & Tutorials, vol. 22, no. 3, pp. 2031–2063, 2020.
  4. P. Kairouz, H. B. McMahan, B. Avent, A. Bellet, M. Bennis, A. N. Bhagoji, K. Bonawitz, Z. Charles, G. Cormode, R. Cummings et al., “Advances and open problems in federated learning,” Foundations and Trends® in Machine Learning, vol. 14, no. 1–2, pp. 1–210, 2021.
  5. Y. Zhou, X. Ma, D. Wu, and X. Li, “Communication-efficient and attack-resistant federated edge learning with dataset distillation,” IEEE Transactions on Cloud Computing, 2022.
  6. S. Zeng, Z. Li, H. Yu, Z. Zhang, L. Luo, B. Li, and D. Niyato, “Hfedms: Heterogeneous federated learning with memorable data semantics in industrial metaverse,” IEEE Transactions on Cloud Computing, 2023.
  7. H. Chang, V. Shejwalkar, R. Shokri, and A. Houmansadr, “Cronus: Robust and heterogeneous collaborative learning with black-box knowledge transfer,” arXiv preprint arXiv:1912.11279, 2019.
  8. D. Li and J. Wang, “Fedmd: Heterogenous federated learning via model distillation,” arXiv preprint arXiv:1910.03581, 2019.
  9. T. Lin, L. Kong, S. U. Stich, and M. Jaggi, “Ensemble distillation for robust model fusion in federated learning,” Advances in Neural Information Processing Systems, vol. 33, pp. 2351–2363, 2020.
  10. S. Itahara, T. Nishio, Y. Koda, M. Morikura, and K. Yamamoto, “Distillation-based semi-supervised federated learning for communication-efficient collaborative training with non-iid private data,” IEEE Transactions on Mobile Computing, vol. 22, no. 1, pp. 191–205, 2021.
  11. S. Cheng, J. Wu, Y. Xiao, and Y. Liu, “Fedgems: Federated learning of larger server models via selective knowledge fusion,” arXiv preprint arXiv:2110.11027, 2021.
  12. X. Fang and M. Ye, “Robust federated learning with noisy and heterogeneous clients,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 10 072–10 081.
  13. G. Hinton, O. Vinyals, and J. Dean, “Distilling the knowledge in a neural network,” arXiv preprint arXiv:1503.02531, 2015.
  14. J. Ba and R. Caruana, “Do deep nets really need to be deep?” Advances in neural information processing systems, vol. 27, 2014.
  15. N. Papernot, M. Abadi, U. Erlingsson, I. Goodfellow, and K. Talwar, “Semi-supervised knowledge transfer for deep learning from private training data,” arXiv preprint arXiv:1610.05755, 2016.
  16. N. Papernot, S. Song, I. Mironov, A. Raghunathan, K. Talwar, and Ú. Erlingsson, “Scalable private learning with pate,” arXiv preprint arXiv:1802.08908, 2018.
  17. X. Wang, R. Zhang, Y. Sun, and J. Qi, “Kdgan: Knowledge distillation with generative adversarial networks,” Advances in neural information processing systems, vol. 31, 2018.
  18. R. Anil, G. Pereyra, A. Passos, R. Ormandi, G. E. Dahl, and G. E. Hinton, “Large scale distributed neural network training through online distillation,” arXiv preprint arXiv:1804.03235, 2018.
  19. G. F. Ejigu, S. H. Hong, and C. S. Hong, “Robust federated learning with local mixed co-teaching,” in 2023 International Conference on Information Networking (ICOIN).   IEEE, 2023, pp. 277–281.
  20. M. Jagielski, A. Oprea, B. Biggio, C. Liu, C. Nita-Rotaru, and B. Li, “Manipulating machine learning: Poisoning attacks and countermeasures for regression learning,” in 2018 IEEE symposium on security and privacy (SP).   IEEE, 2018, pp. 19–35.
  21. J. Hayes and O. Ohrimenko, “Contamination attacks and mitigation in multi-party machine learning,” Advances in neural information processing systems, vol. 31, 2018.
  22. L. Muñoz-González, B. Biggio, A. Demontis, A. Paudice, V. Wongrassamee, E. C. Lupu, and F. Roli, “Towards poisoning of deep learning algorithms with back-gradient optimization,” in Proceedings of the 10th ACM workshop on artificial intelligence and security, 2017, pp. 27–38.
  23. G. Baruch, M. Baruch, and Y. Goldberg, “A little is enough: Circumventing defenses for distributed learning,” Advances in Neural Information Processing Systems, vol. 32, 2019.
  24. E. Bagdasaryan, A. Veit, Y. Hua, D. Estrin, and V. Shmatikov, “How to backdoor federated learning,” in International Conference on Artificial Intelligence and Statistics.   PMLR, 2020, pp. 2938–2948.
  25. Y. Wang, X. Ma, Z. Chen, Y. Luo, J. Yi, and J. Bailey, “Symmetric cross entropy for robust learning with noisy labels,” in Proceedings of the IEEE/CVF international conference on computer vision, 2019, pp. 322–330.
  26. D. Cao, S. Chang, Z. Lin, G. Liu, and D. Sun, “Understanding distributed poisoning attack in federated learning,” in 2019 IEEE 25th International Conference on Parallel and Distributed Systems (ICPADS).   IEEE, 2019, pp. 233–239.
  27. X. Lyu, Y. Han, W. Wang, J. Liu, B. Wang, J. Liu, and X. Zhang, “Poisoning with cerberus: stealthy and colluded backdoor attack against federated learning,” in Thirty-Seventh AAAI Conference on Artificial Intelligence, 2023.
  28. P. Blanchard, E. M. El Mhamdi, R. Guerraoui, and J. Stainer, “Machine learning with adversaries: Byzantine tolerant gradient descent,” Advances in neural information processing systems, vol. 30, 2017.
  29. V. Shejwalkar and A. Houmansadr, “Manipulating the byzantine: Optimizing model poisoning attacks and defenses for federated learning,” in NDSS, 2021.
  30. C. Fung, C. J. Yoon, and I. Beschastnikh, “The limitations of federated learning in sybil settings.” in RAID, 2020, pp. 301–316.
  31. A. P. Sundar, F. Li, X. Zou, and T. Gao, “Distributed swift and stealthy backdoor attack on federated learning,” in 2022 IEEE International Conference on Networking, Architecture and Storage (NAS).   IEEE, 2022, pp. 1–8.
  32. L. Lyu, H. Yu, X. Ma, C. Chen, L. Sun, J. Zhao, Q. Yang, and S. Y. Philip, “Privacy and robustness in federated learning: Attacks and defenses,” IEEE transactions on neural networks and learning systems, 2022.
  33. M. Fang, X. Cao, J. Jia, and N. Z. Gong, “Local model poisoning attacks to byzantine-robust federated learning,” in Proceedings of the 29th USENIX Conference on Security Symposium, 2020, pp. 1623–1640.
  34. M. Yang, H. Cheng, F. Chen, X. Liu, M. Wang, and X. Li, “Model poisoning attack in differential privacy-based federated learning,” Information Sciences, vol. 630, pp. 158–172, 2023.
  35. K. Wei, J. Li, M. Ding, C. Ma, H. H. Yang, F. Farokhi, S. Jin, T. Q. Quek, and H. V. Poor, “Federated learning with differential privacy: Algorithms and performance analysis,” IEEE Transactions on Information Forensics and Security, vol. 15, pp. 3454–3469, 2020.
  36. S. Kullback and R. A. Leibler, “On information and sufficiency,” The annals of mathematical statistics, vol. 22, no. 1, pp. 79–86, 1951.
  37. X. Cao and N. Z. Gong, “Mpaf: Model poisoning attacks to federated learning based on fake clients,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 3396–3404.
  38. C. Xie, O. Koyejo, and I. Gupta, “Fall of empires: Breaking byzantine-tolerant sgd by inner product manipulation,” in Uncertainty in Artificial Intelligence.   PMLR, 2020, pp. 261–270.
  39. H. Li, X. Sun, and Z. Zheng, “Learning to attack federated learning: A model-based reinforcement learning attack framework,” Advances in Neural Information Processing Systems, vol. 35, pp. 35 007–35 020, 2022.
  40. Y. Yu, Q. Liu, L. Wu, R. Yu, S. L. Yu, and Z. Zhang, “Untargeted attack against federated recommendation systems via poisonous item embeddings and the defense,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 37, no. 4, 2023, pp. 4854–4863.
  41. A. N. Bhagoji, S. Chakraborty, P. Mittal, and S. Calo, “Analyzing federated learning through an adversarial lens,” in International Conference on Machine Learning.   PMLR, 2019, pp. 634–643.
  42. C. Xie, K. Huang, P.-Y. Chen, and B. Li, “Dba: Distributed backdoor attacks against federated learning,” in International conference on learning representations, 2019.
  43. O. Suciu, R. Marginean, Y. Kaya, H. Daume III, and T. Dumitras, “When does machine learning {{\{{FAIL}}\}}? generalized transferability for evasion and poisoning attacks,” in 27th USENIX Security Symposium (USENIX Security 18), 2018, pp. 1299–1316.
  44. B. Biggio, L. Didaci, G. Fumera, and F. Roli, “Poisoning attacks to compromise face templates,” in 2013 international conference on biometrics (ICB).   IEEE, 2013, pp. 1–7.
  45. L. Deng, “The mnist database of handwritten digit images for machine learning research,” IEEE Signal Processing Magazine, vol. 29, no. 6, pp. 141–142, 2012.
  46. S. Lloyd, “Least squares quantization in pcm,” IEEE transactions on information theory, vol. 28, no. 2, pp. 129–137, 1982.
  47. U. Von Luxburg, “A tutorial on spectral clustering,” Statistics and computing, vol. 17, pp. 395–416, 2007.
  48. Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, “Gradient-based learning applied to document recognition,” Proceedings of the IEEE, vol. 86, no. 11, pp. 2278–2324, 1998.
  49. G. E. Hinton, N. Srivastava, A. Krizhevsky, I. Sutskever, and R. R. Salakhutdinov, “Improving neural networks by preventing co-adaptation of feature detectors,” arXiv preprint arXiv:1207.0580, 2012.
  50. D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” arXiv preprint arXiv:1412.6980, 2014.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now