Papers
Topics
Authors
Recent
Search
2000 character limit reached

Towards Efficient Secure Aggregation in FL: Partial Vector Freezing for Cost Compression

Published 8 Dec 2023 in cs.CR | (2312.04920v2)

Abstract: Secure aggregation of user vectors has become a critical issue in the field of federated learning. Many Secure Aggregation Protocols (SAP) face exorbitant computation costs, which severely limit their applicability. We uncover that current endeavors to reduce computation costs tend to overlook a crucial fact: a considerable portion of SAP's computation burden stems from processing each entry in the private vectors. Given this observation, we propose PVF, a portable module for compressing computation costs. PVF is able to freeze'' a substantial portion of the private vector through specific linear transformations, only requiring $\frac{1}{\lambda}$ of the original vector to participate in SAP. Eventually, users canthaw'' the public sum of the ``frozen entries" by the result of SAP. To enhance functionality, we introduce extensions that can enforce consistency constraints on users' original vectors, verify aggregated results, and enhance security when a portion of the private vector is known to the server. We demonstrate that PVF can seamlessly integrate with various SAP and prove that it poses no threat to user privacy in the semi-honest and active adversary settings. We select $8$ baselines, encompassing $6$ distinct types of SAP, and explore the acceleration effects of PVF on these SAP. Empirical investigations indicate that when $\lambda=100$, PVF yields up to $99.5\times$ speedup and up to $32.3\times$ communication reduction, with the potential to approach nearly $1000\times$ acceleration as $\lambda$ increases.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (52)
  1. B. Liu, M. Ding, S. Shaham, W. Rahayu, F. Farokhi, and Z. Lin, “When machine learning meets privacy: A survey and outlook,” ACM Computing Surveys (CSUR), vol. 54, no. 2, pp. 1–36, 2021.
  2. B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Arcas, “Communication-efficient learning of deep networks from decentralized data,” in Artificial intelligence and statistics.   PMLR, 2017, pp. 1273–1282.
  3. L. Zhu, Z. Liu, and S. Han, “Deep leakage from gradients,” Advances in neural information processing systems, vol. 32, 2019.
  4. J. Geiping, H. Bauermeister, H. Dröge, and M. Moeller, “Inverting gradients-how easy is it to break privacy in federated learning?” Advances in Neural Information Processing Systems, vol. 33, pp. 16 937–16 947, 2020.
  5. Z. Liu, J. Guo, W. Yang, J. Fan, K.-Y. Lam, and J. Zhao, “Privacy-preserving aggregation in federated learning: A survey,” IEEE Transactions on Big Data, 2022.
  6. Y. Xu, C. Peng, W. Tan, Y. Tian, M. Ma, and K. Niu, “Non-interactive verifiable privacy-preserving federated learning,” Future Generation Computer Systems, vol. 128, pp. 365–380, 2022.
  7. E. Sotthiwat, L. Zhen, Z. Li, and C. Zhang, “Partially encrypted multi-party computation for federated learning,” 2021 IEEE/ACM 21st International Symposium on Cluster, Cloud and Internet Computing (CCGrid), pp. 828–835, 2021.
  8. K. Bonawitz, V. Ivanov, B. Kreuter, A. Marcedone, H. B. McMahan, S. Patel, D. Ramage, A. Segal, and K. Seth, “Practical secure aggregation for privacy-preserving machine learning,” Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security, 2017.
  9. J. Bell, K. A. Bonawitz, A. Gascón, T. Lepoint, and M. Raykova, “Secure single-server aggregation with (poly)logarithmic overhead,” Proceedings of the 2020 ACM SIGSAC Conference on Computer and Communications Security, 2020.
  10. Z. Liu, J. Guo, K.-Y. Lam, and J. Zhao, “Efficient dropout-resilient aggregation for privacy-preserving machine learning,” IEEE Transactions on Information Forensics and Security, vol. 18, pp. 1839–1854, 2022.
  11. Z. Wei, Q. Pei, N. Zhang, X. Liu, C. Wu, and A. Taherkordi, “Lightweight federated learning for large-scale iot devices with privacy guarantee,” IEEE Internet of Things Journal, vol. 10, pp. 3179–3191, 2023.
  12. T. Eltaras, F. Sabry, W. Labda, K. Alzoubi, and Q. Malluhi, “Efficient verifiable protocol for privacy-preserving aggregation in federated learning,” IEEE Transactions on Information Forensics and Security, 2023.
  13. Y. Aono, T. Hayashi, L. Wang, S. Moriai et al., “Privacy-preserving deep learning via additively homomorphic encryption,” IEEE Transactions on Information Forensics and Security, vol. 13, no. 5, pp. 1333–1345, 2017.
  14. Y. Li, H. Li, G. Xu, X. Huang, and R. Lu, “Efficient privacy-preserving federated learning with unreliable users,” IEEE Internet of Things Journal, vol. 9, pp. 11 590–11 603, 2022.
  15. K. Wei, J. Li, M. Ding, C. Ma, H. H. Yang, F. Farokhi, S. Jin, T. Q. Quek, and H. V. Poor, “Federated learning with differential privacy: Algorithms and performance analysis,” IEEE Transactions on Information Forensics and Security, vol. 15, pp. 3454–3469, 2020.
  16. R. Hu, Y. Guo, H. Li, Q. Pei, and Y. Gong, “Personalized federated learning with differential privacy,” IEEE Internet of Things Journal, vol. 7, no. 10, pp. 9530–9539, 2020.
  17. S. L. Garfinkel, J. M. Abowd, and S. Powazek, “Issues encountered deploying differential privacy,” in Proceedings of the 2018 Workshop on Privacy in the Electronic Society, 2018, pp. 133–137.
  18. T. Stevens, C. Skalka, C. Vincent, J. Ring, S. Clark, and J. Near, “Efficient differentially private secure aggregation for federated learning via hardness of learning with errors,” in 31st USENIX Security Symposium (USENIX Security 22), 2022, pp. 1379–1395.
  19. J. So, B. Güler, and A. S. Avestimehr, “Turbo-aggregate: Breaking the quadratic aggregation barrier in secure federated learning,” IEEE Journal on Selected Areas in Information Theory, vol. 2, no. 1, pp. 479–489, 2021.
  20. M. Kalikinkar, G. Guang, and L. Chuyi, “Nike-based fast privacy-preserving high-dimensional data aggregation for mobile devices,” University of Waterloo, Waterloo, ON, Canada, Tech. Rep. CACR, vol. 10, p. 2018, 2018.
  21. S. Kadhe, N. Rajaraman, O. O. Koyluoglu, and K. Ramchandran, “Fastsecagg: Scalable secure aggregation for privacy-preserving federated learning,” arXiv preprint arXiv:2009.11248, 2020.
  22. I. Ergun, H. U. Sami, and B. Guler, “Sparsified secure aggregation for privacy-preserving federated learning,” arXiv preprint arXiv:2112.12872, 2021.
  23. S. Lu, R. Li, W. Liu, C. Guan, and X. Yang, “Top-k sparsification with secure aggregation for privacy-preserving federated learning,” Computers & Security, vol. 124, p. 102993, 2023.
  24. C. Hahn, H. Kim, M. Kim, and J. Hur, “Versa: Verifiable secure aggregation for cross-device federated learning,” IEEE Transactions on Dependable and Secure Computing, vol. 20, pp. 36–52, 2023.
  25. J. So, R. E. Ali, B. Güler, J. Jiao, and A. S. Avestimehr, “Securing secure aggregation: Mitigating multi-round privacy leakage in federated learning,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 37, no. 8, 2023, pp. 9864–9873.
  26. Z. Liu, H.-Y. Lin, and Y. Liu, “Long-term privacy-preserving aggregation with user-dynamics for federated learning,” IEEE Transactions on Information Forensics and Security, vol. 18, pp. 2398–2412, 2023.
  27. D. Pasquini, D. Francati, and G. Ateniese, “Eluding secure aggregation in federated learning via model inconsistency,” in Proceedings of the 2022 ACM SIGSAC Conference on Computer and Communications Security, 2022, pp. 2429–2443.
  28. X. Guo, Z. Liu, J. Li, J. Gao, B. Hou, C. Dong, and T. Baker, “Verifl: Communication-efficient and fast verifiable aggregation for federated learning,” IEEE Transactions on Information Forensics and Security, vol. 16, pp. 1736–1751, 2021.
  29. J. Bell, A. Gascón, T. Lepoint, B. Li, S. Meiklejohn, M. Raykova, and C. Yun, “{{\{{ACORN}}\}}: Input validation for secure aggregation,” in 32nd USENIX Security Symposium (USENIX Security 23), 2023, pp. 4805–4822.
  30. H. Lycklama, L. Burkhalter, A. Viand, N. Küchler, and A. Hithnawi, “Rofl: Robustness of secure federated learning,” in 2023 IEEE Symposium on Security and Privacy (SP).   IEEE, 2023, pp. 453–476.
  31. Y. Ma, J. Woods, S. Angel, A. Polychroniadou, and T. Rabin, “Flamingo: Multi-round single-server secure aggregation with applications to private federated learning,” 2023 IEEE Symposium on Security and Privacy (SP), pp. 477–496, 2023.
  32. H. Mozaffari, V. Shejwalkar, and A. Houmansadr, “Every vote counts: Ranking-based training of federated learning to resist poisoning attacks,” in USENIX Security Symposium, 2023.
  33. T. P. Pedersen, “Non-interactive and information-theoretic secure verifiable secret sharing,” in Annual international cryptology conference.   Springer, 1991, pp. 129–140.
  34. J. Howard and S. Ruder, “Universal language model fine-tuning for text classification,” in Annual Meeting of the Association for Computational Linguistics, 2018.
  35. M. Rathee, C. Shen, S. Wagh, and R. A. Popa, “Elsa: Secure aggregation for federated learning with malicious actors,” in 2023 IEEE Symposium on Security and Privacy (SP).   IEEE, 2023, pp. 1961–1979.
  36. Y. Wang, A. Zhang, S.-L. Wu, and S. Yu, “Vosa: Verifiable and oblivious secure aggregation for privacy-preserving federated learning,” IEEE Transactions on Dependable and Secure Computing, vol. 20, pp. 3601–3616, 2023.
  37. T. A. Eltaras, F. Sabry, W. Labda, K. Alzoubi, and Q. Ahmedeltaras, “Efficient verifiable protocol for privacy-preserving aggregation in federated learning,” IEEE Transactions on Information Forensics and Security, vol. 18, pp. 2977–2990, 2023.
  38. A. Shamir, “How to share a secret,” Communications of the ACM, vol. 22, no. 11, pp. 612–613, 1979.
  39. Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, “Gradient-based learning applied to document recognition,” Proceedings of the IEEE, vol. 86, no. 11, pp. 2278–2324, 1998.
  40. L. Deng, “The mnist database of handwritten digit images for machine learning research,” IEEE Signal Processing Magazine, vol. 29, no. 6, pp. 141–142, 2012.
  41. Y. Lindell, “Secure multiparty computation,” Communications of the ACM, vol. 64, no. 1, pp. 86–96, 2020.
  42. D. Boer and S. Kramer, “Secure sum outperforms homomorphic encryption in (current) collaborative deep learning,” arXiv preprint arXiv:2006.02894, 2020.
  43. T. Jahani-Nezhad, M. A. Maddah-ali, S. Li, and G. Caire, “Swiftagg: Communication-efficient and dropout-resistant secure aggregation for federated learning with worst-case security guarantees,” 2022 IEEE International Symposium on Information Theory (ISIT), pp. 103–108, 2022.
  44. Y. Zheng, S. Lai, Y. Liu, X. Yuan, X. Yi, and C. Wang, “Aggregation service for federated learning: An efficient, secure, and more resilient realization,” IEEE Transactions on Dependable and Secure Computing, vol. 20, pp. 988–1001, 2022.
  45. P. Kairouz, Z. Liu, and T. Steinke, “The distributed discrete gaussian mechanism for federated learning with secure aggregation,” in International Conference on Machine Learning, 2021.
  46. I. Ergun, H. U. Sami, and B. Guler, “Sparsified secure aggregation for privacy-preserving federated learning,” ArXiv, vol. abs/2112.12872, 2021.
  47. Z. Liu, S. Chen, J. Ye, J. Fan, H. Li, and X. Li, “Sash: Efficient secure aggregation based on shprg for federated learning,” in Uncertainty in Artificial Intelligence.   PMLR, 2022, pp. 1243–1252.
  48. F. Tang, W. Wu, J. Liu, H. Wang, and M. Xian, “Privacy-preserving distributed deep learning via homomorphic re-encryption,” Electronics, 2019.
  49. H. S. A. Fang and Q. Qian, “Privacy preserving machine learning with homomorphic encryption and federated learning,” Future Internet, vol. 13, p. 94, 2021.
  50. S. Truex, N. Baracaldo, A. Anwar, T. Steinke, H. Ludwig, R. Zhang, and Y. Zhou, “A hybrid approach to privacy-preserving federated learning,” in Proceedings of the 12th ACM workshop on artificial intelligence and security, 2019, pp. 1–11.
  51. J. Ma, S.-A. Naas, S. Sigg, and X. Lyu, “Privacy-preserving federated learning based on multi-key homomorphic encryption,” International Journal of Intelligent Systems, vol. 37, no. 9, pp. 5880–5901, 2022.
  52. R. C. Geyer, T. Klein, and M. Nabi, “Differentially private federated learning: A client level perspective,” ArXiv, vol. abs/1712.07557, 2017.

Summary

No one has generated a summary of this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Sign Up to Summarize

Paper to Video (Beta)

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Collections

Sign up for free to add this paper to one or more collections.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up