Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
149 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Flamingo: Multi-Round Single-Server Secure Aggregation with Applications to Private Federated Learning (2308.09883v4)

Published 19 Aug 2023 in cs.CR and cs.LG

Abstract: This paper introduces Flamingo, a system for secure aggregation of data across a large set of clients. In secure aggregation, a server sums up the private inputs of clients and obtains the result without learning anything about the individual inputs beyond what is implied by the final sum. Flamingo focuses on the multi-round setting found in federated learning in which many consecutive summations (averages) of model weights are performed to derive a good model. Previous protocols, such as Bell et al. (CCS '20), have been designed for a single round and are adapted to the federated learning setting by repeating the protocol multiple times. Flamingo eliminates the need for the per-round setup of previous protocols, and has a new lightweight dropout resilience protocol to ensure that if clients leave in the middle of a sum the server can still obtain a meaningful result. Furthermore, Flamingo introduces a new way to locally choose the so-called client neighborhood introduced by Bell et al. These techniques help Flamingo reduce the number of interactions between clients and the server, resulting in a significant reduction in the end-to-end runtime for a full training session over prior work. We implement and evaluate Flamingo and show that it can securely train a neural network on the (Extended) MNIST and CIFAR-100 datasets, and the model converges without a loss in accuracy, compared to a non-private federated learning system.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (77)
  1. Cloudflare randomness beacon. https://developers.cloudflare.com/randomness-beacon/.
  2. TensorFlow: Large-scale machine learning on heterogeneous systems. Technical report, 2015. https://www.tensorflow.org/.
  3. Bingo: Adaptivity and asynchrony in verifiable secret sharing and distributed key generation. In Proceedings of the International Cryptology Conference (CRYPTO), 2023.
  4. Prio+: Privacy preserving aggregate statistics via boolean shares. In C. Galdi and S. Jarecki, editors, Proceedings of the International Conference on Security and Cryptography for Networks(SCN), 2022.
  5. Aggregate measurement via oblivious shuffling. Cryptology ePrint Archive, Paper 2021/1490, 2021. https://eprint.iacr.org/2021/1490.
  6. Efficient representation of numerical optimization problems for SNARKs. In Proceedings of the USENIX Security Symposium, 2022.
  7. How to backdoor federated learning. In Proceedings of the Artificial Intelligence and Statistics Conference (AISTATS), 2020.
  8. Non-interactive secure multiparty computation. In Proceedings of the International Cryptology Conference (CRYPTO), 2014.
  9. Secure single-server aggregation with (poly) logarithmic overhead. In Proceedings of the ACM Conference on Computer and Communications Security (CCS), 2020.
  10. Acorn: Input validation for secure aggregation. Cryptology ePrint Archive, Paper 2022/1461, 2022. https://eprint.iacr.org/2022/1461.
  11. Practical secure aggregation for privacy-preserving machine learning. In Proceedings of the ACM Conference on Computer and Communications Security (CCS), 2017.
  12. Lightweight techniques for private heavy hitters. In Proceedings of the IEEE Symposium on Security and Privacy (S&P), 2021.
  13. Efficient indifferentiable hashing into ordinary elliptic curves. In Proceedings of the International Cryptology Conference (CRYPTO), 2010.
  14. ABIDES: Agent-based interactive discrete event simulation environment. https://github.com/abides-sim/abides, 2020.
  15. ABIDES: Towards high-fidelity multi-agent market simulation. In Proceedings of the 2020 ACM SIGSIM Conference on Principles of Advanced Discrete Simulation, 2020.
  16. Leaf: A benchmark for federated settings. https://github.com/TalwalkarLab/leaf.
  17. Leaf: A benchmark for federated settings. arXiv preprint arXiv:1812.01097, 2018. https://arxiv.org/abs/1812.01097.
  18. J. Canny and S. Sorkin. Practical large-scale distributed key generation. In Proceedings of the International Conference on the Theory and Applications of Cryptographic Techniques (EUROCRYPT), 2004.
  19. G. Castagnos and F. Laguillaumie. Linearly homomorphic encryption from ddh. Cryptology ePrint Archive, Paper 2015/047, 2015. https://eprint.iacr.org/2015/047.
  20. Privacy-preserving stream aggregation with fault tolerance. In Proceedings of the International Financial Cryptography Conference, 2011.
  21. Seemless: Secure end-to-end encrypted messaging with less trust. In Proceedings of the ACM Conference on Computer and Communications Security (CCS), 2019.
  22. D. L. Chaum. The dining cryptographers problem: Unconditional sender and recipient untraceability. Journal of Cryptology, 1(1), 1988.
  23. Eiffel: Ensuring integrity for federated learning. In Proceedings of the ACM Conference on Computer and Communications Security (CCS), 2022.
  24. H. Corrigan-Gibbs and D. Boneh. Prio: Private, robust, and scalable computation of aggregate statistics. In Proceedings of the USENIX Symposium on Networked Systems Design and Implementation (NSDI), 2017.
  25. Spurt: Scalable distributed randomness beacon with transparent setup. In Proceedings of the IEEE Symposium on Security and Privacy (S&P), 2021.
  26. Practical asynchronous distributed key generation. In Proceedings of the IEEE Symposium on Security and Privacy (S&P), 2022.
  27. PROV-FL: Privacy-preserving round optimal verifiable federated learning. In Proceedings of the ACM Workshop on Artificial Intelligence and Security, 2022.
  28. Y. Desmedt and Y. Frankel. Threshold cryptosystems. In Proceedings of the International Cryptology Conference (CRYPTO), 1989.
  29. PrivEx: Private collection of traffic statistics for anonymous communication networks. In Proceedings of the ACM Conference on Computer and Communications Security (CCS), 2014.
  30. How Much Privacy Does Federated Learning with Secure Aggregation Guarantee? In Proceedings of the Privacy Enhancing Technologies Symposium (PETS), 2023.
  31. P. Feldman. A practical scheme for non-interactive verifiable secret sharing. In Proceedings of the IEEE Symposium on Foundations of Computer Science (FOCS), 1987.
  32. Threshold rsa for dynamic and adhoc groups. In Proceedings of the International Conference on the Theory and Applications of Cryptographic Techniques (EUROCRYPT), 2008.
  33. Secure distributed key generation for discrete-log based cryptosystems. In Journal of Cryptology, 2006.
  34. YOSO: You only speak once / secure MPC with stateless ephemeral roles. In Proceedings of the International Cryptology Conference (CRYPTO), 2021.
  35. E. N. Gilbert. Random graphs. In The Annals of Mathematical Statistics, 1959.
  36. MicroFedML: Privacy preserving federated learning for small weights. Cryptology ePrint Archive, Paper 2022/714, 2022. https://eprint.iacr.org/2022/714.
  37. Cubic: A new tcp-friendly high-speed tcp variant. ACM SIGOPS operating systems review, 2008.
  38. Best possible information-theoretic MPC. In Proceedings of the Theory of Cryptography Conference (TCC), 2018.
  39. Hashing to elliptic curves. https://www.ietf.org/archive/id/draft-irtf-cfrg-hash-to-curve-10.html, 2021.
  40. Proactive secret sharing or how to cope with perpetual leakage. In Proceedings of the International Cryptology Conference (CRYPTO), 1995.
  41. Merkle2: A low-latency transparency log system. In Proceedings of the IEEE Symposium on Security and Privacy (S&P), 2021.
  42. FastSecAgg: Scalable secure aggregation for privacy-preserving federated learning. In ICML Workshop on Federated Learning for User Privacy and Data Confidentiality, 2020.
  43. Advances and open problems in federated learning. In Foundations and Trends in Machine Learning, 2021.
  44. Asynchronous distributed key generation for computationally-secure randomness, consensus, and threshold signatures. In Proceedings of the ACM Conference on Computer and Communications Security (CCS), 2020.
  45. Churp: Dynamic-committee proactive secret sharing. In Proceedings of the ACM Conference on Computer and Communications Security (CCS), 2019.
  46. A. Krizhevsky. Learning multiple layers of features from tiny images. Technical report, 2009. https://www.cs.toronto.edu/~kriz/learning-features-2009-TR.pdf.
  47. The CIFAR-100 dataset. https://www.cs.toronto.edu/~kriz/cifar.html.
  48. Oort: Efficient federated learning via guided participant selection. In Proceedings of the USENIX Symposium on Operating Systems Design and Implementation (OSDI), 2021.
  49. Aardvark: An asynchronous authenticated dictionary with short proofs. In Proceedings of the USENIX Security Symposium, 2022.
  50. RoFL: Robustness of secure federated learning. In Proceedings of the IEEE Symposium on Security and Privacy (S&P), 2023.
  51. SoK: Secure aggregation based on cryptographic schemes for federated learning. In Proceedings of the Privacy Enhancing Technologies Symposium (PETS), 2023.
  52. Communication-efficient learning of deep networks from decentralized data. In Proceedings of the Artificial Intelligence and Statistics Conference (AISTATS), 2017.
  53. CONIKS: bringing key transparency to end users. In Proceedings of the USENIX Security Symposium, 2015.
  54. Efficient private statistics with succinct sketches. In Proceedings of the Network and Distributed System Security Symposium (NDSS), 2016.
  55. Exploiting unintended feature leakage in collaborative learning. In Proceedings of the IEEE Symposium on Security and Privacy (S&P), 2019.
  56. Eluding secure aggregation in federated learning via model inconsistency. In Proceedings of the ACM Conference on Computer and Communications Security (CCS), 2022.
  57. T. Pedersen. A threshold cryptosystem without a trusted party. In Proceedings of the International Conference on the Theory and Applications of Cryptographic Techniques (EUROCRYPT), 1991.
  58. VPriv: Protecting privacy in location-based vehicular services. In Proceedings of the USENIX Security Symposium, 2009.
  59. Privacy and accountability for location-based aggregate statistics. In Proceedings of the ACM Conference on Computer and Communications Security (CCS), 2011.
  60. Mycelium: Large-scale distributed graph queries with differential privacy. In Proceedings of the ACM Symposium on Operating Systems Principles (SOSP), 2021.
  61. Honeycrisp: Large-scale differentially private aggregation without a trusted core. In Proceedings of the ACM Symposium on Operating Systems Principles (SOSP), 2019.
  62. Orchard: Differentially private analytics at scale. In Proceedings of the USENIX Symposium on Operating Systems Design and Implementation (OSDI), 2020.
  63. Privacy-preserving aggregation of time-series data. In Proceedings of the Network and Distributed System Security Symposium (NDSS), 2011.
  64. V. Shoup and R. Gennaro. Securing threshold cryptosystems against chosen ciphertext attack. In Journal of Cryptology, 2002.
  65. Turbo-aggregate: Breaking the quadratic aggregation barrier in secure federated learning. In Journal on Selected Areas in Information Theory, 2021.
  66. LightSecAgg: a lightweight and versatile design for secure aggregation in federated learning. In Proceedings of Machine Learning and Systems, 2022.
  67. Efficient differentially private secure aggregation for federated learning via hardness of learning with errors. In Proceedings of the USENIX Security Symposium, 2022.
  68. Transparency logs via append-only authenticated dictionaries. In Proceedings of the ACM Conference on Computer and Communications Security (CCS), 2019.
  69. A hybrid approach to privacy-preserving federated learning. In Proceedings of the ACM workshop on artificial intelligence and security, 2019.
  70. VeRSA: Verifiable registries with efficient client audits from RSA authenticated dictionaries. In Proceedings of the ACM Conference on Computer and Communications Security (CCS), 2022.
  71. Transparency dictionaries with succinct proofs of correct operation. In Proceedings of the Network and Distributed System Security Symposium (NDSS), 2022.
  72. R. S. Wahby and D. Boneh. Fast and simple constant-time hashing to the BLS12-381 elliptic curve. In Proceedings of the Conference on Cryptographic Hardware and Embedded Systems (CHES), 2019.
  73. Eavesdrop the composition proportion of training labels in federated learning. arXiv:1910/06044, 2023. https://arxiv.org/abs/1910.06044.
  74. H. Yuan and T. Ma. Federated accelerated stochastic gradient descent. In Neural Information Processing Systems (NeurIPS), 2020.
  75. Ibex: Privacy-preserving ad conversion tracking and bidding. In Proceedings of the ACM Conference on Computer and Communications Security (CCS), 2022.
  76. Addax: A fast, private, and accountable ad exchange infrastructure. In Proceedings of the USENIX Symposium on Networked Systems Design and Implementation (NSDI), 2023.
  77. Deep leakage from gradients. In Neural Information Processing Systems (NeurIPS), 2019.
Citations (38)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets