Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
162 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

A Meta-learning Framework for Tuning Parameters of Protection Mechanisms in Trustworthy Federated Learning (2305.18400v3)

Published 28 May 2023 in cs.LG and cs.AI

Abstract: Trustworthy Federated Learning (TFL) typically leverages protection mechanisms to guarantee privacy. However, protection mechanisms inevitably introduce utility loss or efficiency reduction while protecting data privacy. Therefore, protection mechanisms and their parameters should be carefully chosen to strike an optimal tradeoff between \textit{privacy leakage}, \textit{utility loss}, and \textit{efficiency reduction}. To this end, federated learning practitioners need tools to measure the three factors and optimize the tradeoff between them to choose the protection mechanism that is most appropriate to the application at hand. Motivated by this requirement, we propose a framework that (1) formulates TFL as a problem of finding a protection mechanism to optimize the tradeoff between privacy leakage, utility loss, and efficiency reduction and (2) formally defines bounded measurements of the three factors. We then propose a meta-learning algorithm to approximate this optimization problem and find optimal protection parameters for representative protection mechanisms, including Randomization, Homomorphic Encryption, Secret Sharing, and Compression. We further design estimation algorithms to quantify these found optimal protection parameters in a practical horizontal federated learning setting and provide a theoretical analysis of the estimation error.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (51)
  1. Deep learning with differential privacy. In Proceedings of the 2016 ACM SIGSAC conference on computer and communications security. ACM, New York, NY, USA, 308–318.
  2. Privacy-preserving deep learning via additively homomorphic encryption. IEEE Transactions on Information Forensics and Security 13, 5 (2017), 1333–1345.
  3. From robustness to privacy and back. arXiv preprint arXiv:2302.01855 (2023).
  4. G.R. Blakley. 1979. Safeguarding cryptographic keys. In Proceedings of the 1979 AFIPS National Computer Conference. AFIPS Press, Monval, NJ, USA, 313–317.
  5. Practical secure aggregation for privacy-preserving machine learning. In proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security. 1175–1191.
  6. Secureboost: A lossless federated learning framework. IEEE Intelligent Systems (2021).
  7. Flávio du Pin Calmon and Nadia Fawaz. 2012. Privacy against statistical inference. In 2012 50th annual Allerton conference on communication, control, and computing (Allerton). IEEE, 1401–1408.
  8. Adaptive subgradient methods for online learning and stochastic optimization. Journal of machine learning research 12, 7 (2011).
  9. Local privacy and minimax bounds: Sharp rates for probability estimation. arXiv preprint arXiv:1305.6000 (2013).
  10. Calibrating noise to sensitivity in private data analysis. In Theory of cryptography conference. Springer, 265–284.
  11. The algorithmic foundations of differential privacy. Foundations and Trends in Theoretical Computer Science 9, 3-4 (2014), 211–407.
  12. Bayesian privacy. Theoretical Economics 16, 4 (2021), 1557–1603.
  13. Dominik Maria Endres and Johannes E Schindelin. 2003. A new metric for probability distributions. IEEE Transactions on Information theory 49, 7 (2003), 1858–1860.
  14. Haokun Fang and Quan Qian. 2021. Privacy preserving machine learning with homomorphic encryption and federated learning. Future Internet 13, 4 (2021), 94.
  15. On the theory and practice of privacy-preserving Bayesian data analysis. arXiv preprint arXiv:1603.07294 (2016).
  16. Model inversion attacks that exploit confidence information and basic countermeasures. In Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security. 1322–1333.
  17. Inverting Gradients–How easy is it to break privacy in federated learning? arXiv preprint arXiv:2003.14053 (2020).
  18. Craig Gentry. 2009. A fully homomorphic encryption scheme. Stanford university.
  19. Differentially private federated learning: A client level perspective. arXiv preprint arXiv:1712.07557 (2017).
  20. David E. Goldberg. 1989. Genetic Algorithms in Search, Optimization and Machine Learning (1st ed.). Addison-Wesley Longman Publishing Co., Inc., USA.
  21. Federated Deep Learning with Bayesian Privacy. arXiv preprint arXiv:2109.13012 (2021).
  22. Otkrist Gupta and Ramesh Raskar. 2018. Distributed learning of deep neural network over multiple agents. Journal of Network and Computer Applications 116 (2018), 1–8.
  23. Optimizing Privacy, Utility and Efficiency in Constrained Multi-Objective Federated Learning. arXiv preprint arXiv:2305.00312 (2023).
  24. Privacy-preserving Federated Adversarial Domain Adaptation over Feature Groups for Interpretability. IEEE Transactions on Big Data (2022), 1–12.
  25. A framework for evaluating privacy-utility trade-off in vertical federated learning. arXiv preprint arXiv:2209.03885 (2022).
  26. Federated optimization: Distributed machine learning for on-device intelligence. arXiv preprint arXiv:1610.02527 (2016).
  27. Federated learning: Strategies for improving communication efficiency. arXiv preprint arXiv:1610.05492 (2016).
  28. Communication-efficient learning of deep networks from decentralized data. In Artificial Intelligence and Statistics. PMLR, 1273–1282.
  29. Federated learning of deep networks using model averaging. arXiv preprint arXiv:1602.05629 (2016).
  30. Rajeev Motwani and Prabhakar Raghavan. 1996. Randomized algorithms. ACM Computing Surveys (CSUR) 28, 1 (1996), 33–37.
  31. Frank Nielsen. 2019. On the Jensen–Shannon symmetrization of distances relying on abstract means. Entropy 21, 5 (2019), 485.
  32. Fast federated learning by balancing communication trade-offs. IEEE Transactions on Communications 69, 8 (2021), 5168–5182.
  33. Pascal Paillier. 1999. Public-key cryptosystems based on composite degree residuosity classes. In International conference on the theory and applications of cryptographic techniques. Springer, 223–238.
  34. Borzoo Rassouli and Deniz Gündüz. 2019. Optimal utility-privacy trade-off with total variation distance as a privacy measure. IEEE Transactions on Information Forensics and Security 15 (2019), 594–603.
  35. Learning representations by back-propagating errors. nature 323, 6088 (1986), 533–536.
  36. Adi Shamir. 1979. How to Share a Secret. Commun. ACM 22, 11 (nov 1979), 612–613. https://doi.org/10.1145/359168.359176
  37. Aleksei Triastcyn and Boi Faltings. 2020. Bayesian Differential Privacy for Machine Learning. In Proceedings of the 37th International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 119), Hal Daumé III and Aarti Singh (Eds.). PMLR, 9583–9592. https://proceedings.mlr.press/v119/triastcyn20a.html
  38. A hybrid approach to privacy-preserving federated learning. In Proceedings of the 12th ACM Workshop on Artificial Intelligence and Security. 1–11.
  39. LDP-Fed: Federated learning with local differential privacy. In Proceedings of the Third ACM International Workshop on Edge Systems, Analytics and Networking. 61–66.
  40. See through Gradients: Image Batch Recovery via GradInversion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 16337–16346.
  41. BatchCrypt: Efficient Homomorphic Encryption for Cross-Silo Federated Learning. In 2020 USENIX Annual Technical Conference (USENIX ATC 20). USENIX Association, 493–506. https://www.usenix.org/conference/atc20/presentation/zhang-chengliang
  42. PEFL: A privacy-enhanced federated learning scheme for big data analytics. In 2019 IEEE Global Communications Conference (GLOBECOM). IEEE, 1–6.
  43. Towards Achieving Near-optimal Utility for Privacy-Preserving Federated Learning via Data Generation and Parameter Distortion. arXiv preprint arXiv:2305.04288 (2023).
  44. A Game-theoretic Framework for Federated Learning. arXiv preprint arXiv:2304.05836 (2023).
  45. No free lunch theorem for security and utility in federated learning. ACM Transactions on Intelligent Systems and Technology 14, 1 (2022), 1–35.
  46. Probably approximately correct federated learning. arXiv preprint arXiv:2304.04641 (2023).
  47. Trading Off Privacy, Utility and Efficiency in Federated Learning. ACM Trans. Intell. Syst. Technol. (2023).
  48. Theoretically Principled Federated Learning for Balancing Privacy and Utility. arXiv preprint arXiv:2305.15148 (2023).
  49. idlg: Improved deep leakage from gradients. arXiv preprint arXiv:2001.02610 (2020).
  50. Ligeng Zhu and Song Han. 2020. Deep leakage from gradients. In Federated Learning. Springer, 17–31.
  51. Deep Leakage from Gradients. In Annual Conference on Neural Information Processing Systems (NeurIPS).
Citations (5)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now