Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
129 tokens/sec
GPT-4o
28 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Distributed Least-Squares Optimization Solvers with Differential Privacy (2403.01435v1)

Published 3 Mar 2024 in math.OC, cs.SY, and eess.SY

Abstract: This paper studies the distributed least-squares optimization problem with differential privacy requirement of local cost functions, for which two differentially private distributed solvers are proposed. The first is established on the distributed gradient tracking algorithm, by appropriately perturbing the initial values and parameters that contain the privacy-sensitive data with Gaussian and truncated Laplacian noises, respectively. Rigorous proofs are established to show the achievable trade-off between the ({\epsilon}, {\delta})-differential privacy and the computation accuracy. The second solver is established on the combination of the distributed shuffling mechanism and the average consensus algorithm, which enables each agent to obtain a noisy version of parameters characterizing the global gradient. As a result, the least-squares optimization problem can be eventually solved by each agent locally in such a way that any given ({\epsilon}, {\delta})-differential privacy requirement can be preserved while the solution may be computed with the accuracy independent of the network size, which makes the latter more suitable for large-scale distributed least-squares problems. Numerical simulations are presented to show the effectiveness of both solvers.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (42)
  1. A. Nedic, “Distributed gradient methods for convex machine learning problems in networks: Distributed optimization,” IEEE Signal Processing Magazine, vol. 37, no. 3, pp. 92–101, 2020.
  2. L. Yan, J. L. Webber, A. Mehbodniya, B. Moorthy, S. Sivamani, S. Nazir, and M. Shabaz, “Distributed optimization of heterogeneous uav cluster pid controller based on machine learning,” Computers and Electrical Engineering, vol. 101, p. 108059, 2022.
  3. P. Wan and M. D. Lemmon, “Event-triggered distributed optimization in sensor networks,” in 2009 International Conference on Information Processing in Sensor Networks, 2009, pp. 49–60.
  4. I. Lobel and A. Ozdaglar, “Distributed subgradient methods for convex optimization over random networks,” IEEE Transactions on Automatic Control, vol. 56, no. 6, pp. 1291–1306, 2011.
  5. I. Notarnicola, M. Bin, L. Marconi, and G. Notarstefano, “The gradient tracking is a distributed integral action,” IEEE Transactions on Automatic Control, pp. 1–8, 2023.
  6. D. Jakovetić, J. Xavier, and J. M. F. Moura, “Fast distributed gradient methods,” IEEE Transactions on Automatic Control, vol. 59, no. 5, pp. 1131–1146, 2014.
  7. A. Nedić and A. Olshevsky, “Distributed optimization over time-varying directed graphs,” IEEE Transactions on Automatic Control, vol. 60, no. 3, pp. 601–615, 2015.
  8. S. Yang, Q. Liu, and J. Wang, “Distributed optimization based on a multiagent system in the presence of communication delays,” IEEE Transactions on Systems, Man, and Cybernetics: Systems, vol. 47, no. 5, pp. 717–728, 2017.
  9. K. Srivastava and A. Nedic, “Distributed asynchronous constrained stochastic optimization,” IEEE Journal of Selected Topics in Signal Processing, vol. 5, no. 4, pp. 772–790, 2011.
  10. C. Zhang, M. Ahmad, and Y. Wang, “Admm based privacy-preserving decentralized optimization,” IEEE Transactions on Information Forensics and Security, vol. 14, no. 3, pp. 565–580, 2019.
  11. A. Nedic, A. Ozdaglar, and P. A. Parrilo, “Constrained consensus and optimization in multi-agent networks,” IEEE Transactions on Automatic Control, vol. 55, no. 4, pp. 922–938, 2010.
  12. D. Jakovetić, “A unification and generalization of exact distributed first-order methods,” IEEE Transactions on Signal and Information Processing over Networks, vol. 5, no. 1, pp. 31–46, 2019.
  13. D. Varagnolo, F. Zanella, A. Cenedese, G. Pillonetto, and L. Schenato, “Newton-raphson consensus for distributed convex optimization,” IEEE Transactions on Automatic Control, vol. 61, no. 4, pp. 994–1009, 2015.
  14. A. Nedic, A. Olshevsky, and W. Shi, “Achieving geometric convergence for distributed optimization over time-varying graphs,” SIAM Journal on Optimization, vol. 27, no. 4, pp. 2597–2633, 2017.
  15. J. Xu, S. Zhu, Y. C. Soh, and L. Xie, “Convergence of asynchronous distributed gradient methods over stochastic networks,” IEEE Transactions on Automatic Control, vol. 63, no. 2, pp. 434–448, 2017.
  16. G. Scutari and Y. Sun, “Distributed nonconvex constrained optimization over time-varying digraphs,” Mathematical Programming, vol. 176, pp. 497–544, 2019.
  17. S. Pu, W. Shi, J. Xu, and A. Nedić, “Push–pull gradient methods for distributed optimization in networks,” IEEE Transactions on Automatic Control, vol. 66, no. 1, pp. 1–16, 2020.
  18. S. Pu and A. Nedić, “Distributed stochastic gradient tracking methods,” Mathematical Programming, vol. 187, pp. 409–457, 2021.
  19. Y. Pan, P. Xiao, Y. He, Z. Shao, and Z. Li, “Mulls: Versatile lidar slam via multi-metric linear least square,” in 2021 IEEE International Conference on Robotics and Automation (ICRA), 2021, pp. 11 633–11 640.
  20. Y. Zheng and Q. Liu, “A review of distributed optimization: Problems, models and algorithms,” Neurocomputing, vol. 483, pp. 446–459, 2022.
  21. T. Yang, X. Yi, J. Wu, Y. Yuan, D. Wu, Z. Meng, Y. Hong, H. Wang, Z. Lin, and K. H. Johansson, “A survey of distributed optimization,” Annual Reviews in Control, vol. 47, pp. 278–305, 2019.
  22. C.-N. Hang, Y.-Z. Tsai, P.-D. Yu, J. Chen, and C.-W. Tan, “Privacy-enhancing digital contact tracing with machine learning for pandemic response: A comprehensive review,” Big Data and Cognitive Computing, vol. 7, no. 2, 2023.
  23. Y. Lu and M. Zhu, “Privacy preserving distributed optimization using homomorphic encryption,” Automatica, vol. 96, pp. 314–325, 2018.
  24. S. Mao, Y. Tang, Z. Dong, K. Meng, Z. Y. Dong, and F. Qian, “A privacy preserving distributed optimization algorithm for economic dispatch over time-varying directed networks,” IEEE Transactions on Industrial Informatics, vol. 17, no. 3, pp. 1689–1701, 2021.
  25. Y. Wang and A. Nedić, “Tailoring gradient methods for differentially-private distributed optimization,” IEEE Transactions on Automatic Control, pp. 1–16, 2023.
  26. Y. Lou, L. Yu, S. Wang, and P. Yi, “Privacy preservation in distributed subgradient optimization algorithms,” IEEE Transactions on Cybernetics, vol. 48, no. 7, pp. 2154–2165, 2018.
  27. C. Dwork, F. McSherry, K. Nissim, and A. Smith, “Calibrating noise to sensitivity in private data analysis,” in Theory of Cryptography, S. Halevi and T. Rabin, Eds.   Berlin, Heidelberg: Springer Berlin Heidelberg, 2006, pp. 265–284.
  28. J. Le Ny and G. J. Pappas, “Differentially private filtering,” IEEE Transactions on Automatic Control, vol. 59, no. 2, pp. 341–354, 2013.
  29. E. Nozari, P. Tallapragada, and J. Cortés, “Differentially private average consensus: Obstructions, trade-offs, and optimal algorithm design,” Automatica, vol. 81, pp. 221–231, 2017.
  30. M. Ruan, H. Gao, and Y. Wang, “Secure and privacy-preserving consensus,” IEEE Transactions on Automatic Control, vol. 64, no. 10, pp. 4035–4049, 2019.
  31. Z. Huang, S. Mitra, and N. Vaidya, “Differentially private distributed optimization,” in Proceedings of the 16th International Conference on Distributed Computing and Networking, ser. ICDCN ’15.   New York, NY, USA: Association for Computing Machinery, 2015.
  32. Y. Wang and T. Başar, “Decentralized nonconvex optimization with guaranteed privacy and accuracy,” Automatica, vol. 150, p. 110858, 2023.
  33. E. Nozari, P. Tallapragada, and J. Cortés, “Differentially private distributed convex optimization via functional perturbation,” IEEE Transactions on Control of Network Systems, vol. 5, no. 1, pp. 395–408, 2018.
  34. S. Han, U. Topcu, and G. J. Pappas, “Differentially private distributed constrained optimization,” IEEE Transactions on Automatic Control, vol. 62, no. 1, pp. 50–64, 2017.
  35. M. T. Hale and M. Egerstedt, “Cloud-enabled differentially private multiagent optimization with constraints,” IEEE Transactions on Control of Network Systems, vol. 5, no. 4, pp. 1693–1706, 2018.
  36. C. Dwork, A. Roth et al., “The algorithmic foundations of differential privacy.” Found. Trends Theor. Comput. Sci., vol. 9, no. 3-4, pp. 211–407, 2014.
  37. Q. Geng, W. Ding, R. Guo, and S. Kumar, “Tight analysis of privacy and utility tradeoff in approximate differential privacy,” in Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics, ser. Proceedings of Machine Learning Research, S. Chiappa and R. Calandra, Eds., vol. 108.   PMLR, 26–28 Aug 2020, pp. 89–99.
  38. B. Balle and Y.-X. Wang, “Improving the gaussian mechanism for differential privacy: Analytical calibration and optimal denoising,” in International Conference on Machine Learning.   PMLR, 2018, pp. 394–403.
  39. L. Wang, W. Liu, F. Guo, Z. Qiao, and Z. Wu, “Differentially private average consensus with improved accuracy-privacy trade-off,” arXiv:2309.08464, 2023.
  40. Z. Qiao, F. Guo, X. Pan, Y. Sun, and L. Wang, “Distributed load shedding via differentially private average consensus algorithm,” in 2022 34th Chinese Control and Decision Conference (CCDC), 2022, pp. 1503–1508.
  41. P. Paillier, “Public-key cryptosystems based on composite degree residuosity classes,” in International conference on the theory and applications of cryptographic techniques.   Springer, 1999, pp. 223–238.
  42. F. McSherry, “Privacy integrated queries: An extensible platform for privacy-preserving data analysis,” Commun. ACM, vol. 53, no. 9, p. 89–97, sep 2010.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now