Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
126 tokens/sec
GPT-4o
47 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Distributed Solvers for Network Linear Equations with Scalarized Compression (2401.06332v2)

Published 12 Jan 2024 in eess.SY and cs.SY

Abstract: Distributed computing is fundamental to multi-agent systems, with solving distributed linear equations as a typical example. In this paper, we study distributed solvers for network linear equations over a network with node-to-node communication messages compressed as scalar values. Our key idea lies in a dimension compression scheme that includes a dimension-compressing vector and a data unfolding step. The compression vector applies to individual node states as an inner product to generate a real-valued message for node communication. In the unfolding step, such scalar message is then plotted along the subspace generated by the compression vector for the local computations. We first present a compressed consensus flow that relies only on such scalarized communication, and show that linear convergence can be achieved with well excited signals for the compression vector. We then employ such a compressed consensus flow as a fundamental consensus subroutine to develop distributed continuous-time and discrete-time solvers for network linear equations, and prove their linear convergence properties under scalar node communications. With scalar communications, a direct benefit would be the reduced node-to-node communication channel burden for distributed computing. Numerical examples are presented to illustrate the effectiveness of the established theoretical results.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (35)
  1. M. Mesbahi and M. Egerstedt. Graph Theoretic Methods in Multiagent Networks. Princeton University Press, 2010.
  2. S. Martinez, J. Cortés, and F. Bullo, “Motion coordination with distributed information,” IEEE Control Syst. Mag., vol. 27, no. 4, pp. 75-88, 2007.
  3. S. Kar, J. M. F. Moura and K. Ramanan, “Distributed parameter estimation in sensor networks: nonlinear observation models and imperfect communication,” IEEE Trans. Information Theory, vol. 58, no. 6, pp. 3575-52, 2012.
  4. A. G. Dimakis, S. Kar, J. M. F. Moura, M. G. Rabbat, and A Scaglione, “Gossip algorithms for distributed signal processing,” Proceedings of IEEE, vol. 98, no. 11, pp. 1847-1864, 2010.
  5. J. Lu and C. Y. Tang, “Distributed asynchronous algorithms for solving positive definite linear equations over networks - Part II: wireless networks. In Proc. first IFAC Workshop on Estimation and Control of Networked Systems, pp. 258-264, 2009.
  6. J. Lu and C. Y. Tang, “Distributed asynchronous algorithms for solving positive definite linear equations over networks - Part I: agent networks,” In Proc. first IFAC Workshop on Estimation and Control of Networked Systems, pp. 22-26, 2009.
  7. J. Liu, S. Mou, and A. S. Morse, “An asynchronous distributed algorithm for solving a linear algebraic equation,” In Proceedings of the 2013 IEEE Conference on Decision and Control, pp. 5409-5414, 2013.
  8. S. Mou and A. S. Morse, “A fixed-neighbor, distributed algorithm for solving a linear algebraic equation,” In Proceedings of the 2013 European Control Conference, pp. 2269-2273, 2013.
  9. C. E. Lee, A. Ozdaglar, and D. Shah, “Solving systems of linear equations: locally and asynchronously,” preprint arXiv 1411.2647, 2014.
  10. R. Tutunov, H. B. Ammar, and A. Jadbabaie, “A fast distributed solver for symmetric diagonally dominant linear equations,” preprint arXiv 1502.03158, 2015.
  11. S. Mou, J. Liu, and A. S. Morse, “A distributed algorithm for solving a linear algebraic equation,” IEEE Transactions on Automatic Control, vol. 60, no. 11, pp. 2863–2878, 2015.
  12. C. Hegde, F. Keinert, and E. Weber, “A Kaczmarz algorithm for solving tree based distributed systems of equations,” preprint arXiv 1904.05732, 2019.
  13. T. Yang, J. George, J. Qin, X. Yi, and J. Wu, “Distributed least squares solver for network linear equations,” Automatica, vol. 113, 108798, 2020.
  14. L. Ljung. “Analysis of recursive stochastic algorithms,” IEEE Trans. on Automatic Control, vol. 22, no.4, pp. 551–575, 1977.
  15. J. Wang and N. Elia, “Control approach to distributed optimization,” The 48th Annual Allerton Conference, pp. 557-561, 2010.
  16. B. Gharesifard and J. Cortés, “Distributed continuous-time convex optimization on weight-balanced digraphs,” IEEE Trans. Automatic Control, vol. 59, no. 3, pp. 781-786, 2014.
  17. B. Anderson, S. Mou, A. S. Morse, and U. Helmke, “Decentralized gradient algorithm for solution of a linear equation,” Numerical Algebra, Control and Optimization, vol. 6, no. 3, pp. 319–328, 2015.
  18. G. Shi, B. D. Anderson and U. Helmke, “Network flows that solve linear equations,” IEEE Trans. Autom. Control, vol. 62, no. 6, pp. 2659–2674, 2017.
  19. Doan, Thinh T, S. Theja Maguluri and J. Romberg , “Fast convergence rates of distributed subgradient methods with adaptive quantization,” IEEE Trans. Autom. Control, vol. 66, no. 5, pp. 2191–2205, 2021.
  20. X. Liu, Y. Li, R. Wang, J. Tang, and M. Yan, “Linear convergent decentralized optimization with compression,” in International Conference on Learning Representations, 2021.
  21. A. Beznosikov, S. Horvath, P. Richtarik, and M. Safaryan, “On biased compression for distributed learning”. arXiv preprint arXiv:2002.12410, 2020.
  22. X. Yi, S. Zhang, T. Yang, T. Chai and K. H. Johansson, “Communication Compression for Distributed Nonconvex Optimization,” IEEE Transactions on Automatic Control, vol. 68, no. 9, pp. 5477–5492, 2022.
  23. B. D. O. Anderson, “Exponential stability of linear equations arising in adaptive identification,” IEEE Trans. Autom. Control, vol. 22, no. 1, pp. 83-88, 1977.
  24. A. P. Morgan and K. S. Narendra, “On the uniform asymptotic stability of certain linear nonautonomous differential equations,” SIAM J. Control Optim., vol. 15, no. 1, pp. 5-24, 1977.
  25. S. S. Vempala, R. Wang, and D. P. Woodruff, “The communication complexity of optimization,” in Proceedings of the Thirty-First Annual ACM-SIAM Symposium on Discrete Algorithms (SODA), pp. 1733- 1752, 2020.
  26. S. Magnusson , H. Shokri-Ghadikolaei, and N. Li, “On maintaining linear convergence of distributed learning and optimization under limited communication,” IEEE Transactions on Signal Processing, vol. 68, pp. 6101-6116, 2020.
  27. E J. Candes, “The restricted isometry property and its implications for compressed sensing,” Comptes rendus mathematique, vol. 346, pp. 589-592, 2008.
  28. A. Loria, and E. Panteley, “Uniform exponential stability of linear time-varying systems: revisited,” Systems & Control Letters, vol.47, pp. 13-24, 2002.
  29. F. Pasqualetti, R. Carli, and F. Bullo. “Distributed estimation via iterative projections with application to power network monitoring.” Automatica, vol. 48, no.5, pp. 747-758, 2012.
  30. Y. Pan, P. Xiao, Y. He, Z. Shao, and Z. Li. “MULLS: Versatile LiDAR SLAM via multi-metric linear least square.” In Proceedings of IEEE International Conference on Robotics and Automation pp. 11633-11640, 2021.
  31. P. Yi, J. Lei, J. Chen, Y. Hong and G. Shi. “Distributed linear equations over random networks.” IEEE Transactions on Automatic Control, vol.68, no. 4, pp. 2607-2614, 2023.
  32. P. Wang, S. Mou, J. Lian, and W. Ren. “Solving a system of linear equations: From centralized to distributed algorithms.” Annual Reviews in Control, vol. 47, pp. 306-322, 2019.
  33. T. T. Doan, S. T. Maguluri, and J. Romberg, “Convergence rates of distributed gradient methods under random quantization: A stochastic approximation approach,” IEEE Transactions on Automatic Control, vol. 66, no. 10, pp. 4469–4484, 2020
  34. A. Koloskova, S. Stich, and M. Jaggi, “Decentralized stochastic optimization and gossip algorithms with compressed communication,” in International Conference on Machine Learning, pp. 3478–3487, 2019.
  35. A. Nedic, A. Olshevsky, A. Ozdaglar, and J. N. Tsitsiklis, “Distributed subgradient methods and quantization effects,” in IEEE Conference on Decision and Control, pp. 4177–4184, 2008.
Citations (2)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now