Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
149 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Projection-based Prediction-Correction Method for Distributed Consensus Optimization (2309.09819v3)

Published 18 Sep 2023 in math.OC, cs.SY, and eess.SY

Abstract: Within the realm of industrial technology, optimization methods play a pivotal role and are extensively applied across various sectors, including transportation engineering, robotics, and machine learning. With the surge in data volumes, there is an increasing demand for solving large-scale problems, which in turn has spurred the development of distributed optimization methods. These methods rely on the collaborative efforts of numerous dispersed devices to achieve the collective goals of the system. This study focuses on the exploration of distributed consensus optimization problems with convex set constraints within networks. The paper introduces a novel Adaptive Projection Prediction-Correction Method (PPCM), inspired by the proximal point algorithm and incorporating the theory of variational inequalities. As a contraction algorithm with notable convergence performance, PPCM is particularly suited for decentralized network environments. Moreover, the selection of parameters for this method is both straightforward and intuitive, avoiding the complexities of intricate parameter tuning. Comprehensive theoretical analysis and empirical testing have validated the effectiveness of PPCM. When applied to problems such as distributed linear least squares, logistic regression, and support vector machines, PPCM demonstrates superior performance, achieving computation speeds over ten times faster than built-in Python functions while maintaining high precision. In conclusion, this research provides a valuable distributed consensus optimization technique, both theoretically and practically.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (42)
  1. W. Su, Traffic engineering and time-varying convex optimization. The Pennsylvania State University, 2009.
  2. R. Mohebifard and A. Hajbabaie, “Distributed optimization and coordination algorithms for dynamic traffic metering in urban street networks,” IEEE Transactions on Intelligent Transportation Systems, vol. 20, no. 5, pp. 1930–1941, 2018.
  3. D. Teodorović, “Swarm intelligence systems for transportation engineering: Principles and applications,” Transportation Research Part C: Emerging Technologies, vol. 16, no. 6, pp. 651–667, 2008.
  4. J. Chen and S. Kai, “Cooperative transportation control of multiple mobile manipulators through distributed optimization,” Science China Information Sciences, vol. 61, pp. 1–17, 2018.
  5. R. Tron, J. Thomas, G. Loianno, K. Daniilidis, and V. Kumar, “A distributed optimization framework for localization and formation control: Applications to vision-based measurements,” IEEE Control Systems Magazine, vol. 36, no. 4, pp. 22–44, 2016.
  6. Z. Li, S. S. Ge, and S. Liu, “Contact-force distribution optimization and control for quadruped robots using both gradient and adaptive neural networks,” IEEE Transactions on Neural Networks and Learning Systems, vol. 25, no. 8, pp. 1460–1473, 2013.
  7. Z. Zhang and M.-Y. Chow, “Convergence analysis of the incremental cost consensus algorithm under different communication network topologies in a smart grid,” IEEE Transactions on Power Systems, vol. 27, no. 4, pp. 1761–1768, 2012.
  8. P. Braun, L. Grüne, C. M. Kellett, S. R. Weller, and K. Worthmann, “A distributed optimization algorithm for the predictive control of smart grids,” IEEE Transactions on Automatic Control, vol. 61, no. 12, pp. 3898–3911, 2016.
  9. W. Yu, C. Li, X. Yu, G. Wen, and J. Lü, “Economic power dispatch in smart grids: a framework for distributed optimization and consensus dynamics,” Science China Information Sciences, vol. 61, pp. 1–16, 2018.
  10. G. Joshi, “Distributed optimization in machine learning,” in Optimization Algorithms for Distributed Machine Learning, pp. 1–12, Springer, 2022.
  11. J. Qiu, Q. Wu, G. Ding, Y. Xu, and S. Feng, “A survey of machine learning for big data processing,” EURASIP Journal on Advances in Signal Processing, vol. 2016, pp. 1–16, 2016.
  12. S. Sun, Z. Cao, H. Zhu, and J. Zhao, “A survey of optimization methods from a machine learning perspective,” IEEE Transactions on Cybernetics, vol. 50, no. 8, pp. 3668–3681, 2019.
  13. T. Yang, J. Lu, D. Wu, J. Wu, G. Shi, Z. Meng, and K. H. Johansson, “A distributed algorithm for economic dispatch over time-varying directed networks with delays,” IEEE Transactions on Industrial Electronics, vol. 64, no. 6, pp. 5095–5106, 2016.
  14. A. Nedić and J. Liu, “Distributed optimization for control,” Annual Review of Control, Robotics, and Autonomous Systems, vol. 1, pp. 77–103, 2018.
  15. K. M. Lynch and F. C. Park, Modern robotics. Cambridge University Press, 2017.
  16. M. Rabbat and R. Nowak, “Distributed optimization in sensor networks,” in Proceedings of the 3rd International Symposium on Information Processing in Sensor Networks, pp. 20–27, 2004.
  17. D. Minoli, K. Sohraby, and B. Occhiogrosso, “Iot considerations, requirements, and architectures for smart buildings—energy optimization and next-generation building management systems,” IEEE Internet of Things Journal, vol. 4, no. 1, pp. 269–283, 2017.
  18. J. Davis, T. Edgar, R. Graybill, P. Korambath, B. Schott, D. Swink, J. Wang, and J. Wetzel, “Smart manufacturing,” Annual Review of Chemical and Biomolecular Engineering, vol. 6, pp. 141–160, 2015.
  19. S. Nabavi, J. Zhang, and A. Chakrabortty, “Distributed optimization algorithms for wide-area oscillation monitoring in power systems using interregional pmu-pdc architectures,” IEEE Transactions on Smart Grid, vol. 6, no. 5, pp. 2529–2538, 2015.
  20. R. Zieliński, “Pj huber; robust statistics,” Mathematica Applicanda, vol. 11, no. 23, 1983.
  21. J. C. Chen, K. Yao, and R. E. Hudson, “Source localization and beamforming,” IEEE Signal Processing Magazine, vol. 19, no. 2, pp. 30–39, 2002.
  22. X. Sheng and Y.-H. Hu, “Energy based acoustic source localization,” in Information Processing in Sensor Networks, pp. 285–300, Springer, 2003.
  23. R. D. Nowak, “Distributed em algorithms for density estimation and clustering in sensor networks,” IEEE Transactions on Signal Processing, vol. 51, no. 8, pp. 2245–2253, 2003.
  24. H. K. Abeynanda and G. Lanel, “A study on distributed optimization over large-scale networked systems,” Journal of Mathematics, vol. 2021, pp. 1–19, 2021.
  25. A. Nedic and A. Ozdaglar, “Distributed subgradient methods for multi-agent optimization,” IEEE Transactions on Automatic Control, vol. 54, no. 1, pp. 48–61, 2009.
  26. A. Nedic and A. Ozdaglar, “10 cooperative distributed multi-agent,” Convex Optimization in Signal Processing and Communications, vol. 340, 2010.
  27. A. Nedic, A. Ozdaglar, and P. A. Parrilo, “Constrained consensus and optimization in multi-agent networks,” IEEE Transactions on Automatic Control, vol. 55, no. 4, pp. 922–938, 2010.
  28. D. Varagnolo, F. Zanella, A. Cenedese, G. Pillonetto, and L. Schenato, “Newton-raphson consensus for distributed convex optimization,” IEEE Transactions on Automatic Control, vol. 61, no. 4, pp. 994–1009, 2015.
  29. K. I. Tsianos, S. Lawlor, and M. G. Rabbat, “Consensus-based distributed optimization: Practical issues and applications in large-scale machine learning,” in 2012 50th Annual Allerton Conference on Communication, Control, and Computing, pp. 1543–1550, IEEE, 2012.
  30. K. I. Tsianos and M. G. Rabbat, “Distributed consensus and optimization under communication delays,” in 2011 49th Annual Allerton Conference on Communication, Control, and Computing, pp. 974–982, IEEE, 2011.
  31. K. I. Tsianos, S. Lawlor, and M. G. Rabbat, “Push-sum distributed dual averaging for convex optimization,” in 2012 51st IEEE Conference on Decision and Control, pp. 5453–5458, IEEE, 2012.
  32. K. Tsianos, “The role of the network in distributed optimization algorithms: Convergence rates, scalability, communication/computation tradeoffs and communication delays,” 2013.
  33. S. Pu, W. Shi, J. Xu, and A. Nedić, “Push–pull gradient methods for distributed optimization in networks,” IEEE Transactions on Automatic Control, vol. 66, no. 1, pp. 1–16, 2020.
  34. T.-H. Chang, M. Hong, and X. Wang, “Multi-agent distributed optimization via inexact consensus admm,” IEEE Transactions on Signal Processing, vol. 63, no. 2, pp. 482–497, 2014.
  35. R. Merris, “Laplacian matrices of graphs: a survey,” Linear Algebra and its Applications, vol. 197, pp. 143–176, 1994.
  36. B. He, “A class of projection and contraction methods for monotone variational inequalities,” Applied Mathematics and optimization, vol. 35, no. 1, pp. 69–76, 1997.
  37. B. He, “Inexact implicit methods for monotone general variational inequalities,” Mathematical Programming, vol. 86, no. 1, pp. 199–217, 1999.
  38. G. Korplevich, “Ekstragradientnyi metod dlia otyskaniia sedlovykh tchek i drugikh zadach,” Ekonomika I Matematicheski Metody, vol. 12, no. 74, p. 756, 1976.
  39. B. He and L.-Z. Liao, “Improvements of some projection methods for monotone nonlinear variational inequalities,” Journal of Optimization Theory and Applications, vol. 112, pp. 111–128, 2002.
  40. B. He, X. Yuan, and J. J. Zhang, “Comparison of two kinds of prediction-correction methods for monotone variational inequalities,” Computational Optimization and Applications, vol. 27, pp. 247–267, 2004.
  41. B. He, A Globally Linearly Convergent Projection and Contraction Method for a Class of Linear Complementarity Problems. Inst. für Angewandte Math. und Statistik, 1992.
  42. B. He, “A uniform framework of contraction methods for convex optimization and monotone variational inequality,” Sci. Sin. Math, vol. 48, p. 255, 2018.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now