Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
140 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
46 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

On Inhomogeneous Infinite Products of Stochastic Matrices and Applications (2401.14612v1)

Published 26 Jan 2024 in math.OC, cs.SY, and eess.SY

Abstract: With the growth of magnitude of multi-agent networks, distributed optimization holds considerable significance within complex systems. Convergence, a pivotal goal in this domain, is contingent upon the analysis of infinite products of stochastic matrices (IPSMs). In this work, convergence properties of inhomogeneous IPSMs are investigated. The convergence rate of inhomogeneous IPSMs towards an absolute probability sequence $\pi$ is derived. We also show that the convergence rate is nearly exponential, which coincides with existing results on ergodic chains. The methodology employed relies on delineating the interrelations among Sarymsakov matrices, scrambling matrices, and positive-column matrices. Based on the theoretical results on inhomogeneous IPSMs, we propose a decentralized projected subgradient method for time-varying multi-agent systems with graph-related stretches in (sub)gradient descent directions. The convergence of the proposed method is established for convex objective functions, and extended to non-convex objectives that satisfy Polyak-Lojasiewicz conditions. To corroborate the theoretical findings, we conduct numerical simulations, aligning the outcomes with the established theoretical framework.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (32)
  1. G. Hu, Y. Zhu, D. Zhao, M. Zhao, and J. Hao, “Event-triggered communication network with limited-bandwidth constraint for multi-agent reinforcement learning,” IEEE Trans. Neural Netw. Learn. Syst., vol. 34, no. 8, pp. 3966–3978, Aug. 2023.
  2. S. Liu, J. Sun, H. Zhang, and M. Zhai, “Fully distributed event-driven adaptive consensus of unknown linear systems,” IEEE Trans. Neural Netw. Learn. Syst., vol. 34, no. 10, pp. 8007–8016, Oct. 2023.
  3. J. J. R. Liu, J. Lam, and K.-W. Kwok, “Positive consensus of fractional-order multiagent systems over directed graphs,” IEEE Trans. Neural Netw. Learn. Syst., vol. 34, no. 11, pp. 9542–9548, Nov. 2023.
  4. W. Li, H. Zhang, Z. Gao, Y. Wang, and J. Sun, “Fully distributed event/self-triggered bipartite output formation-containment tracking control for heterogeneous multiagent systems,” IEEE Trans. Neural Netw. Learn. Syst., vol. 34, no. 10, pp. 7851–7860, Oct. 2023.
  5. X. Jiang, X. Zeng, J. Sun, and J. Chen, “Distributed proximal gradient algorithm for nonconvex optimization over time-varying networks,” IEEE Trans. Control Netw. Syst., vol. 10, no. 2, pp. 1005–1017, Jun. 2023.
  6. E. Seneta, “Coefficients of ergodicity: Structure and applications,” Adv. Appl. Probab., vol. 11, no. 3, pp. 576–590, Sept. 1979.
  7. J. Wolfowitz, “Products of indecomposable, aperiodic, stochastic matrices,” Proc. Am. Math. Soc., vol. 14, no. 5, pp. 733–737, Oct. 1963.
  8. J. Anthonisse and H. Tijms, “Exponential convergence of products of stochastic matrices,” J. Math. Anal. Appl., vol. 59, no. 2, pp. 360–364, Jun. 1977.
  9. C. W. Wu, “Agreement and consensus problems in groups of autonomous agents with linear dynamics,” in IEEE Intern. Symp. Circuit. Syst., vol. 1, Kobe, Japan, May. 23-26 2005, pp. 292–295.
  10. J. Hajnal and M. S. Bartlett, “Weak ergodicity in non-homogeneous markov chains,” Math. Proc. Cambridge Philos. Soc., vol. 54, no. 2, pp. 233–246, Apr. 1958.
  11. D. Coppersmith and C. W. Wu, “Conditions for weak ergodicity of inhomogeneous markov chains,” Stat. Probab. Lett., vol. 78, no. 17, pp. 3082–3085, Dec. 2008.
  12. Y. Chen, W. Xiong, and F. Li, “Convergence of infinite products of stochastic matrices: A graphical decomposition criterion,” IEEE Trans. Autom. Control, vol. 61, no. 11, pp. 3599–3605, Nov. 2016.
  13. A. Nedic and A. Ozdaglar, “Distributed subgradient methods for multi-agent optimization,” IEEE Trans. Autom. Control, vol. 54, no. 1, pp. 48–61, Jan. 2009.
  14. B. Touri and A. Nedic, “On ergodicity, infinite flow, and consensus in random models,” IEEE Trans. Autom. Control, vol. 56, no. 7, pp. 1593–1605, Jul. 2011.
  15. ——, “On approximations and ergodicity classes in random chains,” IEEE Trans. Autom. Control, vol. 57, no. 11, pp. 2718–2730, Nov. 2012.
  16. B. Liu, W. Lu, L. Jiao, and T. Chen, “Products of generalized stochastic matrices with applications to consensus analysis in networks of multiagents with delays,” IEEE Trans. Cybern., vol. 50, no. 1, pp. 386–399, Jan. 2020.
  17. W. Xia, J. Liu, M. Cao, K. H. Johansson, and T. Başar, “Generalized sarymsakov matrices,” IEEE Trans. Autom. Control, vol. 64, no. 8, pp. 3085–3100, Aug. 2019.
  18. A. Rogozin, C. A. Uribe, A. V. Gasnikov, N. Malkovsky, and A. Nedić, “Optimal distributed convex optimization on slowly time-varying graphs,” IEEE Trans. Control Netw. Syst., vol. 7, no. 2, pp. 829–841, Jun. 2020.
  19. J. Li, C. Gu, Z. Wu, and T. Huang, “Online learning algorithm for distributed convex optimization with time-varying coupled constraints and bandit feedback,” IEEE Trans. Cybern., vol. 52, no. 2, pp. 1009–1020, Feb. 2022.
  20. R. Dixit, A. S. Bedi, and K. Rajawat, “Online learning over dynamic graphs via distributed proximal gradient algorithm,” IEEE Trans. Autom. Control, vol. 66, no. 11, pp. 5065–5079, Nov. 2021.
  21. C. Wu, H. Fang, X. Zeng, Q. Yang, Y. Wei, and J. Chen, “Distributed continuous-time algorithm for time-varying optimization with affine formation constraints,” IEEE Trans. Autom. Control, vol. 68, no. 4, pp. 2615–2622, Apr. 2023.
  22. A. Sundararajan, B. Van Scoy, and L. Lessard, “Analysis and design of first-order distributed optimization algorithms over time-varying graphs,” IEEE Trans. Control Netw. Syst., vol. 7, no. 4, pp. 1597–1608, Dec. 2020.
  23. Y. Wang and T. Başar, “Gradient-tracking-based distributed optimization with guaranteed optimality under noisy information sharing,” IEEE Trans. Autom. Control, vol. 68, no. 8, pp. 4796–4811, Aug. 2023.
  24. T. A. Sarymsakov, “Inhomogeneous markov chains,” Theory Probab. Appl., vol. 6, no. 2, pp. 178–185, 1961.
  25. P.-Y. Chevalier, “Convergent products of stochastic matrices: Algorithms and complexity,” Ph.D. dissertation, Université Catholique de Louvain, 2018.
  26. A. Kolmogoroff, “Zur Theorie der Markoffschen Ketten,” Math. Ann., vol. 112, pp. 155–160, 1936.
  27. A. Nedić and J. Liu, “On convergence rate of weighted-averaging dynamics for consensus problems,” IEEE Trans. Autom. Control, vol. 62, no. 2, pp. 766–781, Feb. 2017.
  28. H. Robbins and D. Siegmund, “A convergence theorem for non negative almost supermartingales and some applications,” Optim. Methods Statist., pp. 233–257, Jun. 1971.
  29. A. Nedic, A. Ozdaglar, and P. A. Parrilo, “Constrained consensus and optimization in multi-agent networks,” IEEE Trans. Autom. Control, vol. 55, no. 4, pp. 922–938, Apr. 2010.
  30. H. Robbins and D. Siegmund, “A convergence theorem for non negative almost supermartingales and some applications,” in Optim. Methods. Stat., J. S. Rustagi, Ed.   Academic Press, Jun. 1971, pp. 233–257.
  31. B. Polyak, “Gradient methods for the minimisation of functionals,” USSR Comput. Math. Math. Phys., vol. 3, no. 4, pp. 864–878, 1963.
  32. B. D. Craven and B. M. Glover, “Invex functions and duality,” J. Aust. Math. Soc., vol. 39, no. 1, pp. 1–20, Aug. 1985.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets