Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
166 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

A First-Order Gradient Approach for the Connectivity Analysis of Markov Chains (2403.11744v2)

Published 18 Mar 2024 in math.OC, cs.SY, and eess.SY

Abstract: Weighted graphs are commonly used to model various complex systems, including social networks, power grids, transportation networks, and biological systems. In many applications, the connectivity of these networks can be expressed through the Mean First Passage Times (MFPTs) of a Markov chain modeling a random walker on the graph. In this paper, we generalize the network metrics based on Markov chains' MFPTs and extend them to networks affected by uncertainty, in which edges may fail and hence not be present according to a pre-determined stochastic model. To find optimally connected Markov chains, we present a parameterization-free method for optimizing the MFPTs of the Markov chain. More specifically, we present an efficient Simultaneous Perturbation Stochastic Approximation (SPSA) algorithm in the context of Markov chain optimization. The proposed algorithm is suitable for both fixed and random networks. Using various numerical experiments, we demonstrate scalability compared to established benchmarks. Importantly, our algorithm finds an optimal solution without requiring prior knowledge of edge failure probabilities, allowing for an online optimization approach.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (45)
  1. J. J. Hunter, “The computation of key properties of Markov chains via perturbations,” Linear Algebra and its Applications, vol. 511, pp. 176–202, 2016.
  2. ——, “The computation of the mean first passage times for Markov chains,” Linear Algebra and its Applications, vol. 549, pp. 100–122, 2018.
  3. T. Chou and M. R. D. Orsogna, “First passage problems in biology,” in First-Passage Phenomena and Their Applications.   World Scientific, 2014, pp. 306–345.
  4. N. Kalantar and D. Segal, “Mean first-passage time and steady-state transfer rate in classical chains,” The Journal of Physical Chemistry C, vol. 123, no. 2, pp. 1021–1031, Dec. 2018.
  5. G. H. Weiss, “First passage time problems in chemical physics,” in Advances in Chemical Physics.   J. Wiley & Sons, 2007, pp. 1–18.
  6. W. Ellens, F. M. Spieksma, P. Van Mieghem, A. Jamakovic, and R. E. Kooij, “Effective graph resistance,” Linear Algebra and its Applications, vol. 435, no. 10, pp. 2491–2506, 2011.
  7. D. J. Klein and M. Randić, “Resistance distance,” Journal of Mathematical Chemistry, vol. 12, pp. 81–95, 1993.
  8. M. Bianchi, J. L. Palacios, A. Torriero, and A. L. Wirkierman, “Kirchhoffian indices for weighted digraphs,” Discrete Applied Mathematics, vol. 255, pp. 142–154, 2019.
  9. A. Ghosh, S. Boyd, and A. Saberi, “Minimizing effective resistance of a graph,” SIAM review, vol. 50, no. 1, pp. 37–66, 2008.
  10. R. Patel, P. Agharkar, and F. Bullo, “Robotic surveillance and Markov chains with minimal weighted Kemeny constant,” IEEE Transactions on Automatic Control, vol. 60, no. 12, pp. 3156–3167, 2015.
  11. X. Duan and F. Bullo, “Markov chain-based stochastic strategies for robotic surveillance,” arXiv, 2020.
  12. J. C. Spall, “Multivariate stochastic approximation using a simultaneous perturbation gradient approximation,” IEEE Transactions on Automatic Control, vol. 37, no. 3, pp. 332–341, 1992.
  13. D. Aldous and J. A. Fill, “Reversible Markov chains and random walks on graphs,” 2002, unfinished monograph, recompiled 2014, available at http://www.stat.berkeley.edu/~aldous/RWG/book.html.
  14. A. Zocca and B. Zwart, “Optimization of stochastic lossy transport networks and applications to power grids,” Stochastic Systems, vol. 11, no. 1, pp. 34–59, Mar. 2021.
  15. X. Wang, Y. Koç, R. E. Kooij, and P. Van Mieghem, “A network approach for power grid robustness against cascading failures,” in 2015 7th International Workshop on Reliable Networks Design and Modeling (RNDM).   IEEE, 2015, pp. 208–214.
  16. F. Dorfler and F. Bullo, “Synchronization of power networks: Network reduction and effective resistance,” IFAC Proceedings Volumes, vol. 43, no. 19, pp. 197–202, 2010.
  17. A. Tizghadam and A. Leon-Garcia, “Betweenness centrality and resistance distance in communication networks,” IEEE network, vol. 24, no. 6, pp. 10–16, 2010.
  18. D. F. Rueda, E. Calle, and J. L. Marzo, “Robustness comparison of 15 real telecommunication networks: Structural and centrality measurements,” Journal of Network and Systems Management, vol. 25, pp. 269–289, 2017.
  19. C. Yang, J. Mao, X. Qian, and P. Wei, “Designing robust air transportation networks via minimizing total effective resistance,” IEEE Transactions on Intelligent Transportation Systems, vol. 20, no. 6, pp. 2353–2366, 2018.
  20. J. Berkhout and B. F. Heidergott, “Analysis of Markov influence graphs,” Operations Research, vol. 67, no. 3, pp. 892–904, 2019.
  21. S. Yilmaz, E. Dudkina, M. Bin, E. Crisostomi, P. Ferraro, R. Murray-Smith, T. Parisini, L. Stone, and R. Shorten, “Kemeny-based testing for covid-19,” PLOS ONE, vol. 15, no. 11, p. e0242401, 2020.
  22. N. Litvak and V. Ejov, “Markov chains and optimality of the Hamiltonian cycle,” Mathematics of Operations Research, vol. 34, no. 1, pp. 71–82, 2009.
  23. S. Kirkland, “Fastest expected time to mixing for a Markov chain on a directed graph,” Linear Algebra and its Applications, vol. 433, no. 11-12, pp. 1988–1996, 2010.
  24. J. Shi and J. C. Spall, “SQP-based projection SPSA algorithm for stochastic optimization with inequality constraints,” in 2021 American control conference (ACC).   IEEE, 2021, pp. 1244–1249.
  25. B. Heidergott and A. Hordijk, “Taylor series expansions for stationary Markov chains,” Advances in Applied Probability, vol. 35, no. 4, pp. 1046–1070, 2003.
  26. N. Leder, B. Heidergott, and A. Hordijk, “An approximation approach for the deviation matrix of continuous-time Markov processes with application to Markov decision theory,” Operations Research, vol. 58, no. 4-part-1, pp. 918–932, 2010.
  27. P. Sadegh, “Constrained optimization via stochastic approximation with a simultaneous perturbation gradient approximation,” Automatica, vol. 33, no. 5, p. 889–892, 1997.
  28. G. Perez, M. Barlaud, L. Fillatre, and J.-C. Régin, “A filtered bucket-clustering method for projection onto the simplex and the 𝒍1subscript𝒍1\boldsymbol{l}_{1}bold_italic_l start_POSTSUBSCRIPT 1 end_POSTSUBSCRIPT ball,” Mathematical Programming, vol. 182, no. 1-2, pp. 445–464, 2020.
  29. J. P. Boyle and R. L. Dykstra, “A method for finding projections onto the intersection of convex sets in Hilbert spaces,” in Advances in Order Restricted Statistical Inference: Proceedings of the Symposium on Order Restricted Statistical Inference held in Iowa City, Iowa, September 11–13, 1985.   Springer, 1986, pp. 28–47.
  30. H. H. Bauschke, R. S. Burachik, D. B. Herman, and C. Y. Kaya, “On Dykstra’s algorithm: finite convergence, stalling, and the method of alternating projections,” Optimization Letters, vol. 14, pp. 1975–1987, 2020.
  31. T. Papilloud and M. Keiler, “Vulnerability patterns of road network to extreme floods based on accessibility measures,” Transportation research part D: transport and environment, vol. 100, p. 103045, 2021.
  32. M. Panteli and P. Mancarella, “Modeling and evaluating the resilience of critical electrical power infrastructure to extreme weather events,” IEEE Systems Journal, vol. 11, no. 3, pp. 1733–1742, 2015.
  33. S. Soltan, D. Mazauric, and G. Zussman, “Analysis of failures in power grids,” IEEE Transactions on Control of Network Systems, vol. 4, no. 2, pp. 288–300, 2015.
  34. P. Kubat, “Estimation of reliability for communication/computer networks simulation/analytic approach,” IEEE Transactions on Communications, vol. 37, no. 9, pp. 927–933, 1989.
  35. M. Seder, K. Macek, and I. Petrovic, “An integrated approach to real-time mobile robot control in partially known indoor environments,” in 31st Annual Conference of IEEE Industrial Electronics Society, 2005. IECON 2005.   IEEE, 2005, pp. 6–pp.
  36. A. Šelek, M. Seder, and I. Petrović, “Smooth autonomous patrolling for a differential-drive mobile robot in dynamic environments,” Sensors, vol. 23, no. 17, p. 7421, 2023.
  37. S. Neumayer and E. Modiano, “Network reliability with geographically correlated failures,” in 2010 Proceedings IEEE INFOCOM.   IEEE, 2010, pp. 1–9.
  38. J. K. Moore et al., “cyipopt,” https://cyipopt.readthedocs.io/en/stable/index.html/.
  39. B. T. Polyak, “New stochastic approximation type procedures,” Automat. i Telemekh, vol. 7, no. 98-107, p. 2, 1990.
  40. D. Ruppert, “Efficient estimations from a slowly convergent robbins-monro process,” Cornell University Operations Research and Industrial Engineering, Tech. Rep., 1988.
  41. M. Chen, “Generating nonnegatively correlated binary random variates,” The Stata Journal: Promoting communications on statistics and Stata, vol. 15, no. 1, p. 301–308, Apr. 2015. [Online]. Available: http://dx.doi.org/10.1177/1536867X1501500118
  42. E. Jondeau and M. Rockinger, “Optimal portfolio allocation under higher moments,” European Financial Management, vol. 12, no. 1, pp. 29–55, 2006.
  43. M. Patriksson, “A survey on the continuous nonlinear resource allocation problem,” European Journal of Operational Research, vol. 185, no. 1, pp. 1–46, 2008.
  44. I. Lukovits, S. Nikolić, and N. Trinajstić, “Resistance distance in regular graphs,” International Journal of Quantum Chemistry, vol. 71, no. 3, pp. 217–225, 1999.
  45. H. J. Kushner and G. G. Yin, “Stochastic approximation and recursive algorithms and applications,” Stochastic Modelling and Applied Probability, 2003.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now