Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
167 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

IMPaCT: Interval MDP Parallel Construction for Controller Synthesis of Large-Scale Stochastic Systems (2401.03555v2)

Published 7 Jan 2024 in eess.SY and cs.SY

Abstract: This paper is concerned with developing a software tool, called IMPaCT, for the parallelized verification and controller synthesis of large-scale stochastic systems using interval Markov chains (IMCs) and interval Markov decision processes (IMDPs), respectively. The tool serves to (i) construct IMCs/IMDPs as finite abstractions of underlying original systems, and (ii) leverage interval iteration algorithms for formal verification and controller synthesis over infinite-horizon properties, including safety, reachability, and reach-avoid, while offering convergence guarantees. IMPaCT is developed in C++ and designed using AdaptiveCpp, an independent open-source implementation of SYCL, for adaptive parallelism over CPUs and GPUs of all hardware vendors, including Intel and NVIDIA. IMPaCT stands as the first software tool for the parallel construction of IMCs/IMDPs, empowered with the capability to leverage high-performance computing platforms and cloud computing services. Specifically, parallelism offered by IMPaCT effectively addresses the challenges arising from the state-explosion problem inherent in discretization-based techniques applied to large-scale stochastic systems. We benchmark IMPaCT on several physical case studies, adopted from the ARCH tool competition for stochastic models, including a 2-dimensional robot, a 3-dimensional autonomous vehicle, a 5-dimensional room temperature system, and a 7-dimensional building automation system. To show the scalability of our tool, we also employ IMPaCT for the formal analysis of a 14-dimensional case study.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (52)
  1. ARCH-COMP20 Category Report: Stochastic Models. 2020.
  2. ARCH-COMP23 Category report: stochastic models. In International Workshop on Applied Verification of Continuous and Hybrid Systems (ARCH), EPiC Series in Computing, pages 126–150. EasyChair, 2023.
  3. ARCH-COMP22 category report: stochastic models. 90:113–141, 2022.
  4. A. Alpay and V. Heuveline. SYCL beyond OpenCL: The Architecture, Current State and Future Direction of HipSYCL. In Proceedings of the International Workshop on OpenCL, 2020.
  5. A. Alpay and V. Heuveline. One Pass to Bind Them: The First Single-Pass SYCL Compiler with Unified Code Representation Across Backends. In Proceedings of the 2023 International Workshop on OpenCL, 2023.
  6. Probabilistic reachability and safety for controlled discrete-time stochastic hybrid systems. Automatica, 44(11):2724–2734, 2008.
  7. C. Baier and J.P. Katoen. Principles of model checking. MIT Press, 2008.
  8. Ensuring the reliability of your model checker: Interval iteration for Markov decision processes. In International Conference on Computer Aided Verification, pages 160–180, 2017.
  9. Formal methods for discrete-time dynamical systems, volume 89. 2017.
  10. N. Cauchi and A. Abate. Benchmarks for cyber-physical systems: A modular model library for building automation systems. IFAC-PapersOnLine, 51(16):49–54, 2018.
  11. N. Cauchi and A. Abate. StocHy: automated verification and synthesis of stochastic processes. In 25th International Conference on Tools and Algorithms for the Construction and Analysis of Systems (TACAS), 2019.
  12. On the complexity of model checking interval-valued discrete time Markov chains. Information Processing Letters, 113(7):210–216, 2013.
  13. M. Dutreix and S. Coogan. Specification-guided verification and abstraction refinement of mixed monotone stochastic systems. IEEE Transactions on Automatic Control, 66(7):2975–2990, 2020.
  14. Interval Markov Decision Processes with Continuous Action-Spaces. In Proceedings of the 26th ACM International Conference on Hybrid Systems: Computation and Control, pages 1–10, 2023.
  15. Bounded-parameter Markov decision processes. Artificial Intelligence, 122(1):71–109, 2000.
  16. B. Gough. GNU scientific library reference manual. Network Theory Ltd., 2009.
  17. A. Hartmanns and H. Hermanns. Modest Toolset: An integrated environment for quantitative modelling and verification. In International Conference on Tools and Algorithms for the Construction and Analysis of Systems, pages 593–598, 2014.
  18. A compositional modelling and analysis framework for stochastic hybrid systems. Formal Methods in System Design, 43(2):191–232, 2013.
  19. Compositional bisimulation minimization for interval Markov decision processes. In 10th International Conference on Language and Automata Theory and Applications, pages 114–126, 2016.
  20. S. Haddad and B. Monmege. Reachability in MDPs: Refining convergence of value iteration. In 8th International Workshop on Reachability Problems, pages 125–137, 2014.
  21. S. Haddad and B. Monmege. Interval iteration algorithm for MDPs and IMDPs. Theoretical Computer Science, 735:111–131, 2018.
  22. S. G. Johnson. The NLopt nonlinear-optimization package. https://github.com/stevengj/nlopt, 2007.
  23. Approximations of stochastic hybrid systems. IEEE Transactions on Automatic Control, 54(6):1193–1203, 2009.
  24. Safe Learning for Uncertainty-Aware Planning via Interval MDP Abstraction. IEEE Control Systems Letters, 6:2641–2646, 2022.
  25. O. Kallenberg. Foundations of Modern Probability, volume 3. Springer, 2021.
  26. PRISM 4.0: Verification of probabilistic real-time systems. In 23rd International Conference on Computer Aided Verification, pages 585–591, 2011.
  27. M. Khaled and M. Zamani. PFaces: An Acceleration Ecosystem for Symbolic Control. In Proceedings of the 22nd ACM International Conference on Hybrid Systems: Computation and Control, pages 252–257, 2019.
  28. Formal verification and synthesis for discrete-time stochastic systems. IEEE Transactions on Automatic Control, 60(8):2031–2045, 2015.
  29. A. Lavaei and E. Frazzoli. Scalable synthesis of finite MDPs for large-scale stochastic switching systems. In 2022 IEEE 61st Conference on Decision and Control (CDC), pages 7510–7515, 2022.
  30. AMYTISS: Parallelized automated controller synthesis for large-scale stochastic systems. In Proc. 32nd International Conference on Computer Aided Verification (CAV), LNCS, 2020.
  31. Automated verification and synthesis of stochastic hybrid systems: A survey. Automatica, 146, 2022.
  32. Constructing MDP abstractions using data with formal guarantees. IEEE Control Systems Letters, 7:460–465, 2022.
  33. From dissipativity theory to compositional construction of finite Markov decision processes. In Proceedings of the 21st ACM International Conference on Hybrid Systems: Computation and Control, pages 21–30, 2018.
  34. A. Makhorin. GLPK (GNU linear programming kit). http://www. gnu. org/s/glpk/glpk. html, 2008.
  35. Monte Carlo methods in statistical physics. Clarendon Press, 1999.
  36. A. Nejati. Formal Verification and Control of Stochastic Hybrid Systems: Model-based and Data-driven Techniques. PhD thesis, Department of Electrical Engineering, Technische Universität München, 2023.
  37. Compositional abstraction-based synthesis for continuous-time stochastic hybrid systems. European Journal of Control, 57:82–94, 2021.
  38. Learning robust policies for uncertain parametric markov decision processes. arXiv: 2312.06344, 2023.
  39. T. Harvey Rowan. Functional stability analysis of numerical algorithms. PhD thesis, Department of Computer Science, University of Texas at Austin, 1990.
  40. C. Sanderson and R. Curtin. Armadillo: a template-based C++ library for linear algebra. Journal of Open Source Software, 1(2):26, 2016.
  41. C. Sanderson and R. Curtin. A user-friendly hybrid sparse matrix class in C++. In 6th International Conference on Mathematical Software, pages 422–430, 2018.
  42. FAUST𝟤𝟤{}^{\textsf{2}}start_FLOATSUPERSCRIPT 2 end_FLOATSUPERSCRIPT: Formal abstractions of uncountable-state stochastic processes. In TACAS’15, volume 9035 of Lecture Notes in Computer Science, pages 272–286. 2015.
  43. Model-checking Markov chains in the presence of uncertainties. In 12th International Conference Tools and Algorithms for the Construction and Analysis of Systems, pages 394–410, 2006.
  44. F. Shmarov and P. Zuliani. ProbReach: verified probabilistic delta-reachability for stochastic hybrid systems. In Proceedings of the 18th International Conference on Hybrid Systems: Computation and Control, pages 134–139, 2015.
  45. P. Tabuada. Verification and control of hybrid systems: a symbolic approach. Springer Science & Business Media, 2009.
  46. The HDF Group. Hierarchical Data Format, version 5. https://www.hdfgroup.org/HDF5/, 1997-2023.
  47. Quantitative automata-based controller synthesis for non-autonomous stochastic hybrid systems. In Proceedings of the 16th ACM International Conference on Hybrid Systems: Computation and Control, pages 293–302, 2013.
  48. SReachTools: A MATLAB stochastic reachability toolbox. In Proceedings of the 22nd ACM International Conference on Hybrid Systems: Computation and Control, pages 33–38, 2019.
  49. SySCoRe: Synthesis via stochastic coupling relations. In Proceedings of the 26th ACM International Conference on Hybrid Systems: Computation and Control, pages 1–11, 2023.
  50. Satisfiability bounds for ω𝜔\omegaitalic_ω-regular properties in bounded-parameter Markov decision processes. In 2019 IEEE 58th Conference on Decision and Control (CDC), pages 2284–2291, 2019.
  51. SReach: A probabilistic bounded delta-reachability analyzer for stochastic hybrid systems. In Proceedings of the International Conference on Computational Methods in Systems Biology, pages 15–27, 2015.
  52. Symbolic control of stochastic systems via approximately bisimilar finite abstractions. IEEE Transactions on Automatic Control, 59(12):3135–3150, 2014.
Citations (3)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now