Papers
Topics
Authors
Recent
2000 character limit reached

Neural Abstraction-Based Controller Synthesis and Deployment (2307.03783v1)

Published 7 Jul 2023 in eess.SY, cs.LG, and cs.SY

Abstract: Abstraction-based techniques are an attractive approach for synthesizing correct-by-construction controllers to satisfy high-level temporal requirements. A main bottleneck for successful application of these techniques is the memory requirement, both during controller synthesis and in controller deployment. We propose memory-efficient methods for mitigating the high memory demands of the abstraction-based techniques using neural network representations. To perform synthesis for reach-avoid specifications, we propose an on-the-fly algorithm that relies on compressed neural network representations of the forward and backward dynamics of the system. In contrast to usual applications of neural representations, our technique maintains soundness of the end-to-end process. To ensure this, we correct the output of the trained neural network such that the corrected output representations are sound with respect to the finite abstraction. For deployment, we provide a novel training algorithm to find a neural network representation of the synthesized controller and experimentally show that the controller can be correctly represented as a combination of a neural network and a look-up table that requires a substantially smaller memory. We demonstrate experimentally that our approach significantly reduces the memory requirements of abstraction-based methods. For the selected benchmarks, our approach reduces the memory requirements respectively for the synthesis and deployment by a factor of $1.31\times 105$ and $7.13\times 103$ on average, and up to $7.54\times 105$ and $3.18\times 104$. Although this reduction is at the cost of increased off-line computations to train the neural networks, all the steps of our approach are parallelizable and can be implemented on machines with higher number of processing units to reduce the required computational time.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (40)
  1. Rajeev Alur. 2015. Principles of cyber-physical systems. MIT press.
  2. C. Baier and J.-P. Katoen. 2008. Principles of model checking. MIT press.
  3. Formal methods for discrete-time dynamical systems. Vol. 15. Springer.
  4. D.P. Bertsekas. 1999. Nonlinear Programming. Athena Scientific.
  5. Justin Boyan and Andrew Moore. 1994. Generalization in Reinforcement Learning: Safely Approximating the Value Function. In Advances in Neural Information Processing Systems, Vol. 7. MIT Press.
  6. Bryant. 1986. Graph-Based Algorithms for Boolean Function Manipulation. IEEE Trans. Comput. C-35, 8 (1986), 677–691. https://doi.org/10.1109/TC.1986.1676819
  7. Symbolic Abstractions From Data: A PAC Learning Approach. arXiv preprint arXiv:2104.13901 (2021).
  8. Reachability analysis for neural feedback systems using regressive polynomial rule inference. In Proceedings of the 22nd ACM International Conference on Hybrid Systems: Computation and Control. ACM. https://doi.org/10.1145/3302504.3311807
  9. Antoine Girard. 2012. Low-Complexity Switching Controllers for Safety using Symbolic Models. IFAC Proceedings Volumes 45, 9 (2012), 82–87. https://doi.org/10.3182/20120606-3-NL-3011.00022 4th IFAC Conference on Analysis and Design of Hybrid Systems.
  10. Learning an Approximate Model Predictive Controller With Guarantees. IEEE Control. Syst. Lett. 2, 3 (2018), 543–548. https://doi.org/10.1109/LCSYS.2018.2843682
  11. Multi-Layered Abstraction-Based Controller Synthesis for Continuous-Time Systems. In Proceedings of the 21st International Conference on Hybrid Systems: Computation and Control (part of CPS Week). ACM. https://doi.org/10.1145/3178126.3178143
  12. POLAR: A Polynomial Arithmetic Framework for Verifying Neural-Network Controlled Systems. In Automated Technology for Verification and Analysis. Springer International Publishing, 414–430. https://doi.org/10.1007/978-3-031-19992-9_27
  13. ReachNN. ACM Transactions on Embedded Computing Systems 18, 5s (Oct. 2019), 1–22. https://doi.org/10.1145/3358228
  14. Verisig. In Proceedings of the 22nd ACM International Conference on Hybrid Systems: Computation and Control. ACM. https://doi.org/10.1145/3302504.3311806
  15. Lazy controller synthesis for monotone transition systems and directed safety specifications. Autom. 135 (2022), 109993. https://doi.org/10.1016/j.automatica.2021.109993
  16. Deep Neural Network Compression for Aircraft Collision Avoidance Systems. Journal of Guidance, Control, and Dynamics 42, 3 (March 2019), 598–608. https://doi.org/10.2514/1.g003724
  17. Policy compression for aircraft collision avoidance systems. In 2016 IEEE/AIAA 35th Digital Avionics Systems Conference (DASC). 1–10. https://doi.org/10.1109/DASC.2016.7778091
  18. Data-Driven Abstraction-Based Control Synthesis. https://doi.org/10.48550/ARXIV.2206.08069
  19. Synthesis of Symbolic Controllers: A Parallelized and Sparsity-Aware Approach. In Tools and Algorithms for the Construction and Analysis of Systems - 25th International Conference, TACAS 2019, Held as Part of the European Joint Conferences on Theory and Practice of Software, ETAPS 2019, Prague, Czech Republic, April 6-11, 2019, Proceedings, Part II (Lecture Notes in Computer Science), Tomás Vojnar and Lijun Zhang (Eds.), Vol. 11428. Springer, 265–281. https://doi.org/10.1007/978-3-030-17465-1_15
  20. Mahmoud Khaled and Majid Zamani. 2019. pFaces: an acceleration ecosystem for symbolic control. In Proceedings of the 22nd ACM International Conference on Hybrid Systems: Computation and Control, HSCC 2019, Montreal, QC, Canada, April 16-18, 2019. ACM, 252–257. https://doi.org/10.1145/3302504.3311798
  21. Mykel J. Kochenderfer and Nicholas Monath. 2013. Compression of Optimal Value Functions for Markov Decision Processes. In 2013 Data Compression Conference. 501–501. https://doi.org/10.1109/DCC.2013.81
  22. Probabilistic model checking in practice: Case studies with PRISM. ACM SIGMETRICS Performance Evaluation Review 32, 4 (2005), 16–21.
  23. AMYTISS: Parallelized automated controller synthesis for large-scale stochastic systems. In Computer Aided Verification: 32nd International Conference, CAV 2020, Los Angeles, CA, USA, July 21–24, 2020, Proceedings, Part II 32. Springer, 461–474.
  24. Edward Ashford Lee and Sanjit Arunkumar Seshia. 2016. Introduction to embedded systems: A cyber-physical systems approach. MIT Press.
  25. E. Macoveiciuc and G. Reissig. 2022. On-the-fly symbolic synthesis with memory reduction guarantees. IEEE Trans. Automat. Control (2022), 1–8. https://doi.org/10.1109/TAC.2022.3188483
  26. On abstraction-based controller design with output feedback. In HSCC ’20: 23rd ACM International Conference on Hybrid Systems: Computation and Control, Sydney, New South Wales, Australia, April 21-24, 2020, Aaron D. Ames, Sanjit A. Seshia, and Jyotirmoy Deshmukh (Eds.). ACM, 15:1–15:11.
  27. Efficient Data-Driven Abstraction of Monotone Systems with Disturbances. IFAC-PapersOnLine 54, 5 (2021), 49–54. 7th IFAC Conference on Analysis and Design of Hybrid Systems ADHS 2021.
  28. Feedback refinement relations for the synthesis of symbolic controllers. IEEE TAC 62, 4 (2016), 1781–1796.
  29. Pessoa 2.0. In Proceedings of the 14th international conference on Hybrid systems: computation and control. ACM. https://doi.org/10.1145/1967701.1967748
  30. Sebastian Ruder. 2016. An overview of gradient descent optimization algorithms. ArXiv abs/1609.04747 (2016).
  31. Matthias Rungger and Olaf Stursberg. 2012. On-the-fly model abstraction for controller synthesis. In 2012 American Control Conference (ACC). 2645–2650. https://doi.org/10.1109/ACC.2012.6314891
  32. M. Rungger and M. Zamani. 2016. SCOTS: A Tool for the Synthesis of Symbolic Controllers. In HSCC. 99–104.
  33. Memory-Efficient Mixed-Precision Implementations for Robust Explicit Model Predictive Control. ACM Trans. Embed. Comput. Syst. 18, 5s (2019), 100:1–100:19. https://doi.org/10.1145/3358223
  34. Resilient abstraction-based controller design. In HSCC ’20: 23rd ACM International Conference on Hybrid Systems: Computation and Control, Sydney, New South Wales, Australia, April 21-24, 2020. ACM, 33:1–33:2.
  35. Richard S. Sutton and Andrew G. Barto. 2018. Reinforcement Learning: An Introduction (second ed.).
  36. Paulo Tabuada. 2009. Verification and Control of Hybrid Systems: A Symbolic Approach (1st ed.). Springer Publishing Company, Incorporated.
  37. Safety Verification of Cyber-Physical Systems with Reinforcement Learning Control. ACM Transactions on Embedded Computing Systems 18, 5s (Oct. 2019), 1–22. https://doi.org/10.1145/3358230
  38. NNV: The Neural Network Verification Tool for Deep Neural Networks and Learning-Enabled Cyber-Physical Systems. In Computer Aided Verification, Shuvendu K. Lahiri and Chao Wang (Eds.). Springer International Publishing, Cham, 3–17.
  39. J.N. Tsitsiklis and B. Van Roy. 1997. An analysis of temporal-difference learning with function approximation. IEEE Trans. Automat. Control 42, 5 (1997), 674–690.
  40. Optimal Symbolic Controllers Determinization for BDD storage. IFAC-PapersOnLine 51, 16 (2018), 1–6. https://doi.org/10.1016/j.ifacol.2018.08.001 6th IFAC Conference on Analysis and Design of Hybrid Systems ADHS 2018.
Citations (1)

Summary

We haven't generated a summary for this paper yet.

Whiteboard

Paper to Video (Beta)

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Continue Learning

We haven't generated follow-up questions for this paper yet.

Collections

Sign up for free to add this paper to one or more collections.