Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
169 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Safety Guarantees for Neural Network Dynamic Systems via Stochastic Barrier Functions (2206.07811v7)

Published 15 Jun 2022 in eess.SY, cs.RO, and cs.SY

Abstract: Neural Networks (NNs) have been successfully employed to represent the state evolution of complex dynamical systems. Such models, referred to as NN dynamic models (NNDMs), use iterative noisy predictions of NN to estimate a distribution of system trajectories over time. Despite their accuracy, safety analysis of NNDMs is known to be a challenging problem and remains largely unexplored. To address this issue, in this paper, we introduce a method of providing safety guarantees for NNDMs. Our approach is based on stochastic barrier functions, whose relation with safety are analogous to that of Lyapunov functions with stability. We first show a method of synthesizing stochastic barrier functions for NNDMs via a convex optimization problem, which in turn provides a lower bound on the system's safety probability. A key step in our method is the employment of the recent convex approximation results for NNs to find piece-wise linear bounds, which allow the formulation of the barrier function synthesis problem as a sum-of-squares optimization program. If the obtained safety probability is above the desired threshold, the system is certified. Otherwise, we introduce a method of generating controls for the system that robustly maximizes the safety probability in a minimally-invasive manner. We exploit the convexity property of the barrier function to formulate the optimal control synthesis problem as a linear program. Experimental results illustrate the efficacy of the method. Namely, they show that the method can scale to multi-dimensional NNDMs with multiple layers and hundreds of neurons per layer, and that the controller can significantly improve the safety probability.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (47)
  1. Principles of model checking. MIT Press, 2008.
  2. Temporal logic motion planning and control with probabilistic satisfaction guarantees. IEEE Transactions on Robotics, 28(2):396–409, 2011.
  3. Formal and efficient synthesis for continuous-time linear stochastic hybrid processes. IEEE Transactions on Automatic Control, 2020.
  4. Formal methods for control synthesis: An optimization perspective. Annual Review of Control, Robotics, and Autonomous Systems, 2:115–140, 2019.
  5. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational physics, 378:686–707, 2019.
  6. John Cristian Borges Gamboa. Deep learning for time-series analysis. arXiv preprint arXiv:1701.01887, 2017.
  7. A framework for worst-case and stochastic safety verification using barrier certificates. IEEE Transactions on Automatic Control, 52(8):1415–1428, 2007.
  8. A barrier function approach to finite-time stochastic system verification and control. Automatica, 125:109439, 2021.
  9. Neural network dynamics for model-based deep reinforcement learning with model-free fine-tuning. In 2018 IEEE International Conference on Robotics and Automation (ICRA), pages 4063–4069. IEEE, 2017.
  10. Neural ordinary differential equations. Advances in neural information processing systems, 31, 2018.
  11. Deep reinforcement learning in a handful of trials using probabilistic dynamics models. In Advances in Neural Information Processing Systems, pages 4754–4765, 2018.
  12. Machine learning for soft robotic sensing and control. Advanced Intelligent Systems, 2(6):1900171, 2020.
  13. Control barrier functions: Theory and applications. In 2019 18th European Control Conference (ECC), pages 3420–3431, 2019.
  14. Safety verification of hybrid systems using barrier certificates. In International Workshop on Hybrid Systems: Computation and Control, pages 477–492. Springer, 2004.
  15. Control barrier function based quadratic programs with application to adaptive cruise control. In 53rd IEEE Conference on Decision and Control, pages 6271–6278. IEEE, 2014.
  16. Explaining and harnessing adversarial examples. CoRR, abs/1412.6572, 2015.
  17. Reluplex: An efficient SMT solver for verifying deep neural networks. In International Conference on Computer Aided Verification, pages 97–117. Springer, 2017.
  18. Efficient neural network robustness certification with general activation functions. Advances in neural information processing systems, 31, 2018.
  19. Ruediger Ehlers. Formal verification of piece-wise linear feed-forward neural networks. In International Symposium on Automated Technology for Verification and Analysis, pages 269–286. Springer, 2017.
  20. Infinite time horizon safety of bayesian neural networks. Advances in Neural Information Processing Systems, 34, 2021.
  21. On the verification of neural odes with stochastic guarantees. In AAAI, 2021.
  22. Safe control with learned certificates: A survey of neural lyapunov, barrier, and contraction methods. arXiv preprint arXiv:2202.11762, 2022.
  23. Certification of iterative predictions in bayesian neural networks. Uncertanity in Artificial Intelligence (UAI), 2021.
  24. Formal control synthesis for stochastic neural network dynamic models. IEEE Control Systems Letters, 2022.
  25. Safe control with neural network dynamic models. In Learning for Dynamics and Control Conference, pages 739–750. PMLR, 2022.
  26. Stochastic optimal control: the discrete-time case. Athena Scientific, 2004.
  27. Murray Marshall. Positive polynomials and sums of squares. American Mathematical Soc., 2008.
  28. Harold J Kushner. Stochastic stability and control. Technical report, Brown Univ Providence RI, 1967.
  29. Hilbert. Ueber die darstellung definiter formen als summen von formenquadraten. Mathematische Annalen, 32:342–350, 1888.
  30. Semidefinite programming. SIAM review, 38(1):49–95, 1996.
  31. Gilbert Stengle. A nullstellensatz and a positivstellensatz in semialgebraic geometry. Mathematische Annalen, 207:87–98, 1974.
  32. Efficient formal safety analysis of neural networks. In Advances in Neural Information Processing Systems, pages 6367–6377, 2018.
  33. Formal security analysis of neural networks using symbolic intervals. In 27th USENIX Security Symposium (USENIX Security 18), pages 1599–1614, 2018.
  34. Beta-crown: Efficient bound propagation with per-neuron split constraints for complete and incomplete neural network robustness verification. Advances in Neural Information Processing Systems, 34:29909–29921, 2021.
  35. Fast and complete: Enabling complete neural network verification with rapid and massively parallel incomplete verifiers. International Conference on Learning Representations, 2020.
  36. Finite-time regional verification of stochastic non-linear systems. The International Journal of Robotics Research, 31(7):901–923, 2012.
  37. JG Liao and Arthur Berg. Sharpening jensen’s inequality. The American Statistician, 2018.
  38. Sos-convex lyapunov functions with applications to nonlinear switched systems. In Proceedings of the IEEE Conference on Decision and Control, 2013.
  39. Data-driven science and engineering: Machine learning, dynamical systems, and control. Cambridge University Press, 2022.
  40. Gym leaderboard. https://github.com/openai/gym/wiki/Leaderboard.
  41. Pybullet, a python module for physics simulation for games, robotics and machine learning. http://pybullet.org, 2016–2021.
  42. Github code: NeuralNetControlBarrier. https://github.com/aria-systems-group/NeuralNetControlBarrier.
  43. TensorFlow: Large-scale machine learning on heterogeneous systems, 2015. Software available from tensorflow.org.
  44. Polynomial and moment optimization in julia and jump. In JuliaCon, 2019.
  45. Sum-of-squares optimization in Julia. In The First Annual JuMP-dev Workshop, 2017.
  46. Husky urdf clearpath. https://github.com/bulletphysics/bullet3/blob/master/data/husky/husky.urdf.
  47. Guided safe shooting: model based reinforcement learning with safety constraints. arXiv preprint arXiv:2206.09743, 2022.
Citations (21)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
Youtube Logo Streamline Icon: https://streamlinehq.com

YouTube