Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
121 tokens/sec
GPT-4o
9 tokens/sec
Gemini 2.5 Pro Pro
47 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Trajectory Generation, Control, and Safety with Denoising Diffusion Probabilistic Models (2306.15512v1)

Published 27 Jun 2023 in cs.RO

Abstract: We present a framework for safety-critical optimal control of physical systems based on denoising diffusion probabilistic models (DDPMs). The technology of control barrier functions (CBFs), encoding desired safety constraints, is used in combination with DDPMs to plan actions by iteratively denoising trajectories through a CBF-based guided sampling procedure. At the same time, the generated trajectories are also guided to maximize a future cumulative reward representing a specific task to be optimally executed. The proposed scheme can be seen as an offline and model-based reinforcement learning algorithm resembling in its functionalities a model-predictive control optimization scheme with receding horizon in which the selected actions lead to optimal and safe trajectories.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (33)
  1. Safe Policy Synthesis in Multi-Agent POMDPs via Discrete-Time Barrier Functions. Proceedings of the IEEE Conference on Decision and Control, 2019-Decem:4797–4803, 2019. ISSN 25762370. doi:10.1109/CDC40024.2019.9030241.
  2. Control Barrier Function Based Quadratic Programs for Safety Critical Systems. IEEE Transactions on Automatic Control, 62(8):3861–3876, 2017. ISSN 00189286. doi:10.1109/TAC.2016.2638961.
  3. Control barrier functions: Theory and applications. 2019 18th European Control Conference, ECC 2019, pp. 3420–3431, 2019. doi:10.23919/ECC.2019.8796030.
  4. Unsupervised Representation Learning in Deep Reinforcement Learning: A Review. 8 2022. URL http://arxiv.org/abs/2208.14226.
  5. OpenAI Gym. 2016. URL http://arxiv.org/abs/1606.01540.
  6. Data-Driven Science and Engineering: Machine Learning, Dynamical Systems, and Control. 2022. URL https://books.google.nl/books?hl=en&lr=&id=rxNkEAAAQBAJ&oi=fnd&pg=PR9&dq=Data+Driven+Science+and+Engineering&ots=kmG_U1Jx3p&sig=QbxQQ6OrTC3qAcRwEccAMgNqS4U&redir_esc=y#v=onepage&q=Data%20Driven%20Science%20and%20Engineering&f=false.
  7. End-to-End Safe Reinforcement Learning through Barrier Functions for Safety-Critical Continuous Control Tasks. Proceedings of the AAAI Conference on Artificial Intelligence, 33(01):3387–3395, 7 2019. ISSN 2374-3468. doi:10.1609/AAAI.V33I01.33013387. URL https://ojs.aaai.org/index.php/AAAI/article/view/4213.
  8. Safe Control With Learned Certificates: A Survey of Neural Lyapunov, Barrier, and Contraction Methods for Robotics and Control. IEEE Transactions on Robotics, pp.  1–19, 2023. ISSN 1552-3098. doi:10.1109/TRO.2022.3232542.
  9. Diffusion Models Beat GANs on Image Synthesis. In Advances in Neural Information Processing Systems, volume 11, pp.  8780–8794, 2021. ISBN 9781713845393.
  10. Generative Adversarial Networks. COMMUNICATIONS OF THE ACM, 63(11), 2020. doi:10.1145/3422622.
  11. Soft Actor-Critic Algorithms and Applications. 2018. URL http://arxiv.org/abs/1812.05905.
  12. Denoising diffusion probabilistic models. In Advances in Neural Information Processing Systems, volume 2020-Decem, 2020. URL https://github.com/hojonathanho/diffusion.
  13. Planning with Diffusion for Flexible Behavior Synthesis. 2022. URL http://arxiv.org/abs/2205.09991.
  14. Reinforcement Learning: A Survey. Journal of Artificial Intelligence Research, 4:237–285, 5 1996. ISSN 1076-9757. doi:10.1613/JAIR.301. URL https://www.jair.org/index.php/jair/article/view/10166.
  15. Model-Based Reinforcement Learning for Atari. In Reinforcement Learning for Cyber-Physical Systems, pp. 69–92. 3 2019. doi:10.1201/9781351006620-4. URL http://arxiv.org/abs/1903.00374.
  16. Adam: A method for stochastic optimization. In 3rd International Conference on Learning Representations, ICLR 2015 - Conference Track Proceedings, 2015.
  17. Auto-encoding variational bayes. In 2nd International Conference on Learning Representations, ICLR 2014 - Conference Track Proceedings, 2014.
  18. Deep learning. Nature 2015 521:7553, 521(7553):436–444, 5 2015. ISSN 1476-4687. doi:10.1038/nature14539. URL https://www.nature.com/articles/nature14539.
  19. Li, Y. Deep Reinforcement Learning: An Overview. 1 2017. URL https://arxiv.org/abs/1701.07274v6.
  20. Decoupled weight decay regularization. In 7th International Conference on Learning Representations, ICLR 2019, 2019. URL https://github.com/loshchil/AdamW-and-SGDW.
  21. Joint Synthesis of Safety Certificate and Safe Control Policy Using Constrained Reinforcement Learning, 5 2022. ISSN 2640-3498. URL https://proceedings.mlr.press/v168/ma22a.html.
  22. Misra, D. Mish: A Self Regularized Non-Monotonic Activation Function. 8 2019. URL https://arxiv.org/abs/1908.08681v3http://arxiv.org/abs/1908.08681.
  23. Improved Denoising Diffusion Probabilistic Models. 2021. URL https://github.com/openai/http://arxiv.org/abs/2102.09672.
  24. PyTorch: An Imperative Style, High-Performance Deep Learning Library. 2019.
  25. Rasmussen, C. E. Gaussian Processes in Machine Learning. pp.  63–71. 2004. doi:10.1007/978-3-540-28650-9_4. URL http://link.springer.com/10.1007/978-3-540-28650-9_4.
  26. Safety-Critical Kinematic Control of Robotic Systems. Proceedings of the American Control Conference, 2021-May:14–19, 2021. ISSN 07431619. doi:10.23919/ACC50511.2021.9482954.
  27. Deep unsupervised learning using nonequilibrium thermodynamics. In 32nd International Conference on Machine Learning, ICML 2015, volume 3, pp.  2246–2255, 2015. ISBN 9781510810587.
  28. Generative modeling by estimating gradients of the data distribution. In Advances in Neural Information Processing Systems, volume 32, 2019.
  29. Improved techniques for training score-based generative models. In Advances in Neural Information Processing Systems, volume 2020-Decem, 2020.
  30. Reinforcement Learning: An introduction, 2018. URL https://mitpress.ublish.com/ebook/reinforcement-learning-an-introduction-2-preview/2351/Cover.
  31. Group Normalization. International Journal of Computer Vision, 128(3):742–755, 2020. ISSN 15731405. doi:10.1007/s11263-019-01198-w.
  32. Discrete-Time Control Barrier Function: High-Order Case and Adaptive Case. IEEE Transactions on Cybernetics, 53(5):3231–3239, 5 2023. ISSN 2168-2267. doi:10.1109/TCYB.2022.3170607.
  33. Safety-Critical Model Predictive Control with Discrete-Time Control Barrier Function. Proceedings of the American Control Conference, 2021-May:3882–3889, 2021. ISSN 07431619. doi:10.23919/ACC50511.2021.9483029.
Citations (7)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
Github Logo Streamline Icon: https://streamlinehq.com

GitHub

  1. GitHub - nicob15/Trajectory-Generation-Control-and-Safety-with-Denoising-Diffusion-Probabilistic-Models (11 stars)