Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
162 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

PDE Control Gym: A Benchmark for Data-Driven Boundary Control of Partial Differential Equations (2405.11401v2)

Published 18 May 2024 in eess.SY, cs.AI, cs.CE, cs.LG, cs.SY, and math.OC

Abstract: Over the last decade, data-driven methods have surged in popularity, emerging as valuable tools for control theory. As such, neural network approximations of control feedback laws, system dynamics, and even Lyapunov functions have attracted growing attention. With the ascent of learning based control, the need for accurate, fast, and easy-to-use benchmarks has increased. In this work, we present the first learning-based environment for boundary control of PDEs. In our benchmark, we introduce three foundational PDE problems - a 1D transport PDE, a 1D reaction-diffusion PDE, and a 2D Navier-Stokes PDE - whose solvers are bundled in an user-friendly reinforcement learning gym. With this gym, we then present the first set of model-free, reinforcement learning algorithms for solving this series of benchmark problems, achieving stability, although at a higher cost compared to model-based PDE backstepping. With the set of benchmark environments and detailed examples, this work significantly lowers the barrier to entry for learning-based PDE control - a topic largely unexplored by the data-driven control community. The entire benchmark is available on Github along with detailed documentation and the presented reinforcement learning models are open sourced.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (45)
  1. Lipschitz continuity in model-based reinforcement learning, 2018.
  2. Combining prior knowledge and data for robust controller design. IEEE Transactions on Automatic Control, 68(8):4618–4633, 2023. 10.1109/TAC.2022.3209342.
  3. Neural operators for bypassing gain and control computations in PDE backstepping. IEEE Transactions on Automatic Control, pages 1–16, 2023a. 10.1109/TAC.2023.3347499.
  4. Operator learning for nonlinear adaptive control. In Proceedings of The 5th Annual Learning for Dynamics and Control Conference (L4DC), volume 211 of Proceedings of Machine Learning Research, pages 346–357. PMLR, 15–16 Jun 2023b.
  5. Safe learning in robotics: From learning-based control to safe reinforcement learning. Annual Review of Control, Robotics, and Autonomous Systems, 5:411–444, 2022.
  6. Reinforcement learning for selective key applications in power systems: Recent advances and future challenges. IEEE Transactions on Smart Grid, 13(4):2935–2958, 2022.
  7. Neuron growth control and estimation by pde backstepping. Automatica, 165:111669, 2024. ISSN 0005-1098. https://doi.org/10.1016/j.automatica.2024.111669. URL https://www.sciencedirect.com/science/article/pii/S0005109824001626.
  8. Backstepping stabilization of an underactuated 3 × 3 linear hyperbolic system of fluid flow equations. In 2012 American Control Conference (ACC), pages 3365–3370, 2012.
  9. Benchmarking deep reinforcement learning for continuous control. In International conference on machine learning, pages 1329–1338. PMLR, 2016.
  10. Stability constrained reinforcement learning for decentralized real-time voltage control. IEEE Transactions on Control of Network Systems, pages 1–12, 2023. 10.1109/TCNS.2023.3338240.
  11. Max D Gunzburger. Perspectives in flow control and optimization. SIAM, 2002.
  12. Towards multi-spatiotemporal-scale generalized PDE modeling. arXiv:2209.15616, 2022.
  13. Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor. In International conference on machine learning, pages 1861–1870. PMLR, 2018a.
  14. Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor. In Jennifer Dy and Andreas Krause, editors, Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Research, pages 1861–1870. PMLR, 10–15 Jul 2018b. URL https://proceedings.mlr.press/v80/haarnoja18b.html.
  15. Learning to control PDEs with differentiable physics. In International Conference on Learning Representations (ICLR), 2020.
  16. Miroslav Krstic Huan Yu. Traffic Congestion Control by PDE Backstepping. Birkhäuser Cham, 2023.
  17. PDE backstepping control of one-dimensional heat equation with time-varying domain. Automatica, 54:41–48, 2015.
  18. Optimum aerodynamic design using the Navier-Stokes equations. Theoretical and Computational Fluid Dynamics, 10(1):213–237, Jan 1998.
  19. Deep reinforcement learning for autonomous driving: A survey. IEEE Transactions on Intelligent Transportation Systems, 23(6):4909–4926, 2021.
  20. Materials Phase Change PDE Control & Estimation. Birkhäuser, 2020.
  21. Imagenet classification with deep convolutional neural networks. In F. Pereira, C.J. Burges, L. Bottou, and K.Q. Weinberger, editors, Advances in Neural Information Processing Systems, volume 25, 2012.
  22. Backstepping boundary control for first-order hyperbolic PDEs and application to systems with actuator and sensor delays. Systems & Control Letters, 57(9):750–758, 2008a.
  23. Boundary Control of PDEs. Society for Industrial and Applied Mathematics, Philadelphia, PA, 2008b.
  24. Neural operators of backstepping controller and observer gain functions for reaction–diffusion PDEs. Automatica, 164:111649, 2024. ISSN 0005-1098. https://doi.org/10.1016/j.automatica.2024.111649. URL https://www.sciencedirect.com/science/article/pii/S0005109824001420.
  25. An improvement of fractional step methods for the incompressible Navier-Stokes equations. Journal of computational physics, 92(2):369–379, 1991.
  26. Randall J. LeVeque. Numerical methods for conservation laws (2. ed.). Lectures in mathematics. Birkhäuser, 1992. ISBN 978-3-7643-2723-1.
  27. Fourier neural operator for parametric partial differential equations. In International Conference on Learning Representations (ICLR), 2021.
  28. Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators. Nature Machine Intelligence, 3(3):218–229, 2021.
  29. Fluid control using the adjoint method. ACM Transactions On Graphics (TOG), 23(3):449–456, 2004.
  30. Nerf: Representing scenes as neural radiance fields for view synthesis. In ECCV, 2020.
  31. Adaptive partial differential equation observer for battery state-of-charge/state-of-health estimation via an electrochemical model. Journal of Dynamic Systems, Measurement, and Control, 136(1):011015, 2014.
  32. Optimal control of PDEs using physics-informed neural networks. Journal of Computational Physics, 473:111731, 2023.
  33. Optimal feedback control of the incompressible Navier-Stokes-equations using reduced order models. In 2015 54th IEEE Conference on Decision and Control (CDC), pages 2519–2524. IEEE, 2015.
  34. Neural operators for delay-compensating control of hyperbolic PIDEs. arXiv:2307.11436, 2023.
  35. Stable-Baselines3: Reliable Reinforcement Learning Implementations. Journal of Machine Learning Research, 22(268):1–8, 2021.
  36. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational Physics, 378:686–707, 2019.
  37. Proximal policy optimization algorithms. arXiv:1707.06347, 2017.
  38. Machine learning accelerated PDE backstepping observers. In 2022 IEEE 61st Conference on Decision and Control (CDC), pages 5423–5428, 2022.
  39. A. Smyshlyaev and M. Krstic. Closed-form boundary state feedbacks for a class of 1-D partial integro-differential equations. IEEE Transactions on Automatic Control, 49(12):2185–2202, 2004.
  40. A. Smyshlyaev and M. Krstic. Adaptive Control of Parabolic PDEs. Princeton University Press, 2010.
  41. PDEBench: an extensive benchmark for scientific machine learning. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, editors, Advances in Neural Information Processing Systems, volume 35, pages 1596–1611, 2022.
  42. Deepmind control suite. CoRR, abs/1801.00690, 2018.
  43. The power series method to compute backstepping kernel gains: Theory and practice. In 2023 62nd IEEE Conference on Decision and Control (CDC), pages 8162–8169, 2023. 10.1109/CDC49753.2023.10384080.
  44. Modeling information diffusion in online social networks with partial differential equations, volume 7. Springer Nature, 2020.
  45. Reinforcement learning versus PDE backstepping and PI control for congested freeway traffic. IEEE Transactions on Control Systems Technology, 30(4):1595–1611, 2022.
Citations (5)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets