CL-DiffPhyCon: Closed-loop Diffusion Control of Complex Physical Systems (2408.03124v3)
Abstract: The control problems of complex physical systems have broad applications in science and engineering. Previous studies have shown that generative control methods based on diffusion models offer significant advantages for solving these problems. However, existing generative control approaches face challenges in both performance and efficiency when extended to the closed-loop setting, which is essential for effective control. In this paper, we propose an efficient Closed-Loop Diffusion method for Physical systems Control (CL-DiffPhyCon). By employing an asynchronous denoising framework for different physical time steps, CL-DiffPhyCon generates control signals conditioned on real-time feedback from the system with significantly reduced computational cost during sampling. Additionally, the control process could be further accelerated by incorporating fast sampling techniques, such as DDIM. We evaluate CL-DiffPhyCon on two tasks: 1D Burgers' equation control and 2D incompressible fluid control. The results demonstrate that CL-DiffPhyCon achieves superior control performance with significant improvements in sampling efficiency. The code can be found at https://github.com/AI4Science-WestlakeU/CL_DiffPhyCon.
- Is conditional generative modeling all you need for decision making? In The Eleventh International Conference on Learning Representations, 2022.
- Controlling rayleigh–bénard convection via reinforcement learning. Journal of Turbulence, 21(9-10):585–605, 2020.
- Magnetic control of tokamak plasmas through deep reinforcement learning. Nature, 602(7897):414–419, 2022.
- Diffusion models beat gans on image synthesis. Advances in neural information processing systems, 34:8780–8794, 2021.
- MA Elhawary. Deep reinforcement learning for active flow control around a circular cylinder using unsteady-mode plasma actuators. arXiv preprint arXiv:2012.10165, 2020.
- How to control hydrodynamic force on fluidic pinball via deep reinforcement learning. Physics of Fluids, 35(4), 2023.
- Efficient navigation of a robotic fish swimming across the vortical flow field. arXiv preprint arXiv:2405.14251, 2024.
- Deep reinforcement learning for the control of conjugate heat transfer. Journal of Computational Physics, 436:110317, 2021.
- Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840–6851, 2020.
- Video diffusion models. Advances in Neural Information Processing Systems, 35:8633–8646, 2022.
- Learning to control pdes with differentiable physics. In International Conference on Learning Representations, 2020.
- Solving PDE-Constrained Control Problems Using Operator Learning. AAAI, 36(4):4504–4512, June 2022. ISSN 2374-3468, 2159-5399. doi: 10.1609/aaai.v36i4.20373. URL https://ojs.aaai.org/index.php/AAAI/article/view/20373.
- Planning with diffusion for flexible behavior synthesis. Proceedings of Machine Learning Research, 162:9902–9915, 17–23 Jul 2022a.
- Planning with diffusion for flexible behavior synthesis. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvari, Gang Niu, and Sivan Sabato (eds.), Proceedings of the 39th International Conference on Machine Learning, volume 162, pp. 9902–9915. PMLR, 17–23 Jul 2022b.
- Hierarchical task and motion planning in the now. In 2011 IEEE International Conference on Robotics and Automation, pp. 1470–1477. IEEE, 2011.
- Pid control system analysis and design. IEEE Control Systems Magazine, 26(1):32–41, 2006. doi: 10.1109/MCS.2006.1580152.
- Optimal control of pdes using physics-informed neural networks. Journal of Computational Physics, 473:111731, 2023.
- Synchronisation through learning for two self-propelled swimmers. Bioinspiration & biomimetics, 12(3):036001, 2017.
- Nils Thuerey Philipp Holl, Vladlen Koltun. Learning to control pdes with differentiable physics. arXiv:2001.07457, Jan 2020. URL https://arxiv.org/abs/2001.07457.
- Dean A Pomerleau. Alvinn: An autonomous land vehicle in a neural network. Advances in neural information processing systems, 1, 1988.
- Gencast: Diffusion-based ensemble forecasting for medium-range weather. arXiv preprint arXiv:2312.15796, 2023.
- Artificial neural networks trained through deep reinforcement learning discover control strategies for active flow control. Journal of fluid mechanics, 865:281–302, 2019.
- Stochastic optimal robust design of a new multi-stage under-frequency load shedding system considering renewable energy sources. International Journal of Electrical Power & Energy Systems, 118:105735, 2020.
- Magnetic quincke rollers with tunable single-particle dynamics and collective states. Science Advances, 9(26):eadh2522, 2023.
- Score-based generative modeling through stochastic differential equations. arXiv preprint arXiv:2011.13456, 2020.
- Efficient collective swimming by harnessing vortices through deep reinforcement learning. Proceedings of the National Academy of Sciences, 115(23):5849–5854, 2018.
- Learn to flap: foil non-parametric path planning via deep reinforcement learning. Journal of Fluid Mechanics, 984:A9, 2024.
- Generative PDE Control. In ICLR 2024 Workshop on AI4DifferentialEquations In Science, March 2024a. URL https://openreview.net/forum?id=vaKnCahjdj.
- A generative approach to control complex physical systems. arXiv preprint arXiv:2407.06494, 2024b.
- Compositional generative inverse design. In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=wmX0CqFSd7.
- Ar-diffusion: Auto-regressive diffusion model for text generation. Advances in Neural Information Processing Systems, 36:39957–39974, 2023.
- Adaptive online replanning with diffusion models. Advances in Neural Information Processing Systems, 36, 2024.
- A numerical study of fish adaption behaviors in complex environments with a deep reinforcement learning and immersed boundary–lattice boltzmann method. Scientific Reports, 11(1):1691, 2021.
- Behavior proximal policy optimization. arXiv preprint arXiv:2302.11312, 2023.