Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
139 tokens/sec
GPT-4o
8 tokens/sec
Gemini 2.5 Pro Pro
47 tokens/sec
o3 Pro
5 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

A Comparison of Mesh-Free Differentiable Programming and Data-Driven Strategies for Optimal Control under PDE Constraints (2310.02286v1)

Published 2 Oct 2023 in cs.LG, cs.AI, and math.OC

Abstract: The field of Optimal Control under Partial Differential Equations (PDE) constraints is rapidly changing under the influence of Deep Learning and the accompanying automatic differentiation libraries. Novel techniques like Physics-Informed Neural Networks (PINNs) and Differentiable Programming (DP) are to be contrasted with established numerical schemes like Direct-Adjoint Looping (DAL). We present a comprehensive comparison of DAL, PINN, and DP using a general-purpose mesh-free differentiable PDE solver based on Radial Basis Functions. Under Laplace and Navier-Stokes equations, we found DP to be extremely effective as it produces the most accurate gradients; thriving even when DAL fails and PINNs struggle. Additionally, we provide a detailed benchmark highlighting the limited conditions under which any of those methods can be efficiently used. Our work provides a guide to Optimal Control practitioners and connects them further to the Deep Learning community.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (51)
  1. Felipe Miguel Aparicio Acosta. 1995. Radial basis function and related models: an overview. Signal Processing 45, 1 (1995), 37–58.
  2. Grégoire Allaire. 2015. A review of adjoint methods for sensitivity analysis, uncertainty quantification and optimization in numerical codes. Ingénieurs de l’Automobile 836 (2015), 33–36.
  3. Coexistence and optimal control problems for a degenerate predator–prey model. Journal of mathematical analysis and applications 378, 2 (2011), 528–540.
  4. Hazem Ali Attia and NA Kotb. 1996. MHD flow between two parallel plates with heat transfer. Acta mechanica 117, 1-4 (1996), 215–220.
  5. Automatic differentiation in machine learning: a survey. Journal of Marchine Learning Research 18 (2018), 1–43.
  6. Deep learning-based schemes for singularly perturbed convection-diffusion problems. ESAIM: Proceedings and Surveys 73 (8 2023), 48–67. https://doi.org/10.1051/proc/202373048
  7. JAX-Fluids: A fully-differentiable high-order computational fluid dynamics solver for compressible two-phase flows. Computer Physics Communications 282 (2023), 108527.
  8. Mécanique céleste et contrôle des véhicules spatiaux. Vol. 51. Springer Science & Business Media.
  9. JAX: composable transformations of Python+NumPy programs. http://github.com/google/jax
  10. E.W. Cheney. 1966. Introduction to Approximation Theory. McGraw-Hill Book Company. https://books.google.co.uk/books?id=9uZQAAAAMAAJ
  11. Alexandre Joel Chorin. 1967. The numerical solution of the Navier-Stokes equations for an incompressible fluid. Bull. Amer. Math. Soc. 73, 6 (1967), 928–931.
  12. Scientific machine learning through physics–informed neural networks: Where we are and what’s next. Journal of Scientific Computing 92, 3 (2022), 88.
  13. Bengt Fornberg and Cécile Piret. 2008. A stable algorithm for flat radial basis functions on a sphere. SIAM Journal on Scientific Computing 30, 1 (2008), 60–80.
  14. Bengt Fornberg and Julia Zuev. 2007. The Runge phenomenon and spatially variable shape parameters in RBF interpolation. Computers & Mathematics with Applications 54, 3 (2007), 379–398.
  15. Yarin Gal and Zoubin Ghahramani. 2016. Dropout as a bayesian approximation: Representing model uncertainty in deep learning. In international conference on machine learning. PMLR, 1050–1059.
  16. Christophe Geuzaine and Jean-François Remacle. 2009. Gmsh: A 3-D finite element mesh generator with built-in pre- and post-processing facilities. Internat. J. Numer. Methods Engrg. 79, 11 (2009), 1309–1331. https://doi.org/10.1002/nme.2579 arXiv:https://onlinelibrary.wiley.com/doi/pdf/10.1002/nme.2579
  17. Michael B Giles and Niles A Pierce. 2000. An introduction to the adjoint approach to design. Flow, turbulence and combustion 65 (2000), 393–415.
  18. Andreas Griewank and Andrea Walther. 2008. Evaluating derivatives: principles and techniques of algorithmic differentiation. SIAM.
  19. Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems 25, 1106-1114 (2012), 1.
  20. Learning to Control PDEs with Differentiable Physics. (1 2020). http://arxiv.org/abs/2001.07457
  21. Multilayer feedforward networks are universal approximators. Neural networks 2, 5 (1989), 359–366.
  22. Dojo: A differentiable simulator for robotics. arXiv preprint arXiv:2203.00806 9 (2022).
  23. Edward J Kansa. 1990. Multiquadrics—A scattered data approximation scheme with applications to computational fluid-dynamics—I surface approximations and partial derivative estimates. Computers & Mathematics with applications 19, 8-9 (1990), 127–145.
  24. Patrick Kidger. 2022. On Neural Differential Equations. arXiv:2202.02435 [cs.LG]
  25. Harold W Kuhn and Albert W Tucker. 1951. Nonlinear programming. In Proceedings of the 2nd Berkeley Symposium on Mathematics, Statistics and Probability. University of California Press, Berkeley, 481–492.
  26. Joseph Louis Lagrange. 1853. Mécanique analytique. Vol. 1. Mallet-Bachelier.
  27. A comparison of automatic differentiation and continuous sensitivity analysis for derivatives of differential equation solutions. In 2021 IEEE High Performance Extreme Computing Conference (HPEC). IEEE, 1–9.
  28. Saviz Mowlavi and Saleh Nabi. 2023. Optimal control of PDEs using physics-informed neural networks. J. Comput. Phys. 473 (2023), 111731.
  29. Nonlinear optimal control strategies for buoyancy-driven flows in the built environment. Computers & Fluids 194 (2019), 104313.
  30. Mitsuba 2: A retargetable forward and inverse renderer. ACM Transactions on Graphics (TOG) 38, 6 (2019), 1–17.
  31. Roussel Nzoyem. [n. d.]. Updec. https://github.com/ddrous/Updec/ Third Updec pre-release version..
  32. Mark JL Orr et al. 1996. Introduction to radial basis function networks.
  33. Jooyoung Park and Irwin W Sandberg. 1991. Universal approximation using radial-basis-function networks. Neural computation 3, 2 (1991), 246–257.
  34. The maximum principle. The Mathematical Theory of Optimal Processes. New York: John Wiley and Sons (1962).
  35. Michael James David Powell. 1981. Approximation theory and methods. Cambridge university press.
  36. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational physics 378 (2019), 686–707.
  37. Hidden fluid mechanics: A Navier-Stokes informed deep learning framework for assimilating flow visualization data. arXiv preprint arXiv:1808.04327 (2018).
  38. Learning representations by back-propagating errors. nature 323, 6088 (1986), 533–536.
  39. Amit Sabne. 2020. Xla: Compiling machine learning for peak performance. (2020).
  40. Samuel S Schoenholz and Ekin D Cubuk. 2019. Jax md: End-to-end differentiable, hardware accelerated, molecular dynamics in pure python. (2019).
  41. A High-Order Accurate Meshless Method for Solution of Incompressible Fluid Flow Problems. (10 2020). https://doi.org/10.1016/j.jcp.2021.110623
  42. Shantanu Shahane and Surya Pratap Vanka. 2023. A semi-implicit meshless method for incompressible flows in complex geometries. J. Comput. Phys. 472 (2023), 111715.
  43. Differentiable modelling to unify machine learning and physical models for geosciences. Nature Reviews Earth & Environment (2023), 1–16.
  44. Andrei I Tolstykh. 2000. On using RBF-based differencing formulas for unstructured and mixed structured-unstructured grid calculations. In Proceedings of the 16th IMACS world congress, Vol. 228. Lausanne, 4606–4624.
  45. Emmanuel Trélat. 2005. Contrôle optimal: théorie & applications. Vol. 36. Vuibert Paris.
  46. Modeling, state estimation, and optimal control for the US COVID-19 outbreak. Scientific reports 10, 1 (2020), 10711.
  47. Guido Van Rossum et al. 2007. Python Programming Language.. In USENIX annual technical conference, Vol. 41. Santa Clara, CA, 1–36.
  48. Attention is all you need. Advances in neural information processing systems 30 (2017).
  49. Paul Werbos. 1974. Beyond regression: New tools for prediction and analysis in the behavioral sciences. PhD thesis, Committee on Applied Mathematics, Harvard University, Cambridge, MA (1974).
  50. Paul J Werbos. 1990. Backpropagation through time: what it does and how to do it. Proc. IEEE 78, 10 (1990), 1550–1560.
  51. R. Zamolo and E. Nobile. 2019. Solution of incompressible fluid flow problems with heat transfer by means of an efficient RBF-FD meshless approach. Numerical Heat Transfer, Part B: Fundamentals 75 (1 2019), 19–42. Issue 1. https://doi.org/10.1080/10407790.2019.1580048
Citations (3)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now