Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
169 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Learning of discrete models of variational PDEs from data (2308.05082v2)

Published 9 Aug 2023 in math.NA and cs.NA

Abstract: We show how to learn discrete field theories from observational data of fields on a space-time lattice. For this, we train a neural network model of a discrete Lagrangian density such that the discrete Euler--Lagrange equations are consistent with the given training data. We, thus, obtain a structure-preserving machine learning architecture. Lagrangian densities are not uniquely defined by the solutions of a field theory. We introduce a technique to derive regularisers for the training process which optimise numerical regularity of the discrete field theory. Minimisation of the regularisers guarantees that close to the training data the discrete field theory behaves robust and efficient when used in numerical simulations. Further, we show how to identify structurally simple solutions of the underlying continuous field theory such as travelling waves. This is possible even when travelling waves are not present in the training data. This is compared to data-driven model order reduction based approaches, which struggle to identify suitable latent spaces containing structurally simple solutions when these are not present in the training data. Ideas are demonstrated on examples based on the wave equation and the Schr\"odinger equation.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (70)
  1. C. Offen and S. Ober-Blöbaum, “Learning of discrete models of variational PDEs from data,” Chaos 34, 013104 (2024).
  2. K. Narendra and K. Parthasarathy, “Identification and control of dynamical systems using neural networks,” IEEE Transactions on Neural Networks 1, 4–27 (1990).
  3. J. Han, C. Ma, Z. Ma,  and W. E, “Uniformly accurate machine learning-based hydrodynamic models for kinetic equations,” Proceedings of the National Academy of Sciences 116, 21983–21991 (2019).
  4. E. Dierkes, C. Offen, S. Ober-Blöbaum,  and K. Flaßkamp, “Hamiltonian neural networks with automatic symmetry detection,” Chaos: An Interdisciplinary Journal of Nonlinear Science 33 (2023), 10.1063/5.0142969, 063115.
  5. Y. Lishkova, P. Scherer, S. Ridderbusch, M. Jamnik, P. Liò, S. Ober-Blöbaum,  and C. Offen, “Discrete Lagrangian neural networks with automatic symmetry discovery (accepted),” in Proceedings of IFAC World Congress 2023, Yokohama, Japan (9-14/07/2023) (IFAC-PaperOnLine, 2023) 2211.10830 .
  6. J.-L. Wu, H. Xiao,  and E. Paterson, “Physics-informed machine learning approach for augmenting turbulence models: A comprehensive framework,” Phys. Rev. Fluids 3, 074602 (2018).
  7. J. Kadupitiya, F. Sun, G. Fox,  and V. Jadhao, “Machine learning surrogates for molecular dynamics simulations of soft materials,” Journal of Computational Science 42, 101107 (2020).
  8. G. E. Karniadakis, I. G. Kevrekidis, L. Lu, P. Perdikaris, S. Wang,  and L. Yang, “Physics-informed machine learning,” Nature Reviews Physics 3, 422–440 (2021).
  9. J. E. Marsden and T. S. Ratiu, Introduction to Mechanics and Symmetry: A Basic Exposition of Classical Mechanical Systems (Springer New York, New York, NY, 1999).
  10. E. L. Mansfield, A Practical Guide to the Invariant Calculus, Cambridge Monographs on Applied and Computational Mathematics (Cambridge University Press, 2010).
  11. M. Cranmer, S. Greydanus, S. Hoyer, P. Battaglia, D. Spergel,  and S. Ho, “Lagrangian neural networks,”  (2020).
  12. B. Hamzi and H. Owhadi, “Learning dynamical systems from data: A simple cross-validation perspective, part i: Parametric kernel flows,” Physica D: Nonlinear Phenomena 421, 132817 (2021).
  13. C. Allen-Blanchette, S. Veer, A. Majumdar,  and N. E. Leonard, “LagNetViP: A Lagrangian neural network for video prediction (AAAI 2020 symposium on physics guided ai),”  (2020).
  14. M. Lutter, C. Ritter,  and J. Peters, “Deep Lagrangian Networks: Using physics as model prior for deep learning,” in 7th International Conference on Learning Representations (ICLR) (ICLR, 2019).
  15. H. Schaeffer, “Learning partial differential equations via data discovery and sparse optimization,” Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences 473, 20160446 (2017).
  16. S. H. Rudy, S. L. Brunton, J. L. Proctor,  and J. N. Kutz, “Data-driven discovery of partial differential equations,” Science Advances 3, e1602614 (2017), https://www.science.org/doi/pdf/10.1126/sciadv.1602614 .
  17. T. Tripura and S. Chakraborty, “A bayesian framework for discovering interpretable lagrangian of dynamical systems from data,”  (2023), arXiv:2310.06241 [stat.ML] .
  18. S. Greydanus, M. Dzamba,  and J. Yosinski, “Hamiltonian Neural Networks,” in Advances in Neural Information Processing Systems, Vol. 32, edited by H. Wallach, H. Larochelle, A. Beygelzimer, F. d’Alché Buc, E. Fox,  and R. Garnett (Curran Associates, Inc., 2019).
  19. P. Jin, Z. Zhang, A. Zhu, Y. Tang,  and G. E. Karniadakis, “SympNets: Intrinsic structure-preserving symplectic networks for identifying Hamiltonian systems,” Neural Networks 132, 166–179 (2020).
  20. T. Bertalan, F. Dietrich, I. Mezić ,  and I. G. Kevrekidis, “On learning Hamiltonian systems from data,” Chaos: An Interdisciplinary Journal of Nonlinear Science 29, 121107 (2019).
  21. Y. Chen, B. Xu, T. Matsubara,  and T. Yaguchi, “Variational principle and variational integrators for neural symplectic forms,” in ICML Workshop on New Frontiers in Learning, Control, and Dynamical Systems (2023).
  22. H. Qin, “Machine learning and serving of discrete field theories,” Scientific Reports 10 (2020), 10.1038/s41598-020-76301-0.
  23. S. Ober-Blöbaum and C. Offen, ‘‘Variational learning of Euler–Lagrange dynamics from data,” Journal of Computational and Applied Mathematics 421, 114780 (2023).
  24. J. E. Marsden and M. West, “Discrete mechanics and variational integrators,” Acta Numerica 10, 357–514 (2001).
  25. C. Offen and S. Ober-Blöbaum, “Learning discrete Lagrangians for variational pdes from data and detection of travelling waves,” in Geometric Science of Information, Vol. 14071, edited by F. Nielsen and F. Barbaresco (Springer Nature Switzerland, Cham, 2023) pp. 569–579.
  26. L. Lu, P. Jin, G. Pang, Z. Zhang,  and G. E. Karniadakis, “Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators,” Nature Machine Intelligence 3, 218–229 (2021).
  27. Z. Long, Y. Lu, X. Ma,  and B. Dong, “PDE-net: Learning PDEs from data,” in Proceedings of the 35th International Conference on Machine Learning, Proceedings of Machine Learning Research, Vol. 80, edited by J. Dy and A. Krause (PMLR, 2018) pp. 3208–3216.
  28. Z. Long, Y. Lu,  and B. Dong, “PDE-Net 2.0: Learning pdes from data with a numeric-symbolic hybrid deep network,” Journal of Computational Physics 399, 108925 (2019).
  29. T. Matsubara, A. Ishikawa,  and T. Yaguchi, “Deep energy-based modeling of discrete-time physics,” in Advances in Neural Information Processing Systems, Vol. 33, edited by H. Larochelle, M. Ranzato, R. Hadsell, M. Balcan,  and H. Lin (Curran Associates, Inc., 2020) pp. 13100–13111.
  30. P. Jin, Z. Zhang, I. G. Kevrekidis,  and G. E. Karniadakis, “Learning poisson systems and trajectories of autonomous systems via poisson neural networks,” IEEE Transactions on Neural Networks and Learning Systems 34, 8271–8283 (2023).
  31. S. Eidnes and K. O. Lye, “Pseudo-Hamiltonian neural networks for learning partial differential equations,”  (2023), arXiv:2304.14374 [cs.LG] .
  32. J. Mason, C. Allen-Blanchette, N. Zolman, E. Davison,  and N. Leonard, “Learning interpretable dynamics from images of a freely rotating 3d rigid body,”  (2022).
  33. P. Buchfink, S. Glas,  and B. Haasdonk, “Symplectic model reduction of Hamiltonian systems on nonlinear manifolds and approximation with weakly symplectic autoencoder,” SIAM Journal on Scientific Computing 45, A289–A311 (2023).
  34. K. Carlberg, R. Tuminaro,  and P. Boggs, “Preserving Lagrangian structure in nonlinear model reduction with application to structural dynamics,” SIAM Journal on Scientific Computing 37, B153–B184 (2015).
  35. H. Sharma, H. Mu, P. Buchfink, R. Geelen, S. Glas,  and B. Kramer, “Symplectic model reduction of Hamiltonian systems using data-driven quadratic manifolds,”  (2023), arXiv:2305.15490 [math.NA] .
  36. H. Sharma and B. Kramer, “Preserving lagrangian structure in data-driven reduced-order modeling of large-scale dynamical systems,”  (2022), arXiv:2203.06361 [math.NA] .
  37. H. Sharma, Z. Wang,  and B. Kramer, “Hamiltonian operator inference: Physics-preserving learning of reduced-order models for canonical Hamiltonian systems,” Physica D: Nonlinear Phenomena 431, 133122 (2022).
  38. T. M. Tyranowski and M. Kraus, “Symplectic model reduction methods for the Vlasov equation,” Contributions to Plasma Physics , e202200046 (2022).
  39. R. S. Palais, “The principle of symmetric criticality,” Comm. Math. Phys. 69, 19–30 (1979).
  40. C. G. Torre, “Symmetric Criticality in Classical Field Theory,” AIP Conference Proceedings 1360, 63–74 (2011).
  41. P. J. Olver, Applications of Lie Groups to Differential Equations (Springer US, 1986).
  42. J.-L. Basdevant, Variational Principles in Physics (Springer New York, New York, NY, 2007).
  43. M. Henneaux, “Equations of motion, commutation relations and ambiguities in the lagrangian formalism,” Annals of Physics 140, 45–64 (1982).
  44. G. Marmo and C. Rubano, “On the uniqueness of the Lagrangian description for charged particles in external magnetic field,” Il Nuovo Cimento A 98, 387–399 (1987).
  45. G. Marmo and G. Morandi, “On the inverse problem with symmetries, and the appearance of cohomologies in classical Lagrangian dynamics,” Reports on Mathematical Physics 28, 389–410 (1989).
  46. A. Deriglazov, “On singular Lagrangian underlying the Schrödinger equation,” Physics Letters A 373, 3920–3923 (2009).
  47. J. E. Marsden, S. Pekarsky, S. Shkoller,  and M. West, “Variational methods, multisymplectic geometry and continuum mechanics,” Journal of Geometry and Physics 38, 253–284 (2001).
  48. E. Kieri, C. Lubich,  and H. Walach, “Discretized dynamical low-rank approximation in the presence of small singular values,” SIAM Journal on Numerical Analysis 54, 1020–1038 (2016).
  49. C. Lubich, B. Vandereycken,  and H. Walach, “Time integration of rank-constrained tucker tensors,” SIAM Journal on Numerical Analysis 56, 1273–1290 (2018), https://doi.org/10.1137/17M1146889 .
  50. H. M. Walach, Time integration for the dynamical low-rank approximation of matrices and tensors, Ph.D. thesis, Eberhard Karls Universität Tübingen (2019).
  51. S. Schrammer, On dynamical low-rank integrators for matrix differential equations, Ph.D. thesis, Karlsruher Institut für Technologie (KIT) (2022).
  52. C. Rowley and J. Marsden, “Variational integrators for degenerate lagrangians, with application to point vortices,” in Proceedings of the 41st IEEE Conference on Decision and Control, 2002., Vol. 2 (2002) pp. 1521–1527.
  53. M. Kraus, “Projected variational integrators for degenerate lagrangian systems,”  (2017), arXiv:1708.07356 [math.NA] .
  54. C. L. Ellison, J. M. Finn, J. W. Burby, M. Kraus, H. Qin,  and W. M. Tang, “Degenerate variational integrators for magnetic field line flow and guiding center trajectories,” Physics of Plasmas 25 (2018), 10.1063/1.5022277, 052502.
  55. J. W. Burby, J. M. Finn,  and C. L. Ellison, “Improved accuracy in degenerate variational integrators for guiding centre and magnetic field line flow,” Journal of Plasma Physics 88, 835880201 (2022).
  56. S. Marsland, Machine Learning: An Algorithmic Perspective, Second Edition, 2nd ed. (Chapman & Hall/CRC, 2014).
  57. D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” in 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings, edited by Y. Bengio and Y. LeCun (2015).
  58. J. Mallet-Paret, “Traveling waves in spatially discrete dynamical systems of diffusive type,” in Dynamical Systems: Lectures given at the C.I.M.E. Summer School held in Cetraro, Italy, June 19-26, 2000, edited by J. W. Macki and P. Zecca (Springer Berlin Heidelberg, Berlin, Heidelberg, 2003) pp. 231–298.
  59. H. J. Hupkes and E. S. V. Vleck, “Travelling waves for adaptive grid discretizations of reaction diffusion systems III: Nonlinear theory,” Journal of Dynamics and Differential Equations  (2022), 10.1007/s10884-022-10143-4.
  60. R. I. McLachlan and C. Offen, “Backward error analysis for conjugate symplectic methods,” Journal of Geometric Mechanics 15, 98–115 (2023).
  61. R. I. McLachlan and C. Offen, “Backward error analysis for variational discretisations of pdes,” Journal of Geometric Mechanics 14, 447–471 (2022).
  62. M. Frigo and S. G. Johnson, “The design and implementation of FFTW3,” Proceedings of the IEEE 93, 216–231 (2005), special issue on “Program Generation, Optimization, and Platform Adaptation”.
  63. M. Innes, E. Saba, K. Fischer, D. Gandhi, M. C. Rudilosso, N. M. Joy, T. Karmali, A. Pal,  and V. Shah, “Fashionable modelling with Flux,” CoRR abs/1811.01457 (2018), arXiv:1811.01457 .
  64. M. Innes, “Flux: Elegant machine learning with Julia,” Journal of Open Source Software  (2018), 10.21105/joss.00602.
  65. S. L. Brunton and J. N. Kutz, “Singular value decomposition (svd),” in Data-Driven Science and Engineering: Machine Learning, Dynamical Systems, and Control (Cambridge University Press, 2019) p. 3–46.
  66. L. N. Trefethen and D. Bau, Numerical Linear Algebra (SIAM, 1997).
  67. C. Offen, “Software: Release v1.0 of GitHub repository Christian-Offen/DLNN_pde. identifier: Christian-Offen/DLNN_pde: DLNN_pde, publisher: Zenodo,”  (2023).
  68. P. Deuflhard and A. Hohmann, “Linear eigenvalue problems,” in Numerical Analysis in Modern Scientific Computing: An Introduction (Springer New York, New York, NY, 2003) pp. 119–150.
  69. M. Röhrig-Zöllner, J. Thies,  and A. Basermann, “Performance of the low-rank TT-SVD for large dense tensors on modern multicore CPUs,” SIAM Journal on Scientific Computing 44, C287–C309 (2022).
  70. P. Deuflhard and A. Hohmann, Numerical Analysis in Modern Scientific Computing (Springer New York, 2003).
Citations (3)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now