2000 character limit reached
Mechanistic Neural Networks for Scientific Machine Learning
Published 20 Feb 2024 in cs.LG, cs.AI, and cs.NE | (2402.13077v1)
Abstract: This paper presents Mechanistic Neural Networks, a neural network design for machine learning applications in the sciences. It incorporates a new Mechanistic Block in standard architectures to explicitly learn governing differential equations as representations, revealing the underlying dynamics of data and enhancing interpretability and efficiency in data modeling. Central to our approach is a novel Relaxed Linear Programming Solver (NeuRLP) inspired by a technique that reduces solving linear ODEs to solving linear programs. This integrates well with neural networks and surpasses the limitations of traditional ODE solvers enabling scalable GPU parallel processing. Overall, Mechanistic Neural Networks demonstrate their versatility for scientific machine learning applications, adeptly managing tasks from equation discovery to dynamic systems modeling. We prove their comprehensive capabilities in analyzing and interpreting complex scientific data across various applications, showing significant performance against specialized state-of-the-art methods.
- E. P. Alves and F. Fiuza. Data-driven discovery of reduced plasma physics models from fully kinetic simulations. Physical Review Research, 4(3):033192, 2022.
- B. Amos and J. Z. Kolter. Optnet: Differentiable optimization as a layer in neural networks. In International Conference on Machine Learning, pages 136–145. PMLR, 2017.
- S. Barratt. On the differentiability of the solution to convex optimization problems. 2019. URL http://arxiv.org/abs/1804.05098.
- Model reduction and neural networks for parametric pdes. The SMAI journal of computational mathematics, 7:121–157, 2021.
- J. Borggaard and J. Burns. A pde sensitivity equation method for optimal aerodynamic design. Journal of Computational Physics, 136(2):366–384, 1997.
- Message passing neural pde solvers. In International Conference on Learning Representations, 2021.
- Lie point symmetry data augmentation for neural pde solvers. In International Conference on Machine Learning, pages 2241–2256. PMLR, 2022.
- Discovering governing equations from data by sparse identification of nonlinear dynamical systems. Proceedings of the national academy of sciences, 113(15):3932–3937, 2016.
- Neural ordinary differential equations. Advances in neural information processing systems, 31, 2018.
- Augmented neural odes. Advances in neural information processing systems, 32, 2019.
- Data assimilation in weather forecasting: a case study in pde-constrained optimization. Optimization and Engineering, 10(3):409–426, 2009.
- J. D. Giorgini. Status of the JPL Horizons Ephemeris System. In IAU General Assembly, volume 29, page 2256293, Aug. 2015.
- Sindy-pi: a robust algorithm for parallel implicit sparse identification of nonlinear dynamics. Proceedings of the Royal Society A, 476(2242):20200279, 2020.
- Neural relational inference for interacting systems. In International conference on machine learning, pages 2688–2697. PMLR, 2018.
- Neural operator: Graph kernel network for partial differential equations, 2020a.
- Multipole graph neural operator for parametric partial differential equations. Advances in Neural Information Processing Systems, 33:6755–6766, 2020b.
- Fourier neural operator for parametric partial differential equations. In International Conference on Learning Representations, 2020c.
- Constrained sparse galerkin regression. Journal of Fluid Mechanics, 838:42–67, 2018.
- Data-driven discovery and extrapolation of parameterized pattern-forming dynamics. Bulletin of the American Physical Society, 2022.
- J.-P. Noël and M. Schoukens. F-16 aircraft benchmark based on ground vibration test data. In 2017 Workshop on Nonlinear System Identification Benchmarks, pages 19–23, 2017.
- On second order behaviour in augmented neural odes. Advances in neural information processing systems, 33:5911–5921, 2020.
- Differentiable mathematical programming for object-centric representation learning. In The Eleventh International Conference on Learning Representations, 2023. URL https://openreview.net/forum?id=1J-ZTr7aypY.
- P. Rabinowitz. Applications of linear programming to numerical analysis. 10(2):121–159, 1968. ISSN 0036-1445. doi: 10.1137/1010029. URL https://epubs.siam.org/doi/10.1137/1010029. Publisher: Society for Industrial and Applied Mathematics.
- Universal differential equations for scientific machine learning. arXiv preprint arXiv:2001.04385, 2020.
- M. Raissi. Deep hidden physics models: Deep learning of nonlinear partial differential equations. The Journal of Machine Learning Research, 19(1):932–955, 2018.
- Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational physics, 378:686–707, 2019.
- Data-driven discovery of partial differential equations. Science advances, 3(4):e1602614, 2017.
- D. Ruiz-Balet and E. Zuazua. Neural ode control for classification, approximation, and transport. SIAM Review, 65(3):735–773, 2023. doi: 10.1137/21M1411433. URL https://doi.org/10.1137/21M1411433.
- Odin: Ode-informed regression for parameter and state inference in time-continuous dynamical systems. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 6364–6371, 2020.
- Melding the data-decisions pipeline: Decision-focused learning for combinatorial optimization. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 1658–1665, 2019.
- S. Wright and J. Nocedal. Numerical optimization. Springer Science, 35(67-68):7, 1999.
- J. D. Young. Linear programming applied to linear differential equations. 1961.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.