Space and Time Continuous Physics Simulation From Partial Observations (2401.09198v3)
Abstract: Modern techniques for physical simulations rely on numerical schemes and mesh-refinement methods to address trade-offs between precision and complexity, but these handcrafted solutions are tedious and require high computational power. Data-driven methods based on large-scale machine learning promise high adaptivity by integrating long-range dependencies more directly and efficiently. In this work, we focus on fluid dynamics and address the shortcomings of a large part of the literature, which are based on fixed support for computations and predictions in the form of regular or irregular grids. We propose a novel setup to perform predictions in a continuous spatial and temporal domain while being trained on sparse observations. We formulate the task as a double observation problem and propose a solution with two interlinked dynamical systems defined on, respectively, the sparse positions and the continuous domain, which allows to forecast and interpolate a solution from the initial condition. Our practical implementation involves recurrent GNNs and a spatio-temporal attention observer capable of interpolating the solution at arbitrary locations. Our model not only generalizes to new initial conditions (as standard auto-regressive models do) but also performs evaluation at arbitrary space and time locations. We evaluate on three standard datasets in fluid dynamics and compare to strong baselines, which are outperformed both in classical settings and in the extended new task requiring continuous predictions.
- Interaction networks for learning about objects, relations and physics. Neural Information Processing Systems, 2016.
- Observer design for continuous-time dynamical systems. Annual Reviews in Control, 2022.
- Deep learning for physical processes: Incorporating prior scientific knowledge. Journal of Statistical Mechanics: Theory and Experiment, 2019.
- Magnet: Mesh agnostic neural pde solver. In Neural Information Processing Systems, 2022.
- Message passing neural PDE solvers. In International Conference on Learning Representations, 2022.
- Physics-informed neural networks (pinns) for fluid mechanics: A review. Acta Mechanica Sinica, 2021.
- Neural ordinary differential equations. Neural Information Processing Systems, 2018.
- Learning phrase representations using rnn encoder-decoder for statistical machine translation. arXiv preprint, 2014.
- Filipe de Avila Belbute-Peres and J Zico Kolter. Simple initialization and parametrization of sinusoidal networks via their kernel bandwidth. In International Conference on Learning Representations, 2023.
- Neural-network-based approximations for solving partial differential equations. Communications in Numerical Methods in Engineering, 1994.
- Augmented neural odes. Neural Information Processing Systems, 2019.
- Multiplicative filter networks. In International Conference on Learning Representations, 2021.
- A stable and scalable method for solving initial value pdes with neural networks. In International Conference on Learning Representations, 2023.
- An initial-value problem for testing numerical models of the global shallow-water equations. Tellus A: Dynamic Meteorology and Oceanography, 2004.
- Disentangling physical dynamics from unknown factors for unsupervised video prediction. In Conference on Computer Vision and Pattern Recognition, 2020.
- Predicting physics in mesh-reduced space with temporal attention. In International Conference on Learning Representations, 2021.
- Efficient continuous spatio-temporal simulation with graph spline networks. In Internation Conference on Machine Learning (AI for Science Workshop), 2022.
- Filtered-cophy: Unsupervised learning of counterfactual physics in pixel space. In International Conference on Learning Representation, 2022a.
- Learning reduced nonlinear state-space models: an output-error based canonical approach. In Conference on Decision and Control, 2022b.
- Eagle: Large-scale learning of turbulent fluid dynamics with mesh transformers. In International Conference on Learning Representation, 2023.
- Attentive neural processes. In International Conference on Learning Representations, 2018.
- Machine learning in cardiovascular flows modeling: Predicting arterial blood pressure from non-invasive 4d flow mri data using physics-informed neural networks. Computer Methods in Applied Mechanics and Engineering, 2020.
- Learning latent field dynamics of pdes. In Third Workshop on Machine Learning and the Physical Sciences (NeurIPS 2020), 2020.
- Characterizing possible failure modes in physics-informed neural networks. Neural Information Processing Systems, 2021.
- Artificial neural networks for solving ordinary and partial differential equations. Transactions on Neural Networks, 1998.
- Learning particle dynamics for manipulating rigid bodies, deformable objects, and fluids. In International Conference on Learning Representations, 2018.
- Multipole graph neural operator for parametric partial differential equations. In Neural Information Processing Systems, 2020a.
- Fourier neural operator for parametric partial differential equations. In International Conference on Learning Representations, 2020b.
- Markov neural operators for learning chaotic systems. arXiv preprint, 2021.
- Deeponet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators. arXiv preprint, 2019.
- Physics-informed neural networks with hard constraints for inverse design. Journal on Scientific Computing, 2021.
- Physics-informed neural networks for power systems. In Power & Energy Society General Meeting, 2020.
- Learning mesh-based simulation with graph networks. In International Conference on Learning Representations, 2020.
- A hybrid neural network-first principles approach to process modeling. AIChE Journal, 1992.
- Physics informed deep learning (part i): Data-driven solutions of nonlinear partial differential equations. arXiv preprint, 2017.
- Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational physics, 2019.
- Searching for activation functions. arXiv preprint, 2017.
- Translating images into maps. In International Conference on Robotics and Automation, 2022.
- Learning to simulate complex physics with graph networks. In International Conference on Machine Learning, 2020.
- Implicit neural representations with periodic activation functions. Neural Information Processing Systems, 2020.
- Learned simulators for turbulence. In International Conference on Learning Representations, 2021.
- George Gabriel Stokes. On the Effect of the Internal Friction of Fluids on the Motion of Pendulums. Cambridge University Press, 2009.
- Attention is all you need. Neural Information Processing Systems, 2017.
- When and why pinns fail to train: A neural tangent kernel perspective. Journal of Computational Physics, 2022.
- Learning to accelerate partial differential equations via latent global evolution. Advances in Neural Information Processing Systems, 2022.
- Predictive large-eddy-simulation wall modeling via physics-informed neural networks. Physical Review Fluids, 2019.
- Continuous pde dynamics forecasting with implicit neural representations. In International Conference on Learning Representations, 2022.
- Competitive physics informed networks. In International Conference on Learning Representations, 2023.
- Deep learning-based output tracking via regulation and contraction theory. In International Federation of Automatic Control, 2022.