Learning to correct spectral methods for simulating turbulent flows (2207.00556v2)
Abstract: Despite their ubiquity throughout science and engineering, only a handful of partial differential equations (PDEs) have analytical, or closed-form solutions. This motivates a vast amount of classical work on numerical simulation of PDEs and more recently, a whirlwind of research into data-driven techniques leveraging ML. A recent line of work indicates that a hybrid of classical numerical techniques and machine learning can offer significant improvements over either approach alone. In this work, we show that the choice of the numerical scheme is crucial when incorporating physics-based priors. We build upon Fourier-based spectral methods, which are known to be more efficient than other numerical schemes for simulating PDEs with smooth and periodic solutions. Specifically, we develop ML-augmented spectral solvers for three common PDEs of fluid dynamics. Our models are more accurate (2-4x) than standard spectral solvers at the same resolution but have longer overall runtimes (~2x), due to the additional runtime cost of the neural network component. We also demonstrate a handful of key design principles for combining machine learning and numerical methods for solving PDEs.
- Learning data-driven discretizations for partial differential equations. Proceedings of the National Academy of Sciences, 116, July 2019. ISSN 0027-8424, 1091-6490. doi: 10.1073/pnas.1814058116.
- Two-Dimensional Turbulence. Annual Review of Fluid Mechanics, 44, January 2012. ISSN 0066-4189, 1545-4479. doi: 10.1146/annurev-fluid-120710-101240.
- John P. Boyd. The Erfc-Log Filter and the Asymptotics of the Euler and Vandeven Sequence Accelerations. In Ilin, A V and Scott, L Ridway (ed.), Proceedings of the Third International Conference on Spectral and High Order Methods, Houston, 1996. Houston Journal of Mathematics.
- John P. Boyd. Chebyshev and Fourier Spectral Methods. Dover Books on Mathematics. Dover Publications, Mineola, NY, second edition, 2001. ISBN 0486411834 9780486411835.
- Prognostic Validation of a Neural Network Unified Physics Parameterization. Geophysical Research Letters, 45, 2018. doi: https://doi.org/10.1029/2018GL078510.
- FC-based shock-dynamics solver with neural-network localized artificial-viscosity assignment, 2021.
- Dedalus: A flexible framework for numerical simulations with spectral methods. Phys. Rev. Research, 2, April 2020.
- Spectral Methods: Evolution to Complex Geometries and Applications to Fluid Dynamics. Scientific Computation. Springer Berlin Heidelberg, 2007. ISBN 9783540307280.
- Spectral methods: Fundamentals in single domains. Scientific computation. Springer, Berlin, Germany, 1 edition, April 2006.
- Neural Ordinary Differential Equations. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 31. Curran Associates, Inc., 2018.
- An Algorithm for the Machine Calculation of Complex Fourier Series. Math. Comput., 19, 1965. ISSN 00255718, 10886842.
- Paul A Durbin. Some Recent Developments in Turbulence Closure Modeling. Annu. Rev. Fluid Mech., 50, 2018.
- A multiscale neural network based on hierarchical nested bases. Research in the Mathematical Sciences, 6, Mar. 2019. ISSN 2197-9847. doi: 10.1007/s40687-019-0183-3.
- A posteriori learning for quasi-geostrophic turbulence parametrization. arXiv, April 2022.
- D. Gottlieb and J. S. Hesthaven. Spectral methods for hyperbolic problems. Journal of Computational and Applied Mathematics, 128, March 2001. ISSN 0377-0427. doi: 10.1016/S0377-0427(00)00510-0.
- Numerical Analysis of Spectral Methods. Society for Industrial and Applied Mathematics, 1977. doi: 10.1137/1.9781611970425.
- Accelerating Numerical Solvers for Large-Scale Simulation of Dynamical System via NeurVec, August 2022. arXiv:2208.03680.
- Machine learning–accelerated computational fluid dynamics. Proc. Natl. Acad. Sci. U. S. A., 118, May 2021.
- David A. Kopriva. Implementing Spectral Methods for Partial Differential Equations: Algorithms for Scientists and Engineers. Springer Publishing Company, Incorporated, 1st edition, 2009. ISBN 9048122600.
- Multipole Graph Neural Operator for Parametric Partial Differential Equations. In Proceedings of the 34th International Conference on Neural Information Processing Systems, NIPS’20, 2020.
- Fourier Neural Operator for Parametric Partial Differential Equations. arXiv:2010.08895 [cs, math], May 2021. arXiv: 2010.08895.
- Learned Turbulence Modelling with Differentiable Fluid Solvers. arXiv:2202.06988 [physics], February 2022. arXiv: 2202.06988.
- Splitting methods. Acta Numerica, 11, 2002. doi: 10.1017/S0962492902000053.
- Siddhartha Mishra. A machine learning framework for data driven acceleration of computations of differential equations. Mathematics in Engineering, 1, 2018. ISSN 2640-3501. doi: 10.3934/Mine.2018.1.118.
- Steven A. Orszag. On the Elimination of Aliasing in Finite-Difference Schemes by Filtering High-Wavenumber Components. Journal of the Atmospheric Sciences, 28, September 1971. ISSN 0022-4928, 1520-0469. doi: 10.1175/1520-0469(1971)028<1074:OTEOAI>2.0.CO;2.
- Stephen B. Pope. Turbulent Flows. Cambridge University Press, August 2000. ISBN 9780521598866.
- Stephen B Pope. Ten questions concerning the large-eddy simulation of turbulent flows. New Journal of Physics, 6, March 2004. ISSN 1367-2630. doi: 10.1088/1367-2630/6/1/035.
- Climate model configurations of the ECMWF Integrated Forecasting System (ECMWF-IFS cycle 43r1) for HighResMIP. Geoscientific model development, 11, 2018.
- U-Net: Convolutional Networks for Biomedical Image Segmentation. In Nassir Navab, Joachim Hornegger, William M. Wells, and Alejandro F. Frangi (eds.), Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015, Cham, 2015. Springer International Publishing. ISBN 978-3-319-24574-4.
- Hidetsugu Sakaguchi. Chaotic dynamics of an unstable Burgers equation. Physica D: Nonlinear Phenomena, 129, May 1999. ISSN 0167-2789. doi: 10.1016/S0167-2789(98)00317-0.
- Extreme learning machine for reduced order modeling of turbulent geophysical flows. Physical Review E, 97(4):042322, April 2018. doi: 10.1103/PhysRevE.97.042322. Publisher: American Physical Society.
- Learning to Simulate Complex Physics with Graph Networks. arXiv:2002.09405 [physics, stat], September 2020. arXiv: 2002.09405.
- Learned Coarse Models for Efficient Turbulence Simulation. arXiv:2112.15275 [physics], January 2022. arXiv: 2112.15275.
- Gilbert Strang. On the Construction and Comparison of Difference Schemes. SIAM Journal on Numerical Analysis, 5, 1968. doi: 10.1137/0705041.
- Data-driven subgrid-scale modeling of forced Burgers turbulence using deep learning with generalization to higher Reynolds numbers via transfer learning. Physics of Fluids, 33(3):031702, March 2021. ISSN 1070-6631, 1089-7666. doi: 10.1063/5.0040286. arXiv:2012.06664 [physics].
- Eitan Tadmor. The Exponential Accuracy of Fourier and Chebyshev Differencing Methods. SIAM Journal on Numerical Analysis, 23, 1986. doi: 10.1137/0723001.
- Factorized Fourier Neural Operators. arXiv:2111.13802 [cs], November 2021. arXiv: 2111.13802.
- Lloyd N. Trefethen. Spectral Methods in MATLAB. Society for industrial and applied mathematics (SIAM), 2000.
- Solver-in-the-Loop: Learning from Differentiable Physics to Interact with Iterative PDE-Solvers. In NeurIPS 2020, June 2020.
- Towards Physics-informed Deep Learning for Turbulent Flow Prediction. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 2020.
- David L Williamson. Equivalent finite volume and Eulerian spectral transform horizontal resolutions established from aqua-planet simulations. Tellus Ser. A Dyn. Meteorol. Oceanogr., 60, January 2008.
- P K Yeung and K Ravikumar. Advancing understanding of turbulence through extreme-scale computation: Intermittency and simulations at large problem sizes. Phys. Rev. Fluids, 5, November 2020.
- An easily implemented task-based parallel scheme for the Fourier pseudospectral solver applied to 2D Navier–Stokes turbulence. Computers & Fluids, 33(4):509–520, 2004. ISSN 0045-7930. doi: 10.1016/j.compfluid.2003.06.003.
- Multi-Scale Context Aggregation by Dilated Convolutions. In Yoshua Bengio and Yann LeCun (eds.), 4th International Conference on Learning Representations, ICLR 2016, San Juan, Puerto Rico, May 2-4, 2016, Conference Track Proceedings, 2016.