Uncovering solutions from data corrupted by systematic errors: A physics-constrained convolutional neural network approach (2306.04600v2)
Abstract: Information on natural phenomena and engineering systems is typically contained in data. Data can be corrupted by systematic errors in models and experiments. In this paper, we propose a tool to uncover the spatiotemporal solution of the underlying physical system by removing the systematic errors from data. The tool is the physics-constrained convolutional neural network (PC-CNN), which combines information from both the systems governing equations and data. We focus on fundamental phenomena that are modelled by partial differential equations, such as linear convection, Burgers equation, and two-dimensional turbulence. First, we formulate the problem, describe the physics-constrained convolutional neural network, and parameterise the systematic error. Second, we uncover the solutions from data corrupted by large multimodal systematic errors. Third, we perform a parametric study for different systematic errors. We show that the method is robust. Fourth, we analyse the physical properties of the uncovered solutions. We show that the solutions inferred from the PC-CNN are physical, in contrast to the data corrupted by systematic errors that does not fulfil the governing equations. This work opens opportunities for removing epistemic errors from models, and systematic errors from measurements.
- A. Sciacchitano, D. R. Neal, B. L. Smith, S. O. Warner, P. P. Vlachos, B. Wieneke, and F. Scarano, “Collaborative framework for piv uncertainty quantification: comparative assessment of methods,” Measurement Science and Technology, vol. 26, no. 7, p. 074004, 2015.
- E. M. Zucchelli, E. D. Delande, B. A. Jones, and M. K. Jah, “Multi-fidelity orbit determination with systematic errors,” The Journal of the Astronautical Sciences, vol. 68, pp. 695–727, 2021.
- R. J. Adrian, “Particle-imaging techniques for experimental fluid mechanics,” Annual review of fluid mechanics, vol. 23, no. 1, pp. 261–304, 1991.
- P. Vedula and R. J. Adrian, “Optimal solenoidal interpolation of turbulent vector fields: Application to ptv and super-resolution piv,” Experiments in Fluids, vol. 39, pp. 213–221, 8 2005.
- D. Schiavazzi, F. Coletti, G. Iaccarino, and J. K. Eaton, “A matching pursuit approach to solenoidal filtering of three-dimensional velocity measurements,” Journal of Computational Physics, vol. 263, pp. 206–221, 4 2014.
- D. Kochkov, J. A. Smith, A. Alieva, Q. Wang, M. P. Brenner, and S. Hoyer, “Machine learning–accelerated computational fluid dynamics,” Proceedings of the National Academy of Sciences, vol. 118, no. 21, p. e2101784118, 2021.
- N. Kumar and M. Nachamai, “Noise removal and filtering techniques used in medical images,” Oriental journal of computer science and technology, vol. 10, pp. 103–113, 3 2017.
- M. Raiola, S. Discetti, and A. Ianiro, “On piv random error minimization with optimal pod-based low-order reconstruction,” Experiments in Fluids, vol. 56, 4 2015.
- M. A. Mendez, M. Raiola, A. Masullo, S. Discetti, A. Ianiro, R. Theunissen, and J. M. Buchlin, “Pod-based background removal for particle image velocimetry,” Experimental Thermal and Fluid Science, vol. 80, pp. 181–192, 1 2017.
- P. Vincent, H. Larochelle, Y. Bengio, and P.-A. Manzagol, “Extracting and composing robust features with denoising autoencoders,” Proceedings of the 25th International Conference on Machine Learning, pp. 1096–1103, 2008.
- W. T. Freeman, T. R. Jones, and E. C. Pasztor, “Example-based super-resolution,” IEEE Computer Graphics and Applications, vol. 22, pp. 56–65, 2002.
- J. Yang, J. Wright, T. S. Huang, and Y. Ma, “Image super-resolution via sparse representation,” IEEE Transactions on Image Processing, vol. 19, pp. 2861–2873, 11 2010.
- C. Dong, C. C. Loy, K. He, and X. Tang, “Learning a deep convolutional network for image super-resolution,” Computer Vision – ECCV 2014184, pp. 184–199, 2014. [Online]. Available: http://mmlab.ie.cuhk.edu.hk/projects/SRCNN.html.
- Y. Xie, E. Franz, M. Chu, and N. Thuerey, “Tempogan: A temporally coherent, volumetric gan for super-resolution fluid flow,” ACM Transactions on Graphics, vol. 37, 2018.
- K. Hornik, M. Stinchcombe, and H. White, “Multilayer feedforward networks are universal approximators,” Neural Networks, vol. 2, pp. 359–366, 1989.
- D. X. Zhou, “Universality of deep convolutional neural networks,” Applied and Computational Harmonic Analysis, vol. 48, pp. 787–794, 3 2020.
- I. E. Lagaris, A. Likas, and D. I. Fotiadis, “Artificial neural networks for solving ordinary and partial differential equations,” IEEE Transactions on Neural Networks, vol. 9, pp. 987–1000, 1998.
- M. Raissi, P. Perdikaris, and G. E. Karniadakis, “Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations,” Journal of Computational Physics, vol. 378, pp. 686–707, 2 2019.
- L. A. Rastrigin, “Systems of extremal control,” Nauka, 1974.
- A. Nóvoa and L. Magri, “Real-time thermoacoustic data assimilation,” Journal of Fluid Mechanics, vol. 948, p. A35, 2022.
- Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning,” nature, vol. 521, no. 7553, pp. 436–444, 2015.
- J. Gu, Z. Wang, J. Kuen, L. Ma, A. Shahroudy, B. Shuai, T. Liu, X. Wang, G. Wang, J. Cai, and T. Chen, “Recent advances in convolutional neural networks,” Pattern Recognition, vol. 77, pp. 354–377, 2018. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0031320317304120
- A. Racca, N. A. K. Doan, and L. Magri, “Modelling spatiotemporal turbulent dynamics with the convolutional autoencoder echo state network,” 2022.
- N. A. K. Doan, W. Polifke, and L. Magri, “Short and long-term predictions of chaotic flows and extreme events: a physics-constrained reservoir computing approach,” Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, vol. 477, no. 2253, p. 20210135, 2021.
- L. Magri, “Introduction to neural networks for engineering and computational science,” 1 2023.
- T. Murata, K. Fukami, and K. Fukagata, “Nonlinear mode decomposition with convolutional neural networks for fluid dynamics,” Journal of Fluid Mechanics, vol. 882, p. A13, 2020.
- H. Gao, L. Sun, and J.-X. Wang, “Super-resolution and denoising of fluid flow using physics-informed convolutional neural networks without high-resolution labels,” Physics of Fluids, vol. 33, p. 073603, 7 2021.
- C. M. Bishop, “Neural networks and their applications,” Review of Scientific Instruments, vol. 65, no. 6, pp. 1803–1832, 1994.
- N. Aloysius and M. Geetha, “A review on deep convolutional neural networks,” pp. 0588–0592, 2017.
- D. Kelshaw, “Kolsol,” https://github.com/magrilab/kolsol, 2022.
- H. Bateman, “Some recent researches on the motion of fluids,” Monthly Weather Review, vol. 43, no. 4, pp. 163 – 170, 1915.
- J. Burgers, “A mathematical model illustrating the theory of turbulence,” vol. 1, pp. 171–199, 1948.
- E. D. Fylladitakis, “Kolmogorov flow: Seven decades of history,” Journal of Applied Mathematics and Physics, vol. 6, pp. 2227–2263, 2018.
- D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” 2015.
- A. J. Majda and P. R. Kramer, “Simplified models for turbulent diffusion: theory, numerical modelling, and physical phenomena,” Physics reports, vol. 314, no. 4-5, pp. 237–574, 1999.