Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
110 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
44 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Physics-constrained convolutional neural networks for inverse problems in spatiotemporal partial differential equations (2401.10306v4)

Published 18 Jan 2024 in physics.flu-dyn and cs.LG

Abstract: We propose a physics-constrained convolutional neural network (PC-CNN) to solve two types of inverse problems in partial differential equations (PDEs), which are nonlinear and vary both in space and time. In the first inverse problem, we are given data that is offset by spatially varying systematic error (i.e., the bias, also known as the epistemic uncertainty). The task is to uncover the true state, which is the solution of the PDE, from the biased data. In the second inverse problem, we are given sparse information on the solution of a PDE. The task is to reconstruct the solution in space with high-resolution. First, we present the PC-CNN, which constrains the PDE with a time-windowing scheme to handle sequential data. Second, we analyse the performance of the PC-CNN for uncovering solutions from biased data. We analyse both linear and nonlinear convection-diffusion equations, and the Navier-Stokes equations, which govern the spatiotemporally chaotic dynamics of turbulent flows. We find that the PC-CNN correctly recovers the true solution for a variety of biases, which are parameterised as non-convex functions. Third, we analyse the performance of the PC-CNN for reconstructing solutions from sparse information for the turbulent flow. We reconstruct the spatiotemporal chaotic solution on a high-resolution grid from only 1% of the information contained in it. For both tasks, we further analyse the Navier-Stokes solutions. We find that the inferred solutions have a physical spectral energy content, whereas traditional methods, such as interpolation, do not. This work opens opportunities for solving inverse problems with partial differential equations.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (36)
  1. N. Kumar and M. Nachamai, “Noise removal and filtering techniques used in medical images,” Oriental journal of computer science and technology, vol. 10, no. 1, pp. 103–113, Mar. 2017.
  2. M. Raiola, S. Discetti, and A. Ianiro, “On PIV random error minimization with optimal POD-based low-order reconstruction,” Experiments in Fluids, vol. 56, no. 4, Apr. 2015.
  3. M. A. Mendez, M. Raiola, A. Masullo, S. Discetti, A. Ianiro, R. Theunissen, and J. M. Buchlin, “POD-based background removal for particle image velocimetry,” Experimental Thermal and Fluid Science, vol. 80, pp. 181–192, Jan. 2017.
  4. P. Vincent, H. Larochelle, I. Lajoie, Y. Bengio, and P.-A. Manzagol, “Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion,” Journal of Machine Learning Research, vol. 11, pp. 3371–3408, Dec. 2010.
  5. S. L. Brunton, J. L. Proctor, and J. N. Kutz, “Discovering governing equations from data by sparse identification of nonlinear dynamical systems,” Proceedings of the National Academy of Sciences, vol. 113, no. 15, pp. 3932–3937, 2016.
  6. C. Dong, C. C. Loy, K. He, and X. Tang, “Learning a deep convolutional network for image super-resolution,” in European Conference on Computer Vision, 2014, pp. 184–199.
  7. W. Shi, J. Caballero, F. Huszár, J. Totz, A. P. Aitken, R. Bishop, D. Rueckert, and Z. Wang, “Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network,” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Sep. 2016.
  8. W. Yang, X. Zhang, Y. Tian, W. Wang, J.-H. Xue, and Q. Liao, “Deep learning for single image super-resolution: A brief review,” IEEE Transactions on Multimedia, vol. 21, no. 12, pp. 3106–3121, Dec. 2019.
  9. B. Liu, J. Tang, H. Huang, and X.-Y. Lu, “Deep learning methods for super-resolution reconstruction of turbulent flows,” Physics of Fluids, vol. 32, no. 2, p. 25105, 2020.
  10. K. Hornik, M. Stinchcombe, and H. White, “Multilayer feedforward networks are universal approximators,” Neural Networks, vol. 2, no. 5, pp. 359–366, Jan. 1989.
  11. D. X. Zhou, “Universality of deep convolutional neural networks,” Applied and Computational Harmonic Analysis, vol. 48, no. 2, pp. 787–794, Mar. 2020.
  12. I. E. Lagaris, A. Likas, and D. I. Fotiadis, “Artificial neural networks for solving ordinary and partial differential equations,” IEEE Transactions on Neural Networks, vol. 9, no. 5, pp. 987–1000, 1998.
  13. M. Raissi, P. Perdikaris, and G. E. Karniadakis, “Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations,” Journal of Computational Physics, vol. 378, pp. 686–707, 2019.
  14. NAK. Doan, W. Polifke, and L. Magri, “Short-and long-term predictions of chaotic flows and extreme events: A physics-constrained reservoir computing approach,” Proceedings of the Royal Society A, vol. 477, no. 2253, p. 20210135, 2021.
  15. H. Eivazi and R. Vinuesa, “Physics-informed deep-learning applications to experimental fluid mechanics,” 2022.
  16. A. Krishnapriyan, A. Gholami, S. Zhe, R. Kirby, and M. W. Mahoney, “Characterizing possible failure modes in physics-informed neural networks,” Advances in Neural Information Processing Systems, vol. 34, pp. 26 548–26 560, 2021.
  17. T. G. Grossmann, U. J. Komorowska, J. Latz, and C.-B. Schönlieb, “Can physics-informed neural networks beat the finite element method?” arXiv preprint arXiv:2302.04107, 2023.
  18. D. Kelshaw and L. Magri, “Uncovering solutions from data corrupted by systematic errors: A physics-constrained convolutional neural network approach,” Jun. 2023.
  19. H. Gao, L. Sun, and J.-X. Wang, “Super-resolution and denoising of fluid flow using physics-informed convolutional neural networks without high-resolution labels,” Physics of Fluids, vol. 33, no. 7, p. 073603, Jul. 2021.
  20. A. Sciacchitano, D. R. Neal, B. L. Smith, S. O. Warner, P. P. Vlachos, B. Wieneke, and F. Scarano, “Collaborative framework for PIV uncertainty quantification: Comparative assessment of methods,” Measurement Science and Technology, vol. 26, no. 7, p. 074004, 2015.
  21. A. Nóvoa and L. Magri, “Real-time thermoacoustic data assimilation,” Journal of Fluid Mechanics, vol. 948, p. A35, 2022.
  22. Y. LeCun and Y. Bengio, “Convolutional networks for images, speech, and time series,” in The Handbook of Brain Theory and Neural Networks.   Cambridge, MA, USA: MIT Press, 1998, pp. 255–258.
  23. T. Murata, K. Fukami, and K. Fukagata, “Nonlinear mode decomposition with convolutional neural networks for fluid dynamics,” Journal of Fluid Mechanics, vol. 882, p. A13, 2020.
  24. D. Kelshaw and L. Magri, “Super-resolving sparse observations in partial differential equations: A physics-constrained convolutional neural network approach,” Jun. 2023.
  25. J. Kim and C. Lee, “Prediction of turbulent heat transfer using convolutional neural networks,” Journal of Fluid Mechanics, vol. 882, p. A18, Jan. 2020.
  26. M. Edalatifar, M. B. Tavakoli, M. Ghalambaz, and F. Setoudeh, “Using deep learning to learn physics of conduction heat transfer,” Journal of Thermal Analysis and Calorimetry, vol. 146, no. 3, pp. 1435–1452, Nov. 2021.
  27. L. Magri, “Introduction to neural networks for engineering and computational science,” Jan. 2023.
  28. W. T. Freeman, T. R. Jones, and E. C. Pasztor, “Example-based super-resolution,” IEEE Computer Graphics and Applications, vol. 22, no. 2, pp. 56–65, 2002.
  29. D. Kelshaw, “KolSol,” 2022.
  30. H. Bateman, “Some recent researches on the motion of fluids,” Monthly Weather Review, vol. 43, no. 4, pp. 163–170, 1915.
  31. J. Burgers, “A mathematical model illustrating the theory of turbulence,” ser. Advances in Applied Mechanics, R. Von Mises and T. Von Kármán, Eds.   Elsevier, 1948, vol. 1, pp. 171–199.
  32. E. D. Fylladitakis, “Kolmogorov flow: Seven decades of history,” Journal of Applied Mathematics and Physics, vol. 6, no. 11, pp. 2227–2263, 2018.
  33. L. A. Rastrigin, “Systems of extremal control,” Nauka, 1974.
  34. D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” in 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings, Y. Bengio and Y. LeCun, Eds., 2015.
  35. A. J. Majda and P. R. Kramer, “Simplified models for turbulent diffusion: Theory, numerical modelling, and physical phenomena,” Physics reports, vol. 314, no. 4-5, pp. 237–574, 1999.
  36. F. Ruan and D. McLaughlin, “An efficient multivariate random field generator using the fast Fourier transform,” Advances in Water Resources, vol. 21, no. 5, pp. 385–399, 1998.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (2)
  1. Daniel Kelshaw (9 papers)
  2. Luca Magri (88 papers)

Summary

We haven't generated a summary for this paper yet.

X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets