Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
144 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Physics-Informed Neural Networks for High-Frequency and Multi-Scale Problems using Transfer Learning (2401.02810v2)

Published 5 Jan 2024 in cs.LG, cs.AI, cs.NA, and math.NA

Abstract: Physics-informed neural network (PINN) is a data-driven solver for partial and ordinary differential equations(ODEs/PDEs). It provides a unified framework to address both forward and inverse problems. However, the complexity of the objective function often leads to training failures. This issue is particularly prominent when solving high-frequency and multi-scale problems. We proposed using transfer learning to boost the robustness and convergence of training PINN, starting training from low-frequency problems and gradually approaching high-frequency problems. Through two case studies, we discovered that transfer learning can effectively train PINN to approximate solutions from low-frequency problems to high-frequency problems without increasing network parameters. Furthermore, it requires fewer data points and less training time. We elaborately described our training strategy, including optimizer selection, and suggested guidelines for using transfer learning to train neural networks for solving more complex problems.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (47)
  1. Modeling the wave equation using physics-informed neural networks enhanced with attention to loss weights. In ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1–5, June 2023.
  2. Wave equation modeling via physics-informed neural networks: Models of soft and hard constraints for initial and boundary conditions. Sensors, 23(5), 2023.
  3. Physics-informed neural networks: Theory and applications. In Timon Rabczuk and Klaus-Jürgen Bathe, editors, Machine Learning in Modeling and Simulation: Methods and Applications, pages 179–218, Cham, 2023. Springer International Publishing.
  4. Automatic differentiation in machine learning: a survey. Journal of Marchine Learning Research, 18:1–43, 2018.
  5. A limited memory algorithm for bound constrained optimization. SIAM Journal on Scientific Computing, 16(5):1190–1208, 1995.
  6. Physics-informed neural networks (pinns) for fluid mechanics: a review. Acta Mechanica Sinica, 37(12):1727–1738, Dec 2021.
  7. Towards understanding the spectral bias of deep learning. In Zhi-Hua Zhou, editor, Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence (IJCAI 2021), IJCAI International Joint Conference on Artificial Intelligence, pages 2205–2211. International Joint Conferences on Artificial Intelligence, August 2021. 30th International Joint Conference on Artificial Intelligence (IJCAI 2021), IJCAI-21 ; Conference date: 19-08-2021 Through 27-08-2021.
  8. Souvik Chakraborty. Transfer learning based multi-fidelity physics informed deep neural network. Journal of Computational Physics, 426:109942, 2021.
  9. Transfer learning-based physics-informed convolutional neural network for simulating flow in porous media with time-varying controls, 2023.
  10. Transfer learning enhanced physics informed neural network for phase-field modeling of fracture. Theoretical and Applied Fracture Mechanics, 106:102447, 2020.
  11. Sciann: A keras/tensorflow wrapper for scientific computations and physics-informed deep learning using artificial neural networks. Computer Methods in Applied Mechanics and Engineering, 373:113552, 2021.
  12. A physics-informed deep learning framework for inversion and surrogate modeling in solid mechanics. Computer Methods in Applied Mechanics and Engineering, 379:113741, 2021.
  13. Nvidia simnet™: An ai-accelerated multi-physics simulation framework. In Maciej Paszynski, Dieter Kranzlmüller, Valeria V. Krzhizhanovskaya, Jack J. Dongarra, and Peter M.A. Sloot, editors, Computational Science – ICCS 2021, pages 447–461, Cham, 2021. Springer International Publishing.
  14. Physics informed machine learning: Seismic wave equation. Geoscience Frontiers, 11(6):1993–2001, 2020.
  15. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
  16. Transfer learning enhanced full waveform inversion, 2023.
  17. Characterizing possible failure modes in physics-informed neural networks. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan, editors, Advances in Neural Information Processing Systems, volume 34, pages 26548–26560. Curran Associates, Inc., 2021.
  18. Deep learning. Nature, 521:436–444, 2015.
  19. Solving elastodynamics via physics-informed neural network frequency domain method. International Journal of Mechanical Sciences, 258:108575, 2023.
  20. On the limited memory bfgs method for large scale optimization. Mathematical programming, 45(1-3):503–528, 1989.
  21. Deepxde: A deep learning library for solving differential equations. SIAM Review, 63(1):208–228, 2021.
  22. Influence of activation functions on the convergence of physics-informed neural networks for 1d wave equation. In Jiří Mikyška, Clélia de Mulatier, Maciej Paszynski, Valeria V. Krzhizhanovskaya, Jack J. Dongarra, and Peter M.A. Sloot, editors, Computational Science – ICCS 2023, pages 74–88, Cham, 2023. Springer Nature Switzerland.
  23. Stefano Markidis. The old and the new : Can physics-informed deep-learning replace traditional linear solvers? Frontiers In Big Data, 4, 2021. QC 20211217.
  24. Stefano Markidis. The old and the new: Can physics-informed deep-learning replace traditional linear solvers? Frontiers in big Data, 4:669097, 2021.
  25. Solving the wave equation with physics-informed deep learning, 2020.
  26. Deep pre-trained FWI: where supervised learning meets the physics-informed neural networks. Geophysical Journal International, 235(1):119–134, 05 2023.
  27. Numerical wave propagation aided by deep learning. Journal of Computational Physics, 475:111828, 2023.
  28. Jorge Nocedal and Stephen J. Wright. Numerical optimization, pages 1–664. Springer Series in Operations Research and Financial Engineering. Springer Nature, 2006.
  29. Physics-informed deep learning approach for modeling crustal deformation. Nature Communications, 13(1):7092, Nov 2022.
  30. Physics-informed neural network with transfer learning (tl-pinn) based on domain similarity measure for prediction of nuclear reactor transients. Scientific Reports, 13(1):16840, October 2023.
  31. On the spectral bias of neural networks. In Kamalika Chaudhuri and Ruslan Salakhutdinov, editors, Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pages 5301–5310. PMLR, 09–15 Jun 2019.
  32. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational Physics, 378:686–707, 2019.
  33. Physics informed deep learning (part i): Data-driven solutions of nonlinear partial differential equations, 2017.
  34. Physics informed deep learning (part ii): Data-driven discovery of nonlinear partial differential equations. ArXiv, abs/1711.10566, 2017.
  35. Physics-informed deep learning for computational elastodynamics without labeled data. Journal of Engineering Mechanics, 147(8):04021043, 2021.
  36. Physics-informed neural networks (pinns) for wave propagation and full waveform inversions. Journal of Geophysical Research: Solid Earth, 127(5):e2021JB023120, 2022. e2021JB023120 2021JB023120.
  37. Fourier features let networks learn high frequency functions in low dimensional domains. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin, editors, Advances in Neural Information Processing Systems, volume 33, pages 7537–7547. Curran Associates, Inc., 2020.
  38. A transfer learning-physics informed neural network (tl-pinn) for vortex-induced vibration. Ocean Engineering, 266:113101, 2022.
  39. Umair Bin Waheed. Kronecker neural networks overcome spectral bias for pinn-based wavefield computation. IEEE Geoscience and Remote Sensing Letters, 19:1–5, 2022.
  40. On acoustic fields of complex scatters based on physics-informed neural networks. Ultrasonics, 128:106872, 2023.
  41. Understanding and mitigating gradient flow pathologies in physics-informed neural networks. SIAM Journal on Scientific Computing, 43(5):A3055–A3081, 2021.
  42. On the eigenvector bias of fourier feature networks: From regression to solving multi-scale pdes with physics-informed neural networks. Computer Methods in Applied Mechanics and Engineering, 384:113938, 2021.
  43. Transfer learning based physics-informed neural networks for solving inverse problems in engineering structures under different loading scenarios. Computer Methods in Applied Mechanics and Engineering, 405:115852, 2023.
  44. Fwigan: Full-waveform inversion via a physics-informed generative adversarial network. Journal of Geophysical Research: Solid Earth, 128(4):e2022JB025493, 2023. e2022JB025493 2022JB025493.
  45. Wasserstein distance-based full-waveform inversion with a regularizer powered by learned gradient. IEEE Transactions on Geoscience and Remote Sensing, 61:1–13, 2023.
  46. Deep learning for geophysics: Current and future trends. Reviews of Geophysics, 59(3):e2021RG000742, 2021. e2021RG000742 2021RG000742.
  47. Transfer learning-based coupling of smoothed finite element method and physics-informed neural network for solving elastoplastic inverse problems. Mathematics, 11(11), 2023.
Citations (5)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets