SVD-PINNs: Transfer Learning of Physics-Informed Neural Networks via Singular Value Decomposition (2211.08760v2)
Abstract: Physics-informed neural networks (PINNs) have attracted significant attention for solving partial differential equations (PDEs) in recent years because they alleviate the curse of dimensionality that appears in traditional methods. However, the most disadvantage of PINNs is that one neural network corresponds to one PDE. In practice, we usually need to solve a class of PDEs, not just one. With the explosive growth of deep learning, many useful techniques in general deep learning tasks are also suitable for PINNs. Transfer learning methods may reduce the cost for PINNs in solving a class of PDEs. In this paper, we proposed a transfer learning method of PINNs via keeping singular vectors and optimizing singular values (namely SVD-PINNs). Numerical experiments on high dimensional PDEs (10-d linear parabolic equations and 10-d Allen-Cahn equations) show that SVD-PINNs work for solving a class of PDEs with different but close right-hand-side functions.
- Wasserstein generative adversarial networks. In International Conference on Machine Learning, pages 214–223. PMLR, 2017.
- Learning and meta-learning of stochastic advection–diffusion–reaction systems from sparse measurements. European Journal of Applied Mathematics, 32(3):397–420, 2021.
- One-shot transfer learning of physics-informed neural networks. ICML AI4Science Workshop, 2022.
- Deep neural networks for solving extremely large linear systems. arXiv preprint arXiv:2204.00313, 2022.
- Geoffrey Hinton. Rmsprop: Divide the gradient by a running average of its recent magnitude. Neural Networks for Machine Learning, 2012.
- Adaptive activation functions accelerate convergence in deep and physics-informed neural networks. Journal of Computational Physics, 404:109136, 2020.
- Adam: A method for stochastic optimization. International Conference for Learning Representations (ICLR), 2015.
- Imagenet classification with deep convolutional neural networks. Advances in Neural Information Processing Systems, 25, 2012.
- Artificial neural networks for solving ordinary and partial differential equations. IEEE Transactions on Neural Networks, 9(5):987–1000, 1998.
- Learning nonlinear operators via deeponet based on the universal approximation theorem of operators. Nature Machine Intelligence, 3(3):218–229, 2021.
- Deep learning for healthcare: review, opportunities and challenges. Briefings in bioinformatics, 19(6):1236–1246, 2018.
- A survey on transfer learning. IEEE Transactions on Knowledge and Data Engineering, 22(10):1345–1359, 2009.
- fpinns: Fractional physics-informed neural networks. SIAM Journal on Scientific Computing, 41(4):A2603–A2626, 2019.
- A hybrid neural network-first principles approach to process modeling. AIChE Journal, 38(10):1499–1511, 1992.
- Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational Physics, 378:686–707, 2019.
- On the convergence of physics informed neural networks for linear second-order elliptic and parabolic type pdes. on Commun. Comput. Phys., (28):2042–2074, 2020.
- Dgm: A deep learning algorithm for solving partial differential equations. Journal of computational physics, 375:1339–1364, 2018.
- Grammar as a foreign language. Advances in Neural Information Processing Systems, 28, 2015.
- When and why pinns fail to train: A neural tangent kernel perspective. Journal of Computational Physics, 449:110768, 2022.
- Algorithms for solving high dimensional pdes: from nonlinear monte carlo to machine learning. Nonlinearity, 35(1):278, 2021.
- Can transfer neuroevolution tractably solve your differential equations? IEEE Computational Intelligence Magazine, 16(2):14–30, 2021.
- Jinchao Xu. The finite neuron method and convergence analysis. arXiv preprint arXiv:2010.01458, 2020.
- Adversarial uncertainty quantification in physics-informed neural networks. Journal of Computational Physics, 394:136–152, 2019.
- How transferable are features in deep neural networks? Advances in Neural Information Processing Systems, 27, 2014.
- Learning in modal space: Solving time-dependent stochastic pdes using physics-informed neural networks. SIAM Journal on Scientific Computing, 42(2):A639–A665, 2020.
- Quantifying total uncertainty in physics-informed neural networks for solving forward and inverse stochastic problems. Journal of Computational Physics, 397:108850, 2019.