Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
169 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Discovering Physics-Informed Neural Networks Model for Solving Partial Differential Equations through Evolutionary Computation (2405.11208v1)

Published 18 May 2024 in cs.NE and cs.LG

Abstract: In recent years, the researches about solving partial differential equations (PDEs) based on artificial neural network have attracted considerable attention. In these researches, the neural network models are usually designed depend on human experience or trial and error. Despite the emergence of several model searching methods, these methods primarily concentrate on optimizing the hyperparameters of fully connected neural network model based on the framework of physics-informed neural networks (PINNs), and the corresponding search spaces are relatively restricted, thereby limiting the exploration of superior models. This article proposes an evolutionary computation method aimed at discovering the PINNs model with higher approximation accuracy and faster convergence rate. In addition to searching the numbers of layers and neurons per hidden layer, this method concurrently explores the optimal shortcut connections between the layers and the novel parametric activation functions expressed by the binary trees. In evolution, the strategy about dynamic population size and training epochs (DPSTE) is adopted, which significantly increases the number of models to be explored and facilitates the discovery of models with fast convergence rate. In experiments, the performance of different models that are searched through Bayesian optimization, random search and evolution is compared in solving Klein-Gordon, Burgers, and Lam\'e equations. The experimental results affirm that the models discovered by the proposed evolutionary computation method generally exhibit superior approximation accuracy and convergence rate, and these models also show commendable generalization performance with respect to the source term, initial and boundary conditions, equation coefficient and computational domain. The corresponding code is available at https://github.com/MathBon/Discover-PINNs-Model.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (61)
  1. Artificial neural networks for solving ordinary and partial differential equations. IEEE transactions on neural networks 9, 987–1000.
  2. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational physics 378, 686–707.
  3. Mim: A deep mixed residual method for solving high-order partial differential equations. Journal of Computational Physics 452, 110930.
  4. Pfnn: A penalty-free neural network method for solving a class of second-order boundary-value problems on complex geometries. Journal of Computational Physics 428, 110085.
  5. The deep ritz method: a deep learning-based numerical algorithm for solving variational problems. Communications in Mathematics and Statistics 6, 1–12.
  6. Pfnn-2: A domain decomposed penalty-free neural network method for solving partial differential equations. Communications in Computational Physics 32, 980–1006.
  7. hp-vpinns: Variational physics-informed neural networks with domain decomposition. Computer Methods in Applied Mechanics and Engineering 374, 113547.
  8. Varnet: Variational neural networks for the solution of partial differential equations, in: Learning for Dynamics and Control, PMLR. pp. 298–307.
  9. Artificial neural network method for solution of boundary value problems with exact satisfaction of arbitrary boundary conditions. IEEE Transactions on Neural Networks 20, 1221–1233.
  10. Enforcing exact boundary and initial conditions in the deep mixed residual method. CSIAM Transactions on Applied Mathematics 2, 748–775.
  11. A unified deep artificial neural network approach to partial differential equations in complex geometries. Neurocomputing 317, 28–41.
  12. Solving allen-cahn and cahn-hilliard equations using the adaptive physics informed neural networks. Communications in Computational Physics 29, 930–954.
  13. Efficient training of physics-informed neural networks via importance sampling. Computer-Aided Civil and Infrastructure Engineering 36, 962–977.
  14. Das-pinns: A deep adaptive sampling method for solving high-dimensional partial differential equations. Journal of Computational Physics 476, 111868.
  15. Understanding and mitigating gradient flow pathologies in physics-informed neural networks. SIAM Journal on Scientific Computing 43, A3055–A3081.
  16. Multi-objective loss balancing for physics-informed deep learning. arXiv preprint arXiv:2110.09813 .
  17. When and why pinns fail to train: A neural tangent kernel perspective. Journal of Computational Physics 449, 110768.
  18. Parallel physics-informed neural networks via domain decomposition. Journal of Computational Physics 447, 110683.
  19. Extended physics-informed neural networks (xpinns): A generalized space-time domain decomposition based deep learning framework for nonlinear partial differential equations. Communications in Computational Physics 28, 2002–2041.
  20. Deepxde: A deep learning library for solving differential equations. SIAM review 63, 208–228.
  21. Nvidia simnet™: An ai-accelerated multi-physics simulation framework, in: Computational Science–ICCS 2021: 21st International Conference, Krakow, Poland, June 16–18, 2021, Proceedings, Part V, Springer. pp. 447–461.
  22. Idrlnet: A physics-informed neural network library. arXiv preprint arXiv:2107.04320 .
  23. Deep learning method based on physics informed neural network with resnet block for solving fluid flow problems. Water 13, 423.
  24. Phygeonet: Physics-informed geometry-adaptive convolutional neural networks for solving parameterized steady-state pdes on irregular domain. Journal of Computational Physics 428, 110079.
  25. Physics-informed attention-based neural network for solving non-linear partial differential equations. arXiv preprint arXiv:2105.07898 .
  26. Phygnnet: Solving spatiotemporal pdes with physics-informed graph neural network, in: Proceedings of the 2023 2nd Asia Conference on Algorithms, Computing and Machine Learning, pp. 143–147.
  27. Adaptive activation functions accelerate convergence in deep and physics-informed neural networks. Journal of Computational Physics 404, 109136.
  28. Locally adaptive activation functions with slope recovery for deep and physics-informed neural networks. Proceedings of the Royal Society A 476, 20200334.
  29. Random search and reproducibility for neural architecture search, in: Uncertainty in artificial intelligence, PMLR. pp. 367–377.
  30. Evaluating the search phase of neural architecture search, in: International Conference on Learning Representations.
  31. Regularized evolution for image classifier architecture search, in: Proceedings of the aaai conference on artificial intelligence, pp. 4780–4789.
  32. Genetic cnn, in: Proceedings of the IEEE international conference on computer vision, pp. 1379–1388.
  33. Neural architecture search with reinforcement learning, in: International Conference on Learning Representations.
  34. Blockqnn: Efficient block-wise neural network architecture generation. IEEE transactions on pattern analysis and machine intelligence 43, 2314–2328.
  35. Darts: Differentiable architecture search, in: International Conference on Learning Representations.
  36. Proxylessnas: Direct neural architecture search on target task and hardware, in: International Conference on Learning Representations.
  37. Searching for activation functions. arXiv preprint arXiv:1710.05941 .
  38. The quest for the golden activation function. arXiv preprint arXiv:1808.00783 .
  39. Evolutionary optimization of deep learning activation functions, in: Proceedings of the 2020 Genetic and Evolutionary Computation Conference, pp. 289–296.
  40. Discovering parametric activation functions. Neural Networks 148, 48–65.
  41. Autoke: An automatic knowledge embedding framework for scientific machine learning. IEEE Transactions on Artificial Intelligence .
  42. Stochastic deep collocation method based on neural architecture search and transfer learning for heterogeneous porous media. Engineering with Computers , 1–26.
  43. Hyper-parameter tuning of physics-informed neural networks: Application to helmholtz problems. Neurocomputing , 126826.
  44. Autopinn: When automl meets physics-informed neural networks. arXiv preprint arXiv:2212.04058 .
  45. Auto-pinn: Understanding and optimizing physics-informed neural architecture. arXiv preprint arXiv:2205.13748 .
  46. fpinns: Fractional physics-informed neural networks. SIAM Journal on Scientific Computing 41, A2603–A2626.
  47. Quantifying total uncertainty in physics-informed neural networks for solving forward and inverse stochastic problems. Journal of Computational Physics 397, 108850.
  48. Deep residual learning for image recognition, in: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770–778.
  49. Analysis of three-dimensional potential problems in non-homogeneous media with physics-informed deep collocation method using material transfer learning and sensitivity analysis. Engineering with Computers 38, 5423–5444.
  50. A comparison of selection schemes used in evolutionary algorithms. Evolutionary Computation 4, 361–394.
  51. Efficient evolutionary neural architecture search by modular inheritable crossover. Swarm and Evolutionary Computation 64, 100894.
  52. Evolutionary algorithm with dynamic population size for constrained multiobjective optimization. Swarm and Evolutionary Computation 73, 101104.
  53. A proposed iteration optimization approach integrating backpropagation neural network with genetic algorithm. Expert Systems with Applications 42, 146–155.
  54. Practical network blocks design with q-learning. arXiv preprint arXiv:1708.05552 6.
  55. Performance investigations of genetic algorithms on graphics cards. Swarm and Evolutionary Computation 12, 33–47.
  56. Delving deep into rectifiers: Surpassing human-level performance on imagenet classification, in: Proceedings of the IEEE international conference on computer vision, pp. 1026–1034.
  57. Enhanced physics-informed neural networks with augmented lagrangian relaxation method (al-pinns). Neurocomputing , 126424.
  58. An improved data-free surrogate model for solving partial differential equations using deep neural networks. Scientific reports 11, 19507.
  59. A class of compact finite difference schemes for solving the 2d and 3d burgers’ equations. Mathematics and Computers in Simulation 185, 510–534.
  60. Deep learning-based method coupled with small sample learning for solving partial differential equations. Multimedia Tools and Applications 80, 17391–17413.
  61. Elasticity: theory, applications, and numerics. Academic Press.
Citations (2)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now