Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
167 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

A Method for Computing Inverse Parametric PDE Problems with Random-Weight Neural Networks (2210.04338v1)

Published 9 Oct 2022 in math.NA, cs.LG, cs.NA, physics.comp-ph, and physics.flu-dyn

Abstract: We present a method for computing the inverse parameters and the solution field to inverse parametric PDEs based on randomized neural networks. This extends the local extreme learning machine technique originally developed for forward PDEs to inverse problems. We develop three algorithms for training the neural network to solve the inverse PDE problem. The first algorithm (NLLSQ) determines the inverse parameters and the trainable network parameters all together by the nonlinear least squares method with perturbations (NLLSQ-perturb). The second algorithm (VarPro-F1) eliminates the inverse parameters from the overall problem by variable projection to attain a reduced problem about the trainable network parameters only. It solves the reduced problem first by the NLLSQ-perturb algorithm for the trainable network parameters, and then computes the inverse parameters by the linear least squares method. The third algorithm (VarPro-F2) eliminates the trainable network parameters from the overall problem by variable projection to attain a reduced problem about the inverse parameters only. It solves the reduced problem for the inverse parameters first, and then computes the trainable network parameters afterwards. VarPro-F1 and VarPro-F2 are reciprocal to each other in a sense. The presented method produces accurate results for inverse PDE problems, as shown by the numerical examples herein. For noise-free data, the errors for the inverse parameters and the solution field decrease exponentially as the number of collocation points or the number of trainable network parameters increases, and can reach a level close to the machine accuracy. For noisy data, the accuracy degrades compared with the case of noise-free data, but the method remains quite accurate. The presented method has been compared with the physics-informed neural network method.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (60)
  1. J. Berg and K. Nystrom. Neural network augmented inverse problems for PDEs. arXiv:1712.09685, 2018.
  2. J. Berg and K. Nystrom. Data-driven discovery of PDEs in complex datasets. Journal of Computational Physics, 384:239–252, 2019.
  3. A. Bjorck. Numerical Methods for Least Squares Problems. SIAM, 1996.
  4. J. Bongard and H. Lipton. Automated reverse engineering of nonlinear dynamical systems. Proceedings of National Academy of Sciences USA, 104:9943–9948, 2007.
  5. DeepMoD: deep learning for model discovery in noisy data. Journal of Computational Physics, 428:109985, 2021.
  6. A subspace, interior, and conjugate gradient method for large-scale bound-constrained minimization problems. SIAM Journal on Scientific Computing, 21:1–23, 1999.
  7. Discovering governing equations from data by sparse identification of nonlinear dynamical systems. Proceedings of National Academy of Sciences USA, 113:3932–3937, 2016.
  8. Approximate solution of the trust region problem by minimization over two-dimensional subspaces. Math. Programming, 40:247–263, 1988.
  9. Flow over an espresso cup: inferring 3-d velocity and pressure fields from tomographic background oriented schileren via physics-informed neural networks. Journal of Fluid Mechanics, 915:A102, 2021.
  10. Extreme learning machine collocation for the numerical solution of elliptic PDEs with sharp gradients. Computer Methods in Applied Mechanics and Engineering, 387:114188, 2021.
  11. Physics-informed neural networks for inverse problems in nano-optics and metamaterials. Optics Express, 28:11618–11633, 2020.
  12. S. Dong. An efficient algorithm for incompressible N-phase flows. Journal of Computational Physics, 276:691–728, 2014.
  13. S. Dong. Physical formulation and numerical algorithm for simulating N immiscible incompressible fluids involving general order parameters. Journal of Computational Physics, 283:98–128, 2015.
  14. S. Dong. Wall-bounded multiphase flows of N immiscible incompressible fluids: consistency and contact-angle boundary condition. Journal of Computational Physics, 338:21–67, 2017.
  15. S. Dong. Multiphase flows of N immiscible incompressible fluids: a reduction-consistent and thermodynamically-consistent formulation and associated algorithm. Journal of Computational Physics, 361:1–49, 2018.
  16. S. Dong and Z. Li. Local extreme learning machines and domain decomposition for solving linear and nonlinear partial differential equations. Computer Methods in Applied Mechanics and Engineering, 387:114129, 2021. (also arXiv:2012.02895).
  17. S. Dong and Z. Li. A modified batch intrinsic plascity method for pre-training the random coefficients of extreme learning machines. Journal of Computational Physics, 445:110585, 2021. (also arXiv:2103.08042).
  18. S. Dong and J. Yang. Numerical approximation of partial differential equations by a variable projection method with artificial neural networks. Computer Methods in Applued Mechanics and Engineering, 398:115284, 2022. (also arXiv:2201.09989).
  19. S. Dong and J. Yang. On computing the hyperparameter of extreme learning machines: algorithms and applications to computational PDEs, and comparison with classical and high-order finite elements. Journal of Computational Physics, 463:111290, 2022. (also arXiv:2110.14121).
  20. Distributed learning machines for solving forward and inverse problems in partial differential equations. Neurocomputing, 420:299–316, 2021.
  21. V. Dwivedi and B. Srinivasan. Physics informed extreme learning machine (pielm) −-- a rapid method for the numerical solution of partial differential equations. Neurocomputing, 391:96–118, 2020.
  22. Numerical solution and bifurcation analysis of nonlinear partial differential equations with extreme learning machines. Journal of Scientific Computing, 89:44, 2021.
  23. G.H. Golub and V. Pereyra. The differentiation of pseudo-inverse and nonlinear least squares problems whose variables separate. SIAM J. Numer. Anal., 10:413–432, 1973.
  24. G.H. Golub and V. Pereyra. Separable nonlinear least squares: the variable projection method and its applications. Inverse Problems, 19:R1–R26, 2003.
  25. Extreme learning machine: theory and applications. Neurocomputing, 70:489–501, 2006.
  26. Universal approximation using incremental constructive feedforward networks with random hidden nodes. IEEE Transactions on Neural Networks, 17:879–892, 2006.
  27. B. Igelnik and Y.H. Pao. Stochastic choice of basis functions in adaptive function approximation and the functional-link net. IEEE Transactions on Neural Networks, 6:1320–1329, 1995.
  28. Extended physics-informed neural network (XPINNs): A generalized space-time domain decomposition based deep learning framework for nonlinear partial differential equations. Communications in Computational Physics, 28:2002–2041, 2020.
  29. Conservative physics-informed neural networks on discrete domains for conservation laws: applications to forward and inverse problems. Computer Methods in Applied Mechanics and Engineering, 365:113028, 2020.
  30. Physics-informed neural networks for inverse problems in supersonic flows. Journal of Computational Physics, 466:111402, 2022.
  31. Physics-informed machine learning. Nature Reviews Physics, 3:422–440, 2021.
  32. L. Kaufman. A variable projection method for solving separable nonlinear least squares problems. BIT, 15:49–57, 1975.
  33. Coupled time-lapse full-waveform inversion for subsurface flow problems using intrusive automatic differentiation. Water Resour. Res., 56:e2019WR027032, 2020.
  34. PDE-Net 2.0: Learning PDEs from data with a numeric-symbolic hybrid deep network. Journal of Computational Physics, 399:108925, 2019.
  35. DeepXDE: a deep learning library for solving differential equations. SIAM Review, 63:208–228, 2021.
  36. Physics-informed neural networks for high-speed flows. Computer Methods in Applied Mechanics and Engineering, 360:112789, 2020.
  37. Uncovering edge plasma dynamics via deep learning from partial observations. arXiv:2009.05005, 2020.
  38. X. Meng and G.E. Karniadakis. A composite neural network that learns form multi-fidelity data: application to function approximation and inverse pde problems. Journal of Computational Physics, 401:109020, 2020.
  39. Random vector functional link networks for function approximation on manifolds. arXiv:2007.15776, 2020.
  40. A Bayesian approach for data-driven dynamic equation discovery. Journal of Agricultural, Biological and Environmental Statistics, early access, 2022. https://doi.org/10.1007/s13253-022-00514-1.
  41. S. Panghal and M. Kumar. Optimization free neural network approach for solving ordinary and partial differential equations. Engineering with Computers, 37:2989–3002, 2021.
  42. Learning and generalization characteristics of the random vector functional-link net. Neurocomputing, 6:163–180, 1994.
  43. Thermodynamically consistent physics-informed neural networks for hyperbolic systems. Journal of Computational Physics, 449:110754, 2022.
  44. M. Raissi and G.E. Karniadakis. Hidden physics models: machine learning of nonlinear partial differential equations. Journal of Computational Physics, 357:125–141, 2018.
  45. Physics-informed neural networks: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational Physics, 378:686–707, 2019.
  46. Hidden fluid mechanics: learning velocity and pressure fields from flow visualizations. Science, 367:1026–1030, 2020.
  47. Data-driven identification of parametric partial differential equations. SIAM J. Applied Dynamical Systems, 18:643–660, 2019.
  48. Data-driven discovery of partial differential equations. Science Advances, 3:e1602614, 2017.
  49. H. Schaeffer. Learning partial differential equations via data discovery and sparse optimization. Proc. R. Soc. A, 473:20160446, 2017.
  50. Physics-informed extreme theory of functional connections applied to data-driven parameters discovery of epidemiological compartmental models. arXiv:2008.05554, 2020.
  51. Extreme theory of functional connections: a fast physics-informed neural network method for solving ordinary and partial differential equations. Neurocomputing, 457:334–356, 2021.
  52. M. Schmidt and H. Lipton. Distilling free-form natural laws from experimental data. Science, 324:81–85, 2009.
  53. Physics-informed deep neural networks for learning parameters and constitutive relationships in subsurface flow problems. Water Resource Research, 56:e2019WR026731, 2020.
  54. K. Wu and D. Xiu. Data-driven deep learning of partial differential equations in modal space. Journal of Computational Physics, 408:109307, 2020.
  55. Deep-learning of parametric partial differential equations from sparse and noisy data. Physics of Fluids, 33:037132, 2021.
  56. B-PINNs: Bayesian physics-informed neural netwotks for forward and inverse pde problems with noisy data. Journal of Computational Physics, 425:109913, 2021.
  57. A novel improved extreme learning machine algorithm in solving ordinary differential equations by legendre neural network methods. Advances in Differential Equations, 469:1–24, 2018.
  58. Z. Yang and S. Dong. Multiphase flows of N immiscible incompressible fluids: an outflow/open boundary condition and algorithm. Journal of Computational Physics, 366:33–70, 2018.
  59. A-PINN: auxiliary physics informed neural networks for forward and inverse problems of nonlinear integro-differential equations. Journal of Computational Physics, 462:111260, 2022.
  60. S. Zhang and G. Lin. Robust data-driven discovery of governing physical laws with error bars. Prov. R. Soc. A, 474:20180305, 2018.
Citations (14)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now