Papers
Topics
Authors
Recent
2000 character limit reached

Physics-Aware Neural Implicit Solvers for multiscale, parametric PDEs with applications in heterogeneous media (2405.19019v1)

Published 29 May 2024 in stat.ML and cs.LG

Abstract: We propose Physics-Aware Neural Implicit Solvers (PANIS), a novel, data-driven framework for learning surrogates for parametrized Partial Differential Equations (PDEs). It consists of a probabilistic, learning objective in which weighted residuals are used to probe the PDE and provide a source of {\em virtual} data i.e. the actual PDE never needs to be solved. This is combined with a physics-aware implicit solver that consists of a much coarser, discretized version of the original PDE, which provides the requisite information bottleneck for high-dimensional problems and enables generalization in out-of-distribution settings (e.g. different boundary conditions). We demonstrate its capability in the context of random heterogeneous materials where the input parameters represent the material microstructure. We extend the framework to multiscale problems and show that a surrogate can be learned for the effective (homogenized) solution without ever solving the reference problem. We further demonstrate how the proposed framework can accommodate and generalize several existing learning objectives and architectures while yielding probabilistic surrogates that can quantify predictive uncertainty.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (89)
  1. Random heterogeneous materials: microstructure and macroscopic properties. Appl. Mech. Rev., 55(4):B62–B63, 2002.
  2. Random Material Property Fields of 3D Concrete Microstructures Based on CT Image Reconstruction. Materials, 14(6):1423, January 2021. Number: 6 Publisher: Multidisciplinary Digital Publishing Institute.
  3. A digital workflow for learning the reduced-order structure-property linkages for permeability of porous membranes. Acta Materialia, 195:668–680, August 2020.
  4. Key computational modeling issues in integrated computational materials engineering. Computer-Aided Design, 45(1):4–25, 2013.
  5. Systems Approaches to Materials Design: Past, Present, and Future. Annual Review of Materials Research, 49(1):103–126, 2019. _eprint: https://doi.org/10.1146/annurev-matsci-070218-125955.
  6. Fast inverse design of microstructures via generative invariance networks. Nature Computational Science, 1(3):229–238, March 2021.
  7. Perspective: Materials informatics and big data: Realization of the “fourth paradigm” of science in materials science. APL Materials, 4(5):053208, May 2016. Publisher: American Institute of Physics.
  8. Surya R Kalidindi. Hierarchical materials informatics: novel analytics for materials data. Elsevier, 2015.
  9. The high-throughput highway to computational materials design. Nature Materials, 12(3):191–201, March 2013. Number: 3 Publisher: Nature Publishing Group.
  10. Establishing structure-property localization linkages for elastic deformation of three-dimensional high contrast composites using deep learning approaches. Acta Materialia, 166:335–345, March 2019.
  11. Deeponet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators. arXiv preprint arXiv:1910.03193, 2019.
  12. Neural operator: Graph kernel network for partial differential equations. arXiv preprint arXiv:2003.03485, 2020.
  13. Fourier neural operator for parametric partial differential equations. arXiv preprint arXiv:2010.08895, 2020.
  14. Learning deep Implicit Fourier Neural Operators (IFNOs) with applications to heterogeneous material modeling. Computer Methods in Applied Mechanics and Engineering, 398:115296, August 2022.
  15. Physics-informed machine learning. Nature Reviews Physics, 3(6):422–440, 2021.
  16. Special Issue: Big data and predictive computational modeling. Journal of Computational Physics, 321:1252–1254, September 2016. Publisher: Elsevier BV.
  17. Physics informed deep learning (part i): Data-driven solutions of nonlinear partial differential equations. arXiv preprint arXiv:1711.10561, 2017.
  18. Learning the solution operator of parametric partial differential equations with physics-informed deeponets. Science advances, 7(40):eabi8605, 2021.
  19. Random grid neural processes for parametric partial differential equations. In International Conference on Machine Learning, pages 34759–34778. PMLR, 2023.
  20. Physics-constrained deep learning for high-dimensional surrogate modeling and uncertainty quantification without labeled data. Journal of Computational Physics, 394:56–81, 2019.
  21. A probabilistic generative model for semi-supervised training of coarse-grained surrogates and enforcing physical constraints through virtual observables. Journal of Computational Physics, 434:110218, 2021.
  22. Adversarial uncertainty quantification in physics-informed neural networks. Journal of Computational Physics, 394:136–152, 2019.
  23. Incorporating physical constraints in a deep probabilistic machine learning framework for coarse-graining dynamical systems. Journal of Computational Physics, 419:109673, 2020.
  24. Hidden physics models: Machine learning of nonlinear partial differential equations. Journal of Computational Physics, 357:125–141, 2018.
  25. Bing Yu et al. The deep ritz method: a deep learning-based numerical algorithm for solving variational problems. Communications in Mathematics and Statistics, 6(1):1–12, 2018.
  26. Knowledge integration into deep learning in dynamical systems: an overview and taxonomy. Journal of Mechanical Science and Technology, 35:1331–1342, 2021.
  27. Multi-output local gaussian process regression: Applications to uncertainty quantification. Journal of Computational Physics, 231(17):5718–5746, 2012.
  28. Reduced basis methods for partial differential equations: an introduction, volume 92. Springer, 2015.
  29. Certified reduced basis methods for parametrized partial differential equations, volume 590. Springer, 2016.
  30. Bernard Haasdonk. Reduced basis methods for parametrized pdes–a tutorial introduction for stationary and instationary problems. Model reduction and approximation: theory and algorithms, 15:65, 2017.
  31. Reduced-order models for microstructure-sensitive effective thermal conductivity of woven ceramic matrix composites with residual porosity. Composite Structures, 274:114399, October 2021.
  32. Feature engineering for microstructure-property mapping in organic photovoltaics. arXiv:2111.01897 [cond-mat], November 2021. arXiv: 2111.01897.
  33. Surya R. Kalidindi. Feature engineering of material structure for AI-based materials knowledge systems. Journal of Applied Physics, 128(4):041103, July 2020. Publisher: American Institute of Physics.
  34. A multiscale approach for model reduction of random microstructures. Computational Materials Science, 63:269–285, October 2012.
  35. Deep Learning. MIT Press, 2016. http://www.deeplearningbook.org.
  36. Solving high-dimensional partial differential equations using deep learning. Proceedings of the National Academy of Sciences, 115(34):8505–8510, 2018.
  37. Dgm: A deep learning algorithm for solving partial differential equations. Journal of computational physics, 375:1339–1364, 2018.
  38. Deep residual learning and pdes on manifold. arXiv preprint arXiv:1708.05115, 2017.
  39. Conditional deep surrogate models for stochastic, high-dimensional, and multi-fidelity systems. Computational Mechanics, 64:417–434, 2019.
  40. Bayesian deep convolutional encoder–decoder networks for surrogate modeling and uncertainty quantification. Journal of Computational Physics, 366:415–447, 2018.
  41. Deep convolutional encoder-decoder networks for uncertainty quantification of dynamic multiphase flow in heterogeneous media. Water Resources Research, 55(1):703–728, 2019.
  42. Neural operator: Learning maps between function spaces. arXiv preprint arXiv:2108.08481, 2021.
  43. Physics-informed neural operator for learning partial differential equations. arXiv preprint arXiv:2111.03794, 2021.
  44. Wavelet neural operator: a neural operator for parametric partial differential equations. arXiv preprint arXiv:2205.02191, 2022.
  45. Convolutional neural operators for robust and accurate learning of pdes. Advances in Neural Information Processing Systems, 36, 2024.
  46. Spectral Neural Operators. Doklady Mathematics, 108(2):S226–S232, December 2023.
  47. Discovering Symbolic Models from Deep Learning with Inductive Biases. arXiv:2006.11287 [astro-ph, physics:physics, stat], June 2020.
  48. Equivariant flows: exact likelihood generative learning for symmetric densities. In International conference on machine learning, pages 5361–5370. PMLR, 2020.
  49. A physics-aware, probabilistic machine learning framework for coarse-graining high-dimensional systems in the Small Data regime. Journal of Computational Physics, 397:108842, November 2019.
  50. Variational bayes deep operator network: A data-driven bayesian solver for parametric differential equations. arXiv preprint arXiv:2206.05655, 2022.
  51. Fully probabilistic deep models for forward and inverse problems in parametric pdes. Journal of Computational Physics, 491:112369, 2023.
  52. Efficient quantification of composite spatial variability: A multiscale framework that captures intercorrelation. Composite Structures, 323:117462, November 2023.
  53. BA Finlayson, editor. The method of weighted residuals and variational principles, with application in fluid mechanics, heat and mass transfer, Volume 87. Academic Press, New York, February 1972.
  54. hp-VPINNs: Variational physics-informed neural networks with domain decomposition. Computer Methods in Applied Mechanics and Engineering, 374:113547, February 2021.
  55. Variational bayesian inference with stochastic search. arXiv preprint arXiv:1206.6430, 2012.
  56. Variational inference: A review for statisticians. Journal of the American statistical Association, 112(518):859–877, 2017.
  57. Error estimates for physics-informed neural networks approximating the navier–stokes equations. IMA Journal of Numerical Analysis, 44(1):83–119, 2024.
  58. Characterizing possible failure modes in physics-informed neural networks. Advances in Neural Information Processing Systems, 34:26548–26560, 2021.
  59. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational physics, 378:686–707, 2019.
  60. A deep neural network surrogate for high-dimensional random partial differential equations. arXiv preprint arXiv:1806.02957, 2018.
  61. Quasi-monte carlo integration. Journal of computational physics, 122(2):218–230, 1995.
  62. Weak adversarial networks for high-dimensional partial differential equations. Journal of Computational Physics, 411:109409, 2020.
  63. Combining differentiable pde solvers and graph neural networks for fluid flow prediction. In ICML, pages 2402–2411, 2020.
  64. Solver-in-the-loop: Learning from differentiable physics to interact with iterative pde-solvers. Advances in Neural Information Processing Systems, 33:6111–6122, 2020.
  65. Automatic differentiation of algorithms. Journal of Computational and Applied Mathematics, 124(1-2):171–190, 2000.
  66. Automatic differentiation in machine learning: a survey. Journal of machine learning research, 18(153):1–43, 2018.
  67. Antony Jameson. Aerodynamic shape optimization using the adjoint method. Lectures at the Von Karman Institute, Brussels, 2003.
  68. Continuous adjoint methods for turbulent flows, applied to shape and topology optimization: industrial applications. Archives of Computational Methods in Engineering, 23(2):255–299, 2016.
  69. Stochastic variational inference. Journal of Machine Learning Research, 2013.
  70. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
  71. Recent advances in stochastic gradient descent in deep learning. Mathematics, 11(3):682, 2023.
  72. Pushing stochastic gradient towards second-order methods–backpropagation learning with transformations in nonlinearities. In Neural Information Processing: 20th International Conference, ICONIP 2013, Daegu, Korea, November 3-7, 2013. Proceedings, Part I 20, pages 442–449. Springer, 2013.
  73. Variational dropout and the local reparameterization trick. Advances in neural information processing systems, 28, 2015.
  74. Dabao Zhang. A coefficient of determination for generalized linear models. The American Statistician, 71(4):310–316, 2017.
  75. A comprehensive and fair comparison of two neural operators (with practical extensions) based on fair data. Computer Methods in Applied Mechanics and Engineering, 393:114778, 2022.
  76. E Weinan. Principles of multiscale modeling. Cambridge University Press, 2011.
  77. Multiscale methods: averaging and homogenization. Springer Science & Business Media, 2008.
  78. Homogenization techniques for composite media. Homogenization techniques for composite media, 272, 1987.
  79. Homogenization methods and multiscale modeling: nonlinear problems. Encyclopedia of computational mechanics second edition, pages 1–34, 2017.
  80. GFINNs: GENERIC Formalism Informed Neural Networks for Deterministic and Stochastic Dynamical Systems. arXiv:2109.00092 [cs, math], August 2021. arXiv: 2109.00092.
  81. Thermodynamics-informed graph neural networks. arXiv preprint arXiv:2203.01874, 2022.
  82. Multiscale modeling of inelastic materials with thermodynamics-based artificial neural networks (tann). Computer Methods in Applied Mechanics and Engineering, 398:115190, 2022.
  83. Thermodynamics of learning physical phenomena. Archives of Computational Methods in Engineering, 30(8):4653–4666, 2023.
  84. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems, 32, 2019.
  85. The FEniCS project version 1.5. Archive of Numerical Software, 3, 2015.
  86. Model reduction and neural networks for parametric pdes. The SMAI journal of computational mathematics, 7:121–157, 2021.
  87. 4D Remeshing Using a Space-Time Finite Element Method for Elastodynamics Problems. Mathematical and Computational Applications, 23(2):29, June 2018. Number: 2 Publisher: Multidisciplinary Digital Publishing Institute.
  88. Self-supervised optimization of random material microstructures in the small-data regime. npj Computational Materials, 8(1):1–11, March 2022. Number: 1 Publisher: Nature Publishing Group.
  89. Understanding the difficulty of training deep feedforward neural networks. In Proceedings of the thirteenth international conference on artificial intelligence and statistics, pages 249–256. JMLR Workshop and Conference Proceedings, 2010.
Citations (1)

Summary

We haven't generated a summary for this paper yet.

Whiteboard

Paper to Video (Beta)

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Continue Learning

We haven't generated follow-up questions for this paper yet.

Collections

Sign up for free to add this paper to one or more collections.

Tweets

Sign up for free to view the 2 tweets with 2 likes about this paper.