Papers
Topics
Authors
Recent
Search
2000 character limit reached

GradINN: Gradient Informed Neural Network

Published 3 Sep 2024 in cs.LG and cs.AI | (2409.01914v1)

Abstract: We propose Gradient Informed Neural Networks (GradINNs), a methodology inspired by Physics Informed Neural Networks (PINNs) that can be used to efficiently approximate a wide range of physical systems for which the underlying governing equations are completely unknown or cannot be defined, a condition that is often met in complex engineering problems. GradINNs leverage prior beliefs about a system's gradient to constrain the predicted function's gradient across all input dimensions. This is achieved using two neural networks: one modeling the target function and an auxiliary network expressing prior beliefs, e.g., smoothness. A customized loss function enables training the first network while enforcing gradient constraints derived from the auxiliary network. We demonstrate the advantages of GradINNs, particularly in low-data regimes, on diverse problems spanning non time-dependent systems (Friedman function, Stokes Flow) and time-dependent systems (Lotka-Volterra, Burger's equation). Experimental results showcase strong performance compared to standard neural networks and PINN-like approaches across all tested scenarios.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (29)
  1. Spectral and finite difference solutions of the burgers equation. Computers & Fluids, 14(1):23–41, 1986. ISSN 0045-7930. doi: https://doi.org/10.1016/0045-7930(86)90036-8. URL https://www.sciencedirect.com/science/article/pii/0045793086900368.
  2. Automatic differentiation in machine learning: A survey. Journal of Machine Learning Research, 18:1–43, 04 2018.
  3. Alan A. Berryman. The orgins and evolution of predator-prey theory. Ecology, 73(5):1530–1535, 1992. ISSN 00129658, 19399170. URL http://www.jstor.org/stable/1940005.
  4. Physics-informed neural networks (pinns) for fluid mechanics: A review. Acta Mechanica Sinica, 37(12):1727–1738, 2021a.
  5. Physics-Informed Neural Networks for Heat Transfer Problems. Journal of Heat Transfer, 143(6):060801, 04 2021b. ISSN 0022-1481. doi: 10.1115/1.4050542. URL https://doi.org/10.1115/1.4050542.
  6. Comparative study of physics-based modeling and neural network approach to predict cooling in vehicle integrated thermal management system. Energies, 13(20), 2020. ISSN 1996-1073. doi: 10.3390/en13205301. URL https://www.mdpi.com/1996-1073/13/20/5301.
  7. Sobolev training for neural networks. CoRR, abs/1706.04859, 2017. URL http://arxiv.org/abs/1706.04859.
  8. Jerome H Friedman. Multivariate adaptive regression splines. The annals of statistics, 19(1):1–67, 1991.
  9. Understanding the difficulty of training deep feedforward neural networks. Journal of Machine Learning Research, 9:249–256, 2010. ISSN 15324435.
  10. Hamiltonian neural networks, 2019.
  11. Multilayer feedforward networks are universal approximators. Neural Networks, 2(5):359–366, 1989. ISSN 0893-6080. doi: https://doi.org/10.1016/0893-6080(89)90020-8. URL https://www.sciencedirect.com/science/article/pii/0893608089900208.
  12. Applications of physics-informed neural networks in power systems - a review. IEEE Transactions on Power Systems, 38(1):572–588, 2023. doi: 10.1109/TPWRS.2022.3162473.
  13. Universal approximation property of invertible neural networks. Journal of Machine Learning Research, 24(287):1–68, 2023. URL http://jmlr.org/papers/v24/22-0384.html.
  14. Physics-informed machine learning. pages 1–19, 05 2021. doi: 10.1038/s42254-021-00314-5.
  15. Adam: A method for stochastic optimization. In Yoshua Bengio and Yann LeCun, editors, 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings, 2015. URL http://arxiv.org/abs/1412.6980.
  16. Physics-informed neural networks for high-speed flows. Computer Methods in Applied Mechanics and Engineering, 360:112789, 2020. ISSN 0045-7825. doi: https://doi.org/10.1016/j.cma.2019.112789. URL https://www.sciencedirect.com/science/article/pii/S0045782519306814.
  17. Hamiltonian neural networks for solving equations of motion. Physical Review E, 105(6), June 2022. ISSN 2470-0053. doi: 10.1103/physreve.105.065305. URL http://dx.doi.org/10.1103/PhysRevE.105.065305.
  18. Miquel Noguer i Alonso and Daniel Maxwell. Physics-informed neural networks (pinns) in finance. Daniel, Physics-Informed Neural Networks (PINNs) in Finance (October 10, 2023), 2023.
  19. S. Oreški. Comparison of neural network and empirical models for prediction of second virial coefficients for gases. Procedia Engineering, 42:303–312, 2012. ISSN 1877-7058. doi: https://doi.org/10.1016/j.proeng.2012.07.421. URL https://www.sciencedirect.com/science/article/pii/S1877705812028214. CHISA 2012.
  20. Physics-informed neural networks with unknown measurement noise, 2023.
  21. Allan Pinkus. Approximation theory of the mlp model in neural networks. Acta Numerica, 8:143–195, 1999. doi: 10.1017/S0962492900002919.
  22. Universal physics-informed neural networks: symbolic differential operator discovery with sparse data. In Proceedings of the 40th International Conference on Machine Learning, ICML’23. JMLR.org, 2023.
  23. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational Physics, 378:686–707, 2019. ISSN 0021-9991. doi: https://doi.org/10.1016/j.jcp.2018.10.045. URL https://www.sciencedirect.com/science/article/pii/S0021999118307125.
  24. Maziar Raissi. Deep hidden physics models: Deep learning of nonlinear partial differential equations. Journal of Machine Learning Research, 19, 01 2018.
  25. Junuthula Narasimha Reddy. An introduction to the finite element method, volume 3. McGraw-Hill New York, 2013.
  26. A brief review on artificial neural network: Network structures and applications. In 2023 9th International Conference on Advanced Computing and Communication Systems (ICACCS), volume 1, pages 1974–1979, 2023. doi: 10.1109/ICACCS57279.2023.10112753.
  27. Sobolev training for physics informed neural networks. arXiv preprint arXiv:2101.08932, 2021.
  28. Sir George Gabriel Stokes. On the effect of the internal friction of fluids on the motion of pendulums. From the Transactions of the Cambridge Philosophical Society, Vol. IX. p. [8], 18, 12 1850.
  29. Gradient-enhanced physics-informed neural networks for forward and inverse pde problems. Computer Methods in Applied Mechanics and Engineering, 393:114823, 2022. ISSN 0045-7825. doi: https://doi.org/10.1016/j.cma.2022.114823. URL https://www.sciencedirect.com/science/article/pii/S0045782522001438.

Summary

No one has generated a summary of this paper yet.

Sign Up to Summarize

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Generate Now

Collections

Sign up for free to add this paper to one or more collections.

Sign Up