Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
125 tokens/sec
GPT-4o
53 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Improved generalization with deep neural operators for engineering systems: Path towards digital twin (2301.06701v3)

Published 17 Jan 2023 in cs.LG, stat.AP, stat.CO, and stat.ML

Abstract: Neural Operator Networks (ONets) represent a novel advancement in machine learning algorithms, offering a robust and generalizable alternative for approximating partial differential equations (PDEs) solutions. Unlike traditional Neural Networks (NN), which directly approximate functions, ONets specialize in approximating mathematical operators, enhancing their efficacy in addressing complex PDEs. In this work, we evaluate the capabilities of Deep Operator Networks (DeepONets), an ONets implementation using a branch/trunk architecture. Three test cases are studied: a system of ODEs, a general diffusion system, and the convection/diffusion Burgers equation. It is demonstrated that DeepONets can accurately learn the solution operators, achieving prediction accuracy scores above 0.96 for the ODE and diffusion problems over the observed domain while achieving zero shot (without retraining) capability. More importantly, when evaluated on unseen scenarios (zero shot feature), the trained models exhibit excellent generalization ability. This underscores ONets vital niche for surrogate modeling and digital twin development across physical systems. While convection-diffusion poses a greater challenge, the results confirm the promise of ONets and motivate further enhancements to the DeepONet algorithm. This work represents an important step towards unlocking the potential of digital twins through robust and generalizable surrogates.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (48)
  1. Learning nonlinear operators via deeponet based on the universal approximation theorem of operators. Nature Machine Intelligence 2021 3:3, 3:218–229, 3 2021a. ISSN 2522-5839. doi:10.1038/s42256-021-00302-5. URL https://www.nature.com/articles/s42256-021-00302-5.
  2. Neural operator: Learning maps between function spaces. 8 2021. doi:10.48550/arxiv.2108.08481. URL https://arxiv.org/abs/2108.08481v4.
  3. Improved architectures and training algorithms for deep operator networks. Journal of Scientific Computing, 92:1–42, 8 2022. ISSN 15737691. doi:10.1007/S10915-022-01881-0/FIGURES/27. URL https://link.springer.com/article/10.1007/s10915-022-01881-0.
  4. Fourier neural operator with learned deformations for pdes on general geometries. 7 2022a. doi:10.48550/arxiv.2207.05209. URL https://arxiv.org/abs/2207.05209v1.
  5. Multifidelity deep neural operators for efficient learning of partial differential equations with application to fast inverse design of nanoscale heat transport. Physical Review Research, 4, 4 2022a. ISSN 26431564. doi:10.48550/arxiv.2204.06684. URL https://arxiv.org/abs/2204.06684v1.
  6. Assessment of deeponet for reliability analysis of stochastic nonlinear dynamical systems. 1 2022a. doi:10.48550/arxiv.2201.13145. URL https://arxiv.org/abs/2201.13145v1.
  7. Reliable extrapolation of deep neural operators informed by physics or sparse observations. 12 2022. doi:10.48550/arxiv.2212.06347. URL https://arxiv.org/abs/2212.06347v1.
  8. A seamless multiscale operator neural network for inferring bubble dynamics. Journal of Fluid Mechanics, 929:A18, 2021a. doi:10.1017/jfm.2021.866.
  9. Interfacing finite elements with deep neural operators for fast multiscale modeling of mechanics problems. Computer Methods in Applied Mechanics and Engineering, 402:115027, 12 2022. ISSN 0045-7825. doi:10.1016/J.CMA.2022.115027.
  10. Operator learning for predicting multiscale bubble growth dynamics. The Journal of Chemical Physics, 154:104118, 3 2021b. ISSN 0021-9606. doi:10.1063/5.0041203. URL https://aip.scitation.org/doi/abs/10.1063/5.0041203.
  11. U-fno—an enhanced fourier neural operator-based deep-learning model for multiphase flow. Advances in Water Resources, 163:104180, 5 2022. ISSN 0309-1708. doi:10.1016/J.ADVWATRES.2022.104180.
  12. Fourier neural operator approach to large eddy simulation of three-dimensional turbulence. Theoretical and Applied Mechanics Letters, 12:100389, 11 2022b. ISSN 2095-0349. doi:10.1016/J.TAML.2022.100389.
  13. The cost-accuracy trade-off in operator learning with neural networks. Journal of Machine Learning, 1:299–341, 3 2022. ISSN 2790-203X. doi:10.48550/arxiv.2203.13181. URL https://arxiv.org/abs/2203.13181v3.
  14. Structure and distribution metric for quantifying the quality of uncertainty: Assessing gaussian processes, deep neural nets, and deep neural operators for regression. 3 2022. doi:10.48550/arxiv.2203.04515. URL https://arxiv.org/abs/2203.04515v1.
  15. Fourier neural operator for parametric partial differential equations. 10 2020. doi:10.48550/arxiv.2010.08895. URL https://arxiv.org/abs/2010.08895v3.
  16. Wavelet neural operator: a neural operator for parametric partial differential equations. 5 2022. ISSN 00457825. doi:10.48550/arxiv.2205.02191. URL https://arxiv.org/abs/2205.02191v1.
  17. A general backpropagation algorithm for feedforward neural networks learning. IEEE Transactions on Neural Networks, 13:251–254, 1 2002. ISSN 10459227. doi:10.1109/72.977323.
  18. Jürgen Schmidhuber. Deep learning in neural networks: An overview. Neural Networks, 61:85–117, 1 2015. ISSN 0893-6080. doi:10.1016/J.NEUNET.2014.09.003.
  19. George A. Anastassiou. Multivariate hyperbolic tangent neural network approximation. Computers & Mathematics with Applications, 61:809–821, 2 2011. ISSN 0898-1221. doi:10.1016/J.CAMWA.2010.12.029.
  20. Approximation of a function and its derivative with a neural network. Neural Networks, 5:207–220, 1 1992. ISSN 0893-6080. doi:10.1016/S0893-6080(05)80020-6.
  21. Neural-network approximation of piecewise continuous functions: Application to friction compensation. IEEE Transactions on Neural Networks, 13:745–751, 5 2002. ISSN 10459227. doi:10.1109/TNN.2002.1000141.
  22. Deep neural networks motivated by partial differential equations. Journal of Mathematical Imaging and Vision, 62:352–364, 2020.
  23. Fourier neural operator networks: A fast and general solver for the photoacoustic wave equation. page 22102, 8 2021. doi:10.48550/arxiv.2108.09374. URL https://arxiv.org/abs/2108.09374v1.
  24. Multi-fidelity wavelet neural operator with application to uncertainty quantification. 8 2022. doi:10.48550/arxiv.2208.05606. URL https://arxiv.org/abs/2208.05606v1.
  25. Physics-informed deep neural operator networks. 7 2022. doi:10.48550/arxiv.2207.05748. URL https://arxiv.org/abs/2207.05748v2.
  26. A comprehensive and fair comparison of two neural operators (with practical extensions) based on fair data. Computer Methods in Applied Mechanics and Engineering, 393:114778, 4 2022b. ISSN 0045-7825. doi:10.1016/J.CMA.2022.114778.
  27. The general approximation theorem. pages 1271–1274, 11 2002. doi:10.1109/IJCNN.1998.685957.
  28. Universal approximation to nonlinear operators by neural networks with arbitrary activation functions and its application to dynamical systems. IEEE Transactions on Neural Networks, 6:911–917, 1995. ISSN 19410093. doi:10.1109/72.392253.
  29. Learning deep implicit fourier neural operators (ifnos) with applications to heterogeneous material modeling. Computer Methods in Applied Mechanics and Engineering, 398, 3 2022. doi:10.1016/j.cma.2022.115296. URL http://arxiv.org/abs/2203.08205http://dx.doi.org/10.1016/j.cma.2022.115296.
  30. Exponential convergence of deep operator networks for elliptic partial differential equations. 12 2021. doi:10.48550/arxiv.2112.08125. URL https://arxiv.org/abs/2112.08125v2.
  31. DeepXDE: A deep learning library for solving differential equations. SIAM Review, 63(1):208–228, 2021b. doi:10.1137/19M1274067.
  32. Discovering and forecasting extreme events via active learning in neural operators. Nature Computational Science 2022 2:12, 2:823–833, 12 2022. ISSN 2662-8457. doi:10.1038/s43588-022-00376-0. URL https://www.nature.com/articles/s43588-022-00376-0.
  33. Physics-informed neural operator for learning partial differential equations. 11 2021. doi:10.48550/arxiv.2111.03794. URL https://arxiv.org/abs/2111.03794v2.
  34. Development of a digital twin operational platform using python flask. Data-Centric Engineering, 3, 2022.
  35. Hussain Mohammed Dipu Kabir. Non-linear down-sampling and signal reconstruction, without folding. In 2010 Fourth UKSim European Symposium on Computer Modeling and Simulation, pages 142–146. IEEE, 2010a.
  36. Hussain Mohammed Dipu Kabir. A theory of loss-less compression of high quality speech signals with comparison. In 2010 Fourth UKSim European Symposium on Computer Modeling and Simulation, pages 136–141. IEEE, 2010b.
  37. Hussain Mohammed Dipu Kabir et al. Watermarking with fast and highly secured encryption for real-time speech signals. In 2010 IEEE International Conference on Information Theory and Information Security, pages 446–451. IEEE, 2010a.
  38. Hussain Mohammed Dipu Kabir et al. A loss-less compression technique for high quality speech signals and its implementation with mpeg-4 als for better compression. In 2010 IEEE International Conference on Information Theory and Information Security, pages 781–785. IEEE, 2010b.
  39. Surrogate modeling-driven physics-informed multi-fidelity kriging: Path forward to digital twin enabling simulation for accident tolerant fuel. Springer Nature Handbook of Smart Energy Systems, arXiv preprint arXiv:2210.07164, 2022a.
  40. Data-driven multi-scale modeling and robust optimization of composite structure with uncertainty quantification. Springer Nature Handbook of Smart Energy Systems, arXiv preprint arXiv:2210.09055, 2022b.
  41. Explainable, interpretable & trustworthy ai for intelligent digital twin: Case study on remaining useful life. 2023.
  42. Physics-integrated hybrid framework for model form error identification in nonlinear dynamical systems. Mechanical Systems and Signal Processing, 173:109039, 2022b.
  43. Elham Tabassi. Artificial intelligence risk management framework (ai rmf 1.0). 2023.
  44. Aijun Zhang. Machine learning model validation. part 1: Machine learning interpretability. In QU-ML Model Validation Workshop, Session 1, pages 1–35. Springer, June 29, 2022.
  45. Aijun Zhang. Machine learning model validation. part 2: Model diagnostics and validation. In QU-ML Model Validation Workshop, Session 2, pages 1–35. Springer, July 6, 2022.
  46. Explainable, interpretable, and trustworthy ai for an intelligent digital twin: A case study on remaining useful life. Engineering Applications of Artificial Intelligence, 2023.
  47. Physics-informed multi-stage deep learning framework development for digital twin-centred state-based reactor power prediction. arXiv preprint arXiv:2211.13157, 2022.
  48. Leveraging industry 4.0–deep learning, surrogate model and transfer learning with uncertainty quantification incorporated into digital twin for nuclear system. Springer Nature Handbook of Smart Energy Systems, arXiv preprint arXiv:2210.00074, 2022.
Citations (16)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets