$\textit{FastSVD-ML-ROM}$: A Reduced-Order Modeling Framework based on Machine Learning for Real-Time Applications (2207.11842v1)
Abstract: Digital twins have emerged as a key technology for optimizing the performance of engineering products and systems. High-fidelity numerical simulations constitute the backbone of engineering design, providing an accurate insight into the performance of complex systems. However, large-scale, dynamic, non-linear models require significant computational resources and are prohibitive for real-time digital twin applications. To this end, reduced order models (ROMs) are employed, to approximate the high-fidelity solutions while accurately capturing the dominant aspects of the physical behavior. The present work proposes a new ML platform for the development of ROMs, to handle large-scale numerical problems dealing with transient nonlinear partial differential equations. Our framework, mentioned as $\textit{FastSVD-ML-ROM}$, utilizes $\textit{(i)}$ a singular value decomposition (SVD) update methodology, to compute a linear subspace of the multi-fidelity solutions during the simulation process, $\textit{(ii)}$ convolutional autoencoders for nonlinear dimensionality reduction, $\textit{(iii)}$ feed-forward neural networks to map the input parameters to the latent spaces, and $\textit{(iv)}$ long short-term memory networks to predict and forecast the dynamics of parametric solutions. The efficiency of the $\textit{FastSVD-ML-ROM}$ framework is demonstrated for a 2D linear convection-diffusion equation, the problem of fluid around a cylinder, and the 3D blood flow inside an arterial segment. The accuracy of the reconstructed results demonstrates the robustness and assesses the efficiency of the proposed approach.
- Hypersonic fluid–structure interactions in compression corner shock-wave/boundary-layer interaction, AIAA journal 58 (2020) 4090–4105.
- X. Hu, J. Du, Robust nonlinear control design for dynamic positioning of marine vessels with thruster system dynamics, Nonlinear Dynamics 94 (2018) 365–376.
- Coupled fea model with continuum damage mechanics for the degradation of polymer-based coatings on drug-eluting stents, in: 2021 43rd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), IEEE, 2021, pp. 4319–4323.
- Hemodynamics of anterior circulation intracranial aneurysms with daughter blebs: investigating the multidirectionality of blood flow fields, Computer Methods in Biomechanics and Biomedical Engineering (2022) 1–13.
- Immersed boundary finite element method for blood flow simulation, Computers & Fluids 230 (2021) 105162.
- Multipoint high-fidelity cfd-based aerodynamic shape optimization of a 10 mw wind turbine, Wind Energy Science 4 (2019) 163–192.
- F. J. Gonzalez, M. Balajewicz, Deep convolutional recurrent autoencoders for learning low-dimensional feature dynamics of fluid systems, arXiv preprint arXiv:1808.01346 (2018).
- H. F. Lui, W. R. Wolf, Construction of reduced-order models for fluid flows using deep feedforward neural networks, Journal of Fluid Mechanics 872 (2019) 963–994.
- Model order reduction assisted by deep neural networks (rom-net), Advanced Modeling and Simulation in Engineering Sciences 7 (2020) 1–27.
- On closures for reduced order models—a spectrum of first-principle to machine-learned avenues, Physics of Fluids 33 (2021) 091301.
- Characterising the digital twin: A systematic literature review, CIRP Journal of Manufacturing Science and Technology 29 (2020) 36–52.
- Digital twin: Values, challenges and enablers, arXiv preprint arXiv:1910.01719 (2019).
- Ensemble gaussian processes for online learning over graphs with adaptivity and scalability, IEEE Transactions on Signal Processing 70 (2021) 17–30.
- Multiscale modeling meets machine learning: What can we learn?, Archives of Computational Methods in Engineering 28 (2021) 1017–1037.
- Machine learning for fluid mechanics, Annual Review of Fluid Mechanics 52 (2020) 477–508.
- Digital twin technology challenges and applications: A comprehensive review, Remote Sensing 14 (2022) 1335.
- Increasing facility uptime using machine learning and physics-based hybrid analytics in a dynamic digital twin, in: Offshore Technology Conference, OnePetro, 2020.
- M. W. Grieves, Virtually intelligent product systems: digital and physical twins, 2019.
- Accelerating design optimization using reduced order models, arXiv preprint arXiv:1909.11320 (2019).
- S. Pawar, O. San, Data assimilation empowered neural network parametrizations for subgrid processes in geophysical flows, Physical Review Fluids 6 (2021) 050501.
- Optimal reduced space for variational data assimilation, Journal of Computational Physics 379 (2019) 51–69.
- S. Pagani, A. Manzoni, Enabling forward uncertainty quantification and sensitivity analysis in cardiac electrophysiology by reduced order modeling and machine learning, International Journal for Numerical Methods in Biomedical Engineering 37 (2021) e3450.
- An efficient, globally convergent method for optimization under uncertainty using adaptive model reduction and sparse grids, SIAM/ASA Journal on Uncertainty Quantification 7 (2019) 877–912.
- A survey of projection-based model reduction methods for parametric dynamical systems, SIAM review 57 (2015) 483–531.
- Projection-based model reduction: Formulations for physics-based machine learning, Computers & Fluids 179 (2019) 704–717.
- A. Arzani, S. T. Dawson, Data-driven cardiovascular flow modelling: examples and opportunities, Journal of the Royal Society Interface 18 (2021) 20200802.
- Efficient space–time reduced order model for linear dynamical systems in python using less than 120 lines of code, Mathematics 9 (2021) 1690.
- R. Gupta, R. Jaiman, Three-dimensional deep learning-based reduced order model for unsteady flow dynamics with variable reynolds number, Physics of Fluids 34 (2022) 033612.
- Deep learning for reduced order modelling and efficient temporal evolution of fluid simulations, Physics of Fluids 33 (2021) 107101.
- Data-driven variational multiscale reduced order models, Computer Methods in Applied Mechanics and Engineering 373 (2021) 113470.
- A long short-term memory embedding for hybrid uplifted reduced order models, Physica D: Nonlinear Phenomena 409 (2020) 132471.
- Multifidelity computing for coupling full and reduced order models, Plos one 16 (2021) e0246092.
- The adjoint petrov–galerkin method for non-linear model reduction, Computer Methods in Applied Mechanics and Engineering 365 (2020) 112991.
- Galerkin v. least-squares petrov–galerkin projection in nonlinear model reduction, Journal of Computational Physics 330 (2017) 693–734.
- Non-intrusive inference reduced order model for fluids using deep multistep neural network, Mathematics 7 (2019) 757.
- J. Xu, K. Duraisamy, Multi-level convolutional autoencoder networks for parametric prediction of spatio-temporal dynamics, Computer Methods in Applied Mechanics and Engineering 372 (2020) 113379.
- Reduced-order modeling of advection-dominated systems with recurrent neural networks and convolutional autoencoders, Physics of Fluids 33 (2021) 037106.
- Intrusive and non-intrusive reduced order modeling of the rotating thermal shallow water equation, arXiv preprint arXiv:2104.00213 (2021).
- Reduced order modeling for parameterized time-dependent pdes using spatially and memory aware deep learning, Journal of Computational Science 53 (2021) 101408.
- Non-intrusive reduced order modeling of natural convection in porous media using convolutional autoencoders: comparison with linear subspace techniques, Advances in Water Resources (2022) 104098.
- A comprehensive deep learning-based approach to reduced order modeling of nonlinear time-dependent parametrized pdes, Journal of Scientific Computing 87 (2021) 1–36.
- Non intrusive reduced order modeling of parametrized pdes by kernel pod and neural networks, Computers & Mathematics with Applications 104 (2021) 1–13.
- Sindy-pi: a robust algorithm for parallel implicit sparse identification of nonlinear dynamics, Proceedings of the Royal Society A 476 (2020) 20200279.
- A. K. Saibaba, Randomized discrete empirical interpolation method for nonlinear model reduction, SIAM Journal on Scientific Computing 42 (2020) A1582–A1608.
- Operator inference for non-intrusive model reduction of systems with non-polynomial nonlinear terms, Computer Methods in Applied Mechanics and Engineering 372 (2020) 113433.
- D. Hartmann, L. Failer, A differentiable solver approach to operator inference, arXiv preprint arXiv:2107.02093 (2021).
- A gaussian process regression approach within a data-driven pod framework for engineering problems in fluid dynamics, Mathematics in Engineering 4 (2022) 1–16.
- Machine-learning-based reduced-order modeling for unsteady flows around bluff bodies of various shapes, Theoretical and Computational Fluid Dynamics 34 (2020) 367–383.
- Non-intrusive surrogate modeling for parametrized time-dependent partial differential equations using convolutional autoencoders, Engineering Applications of Artificial Intelligence 109 (2022) 104652.
- Convolutional neural network and long short-term memory based reduced order surrogate for minimal turbulent channel flow, Physics of Fluids 33 (2021) 025116.
- Nonlinear mode decomposition with convolutional neural networks for fluid dynamics, Journal of Fluid Mechanics 882 (2020).
- Latent-space time evolution of non-intrusive reduced-order models using gaussian process emulation, Physica D: Nonlinear Phenomena 416 (2021) 132797.
- Projection techniques to update the truncated svd of evolving matrices with applications, in: International Conference on Machine Learning, PMLR, 2021, pp. 5236–5246.
- P. Benardos, G.-C. Vosniakos, Optimizing feedforward artificial neural network architecture, Engineering applications of artificial intelligence 20 (2007) 365–382.
- Long-time prediction of nonlinear parametrized dynamical systems by deep learning-based reduced order models, arXiv preprint arXiv:2201.10215 (2022).
- On the origin of extrusion instabilities: Linear stability analysis of the viscoelastic die swell, Journal of Non-Newtonian Fluid Mechanics 224 (2015) 61–77.
- Finite volume methods, Handbook of numerical analysis 7 (2000) 713–1018.
- T. Liszka, J. Orkisz, The finite difference method at arbitrary irregular grids and its application in applied mechanics, Computers & Structures 11 (1980) 83–95.
- The finite volume method, in: The finite volume method in computational fluid dynamics, Springer, 2016, pp. 103–135.
- Aca/bem for solving large-scale cathodic protection problems, Engineering Analysis with Boundary Elements 106 (2019) 139–148.
- E. J. Sellountos, A single domain velocity-vorticity fast multipole boundary domain element method for three dimensional incompressible fluid flow problems, part ii, Engineering Analysis with Boundary Elements 114 (2020) 74–93.
- Numerical modelling of micro-seismic and infrasound noise radiated by a wind turbine, Soil Dynamics and Earthquake Engineering 99 (2017) 108–123. URL: https://www.sciencedirect.com/science/article/pii/S026772611730297X. doi:https://doi.org/10.1016/j.soildyn.2017.05.001.
- Cathodic protection of a container ship using a detailed bem model, Journal of Marine Science and Engineering 8 (2020) 359.
- An explicit meshless point collocation method for electrically driven magnetohydrodynamics (mhd) flow, Applied Mathematics and Computation 348 (2019) 215–233.
- Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations, Journal of Computational physics 378 (2019) 686–707.
- C. R. Goodall, 13 computation using the qr decomposition (1993).
- C. C. Aggarwal, et al., Neural networks and deep learning, Springer 10 (2018) 978–3.
- Image restoration using convolutional auto-encoders with symmetric skip connections, arXiv preprint arXiv:1606.08921 (2016).
- An efficient compression of ecg signals using deep convolutional autoencoders, Cognitive Systems Research 52 (2018) 198–211.
- Air quality prediction based on lstm-kalman model, in: 2019 IEEE 8th Joint International Information Technology and Artificial Intelligence Conference (ITAIC), IEEE, 2019, pp. 695–699.
- A. T. Mohan, D. V. Gaitonde, A deep learning based approach to reduced order modeling for turbulent flow control using lstm neural networks, arXiv preprint arXiv:1804.09269 (2018).
- Sequence to sequence-video to text, in: Proceedings of the IEEE international conference on computer vision, 2015, pp. 4534–4542.
- Dive into deep learning, arXiv preprint arXiv:2106.11342 (2021).
- S. Yan, Understanding lstm networks, Online). Accessed on August 11 (2015).
- Activation functions: Comparison of trends in practice and research for deep learning, arXiv preprint arXiv:1811.03378 (2018).
- Neural architecture search: A survey, The Journal of Machine Learning Research 20 (2019) 1997–2017.
- D. P. Kingma, J. Ba, Adam: A method for stochastic optimization, arXiv preprint arXiv:1412.6980 (2014).
- N. Ketkar, Stochastic gradient descent, in: Deep learning with Python, Springer, 2017, pp. 113–132.
- L. Yang, A. Shami, On hyperparameter optimization of machine learning algorithms: Theory and practice, Neurocomputing 415 (2020) 295–316.
- {{\{{TensorFlow}}\}}: A system for {{\{{Large-Scale}}\}} machine learning, in: 12th USENIX symposium on operating systems design and implementation (OSDI 16), 2016, pp. 265–283.
- T. V. Gortsas, S. V. Tsinopoulos, A local domain bem for solving transient convection-diffusion-reaction problems, International Journal of Heat and Mass Transfer 194 (2022) 123029.
- M. Stein, Large sample properties of simulations using latin hypercube sampling, Technometrics 29 (1987) 143–151.
- G. Bourantas, V. Burganos, An implicit meshless scheme for the solution of transient non-linear poisson-type equations, Engineering Analysis with Boundary Elements 37 (2013) 1117–1126.
- Using dc pse operator discretization in eulerian meshless collocation methods improves their robustness in complex geometries, Computers & Fluids 136 (2016) 285–300.
- S. A. AL-Bayati, L. C. Wrobel, A novel dual reciprocity boundary element formulation for two-dimensional transient convection–diffusion–reaction problems with variable velocity, Engineering Analysis with Boundary Elements 94 (2018) 60–68.
- U. Guide, Starccm+ version 2020.1, SIEMENS simcenter (2020).