TGPT-PINN: Nonlinear model reduction with transformed GPT-PINNs (2403.03459v1)
Abstract: We introduce the Transformed Generative Pre-Trained Physics-Informed Neural Networks (TGPT-PINN) for accomplishing nonlinear model order reduction (MOR) of transport-dominated partial differential equations in an MOR-integrating PINNs framework. Building on the recent development of the GPT-PINN that is a network-of-networks design achieving snapshot-based model reduction, we design and test a novel paradigm for nonlinear model reduction that can effectively tackle problems with parameter-dependent discontinuities. Through incorporation of a shock-capturing loss function component as well as a parameter-dependent transform layer, the TGPT-PINN overcomes the limitations of linear model reduction in the transport-dominated regime. We demonstrate this new capability for nonlinear model reduction in the PINNs framework by several nontrivial parametric partial differential equations.
- Neural-network-augmented projection-based model order reduction for mitigating the Kolmogorov barrier to reducibility. Journal of Computational Physics, 492:112420, 2023.
- An “empirical interpolation” method: Application to efficient reduced-basis discretization of partial differential equations. C. R. Acad. Sci. Paris, Série I, 339:667–672, 2004.
- Wasserstein model reduction approach for parametrized flow problems in porous media. ESAIM: Proceedings and Surveys, 73:28–47, 2023.
- Automatic differentiation in machine learning: A survey. Journal of Marchine Learning Research, 18:1–43, 2018.
- Convergence rates for greedy algorithms in reduced basis methods. SIAM Journal on Mathematical Analysis, 43(3):1457–1472, 2011.
- A priori convergence of the greedy algorithm for the parametrized reduced basis method. ESAIM: Mathematical Modelling and Numerical Analysis, 46(3):595–603, 2012.
- Model order reduction for problems with large convection effects. Contributions to Partial Differential Equations and Applications, pages 131–150, 2019.
- Y. Chen and S. Koohy. GPT-PINN: Generative pre-trained physics-informed neural networks toward non-intrusive meta-learning of parametric PDEs. Finite Elements in Analysis and Design, 228:104047, 2024.
- A. Cohen and R. DeVore. Approximation of high-dimensional parametric PDEs. Acta Numerica, 24:1–159, 2015.
- W. Dahmen. Compositional sparsity, approximation classes, and parametric transport equations. arXiv:2207.06128, 2022.
- Nonlinear model reduction on metric spaces. Application to one-dimensional conservative PDEs in Wasserstein spaces. ESAIM: Mathematical Modelling and Numerical Analysis, 54(6):2159–2197, 2020.
- Operator inference for non-intrusive model reduction with quadratic manifolds. Computer Methods in Applied Mechanics and Engineering, 403:115717, 2023.
- Efficient reduced-basis treatment of nonaffine and nonlinear partial differential equations. Mathematical Modelling and Numerical Analysis, 41(3):575–605, 2007.
- A. Iollo and D. Lombardi. Advection modes by optimal mass transfer. Physical Review E, 89(2):022923, 2014.
- Efficient nonlinear manifold reduced order model. arXiv:2011.07727, 2020.
- A fast and accurate physics-informed neural network reduced order model with shallow masked autoencoder. Journal of Computational Physics, 451:110841, 2022.
- Model order reduction of combustion processes with complex front dynamics. In Numerical Mathematics and Advanced Applications ENUMATH 2019: European Conference, Egmond aan Zee, The Netherlands, September 30-October 4, pages 803–811. Springer, 2020.
- K. Lee and K. T. Carlberg. Model reduction of dynamical systems on nonlinear manifolds using deep convolutional autoencoders. Journal of Computational Physics, 404:108973, 2020.
- Discontinuity computing using physics-informed neural networks. Journal of Scientific Computing, 98(1):22, 2024.
- N. J. Nair and M. Balajewicz. Transported snapshot model order reduction approach for parametric, steady-state fluid flows containing parameter-dependent shocks. International Journal for Numerical Methods in Engineering, 117(12):1234–1262, 2019.
- M. Ohlberger and S. Rave. Reduced basis methods: Success, limitations and future challenges. arXiv:1511.02021, 2015.
- The neural network shifted-proper orthogonal decomposition: A machine learning approach for non-linear reduction of hyperbolic equations. Computer Methods in Applied Mechanics and Engineering, 392:114687, 2022.
- Automatic differentiation in PyTorch. In 31st Conference on Neural Information Processing Systems, 2017.
- B. Peherstorfer. Model reduction for transport-dominated problems via online adaptive bases and adaptive sampling. SIAM Journal on Scientific Computing, 42(5):A2803–A2836, 2020.
- B. Peherstorfer and K. Willcox. Online adaptive model reduction for nonlinear systems via low-rank updates. SIAM Journal on Scientific Computing, 37(4):A2123–A2150, 2015.
- A. Pinkus. N-Widths in Approximation Theory, volume 7. Springer Science & Business Media, 2012.
- Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational Physics, 378:686–707, 2019.
- The shifted proper orthogonal decomposition: A mode decomposition for multiple transport phenomena. SIAM Journal on Scientific Computing, 40(3):A1322–A1344, 2018.
- Manifold approximations via transported subspaces: Model reduction for transport-dominated problems. SIAM Journal on Scientific Computing, 45(1):A170–A199, 2023.
- T. Taddei. A registration method for model order reduction: Data compression and geometry reduction. SIAM Journal on Scientific Computing, 42(2):A997–A1027, 2020.
- G. Welper. Interpolation of functions with parameter dependent jumps by transformed snapshots. SIAM Journal on Scientific Computing, 39(4):A1225–A1250, 2017.
- PINNsFormer: A transformer-based framework for physics-informed neural networks. arXiv:2307.11833, 2023.