Symplectic Autoencoders for Model Reduction of Hamiltonian Systems (2312.10004v1)
Abstract: Many applications, such as optimization, uncertainty quantification and inverse problems, require repeatedly performing simulations of large-dimensional physical systems for different choices of parameters. This can be prohibitively expensive. In order to save computational cost, one can construct surrogate models by expressing the system in a low-dimensional basis, obtained from training data. This is referred to as model reduction. Past investigations have shown that, when performing model reduction of Hamiltonian systems, it is crucial to preserve the symplectic structure associated with the system in order to ensure long-term numerical stability. Up to this point structure-preserving reductions have largely been limited to linear transformations. We propose a new neural network architecture in the spirit of autoencoders, which are established tools for dimension reduction and feature extraction in data science, to obtain more general mappings. In order to train the network, a non-standard gradient descent approach is applied that leverages the differential-geometric structure emerging from the network design. The new architecture is shown to significantly outperform existing designs in accuracy.
- Optimization algorithms on matrix manifolds. Princeton University Press, 2008.
- A survey of model reduction methods for large-scale systems. Contemp.Math., 280:193–220, 2001.
- VI Arnold. Mathematical methods of classical mechanics. Ann Arbor, 1001:48109, 1978.
- Autoencoders. arXiv preprint arXiv:2003.05991, 2020.
- The real symplectic stiefel and grassmann manifolds: metrics, geodesics and applications. arXiv preprint arXiv:2108.12447, 2021.
- Tensor Analysis on Manifolds. Courier Corporation, 1980.
- Tobias Blickhan. A registration method for reduced basis problems using linear optimal transport. arXiv preprint arXiv:2304.14884, 2023.
- Benedikt Brantner. Generalizing adam to manifolds for efficiently training transformers. arXiv preprint arXiv:2305.16901, 2023.
- Symplectic model reduction of hamiltonian systems on nonlinear manifolds and approximation with weakly symplectic autoencoder. SIAM Journal on Scientific Computing, 45(2):A289–A311, 2023. doi: 10.1137/21M1466657. URL https://doi.org/10.1137/21M1466657.
- The geometry of algorithms with orthogonality constraints. SIAM journal on Matrix Analysis and Applications, 20(2):303–353, 1998.
- A comprehensive deep learning-based approach to reduced order modeling of nonlinear time-dependent parametrized pdes. Journal of Scientific Computing, 87(2):1–36, 2021.
- Riemannian optimization on the symplectic stiefel manifold. SIAM Journal on Optimization, 31(2):1546–1575, 2021.
- Optimization on the symplectic stiefel manifold: Sr decomposition-based retraction and applications. arXiv preprint arXiv:2211.09481, 2022.
- Deep learning. MIT press, 2016.
- Decay of the kolmogorov n-width for wave problems. Applied Mathematics Letters, 96:216–222, 2019.
- Mikhael Gromov. Pseudo holomorphic curves in symplectic manifolds. Inventiones mathematicae, 82(2):307–347, 1985.
- Geometric Numerical integration: structure-preserving algorithms for ordinary differential equations. Springer, 2006.
- Multilayer feedforward networks are universal approximators. Neural networks, 2(5):359–366, 1989.
- A differentiable programming system to bridge machine learning and scientific computing. arXiv preprint arXiv:1907.07587, 2019.
- Sympnets: Intrinsic structure-preserving symplectic networks for identifying hamiltonian systems. Neural Networks, 132:166–179, 2020.
- Optimal unit triangular factorization of symplectic matrices. Linear Algebra and its Applications, 2022.
- Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
- Momentum stiefel optimizer, with applications to suitably-orthogonal attention, and optimal transport. arXiv preprint arXiv:2205.14173, 2022.
- Michael Kraus. Variational integrators in plasma physics. arXiv preprint arXiv:1307.5665, 2013.
- Model order reduction in fluid dynamics: challenges and perspectives. Reduced Order Methods for modeling and computational reduction, pages 235–273, 2014.
- Model reduction of dynamical systems on nonlinear manifolds using deep convolutional autoencoders. Journal of Computational Physics, 404:108973, 2020.
- Simulating hamiltonian dynamics. Cambridge university press, Cambridge, 2004.
- Introduction to symplectic topology, volume 27. Oxford University Press, 2017.
- Reverse-mode automatic differentiation and optimization of gpu kernels via enzyme. In Proceedings of the international conference for high performance computing, networking, storage and analysis, pages 1–16, 2021.
- Symplectic model reduction of hamiltonian systems. SIAM Journal on Scientific Computing, 38(1):A1–A27, 2016.
- Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational physics, 378:686–707, 2019.
- Numerical hamiltonian problems, volume 7. Dover Publications, Mineola, NY, 2018.
- Symplectic model reduction methods for the vlasov equation. Contributions to Plasma Physics, 2022.
- Hongguo Xu. An svd-like matrix decomposition and its applications. Linear Algebra and its Applications, 368:1–24, 2003. ISSN 0024-3795. doi: https://doi.org/10.1016/S0024-3795(03)00370-7. URL https://www.sciencedirect.com/science/article/pii/S0024379503003707.