POD-DL-ROM: enhancing deep learning-based reduced order models for nonlinear parametrized PDEs by proper orthogonal decomposition (2101.11845v1)

Abstract: Deep learning-based reduced order models (DL-ROMs) have been recently proposed to overcome common limitations shared by conventional reduced order models (ROMs) - built, e.g., through proper orthogonal decomposition (POD) - when applied to nonlinear time-dependent parametrized partial differential equations (PDEs). These might be related to (i) the need to deal with projections onto high dimensional linear approximating trial manifolds, (ii) expensive hyper-reduction strategies, or (iii) the intrinsic difficulty to handle physical complexity with a linear superimposition of modes. All these aspects are avoided when employing DL-ROMs, which learn in a non-intrusive way both the nonlinear trial manifold and the reduced dynamics, by relying on deep (e.g., feedforward, convolutional, autoencoder) neural networks. Although extremely efficient at testing time, when evaluating the PDE solution for any new testing-parameter instance, DL-ROMs require an expensive training stage, because of the extremely large number of network parameters to be estimated. In this paper we propose a possible way to avoid an expensive training stage of DL-ROMs, by (i) performing a prior dimensionality reduction through POD, and (ii) relying on a multi-fidelity pretraining stage, where different physical models can be efficiently combined. The proposed POD-DL-ROM is tested on several (both scalar and vector, linear and nonlinear) time-dependent parametrized PDEs (such as, e.g., linear advection-diffusion-reaction, nonlinear diffusion-reaction, nonlinear elastodynamics, and Navier-Stokes equations) to show the generality of this approach and its remarkable computational savings.

Overview of POD-DL-ROM: Enhancing Deep Learning-Based Reduced Order Models for Nonlinear Parametrized PDEs

The paper "POD-DL-ROM: Enhancing Deep Learning-Based Reduced Order Models for Nonlinear Parametrized PDEs by Proper Orthogonal Decomposition," authored by Stefania Fresca and Andrea Manzoni, explores the domain of reduced order modeling (ROM) for complex systems governed by nonlinear, time-dependent parametrized partial differential equations (PDEs). This research addresses the limitations of traditional machine learning-based ROMs by proposing a hybrid approach combining deep learning (DL) techniques with proper orthogonal decomposition (POD), termed as POD-DL-ROM.

Introduction to ROM and DL-ROM

Traditional ROMs, such as those using POD, are well-suited for simplifying the computational complexity of large-scale PDE systems by reducing the dimensionality of the problem. However, when handling nonlinear or time-dependent problems, these methods may require extensive computational resources for projecting the solution onto high-dimensional linear manifolds and managing hyper-reduction strategies. DL-ROMs leverage neural networks to approximate both the nonlinear trial manifold and the reduced dynamics, circumventing the need for explicit projections and enabling efficient testing. Despite their performance in runtime, their training involves a high computational cost due to numerous network parameters that need optimization.

The POD-DL-ROM Framework

The POD-DL-ROM method aims to leverage the strengths of both POD and DL techniques by addressing the large training cost associated with DL-ROMs. Specifically, it involves a prior dimensionality reduction using POD. This is achieved through randomized singular value decomposition (rSVD), which significantly decreases the data dimension before it is fed into deep learning models. This strategic integration maintains the online efficiency of DL-ROMs while reducing the offline computational burden.

Key components of the POD-DL-ROM architecture include:

1. Dimensionality Reduction: POD is utilized for a pre-processing step to reduce the dimensionality of the data while retaining the essential dynamics of the system. The linear manifold dimension (denoted as $N$) is chosen to effectively compress the data from the original full order model (FOM) dimensionality.
2. Deep Learning Structure: The methodology involves two main neural network components. The dynamics on the reduced manifold are modeled through a deep feedforward neural network, and the nonlinear trial manifold is described using the decoder of a convolutional autoencoder (CAE). An encoder function is used during training to map the high-dimensional FOM solutions onto the reduced space.
3. Multi-Fidelity Pretraining: The paper also suggests using pretraining to initialize the deep learning models efficiently. By applying a multi-fidelity approach, the training leverages different model fidelity levels to improve convergence and computational efficiency.

Numerical Experiments and Results

The proposed approach is validated on various complex PDE systems, including linear and nonlinear advection-diffusion-reaction equations, cardiac electrophysiology models, nonlinear elastodynamics, and the unsteady Navier-Stokes equations. These tests demonstrate the following:

• Accuracy and Efficiency: POD-DL-ROMs maintain high numerical accuracy comparable to DL-ROMs but require significantly reduced training time. The testing efficiency is superior, allowing for real-time or faster than real-time prediction capabilities.
• Generalizability and Robustness: Such models can generalize across different Cauchy-Kovalevskaya scenarios, showing robustness to variations in parameter ranges and model configurations.

Implications and Future Directions

The POD-DL-ROM framework offers a noteworthy advancement in ROM technology by combining deep learning's adaptability to capture complex, nonlinear dynamics with the computational expediency of POD. This hybrid approach not only lowers the barrier for simulating sophisticated systems in fields such as fluid dynamics, structural mechanics, and biological processes but also opens new avenues to further explore other DL architectures and POD methods.

In conclusion, the paper makes a significant contribution to ROM methodologies, emphasizing a practical path forward for integrating deep learning in scientific computing applications, thereby setting a precedent for future research focused on efficient and scalable model reduction techniques in various engineering and scientific domains.