Modeling the Dynamics of PDE Systems with Physics-Constrained Deep Auto-Regressive Networks (1906.05747v3)

Abstract: In recent years, deep learning has proven to be a viable methodology for surrogate modeling and uncertainty quantification for a vast number of physical systems. However, in their traditional form, such models can require a large amount of training data. This is of particular importance for various engineering and scientific applications where data may be extremely expensive to obtain. To overcome this shortcoming, physics-constrained deep learning provides a promising methodology as it only utilizes the governing equations. In this work, we propose a novel auto-regressive dense encoder-decoder convolutional neural network to solve and model non-linear dynamical systems without training data at a computational cost that is potentially magnitudes lower than standard numerical solvers. This model includes a Bayesian framework that allows for uncertainty quantification of the predicted quantities of interest at each time-step. We rigorously test this model on several non-linear transient partial differential equation systems including the turbulence of the Kuramoto-Sivashinsky equation, multi-shock formation and interaction with 1D Burgers' equation and 2D wave dynamics with coupled Burgers' equations. For each system, the predictive results and uncertainty are presented and discussed together with comparisons to the results obtained from traditional numerical analysis methods.

• The paper introduces AR-DenseED, a CNN-based auto-regressive network that leverages governing equations as loss constraints to predict PDE dynamics.
• It extends the methodology with BAR-DenseED, which employs SWAG to quantify both epistemic and aleatoric uncertainties in complex non-linear systems.
• The model is applied to chaotic and wave-forming PDEs such as the Kuramoto-Sivashinsky and Burgers’ equations, achieving accurate and computationally efficient simulations.

Overview of Modeling the Dynamics of PDE Systems with Physics-Constrained Deep Auto-Regressive Networks

The paper "Modeling the Dynamics of PDE Systems with Physics-Constrained Deep Auto-Regressive Networks" explores the use of deep learning techniques as a surrogate modeling approach for solving systems governed by partial differential equations (PDEs). The authors, Nicholas Geneva and Nicholas Zabaras, pioneer a novel convolutional neural network model, the auto-regressive dense encoder-decoder (AR-DenseED), to address the challenges associated with the computational demands and data scarcity often encountered in traditional PDE solution methods.

Key Contributions

The primary contributions of the paper are as follows:

1. Model Architecture: The paper introduces AR-DenseED, which leverages the strengths of convolutional neural networks to predict time series in PDE systems by iteratively using history states. This model aims to bypass the need for large datasets by training with zero output data, only using the governing equations as a form of loss constraint.
2. Probabilistic Framework: Extending AR-DenseED, a Bayesian variant called BAR-DenseED is proposed, integrating Stochastic Weight Averaging Gaussian (SWAG) to quantify the epistemic and aleatoric uncertainty in predictions. This approach is significant as it estimates both the model's inherent uncertainty and observational noise, which are crucial for robust application in scientific simulations.
3. Applications and Results: The models are applied to complex non-linear dynamical systems, including the Kuramoto-Sivashinsky equation, 1D Burgers' equation, and 2D coupled Burgers' equation.

• For the Kuramoto-Sivashinsky equation, which is known for its chaotic dynamics, the model produces stable predictions that align well with the statistical features of turbulence.
• The 1D Burgers' equation predictions demonstrate the model's capability to accurately capture wave formations and interactions, showing potential in simulating systems with shock phenomena.
• In the 2D Burgers' equation, the AR-DenseED exhibits excellent predictive capability, efficiently modeling complex interactions at a fraction of the computational cost required by conventional methods like finite element simulations.

Implications and Future Directions

The work's implications are noteworthy for the field of computational science and engineering:

• Efficiency and Scalability: The AR-DenseED demonstrates that convolutional neural networks can serve as efficient surrogates, potentially transforming the computational expense associated with large-scale simulations. This approach could be expanded to more intricate systems, including those of greater dimensionality or more complex physics, such as turbulent fluid dynamics or multi-physics coupling scenarios.
• Flexibility in Initial Conditions: The ability to train using purely physics-constrained methods without explicit training data makes the approach adaptable for scenarios where initial conditions are uncertain or unavailable. This aligns with many real-world applications where full datasets are impractical to obtain.
• Advancements in Uncertainty Quantification: By utilizing Bayesian techniques like SWAG, the framework enriches the toolkit for uncertainty quantification in machine learning models, providing better interpretability and confidence in predictive applications.
• Integration with Data-Driven Methods: The potential integration with data-driven approaches presents an exciting avenue, combining explicit data availability with physics-based models to address incomplete datasets, a common issue in observational and experimental sciences.

Conclusion

Geneva and Zabaras' contributions mark a significant advancement in physics-informed machine learning, offering a promising methodology for addressing complex, data-scarce problems across various scientific fields. By further exploring larger and more complex systems, improving training algorithms, and integrating the hybridization of data and physics-driven models, the research opens up new prospects for the efficient and accurate simulation of dynamical systems governed by PDEs.