Bayesian Deep Convolutional Encoder-Decoder Networks for Surrogate Modeling and Uncertainty Quantification (1801.06879v1)

Abstract: We are interested in the development of surrogate models for uncertainty quantification and propagation in problems governed by stochastic PDEs using a deep convolutional encoder-decoder network in a similar fashion to approaches considered in deep learning for image-to-image regression tasks. Since normal neural networks are data intensive and cannot provide predictive uncertainty, we propose a Bayesian approach to convolutional neural nets. A recently introduced variational gradient descent algorithm based on Stein's method is scaled to deep convolutional networks to perform approximate Bayesian inference on millions of uncertain network parameters. This approach achieves state of the art performance in terms of predictive accuracy and uncertainty quantification in comparison to other approaches in Bayesian neural networks as well as techniques that include Gaussian processes and ensemble methods even when the training data size is relatively small. To evaluate the performance of this approach, we consider standard uncertainty quantification benchmark problems including flow in heterogeneous media defined in terms of limited data-driven permeability realizations. The performance of the surrogate model developed is very good even though there is no underlying structure shared between the input (permeability) and output (flow/pressure) fields as is often the case in the image-to-image regression models used in computer vision problems. Studies are performed with an underlying stochastic input dimensionality up to $4,225$ where most other uncertainty quantification methods fail. Uncertainty propagation tasks are considered and the predictive output Bayesian statistics are compared to those obtained with Monte Carlo estimates.

Bayesian Deep Convolutional Encoder-Decoder Networks for Surrogate Modeling and Uncertainty Quantification

In the investigation presented by Zhu and Zabaras, the focus centers on deploying Bayesian deep convolutional encoder-decoder networks for surrogate modeling and uncertainty quantification in problems driven by stochastic partial differential equations (SPDEs). The approach parallels image-to-image regression methodologies seen in computer vision, and importantly, transitions from deterministic to Bayesian frameworks to grasp the uncertainties inherent in predictive tasks.

Methodology and Technical Foundations

The paper introduces a sophisticated integration of Bayesian inference with deep convolutional networks through a variational gradient descent algorithm rooted in Stein's method. This approach aptly fits scenarios with sparse data availability, ensuring both high predictive accuracy and robust uncertainty quantification. The networks are leveraged to model complex mappings between high-dimensional stochastic inputs and outputs without resorting to intermediate dimensionality reductions typically seen in surrogate modeling.

A noteworthy aspect is the implementation of DenseNet architectures to build an efficient encoder-decoder model. The dense connections facilitate information flow and enhance the network's parameter efficiency, crucial for handling high-dimensional input spaces such as those arising from discretized SPDEs.

Performance and Results

The methodology is systematically evaluated on benchmark uncertainty quantification problems, specifically focusing on flow in heterogeneous porous media. The results are compelling, demonstrating that with as little as 32 training samples, the Bayesian surrogate model achieves competitive results against traditional models like Gaussian processes and various ensemble methods.

Significant emphasis is placed on the model's ability to handle high-dimensional input spaces, such as those of dimensionality 4,225. In these cases, the model showcases robustness where other methodologies typically falter, reflecting the depth and efficiency of the Bayesian convolutional approaches despite minimal data regimes.

Implications and Future Directions

The integration of uncertainty quantification with deep neural networks marks a substantial advancement in surrogate modeling, particularly for complex systems described by SPDEs. The Bayesian framework not only permits the handling of epistemic uncertainties in predictions but also facilitates a more reliable computation of statistical properties of the output response, enhancing decision-making processes in computational physics and engineering.

Looking forward, the research indicates promising pathways for deep Bayesian networks in not only enhancing surrogate modeling but also contributing to a broader range of applications requiring uncertainty quantification. The potential for future exploration includes refining variational inference methods within deep networks, incorporating physics-based constraints into network architectures, and extending these models to differentially complex domains and input-output dependencies.

This work stands as a testament to the capabilities of combining deep learning advancements with Bayesian inference, offering a substantial toolkit for both the modeling and uncertainty quantification of high-dimensional and data-sparse systems.