Uncertainty quantification for deeponets with ensemble kalman inversion (2403.03444v1)
Abstract: In recent years, operator learning, particularly the DeepONet, has received much attention for efficiently learning complex mappings between input and output functions across diverse fields. However, in practical scenarios with limited and noisy data, accessing the uncertainty in DeepONet predictions becomes essential, especially in mission-critical or safety-critical applications. Existing methods, either computationally intensive or yielding unsatisfactory uncertainty quantification, leave room for developing efficient and informative uncertainty quantification (UQ) techniques tailored for DeepONets. In this work, we proposed a novel inference approach for efficient UQ for operator learning by harnessing the power of the Ensemble Kalman Inversion (EKI) approach. EKI, known for its derivative-free, noise-robust, and highly parallelizable feature, has demonstrated its advantages for UQ for physics-informed neural networks [28]. Our innovative application of EKI enables us to efficiently train ensembles of DeepONets while obtaining informative uncertainty estimates for the output of interest. We deploy a mini-batch variant of EKI to accommodate larger datasets, mitigating the computational demand due to large datasets during the training stage. Furthermore, we introduce a heuristic method to estimate the artificial dynamics covariance, thereby improving our uncertainty estimates. Finally, we demonstrate the effectiveness and versatility of our proposed methodology across various benchmark problems, showcasing its potential to address the pressing challenges of uncertainty quantification in DeepONets, especially for practical applications with limited and noisy data.
- Variational inference: A review for statisticians. Journal of the American Statistical Association, 112(518):859–877, apr 2017.
- Lno: Laplace neural operator for solving differential equations, 2023.
- Neural ordinary differential equations, 2019.
- T. Chen and H. Chen. Universal approximation to nonlinear operators by neural networks with arbitrary activation functions and its application to dynamical systems. IEEE Transactions on Neural Networks, 6(4):911–917, 1995.
- Deeponet prediction of linear instability waves in high-speed boundary layers, 2021.
- G. Evensen. The ensemble kalman filter: Theoretical formulation and practical implementation. Ocean dynamics, 53(4):343–367, 2003.
- S. Garg and S. Chakraborty. Vb-deeponet: A bayesian operator learning framework for uncertainty quantification. Engineering Applications of Artificial Intelligence, 118:105685, 2023.
- O. A. Ghattas and D. Sanz-Alonso. Non-asymptotic analysis of ensemble kalman updates: effective dimension and localization. arXiv preprint arXiv:2208.03246, 2022.
- Ib-uq: Information bottleneck based uncertainty quantification for neural function regression and neural operator learning. arXiv preprint arXiv:2302.03271, 2023.
- Ensemble kalman filter for neural network based one-shot inversion. ArXiv, abs/2005.02039, 2020.
- A sequential ensemble kalman filter for atmospheric data assimilation. Monthly Weather Review, 129(1):123–137, 2001.
- Iterated kalman methodology for inverse problems. Journal of Computational Physics, 463:111262, 2022.
- M. A. Iglesias. Iterative regularization for ensemble data assimilation in reservoir models. Computational Geosciences, 19(1):177–212, 2015.
- M. A. Iglesias. A regularizing iterative ensemble kalman method for PDE-constrained inverse problems. Inverse Problems, 32(2):025002, jan 2016.
- Ensemble kalman methods for inverse problems. Inverse Problems, 29(4):045001, mar 2013.
- Normalizing flows: An introduction and review of current methods. IEEE Transactions on Pattern Analysis and Machine Intelligence, 43(11):3964–3979, nov 2021.
- Neural operator: Learning maps between function spaces, 2022.
- Ensemble kalman inversion: a derivative-free technique for machine learning tasks. Inverse Problems, 35(9):095005, aug 2019.
- Fourier neural operator for parametric partial differential equations, 2021.
- Operator learning for predicting multiscale bubble growth dynamics. The Journal of Chemical Physics, 154(10):104118, mar 2021.
- Accelerated replica exchange stochastic gradient langevin diffusion enhanced bayesian deeponet for solving noisy parametric pdes. arXiv preprint arXiv:2111.02484, 2021.
- Dropout ensemble kalman inversion for high dimensional inverse problems, 2023.
- Training physics-based machine-learning parameterizations with gradient-free ensemble kalman methods. Journal of Advances in Modeling Earth Systems, 14(8):e2022MS003105, 2022. e2022MS003105 2022MS003105.
- Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators. Nature Machine Intelligence, 3(3):218–229, mar 2021.
- Deepm&mnet for hypersonics: Predicting the coupled flow and finite-rate chemistry behind a normal shock using neural-network approximation of operators. Journal of Computational Physics, 447:110698, 2021.
- Deeponet-grid-uq: A trustworthy deep operator framework for predicting the power grid’s post-fault trajectories. arXiv preprint arXiv:2202.07176, 2022.
- A local ensemble kalman filter for atmospheric data assimilation. Tellus A: Dynamic Meteorology and Oceanography, 56(5):415–428, 2004.
- A. Pensoneault and X. Zhu. Efficient bayesian physics informed neural networks for inverse problems via ensemble kalman inversion, 2023.
- L. Prechelt. Early Stopping — But When?, pages 53–67. Springer Berlin Heidelberg, Berlin, Heidelberg, 2012.
- Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational Physics, 378:686–707, 2019.
- X. Tong and M. Morzfeld. Localized ensemble kalman inversion. Inverse Problems, 39(6):064002, 2023.
- T. Tripura and S. Chakraborty. Wavelet neural operator for solving parametric partial differential equations in computational mechanics problems. Computer Methods in Applied Mechanics and Engineering, 404:115783, 2023.
- Scalable uncertainty quantification for deep operator networks using randomized priors. Computer Methods in Applied Mechanics and Engineering, 399:115399, 2022.
- Uncertainty quantification for noisy inputs-outputs in physics-informed neural networks and neural operators. arXiv preprint arXiv:2311.11262, 2023.