- The paper presents DeepGaLA, a Bayesian neural network surrogate that employs Laplace approximation for local uncertainty quantification in inverse PDE problems.
- The methodology uses layer randomization and delayed-acceptance MCMC to validate surrogate-induced posteriors and reduce predictive overconfidence.
- Numerical experiments show DeepGaLA's scalability and accuracy compared to GP surrogates, especially in nonlinear and high-dimensional settings.
Neural Network Surrogates with Uncertainty Quantification for Inverse Problems in PDEs
Background and Motivation
Inverse problems associated with partial differential equations (PDEs) are pervasive in computational modeling, especially within Bayesian inference frameworks that stipulate the evaluation of posterior distributions over unknown model parameters given observed, potentially noisy data. Classical numerical methods entail substantial computational burdens, particularly due to repeated evaluations of complex forward models within Markov chain Monte Carlo (MCMC) samplers. Surrogate modeling mitigates this overhead by approximating forward operators, but traditional surrogates (e.g., polynomial chaos expansions, sparse grids, Gaussian processes) encounter scalability bottlenecks as the parameter dimensionality and PDE complexity increase.
Neural network surrogates have emerged as a flexible solution, but conventional architectures—such as DeepONet, Fourier Neural Operators (FNO), and physics-informed neural networks (PINNs)—either require large training datasets or are susceptible to predictive overconfidence. Robust uncertainty quantification is essential for reliable inverse modeling, especially in regimes where data are scarce and extrapolation is required.
Proposed Methodology: DeepGaLA
DeepGaLA (Deep Galerkin via Laplace Approximation) constitutes a Bayesian neural network surrogate designed specifically for parametric PDE inference. It synthesizes the Deep Galerkin Method (DGM) for physics-informed solution approximation with a Laplace-based uncertainty quantification scheme. Key components of the methodology are:
- Bayesian Recasting of DGM: The optimization of the DGM loss corresponds to a maximum a posteriori (MAP) estimate, assuming a Gaussian likelihood. The posterior over model weights is subsequently approximated via the Laplace method, thus enabling tractable local uncertainty quantification.
- Layer Randomization: Uncertainty quantification is realized by randomizing only the final layer weights of the neural network and fitting a Gaussian posterior using the generalized Gauss-Newton approximation. This yields calibrated predictive distributions for test points, even in limited-data regimes.
- Marginal Posterior Approximation: The surrogate-induced marginal posterior incorporates the neural network’s predictive uncertainty, reducing model overconfidence. Two approaches are explored: using the surrogate mean alone, or integrating over the full marginal posterior.
- Validation via DA-MCMC: A short run of delayed-acceptance MCMC serves as a diagnostic for the fidelity of surrogate-induced posteriors, circumventing the need for infeasible full posterior computations.
Theoretical Analysis
The paper rigorously establishes bounds on the deviation between surrogate-approximated posterior distributions and the true posterior. For Gaussian process (GP) surrogates, convergence in the Hellinger distance is achieved as the training dataset grows, with explicit rates depending on the fill distance and kernel properties. For neural network surrogates, convergence rates are derived under certain assumptions, but practical realization depends heavily on architecture choices and available data. The marginal approximation with uncertainty quantification is consistently shown to dominate the mean approximation in low-data scenarios.
Numerical Experiments
The empirical evaluation spans linear and nonlinear PDEs:
- 1D and 2D Elliptic Problems: DeepGaLA is benchmarked against physics-informed GP (PIGP) surrogates. Both surrogates attain comparable accuracy in posterior inference, but DeepGaLA demonstrates superior scalability as parameter dimension increases. Marginal posterior approximations from DeepGaLA exhibit notably less overconfidence than mean approximations in low-data settings.
- Navier–Stokes Nonlinear Problem: DeepGaLA uniquely scales to nonlinear PDE operators where GP surrogates are not applicable. As training data increase, both mean and marginal posterior approximations converge to ground-truth posteriors, with uncertainty estimates concentrating in regions of high prediction error.
- Computational Efficiency: The cost analysis highlights that DeepGaLA’s evaluation time depends primarily on neural architecture rather than dataset size or parameter dimension. In contrast, GP-based surrogates face cubic scaling in the number of training points when computing predictive variance.
- Posterior Validation: Delayed acceptance MCMC systematically quantifies surrogate fidelity by computing the acceptance rate of proposals from the surrogate chain under the fine model. This technique offers an actionable runtime diagnostic for the accuracy of surrogate-induced posteriors.
Implications and Future Directions
The integration of Laplace-based uncertainty quantification into physics-informed neural surrogates enables scalable, calibrated Bayesian inference for high-dimensional and nonlinear PDE inverse problems. Unlike GP-based surrogates, DeepGaLA’s architecture-driven evaluation cost ensures computational tractability as parameter space grows, and its uncertainty quantification mitigates overconfident predictions, especially in out-of-training-distribution contexts.
The practical utility of DA-MCMC as a surrogate validation tool, coupled with robust theoretical justification, positions DeepGaLA as a promising candidate for industrial and scientific inference tasks where ground-truth posteriors are unreachable.
Future research avenues include extending DeepGaLA to fully Bayesian neural networks for comprehensive uncertainty treatment, incorporating adaptive training strategies in high-dimensional regimes, and generalizing the approach to stochastic or multiscale PDEs. Theoretical advances in understanding the convergence properties of neural marginal approximations remain pertinent for advancing reliability guarantees.
Conclusion
DeepGaLA presents a neural network-based surrogate modeling framework for parametric PDE inverse problems, offering uncertainty quantification via Laplace approximation and rigorous posterior validation using delayed acceptance MCMC. Empirical and theoretical results substantiate DeepGaLA’s efficiency, scalability, and reliability—especially in limited-data and nonlinear settings—highlighting its superiority over established Gaussian process surrogates for complex Bayesian inverse problems in scientific computing (2606.20417).