Deep Gaussian Process Priors for Bayesian Inference in Nonlinear Inverse Problems

Abstract: We study the use of a deep Gaussian process (DGP) prior in a general nonlinear inverse problem satisfying certain regularity conditions. We prove that when the data arises from a true parameter $\theta^*$ with a compositional structure, the posterior induced by the DGP prior concentrates around $\theta^*$ as the number of observations increases. The DGP prior accounts for the unknown compositional structure through the use of a hierarchical structure prior. As examples, we show that our results apply to Darcy's problem of recovering the scalar diffusivity from a steady-state heat equation and the problem of determining the attenuation potential in a steady-state Schr\"{o}dinger equation. We further provide a lower bound, proving in Darcy's problem that typical Gaussian priors based on Whittle-Mat\'{e}rn processes (which ignore compositional structure) contract at a polynomially slower rate than the DGP prior for certain diffusivities arising from a generalised additive model.