Multilevel Gaussian Processes with Applications to Inverse Problems

Abstract: We present a multilevel Gaussian process (GP) regression framework for approximating functions that are accessible only through a hierarchy of approximations of increasing accuracy and computational cost. One example is the response surface of a parametrized partial differential equation, where the hierarchy arises from the resolution of the numerical solver. Modeling level increments independently with Matérn-kernel GPs and allocating samples flexibly across discretization levels yields substantial computational savings over single-level GP regression. We derive $L<sup>2$ convergence rates for the posterior mean and variance, accounting for maximum likelihood estimation of the marginal variance and for kernel misspecification. When increment regularity or solver cost is unknown a priori, an adaptive sequential-design algorithm with cost-adapted active learning criteria allocates samples across levels. We apply the surrogate to Bayesian inverse problems, for which Hellinger distance bounds between the true and approximate posteriors are known. Numerical experiments illustrate the theoretical rates and the resulting computational savings.