Diagnostics for Stochastic Gaussian Process Emulators

Abstract: Computer models, also known as simulators, can be computationally expensive to run, and for this reason statistical surrogates, known as emulators, are often used. Any statistical model, including an emulator, should be validated before being used, otherwise resulting decisions can be misguided. We discuss how current methods for validating Gaussian process emulators of deterministic models are insufficient for emulators of stochastic computer models and develop a framework for diagnosing problems in stochastic emulators. These diagnostics are based on independently validating the mean and variance predictions using out-of-sample, replicated, simulator runs. We then also use a building performance simulator as a case study example.