Frequentist Asymptotics of Variational Laplace

Abstract: Variational inference is a general framework to obtain approximations to the posterior distribution in a Bayesian context. In essence, variational inference entails an optimization over a given family of probability distributions to choose the member of this family best approximating the posterior. Variational Laplace, an iterative update scheme motivated by this objective, is widely used in different contexts in the cognitive neuroscience community. However, until now, the theoretical properties of this scheme have not been systematically investigated. Here, we study variational Laplace in the light of frequentist asymptotic statistics. Asymptotical frequentist theory enables one to judge the quality of point estimates by their limit behaviour. We apply this framework to find that point estimates generated by variational Laplace enjoy the desirable properties of asymptotic consistency and efficiency in two toy examples. Furthermore, we derive conditions that are sufficient to establish these properties in a general setting. Besides of point estimates, we also study the frequentist convergence of distributions in the sense of total variation distance, which may be useful to relate variational Laplace both to recent findings regarding variational inference as well as to classical frequentist considerations on the Bayesian posterior. Finally, to illustrate the validity of our theoretical considerations, we conduct simulation experiments in our study examples.