Skew-symmetric approximations of posterior distributions

Abstract: Routinely-implemented deterministic approximations of posterior distributions from, e.g., Laplace method, variational Bayes and expectation-propagation, generally rely on symmetric approximating densities, often taken to be Gaussian. This choice facilitates optimization and inference, but typically affects the quality of the overall approximation. In fact, even in basic parametric models, the posterior distribution often displays asymmetries that yield bias and a reduced accuracy when considering symmetric approximations. Recent research has moved towards more flexible approximating densities that incorporate skewness. However, current solutions are often model-specific, lack general supporting theory, increase the computational complexity of the optimization problem, and do not provide a broadly-applicable solution to include skewness in any symmetric approximation. This article addresses such a gap by introducing a general and provably-optimal strategy to perturb any off-the-shelf symmetric approximation of a generic posterior distribution. Crucially, this novel perturbation is derived without additional optimization steps, and yields a similarly-tractable approximation within the class of skew-symmetric densities that provably enhances the finite-sample accuracy of the original symmetric counterpart. Furthermore, under suitable assumptions, it improves the convergence rate to the exact posterior by at least a $\sqrt{n}$ factor, in asymptotic regimes. These advancements are illustrated in numerical studies focusing on skewed perturbations of state-of-the-art Gaussian approximations.