Convergence Analysis of the Data Augmentation Algorithm for Bayesian Linear Regression with Non-Gaussian Errors (1506.03113v2)

Abstract: Gaussian errors are sometimes inappropriate in a multivariate linear regression setting because, for example, the data contain outliers. In such situations, it is often assumed that the error density is a scale mixture of multivariate normal densities that takes the form $f(\varepsilon) = \int_0^\infty |\Sigma|^{-\frac{1}{2}} u^{\frac{d}{2}} \, \phi_d \big( \Sigma^{-\frac{1}{2}} \sqrt{u} \, \varepsilon \big) \, h(u) \, du$, where $d$ is the dimension of the response, $\phi_d(\cdot)$ is the standard $d$-variate normal density, $\Sigma$ is an unknown $d \times d$ positive definite scale matrix, and $h(\cdot)$ is some fixed mixing density. Combining this alternative regression model with a default prior on the unknown parameters results in a highly intractable posterior density. Fortunately, there is a simple data augmentation (DA) algorithm and a corresponding Haar PX-DA algorithm that can be used to explore this posterior. This paper provides conditions (on $h$) for geometric ergodicity of the Markov chains underlying these Markov chain Monte Carlo (MCMC) algorithms. These results are extremely important from a practical standpoint because geometric ergodicity guarantees the existence of the central limit theorems that form the basis of all the standard methods of calculating valid asymptotic standard errors for MCMC-based estimators. The main result is that, if $h$ converges to 0 at the origin at an appropriate rate, and $\int_0^\infty u^{\frac{d}{2}} \, h(u) \, du < \infty$, then the DA and Haar PX-DA Markov chains are both geometrically ergodic. This result is quite far-reaching. For example, it implies the geometric ergodicity of the DA and Haar PX-DA Markov chains whenever $h$ is generalized inverse Gaussian, log-normal, inverted gamma (with shape parameter larger than $d/2$), or Fr\'{e}chet (with shape parameter larger than $d/2$).