Geometric ergodicity of the Random Walk Metropolis with position-dependent proposal covariance

Abstract: We consider a Metropolis--Hastings method with proposal $\mathcal{N}(x, hG(x)^{-1})$, where $x$ is the current state, and study its ergodicity properties. We show that suitable choices of $G(x)$ can change these compared to the Random Walk Metropolis case $\mathcal{N}(x, h\Sigma)$, either for better or worse. We find that if the proposal variance is allowed to grow unboundedly in the tails of the distribution then geometric ergodicity can be established when the target distribution for the algorithm has tails that are heavier than exponential, but that the growth rate must be carefully controlled to prevent the rejection rate approaching unity. We also illustrate that a judicious choice of $G(x)$ can result in a geometrically ergodic chain when probability concentrates on an ever narrower ridge in the tails, something that is not true for the Random Walk Metropolis.