Optimal Scaling and Shaping of Random Walk Metropolis via Diffusion Limits of Block-I.I.D. Targets

Abstract: This work extends Roberts et al. (1997) by considering limits of Random Walk Metropolis (RWM) applied to block IID target distributions, with corresponding block-independent proposals. The extension verifies the robustness of the optimal scaling heuristic, to tune the acceptance rate to $\approx0.234$, for any choice of proposal shaping. We upgrade the form of weak convergence from a finite-dimensional subprocess to the infinite dimensional process. We show that the optimal shaping (in terms of the decay of autocorrelations of linear functions) is the variance of the target distribution. We show that this choice coincides with the optimal shaping in terms of spectral gaps in special cases where they can be computed. Lastly, we provide some negative guarantees, showing that RWM performance degrades with higher-order dependence. In such cases, no tuning of RWM will yield performance comparable to an IID target.