Simulated Tempering and Swapping on Mean-Field Models

Abstract: Simulated and parallel tempering are families of Markov Chain Monte Carlo algorithms where a temperature parameter is varied during the simulation to overcome bottlenecks to convergence due to multimodality. In this work we introduce and analyze the convergence for a set of new tempering distributions which we call \textit{entropy dampening}. For asymmetric exponential distributions and the mean field Ising model with and external field simulated tempering is known to converge slowly. We show that tempering with entropy dampening distributions mixes in polynomial time for these models. Examining slow mixing times of tempering more closely, we show that for the mean-field 3-state ferromagnetic Potts model, tempering converges slowly regardless of the temperature schedule chosen. On the other hand, tempering with entropy dampening distributions converges in polynomial time to stationarity. Finally we show that the slow mixing can be very expensive practically. In particular, the mixing time of simulated tempering is an exponential factor longer than the mixing time at the fixed temperature.