Extend minibatch acceptance–rejection MCMC to non-i.i.d. posteriors

Extend the acceptance–rejection minibatch MCMC methodology exemplified by TunaMH and TunaMH‑SGLD to posterior distributions with non‑i.i.d. model likelihoods, ensuring the resulting Markov chain preserves the correct stationary distribution while retaining minibatch computational costs.

Background

TunaMH and TunaMH‑SGLD are designed for tall-data posteriors where observations are i.i.d., enabling unbiased Poisson-based estimators for the acceptance ratio that only touch a minibatch per iteration. Many practical Bayesian models violate the i.i.d. assumption (e.g., spatial models, structured dependencies), where full-data evaluations can scale superlinearly with N, making minibatch methods even more appealing.

The authors explicitly identify generalization of their minibatch acceptance–rejection construction beyond the i.i.d. setting as an open question.