Principled fairness criteria for SBC comparisons across algorithms

Determine principled criteria for fair comparison of Markov chain Monte Carlo algorithms using Simulation Based Calibration, including how to select comparable numbers of post burn-in or warm-up samples or target effective sample sizes across parameters when algorithms have differing convergence and efficiency characteristics.

Background

The paper compares Hamiltonian Monte Carlo (HMC) and the KSC off-set mixture MCMC under SBC but uses different numbers of post burn-in samples (999 for HMC and 9,999 for KSC), noting that the original KSC paper used 750,000 draws.

The authors highlight that defining a fair basis for comparison under SBC is not straightforward when algorithms have different mixing behavior and computational constraints.