Bayesian model comparison for simulation-based inference (2207.04037v3)

Abstract: Comparison of appropriate models to describe observational data is a fundamental task of science. The Bayesian model evidence, or marginal likelihood, is a computationally challenging, yet crucial, quantity to estimate to perform Bayesian model comparison. We introduce a methodology to compute the Bayesian model evidence in simulation-based inference (SBI) scenarios (also often called likelihood-free inference). In particular, we leverage the recently proposed learnt harmonic mean estimator and exploit the fact that it is decoupled from the method used to generate posterior samples, i.e. it requires posterior samples only, which may be generated by any approach. This flexibility, which is lacking in many alternative methods for computing the model evidence, allows us to develop SBI model comparison techniques for the three main neural density estimation approaches, including neural posterior estimation (NPE), neural likelihood estimation (NLE), and neural ratio estimation (NRE). We demonstrate and validate our SBI evidence calculation techniques on a range of inference problems, including a gravitational wave example. Moreover, we further validate the accuracy of the learnt harmonic mean estimator, implemented in the HARMONIC software, in likelihood-based settings. These results highlight the potential of HARMONIC as a sampler-agnostic method to estimate the model evidence in both likelihood-based and simulation-based scenarios.

• The paper introduces a learned harmonic mean estimator that provides unbiased, low-variance evidence estimation for likelihood-free simulation-based inference.
• It integrates neural posterior, likelihood, and ratio estimation techniques to extend Bayesian model comparison beyond traditional methods.
• The approach is validated on benchmark problems like linear Gaussian models and gravitational wave analysis, demonstrating its practical efficiency.

Bayesian Model Comparison for Simulation-Based Inference

The paper "Bayesian model comparison for simulation-based inference" by A. Spurio Mancini et al. presents a novel methodology for estimating Bayesian model evidence in contexts where the likelihood is either unavailable or computationally expensive to evaluate, making it particularly suitable for simulation-based inference (SBI). The authors leverage the learned harmonic mean estimator to extend model comparison capabilities to scenarios typically referred to as likelihood-free, which are increasingly common in applications involving complex simulators.

The main contribution of the paper is the integration of the learned harmonic mean estimator with the three predominant neural density estimation approaches for SBI: neural posterior estimation (NPE), neural likelihood estimation (NLE), and neural ratio estimation (NRE). These methodologies are validated and evaluated against scenarios where the true likelihood is available, thereby allowing direct comparison with traditional likelihood-based techniques such as nested sampling employed by MultiNest and PolyChord, as well as evidence estimation via the conventional harmonic mean approach.

Key Contributions

1. Learned Harmonic Mean Estimator: This technique mitigates limitations of the original harmonic mean estimator by learning an approximate posterior distribution with bounded tails, providing an unbiased and low-variance estimate of the Bayesian model evidence. The approach critically decouples evidence estimation from the posterior sampling method, allowing flexibility across different inference frameworks.
2. Application to Neural Density Estimation Methods:
   • NPE: The paper discusses utilizing samples from a learned posterior model and employing an auxiliary learned likelihood function to estimate model evidence. The flexibility of generating samples directly through normalizing flows showcases computational efficiency over traditional MCMC methods, particularly when applied to Bayesian model comparison.
   • NLE and NRE: Common to both approaches is the need for sampling from an unnormalised posterior that has been approximated through density estimators. This often involves additional MCMC steps to retrieve posterior samples, highlighting the computational trade-offs relative to NPE.
3. Practical Demonstration and Validation: The methods are applied to several benchmark problems, including a linear Gaussian model and examples drawn from astrophysical contexts such as gravitational wave analysis. These case studies demonstrate not only the viability of harmonic-aided SBI methodologies in estimating the evidence but also their relative performance compared to traditional nested sampling approaches.

Implications and Future Directions

The methodology introduced in the paper has substantial implications for fields involving high-dimensional data with intractable likelihoods, typically addressed using SBI. By providing a tool for accurate and efficient model comparison, these techniques pave the way for broader adoption of Bayesian inference in complex scientific domains such as cosmology and particle physics.

The paper suggests future avenues of research including enhancing the scalability of these approaches to accommodate larger datasets and more intricate model spaces. Additionally, there is potential for further development in assessing the variance of SBI evidence estimators directly, which would standardize error quantification for more robust model comparison outcomes. These advances would position the harmonic mean estimator as a standard component in the arsenal of computational tools available for simulation-based Bayesian analysis.

In conclusion, the integration of the learned harmonic mean estimator with contemporary neural inference techniques in SBI presents a significant advancement towards more flexible, scalable, and accurate model comparison. The empirical validation across various applications underscores the practicality and potential of these methods in advancing scientific inquiry where traditional likelihood-based approaches are infeasible.