Fast, accurate and lightweight sequential simulation-based inference using Gaussian locally linear mappings (2403.07454v3)
Abstract: Bayesian inference for complex models with an intractable likelihood can be tackled using algorithms performing many calls to computer simulators. These approaches are collectively known as "simulation-based inference" (SBI). Recent SBI methods have made use of neural networks (NN) to provide approximate, yet expressive constructs for the unavailable likelihood function and the posterior distribution. However, the trade-off between accuracy and computational demand leaves much space for improvement. In this work, we propose an alternative that provides both approximations to the likelihood and the posterior distribution, using structured mixtures of probability distributions. Our approach produces accurate posterior inference when compared to state-of-the-art NN-based SBI methods, even for multimodal posteriors, while exhibiting a much smaller computational footprint. We illustrate our results on several benchmark models from the SBI literature and on a biological model of the translation kinetics after mRNA transfection.
- C. Andrieu and G. O. Roberts. The pseudo-marginal approach for efficient Monte Carlo computations. The Annals of Statistics, 2009.
- Particle Markov chain Monte Carlo methods. Journal of the Royal Statistical Society Series B: Statistical Methodology, 72(3):269–342, 2010.
- Normalized Wasserstein distance for mixture distributions with applications in adversarial learning and domain adaptation. arXiv preprint arXiv:1902.00415, 2019.
- Adaptive approximate Bayesian computation. Biometrika, 96(4):983–990, 10 2009. ISSN 0006-3444. doi: 10.1093/biomet/asp052. URL https://doi.org/10.1093/biomet/asp052.
- C. Bishop. Mixture density networks. Technical report, Aston University, 1994.
- PowerAPI: A software library to monitor the energy consumed at the process-level. ERCIM News, 2013.
- Neural approximate sufficient statistics for implicit models. In International Conference on Learning Representations, 2021.
- The frontier of simulation-based inference. Proceedings of the National Academy of Sciences, 117:201912789, 05 2020. doi: 10.1073/pnas.1912789117.
- An adaptive sequential Monte Carlo method for approximate Bayesian computation. Statistics and computing, 22:1009–1020, 2012.
- High-dimensional regression with Gaussian mixtures and partially-latent response variables. Statistics and Computing, 25(5):893–911, mar 2014.
- On contrastive learning for likelihood-free inference. In International conference on machine learning, pages 2771–2781. PMLR, 2020.
- ABCpy: A high-performance computing perspective to approximate Bayesian computation. Journal of Statistical Software, 100:1–38, 2017.
- Amortised likelihood-free inference for expensive time-series simulators with signatured ratio estimation. In International Conference on Artificial Intelligence and Statistics, pages 11131–11144. PMLR, 2022.
- Frugal machine learning. ArXiv, abs/2111.03731, 2021.
- On optimality of kernels for approximate Bayesian computation using sequential Monte Carlo. Statistical applications in genetics and molecular biology, 12(1):87–107, 2013.
- POT: Python optimal transport. The Journal of Machine Learning Research, 22(1):3571–3578, 2021.
- Summary Statistics and Discrepancy Measures for Approximate Bayesian Computation via Surrogate Posteriors. Statistics and Computing, 32(5), oct 2022.
- Generalized Bayesian inference for scientific simulators via amortized cost estimation. arXiv preprint arXiv:2305.15208, 2023.
- Gillespie. Exact stochastic simulation of coupled chemical reactions. The Journal of Physical Chemistry 81 (25), 2340–2361, 1977.
- A. Golightly and D. J. Wilkinson. Bayesian parameter inference for stochastic biochemical network models using particle markov chain monte carlo. Interface focus, 1(6):807–820, 2011.
- Automatic posterior transformation for likelihood-free inference. In International Conference on Machine Learning, pages 2404–2414. PMLR, 2019.
- Adaptive proposal distribution for random walk Metropolis algorithm. Computational Statistics, 14(3):375–395, 1999.
- Likelihood-free MCMC with amortized approximate ratio estimators. In International conference on machine learning, pages 4239–4248. PMLR, 2020.
- Extending approximate Bayesian computation methods to high dimensions via a Gaussian copula model. Computational Statistics & Data Analysis, 106:77–89, 2017.
- Flexible statistical inference for mechanistic models of neural dynamics. Advances in neural information processing systems, 30, 2017.
- Benchmarking simulation-based inference. In International conference on artificial intelligence and statistics, pages 343–351. PMLR, 2021.
- R. M. Neal. Slice sampling. The Annals of Statistics, 31(3):705–767, 2003.
- Demystifying Softmax Gating Function in Gaussian Mixture of Experts. In Thirty-seventh Conference on Neural Information Processing Systems, 2023.
- Approximation results regarding the multiple-output Gaussian gated mixture of linear experts model. Neurocomputing, 366:208–214, 2019.
- Approximations of conditional probability density functions in Lebesgue spaces via mixture of experts models. Journal of Statistical Distributions and Applications, 8:1–15, 2021.
- Handbook of approximate Bayesian computation, chapter High-dimensional ABC. Chapman and Hall/CRC, 2018.
- Likelihood free inference for markov processes: a comparison. Statistical Applications in Genetics and Molecular Biology, 14(2):189–209, 2015a. doi: doi:10.1515/sagmb-2014-0072. URL https://doi.org/10.1515/sagmb-2014-0072.
- Scalable inference for Markov processes with intractable likelihoods. Statistics and Computing, 25:145–156, 2015b.
- G. Papamakarios and I. Murray. Fast ε𝜀\varepsilonitalic_ε-free inference of simulation models with Bayesian conditional density estimation. Advances in Neural Information Processing Systems, 29, 2016.
- Sequential Neural Likelihood: Fast Likelihood-free Inference with Autoregressive Flows. In Proceedings of the Twenty-Second International Conference on Artificial Intelligence and Statistics, volume 89, pages 837–848. PMLR, 16–18 Apr 2019.
- Normalizing flows for probabilistic modeling and inference. The Journal of Machine Learning Research, 22(1):2617–2680, 2021.
- Inverse regression approach to robust nonlinear high-to-low dimensional mapping. Journal of Multivariate Analysis, 163(C):1–14, 2018.
- xLLiM: High Dimensional Locally-Linear Mapping, 2022. R package version 2.2.1.
- ABC of the future. International Statistical Review, 91(2):243–268, 2023.
- U. Picchini and M. Tamborrino. Guided sequential schemes for intractable Bayesian models. arXiv preprint arXiv:2206.12235, 2022.
- Bayesian synthetic likelihood. Journal of Computational and Graphical Statistics, 27(1):1–11, 2018.
- JANA: Jointly amortized neural approximation of complex Bayesian models. In Proceedings of the Thirty-Ninth Conference on Uncertainty in Artificial Intelligence, pages 1695–1706. PMLR, 2023a.
- BayesFlow: Amortized Bayesian workflows with neural networks. arXiv preprint arXiv:2306.16015, 2023b.
- D. Rezende and S. Mohamed. Variational inference with normalizing flows. In International conference on machine learning, pages 1530–1538. PMLR, 2015.
- C. P. Robert and G. Casella. Monte Carlo Statistical Methods. Springer, 2004.
- Green AI. Commun. ACM, 63(12):54–63, 2020.
- pyABC: Efficient and robust easy-to-use approximate bayesian computation. Journal of Open Source Software, 7:4304, 2022.
- Handbook of approximate Bayesian computation. CRC Press, 2018.
- Energy and Policy Considerations for Deep Learning in NLP. In 57th Meeting of Assoc. Computational Linguistics, 2019.
- The computational limits of deep learning. ArXiv, abs/2007.05558, 2022.
- Approximate Bayesian computation scheme for parameter inference and model selection in dynamical systems. Journal of the Royal Society Interface, 6(31):187–202, 2009.
- On the Theory of the Brownian Motion. Phys. Rev., 36:823–841, Sep 1930.
- D. Wilkinson. smfsb: Stochastic Modelling for Systems Biology, 2024. URL https://CRAN.R-project.org/package=smfsb. R package version 1.5.
- D. J. Wilkinson. Summary stats for ABC, 2013. https://darrenjw.wordpress.com/2013/09/01/summary-stats-for-abc/.
- Sequential neural posterior and likelihood approximation. arXiv preprint arXiv:2102.06522, 2021.
- S. N. Wood. Statistical inference for noisy nonlinear ecological dynamic systems. Nature, 466(7310):1102–1104, 2010.
- Henrik Häggström (2 papers)
- Pedro L. C. Rodrigues (6 papers)
- Geoffroy Oudoumanessah (6 papers)
- Florence Forbes (35 papers)
- Umberto Picchini (18 papers)