Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
162 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Neural Likelihood Approximation for Integer Valued Time Series Data (2310.12544v2)

Published 19 Oct 2023 in stat.ML and cs.LG

Abstract: Stochastic processes defined on integer valued state spaces are popular within the physical and biological sciences. These models are necessary for capturing the dynamics of small systems where the individual nature of the populations cannot be ignored and stochastic effects are important. The inference of the parameters of such models, from time series data, is challenging due to intractability of the likelihood. To work at all, current simulation based inference methods require the generation of realisations of the model conditional on the data, which can be both tricky to implement and computationally expensive. In this paper we instead construct a neural likelihood approximation that can be trained using unconditional simulation of the underlying model, which is much simpler. We demonstrate our method by performing inference on a number of ecological and epidemiological models, showing that we can accurately approximate the true posterior while achieving significant computational speed ups compared to current best methods.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (48)
  1. Linda J.S. Allen. A primer on stochastic epidemic models: Formulation, numerical simulation, and analysis. Infectious Disease Modelling, 2:128–142, 2017. doi: 10.1016/j.idm.2017.03.001.
  2. Particle Markov chain Monte Carlo methods. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 72(3):269–342, 2010. doi: 10.1111/j.1467-9868.2009.00736.x.
  3. On Markov chain Monte Carlo methods for tall data. Journal of Machine Learning Research, 18:1–43, 2017.
  4. Automatic Differentiation in Machine Learning: A Survey. Journal of Machine Learning Research, 18(1):5595–5637, January 2017.
  5. Sharp failure rates for the bootstrap particle filter in high dimensions. In Institute of Mathematical Statistics Collections, pp.  318–329. Institute of Mathematical Statistics, Beachwood, Ohio, 2008. doi: 10.1214/074921708000000228.
  6. Christopher M. Bishop. Pattern Recognition and Machine Learning. Information science and statistics. Springer, New York, 2006. ISBN 978-0-387-31073-2.
  7. Andrew J. Black. Importance sampling for partially observed temporal epidemic models. Statistics and Computing, 29:617–630, 2019. doi: 10.1007/s11222-018-9827-1.
  8. Characterising pandemic severity and transmissibility from data collected during first few hundred studies. Epidemics, 19:61–73, 2017. doi: 10.1016/j.epidem.2017.01.004.
  9. JAX: composable transformations of Python+NumPy programs, 2018. URL http://github.com/google/jax.
  10. L. Breuer and D. Baum. An Introduction to Queueing Theory and Matrix-Analytic Methods. Springer-Verlag, 2005. doi: 10.1007/1-4020-3631-0.
  11. An Introduction to Sequential Monte Carlo. Springer, 2020. doi: 10.1007/978-3-030-47845-2.
  12. The frontier of simulation-based inference. Proceedings of the National Academy of Sciences, 117(48):30055, 2020. doi: 10.1073/pnas.1912789117.
  13. The alive particle filter and its use in particle Markov chain Monte Carlo. Stoch. Anal. Appl., 33:943–974, 2015.
  14. Pierre Del Moral. Feynman-Kac Formulae. Springer, New York, 2004. doi: 10.1007/978-1-4684-9393-1.
  15. Sequential Monte Carlo with Highly Informative Observations. SIAM/ASA Journal on Uncertainty Quantification, 3:969–997, 2015. doi: 10.1137/15M1011214.
  16. Efficient implementation of Markov chain Monte Carlo when using an unbiased likelihood estimator. Biometrika, 102:295–313, 2015. doi: 10.1093/biomet/asu075.
  17. Alive SMC22{}^{2}start_FLOATSUPERSCRIPT 2 end_FLOATSUPERSCRIPT: Bayesian model selection for low-count time series models with intractable likelihoods. Biometrics, 72:344–353, 2016. doi: 10.1111/biom.12449.
  18. MADE: Masked Autoencoder for Distribution Estimation. In International Conference on Machine Learning, volume 37, pp.  881–889, 2015.
  19. Daniel T Gillespie. A general method for numerically simulating the stochastic time evolution of coupled chemical reactions. Journal of Computational Physics, 22(4):403–434, 1976. doi: 10.1016/0021-9991(76)90041-3.
  20. Deep Residual Learning for Image Recognition. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp.  770–778, 2016. doi: 10.1109/CVPR.2016.90.
  21. Gaussian Error Linear Units (GELUs). arXiv preprint arXiv:1606.08415, 2016.
  22. A Crisis In Simulation-Based Inference? Beware, Your Posterior Approximations Can Be Unfaithful. Transactions on Machine Learning Research, 2022.
  23. The No-U-Turn Sampler: Adaptively Setting Path Lengths in Hamiltonian Monte Carlo. Journal of Machine Learning Research, 15(47):1593–1623, 2014.
  24. A First Course in Stochastic Processes. Academic Press, second edition, 1975.
  25. Genshiro Kitagawa. Monte Carlo Filter and Smoother for Non-Gaussian Nonlinear State Space Models. Journal of Computational and Graphical Statistics, 5:1–25, 1996. doi: 10.1080/10618600.1996.10474692.
  26. Y. A. Kuznetsov. Elements of applied bifurcation theory. Number v. 112 in Applied mathematical sciences. Springer, New York, 3rd ed edition, 2004. ISBN 978-0-387-21906-6.
  27. Lookahead strategies for sequential monte carlo. Statistical Science, (1):69–94, 2013.
  28. Decoupled Weight Decay Regularization. arXiv preprint arXiv:1711.05101, 2017.
  29. Predator-prey cycles from resonant amplification of demographic stochasticity. Phys. Rev. Lett., 94:218102, 2005. doi: 10.1103/PhysRevLett.94.218102.
  30. Kevin P. Murphy. Probabilistic Machine Learning: Advanced Topics. 2023.
  31. Radford Neal. MCMC Using Hamiltonian Dynamics. Chapman and Hall/CRC, first edition, 2011. doi: 10.1201/b10905.
  32. Art B. Owen. Monte Carlo theory, methods and examples. https://artowen.su.domains/mc/, 2013.
  33. Masked Autoregressive Flow for Density Estimation. In Advances in Neural Information Processing Systems, volume 30, 2017.
  34. Sequential Neural Likelihood: Fast Likelihood-free Inference with Autoregressive Flows. In Proceedings of Machine Learning Research, volume 89, pp.  837–848, 2019.
  35. Composable Effects for Flexible and Accelerated Probabilistic Programming in NumPyro. arXiv preprint arXiv:1912.11554, 2019.
  36. S. M. Ross. Introduction to Probability Models. Academic Press, 7th edition, 2000.
  37. PixelCNN++: A PixelCNN Implementation with Discretized Logistic Mixture Likelihood and Other Modifications. In International Conference on Learning Representations, 2017.
  38. The OR control system of bacteriophage lambda. Journal of Molecular Biology, 181:211–230, 1985. doi: 10.1016/0022-2836(85)90086-5.
  39. On the efficiency of pseudo-marginal random walk metropolis algorithms. Ann. Stats., 43:238–275, 2015.
  40. Chris Sherlock. Direct statistical inference for finite Markov jump processes via the matrix exponential. Computational Statistics, 36:2863–2887, 2021. doi: 10.1007/s00180-021-01102-6.
  41. Time Series Analysis and Its Applications: With R Examples. Springer, 2017. doi: 10.1007/978-3-319-52452-8.
  42. Handbook of Approximate Bayesian Computation. Chapman and Hall/CRC, 2019.
  43. WaveNet: A Generative Model for Raw Audio. arXiv preprint arXiv:1609.03499, 2016.
  44. Pixel Recurrent Neural Networks. In Maria Florina Balcan and Kilian Q. Weinberger (eds.), 33rd International Conference on Machine Learning, volume 48 of Proceedings of Machine Learning Research, pp.  1747–1756, June 2016.
  45. N. G. van Kampen. Stochastic processes in physics and chemistry. Elsevier, Amsterdam, 1992.
  46. Attention is All You Need. In Advances in Nerual Information Processing Systems, pp.  6000–6010, 2017.
  47. Inference of epidemiological parameters from household stratified data. PLoS ONE, 12:e0185910, 2017. doi: 10.1371/journal.pone.0185910.
  48. Darren J. Wilkinson. Stochastic Modelling for Systems Biology, Third Edition. Chapman and Hall/CRC, third edition, 2018. doi: 10.1201/9781351000918.

Summary

We haven't generated a summary for this paper yet.

X Twitter Logo Streamline Icon: https://streamlinehq.com