Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
86 tokens/sec
GPT-4o
11 tokens/sec
Gemini 2.5 Pro Pro
53 tokens/sec
o3 Pro
5 tokens/sec
GPT-4.1 Pro
3 tokens/sec
DeepSeek R1 via Azure Pro
33 tokens/sec
2000 character limit reached

Conditional diffusions for amortized neural posterior estimation (2410.19105v3)

Published 24 Oct 2024 in stat.ML, cs.AI, cs.LG, and stat.AP

Abstract: Neural posterior estimation (NPE), a simulation-based computational approach for Bayesian inference, has shown great success in approximating complex posterior distributions. Existing NPE methods typically rely on normalizing flows, which approximate a distribution by composing many simple, invertible transformations. But flow-based models, while state of the art for NPE, are known to suffer from several limitations, including training instability and sharp trade-offs between representational power and computational cost. In this work, we demonstrate the effectiveness of conditional diffusions coupled with high-capacity summary networks for amortized NPE. Conditional diffusions address many of the challenges faced by flow-based methods. Our results show that, across a highly varied suite of benchmarking problems for NPE architectures, diffusions offer improved stability, superior accuracy, and faster training times, even with simpler, shallower models. Building on prior work on diffusions for NPE, we show that these gains persist across a variety of different summary network architectures. Code is available at https://github.com/TianyuCodings/cDiff.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (28)
  1. A bayesian generative neural network framework for epidemic inference problems. Scientific Reports, 12(1), Nov. 2022. ISSN 2045-2322. doi: 10.1038/s41598-022-20898-x. URL http://dx.doi.org/10.1038/s41598-022-20898-x.
  2. Approximating likelihood ratios with calibrated discriminative classifiers. arXiv preprint arXiv:1506.02169, 2015. URL https://arxiv.org/abs/1506.02169.
  3. The frontier of simulation-based inference. Proceedings of the National Academy of Sciences, 117(48):30055–30062, May 2020. ISSN 1091-6490. doi: 10.1073/pnas.1912789117. URL http://dx.doi.org/10.1073/pnas.1912789117.
  4. Learning to forget: Continual prediction with lstm. Neural computation, 12:2451–71, 10 2000. doi: 10.1162/089976600300015015.
  5. Automatic posterior transformation for likelihood-free inference, 2019. URL https://arxiv.org/abs/1905.07488.
  6. J. Ho and T. Salimans. Classifier-free diffusion guidance. arXiv preprint arXiv:2207.12598, 2022.
  7. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840–6851, 2020.
  8. Elucidating the design space of diffusion-based generative models. Advances in neural information processing systems, 35:26565–26577, 2022.
  9. Variational diffusion models. Advances in neural information processing systems, 34:21696–21707, 2021.
  10. Set transformer: A framework for attention-based permutation-invariant neural networks. In International conference on machine learning, pages 3744–3753. PMLR, 2019a.
  11. Set transformer: A framework for attention-based permutation-invariant neural networks, 2019b. URL https://arxiv.org/abs/1810.00825.
  12. Sampling-based accuracy testing of posterior estimators for general inference. In Proceedings of the 40th International Conference on Machine Learning. PMLR, 2023. URL https://github.com/Ciela-Institute/tarp.
  13. Flexible statistical inference for mechanistic models of neural dynamics, 2017. URL https://arxiv.org/abs/1711.01861.
  14. P. Matthews. A slowly mixing markov chain with implications for gibbs sampling. Statistics & Probability Letters, 17(3):231–236, 1993. ISSN 0167-7152. doi: https://doi.org/10.1016/0167-7152(93)90172-F. URL https://www.sciencedirect.com/science/article/pii/016771529390172F.
  15. Janossy pooling: Learning deep permutation-invariant functions for variable-size inputs, 2019. URL https://arxiv.org/abs/1811.01900.
  16. Sequential neural likelihood: Fast likelihood-free inference with autoregressive flows. In The 22nd International Conference on Artificial Intelligence and Statistics, pages 837–848, 2019a. URL http://proceedings.mlr.press/v89/papamakarios19a.html.
  17. Sequential neural likelihood: Fast likelihood-free inference with autoregressive flows, 2019b. URL https://arxiv.org/abs/1805.07226.
  18. Normalizing flows for probabilistic modeling and inference. Journal of Machine Learning Research, 22(57):1–64, 2021.
  19. Bayesflow: Learning complex stochastic models with invertible neural networks, 2020. URL https://arxiv.org/abs/2003.06281.
  20. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502, 2020a.
  21. Score-based generative modeling through stochastic differential equations. arXiv preprint arXiv:2011.13456, 2020b.
  22. Maximum likelihood training of score-based diffusion models. Advances in neural information processing systems, 34:1415–1428, 2021.
  23. Validating bayesian inference algorithms with simulation-based calibration, 2020. URL https://arxiv.org/abs/1804.06788.
  24. Neural autoregressive distribution estimation. In Journal of Machine Learning Research, volume 17, pages 1–37, 2016. URL https://jmlr.org/papers/volume17/16-272/16-272.pdf.
  25. A. Vaswani. Attention is all you need. Advances in Neural Information Processing Systems, 2017.
  26. Universal approximation of functions on sets, 2021. URL https://arxiv.org/abs/2107.01959.
  27. Deep sets, 2018. URL https://arxiv.org/abs/1703.06114.
  28. Set norm and equivariant skip connections: Putting the deep in deep sets, 2022. URL https://arxiv.org/abs/2206.11925.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets