Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
166 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Energy based diffusion generator for efficient sampling of Boltzmann distributions (2401.02080v2)

Published 4 Jan 2024 in cs.LG, stat.CO, and stat.ML

Abstract: Sampling from Boltzmann distributions, particularly those tied to high-dimensional and complex energy functions, poses a significant challenge in many fields. In this work, we present the Energy-Based Diffusion Generator (EDG), a novel approach that integrates ideas from variational autoencoders and diffusion models. EDG leverages a decoder to transform latent variables from a simple distribution into samples approximating the target Boltzmann distribution, while the diffusion-based encoder provides an accurate estimate of the Kullback-Leibler divergence during training. Notably, EDG is simulation-free, eliminating the need to solve ordinary or stochastic differential equations during training. Furthermore, by removing constraints such as bijectivity in the decoder, EDG allows for flexible network design. Through empirical evaluation, we demonstrate the superior performance of EDG across a variety of complex distribution tasks, outperforming existing methods.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (44)
  1. Flow-based sampling in the lattice schwinger model at criticality. Physical Review D, 106(1):014514, 2022.
  2. Brian DO Anderson. Reverse-time diffusion equation models. Stochastic Processes and their Applications, 12(3):313–326, 1982.
  3. An optimal control perspective on diffusion-based generative modeling. arXiv preprint arXiv:2211.01364, 2022.
  4. The geometric foundations of hamiltonian monte carlo. Bernoulli, 23(4A):2257–2298, 2017.
  5. Sampling using su (n) gauge equivariant flows. Physical Review D, 103(7):074504, 2021.
  6. Neural ordinary differential equations. Advances in neural information processing systems, 31, 2018.
  7. Underdamped langevin mcmc: A non-asymptotic analysis. In Conference on learning theory, pp.  300–323. PMLR, 2018.
  8. Density ratio estimation via infinitesimal classification. In International Conference on Artificial Intelligence and Statistics, pp.  2552–2573. PMLR, 2022.
  9. Density estimation using real nvp. arXiv preprint arXiv:1605.08803, 2016.
  10. D. Dua and C. Graff. Uci machine learning repository. https://archive.ics.uci.edu/ml/index.php, 2017. Accessed: 2022-03-13.
  11. Loris Felardos. Data-free Generation of Molecular Configurations with Normalizing Flows. PhD thesis, Université Grenoble Alpes, 2022.
  12. Learning lattice quantum field theories with equivariant continuous flows. arXiv preprint arXiv:2207.00283, 2022.
  13. A kernel two-sample test. The Journal of Machine Learning Research, 13(1):723–773, 2012.
  14. Auxiliary variational mcmc. In International Conference on Learning Representations, 2018.
  15. W Keith Hastings. Monte carlo sampling methods using markov chains and their applications. Biometrika, 1970.
  16. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840–6851, 2020.
  17. Eigenfold: Generative protein structure prediction with diffusion models. arXiv preprint arXiv:2304.02198, 2023.
  18. An introduction to variational autoencoders. Foundations and Trends® in Machine Learning, 12(4):307–392, 2019.
  19. Equivariant flows: exact likelihood generative learning for symmetric densities. In International conference on machine learning, pp.  5361–5370. PMLR, 2020.
  20. Generalizing hamiltonian monte carlo with neural networks. arXiv preprint arXiv:1711.09268, 2017.
  21. Neural network renormalization group. Physical review letters, 121(26):260601, 2018.
  22. Qiang Liu. Stein variational gradient descent as gradient flow. Advances in neural information processing systems, 30, 2017.
  23. Stein variational gradient descent: A general purpose bayesian inference algorithm. Advances in neural information processing systems, 29, 2016.
  24. Adversarially training mcmc with non-volume-preserving flows. Entropy, 24(3):415, 2022.
  25. Learning interpolations between boltzmann densities. Transactions on Machine Learning Research, 2023.
  26. Radford M Neal et al. Mcmc using hamiltonian dynamics. Handbook of markov chain monte carlo, 2(11):2, 2011.
  27. Boltzmann generators: Sampling equilibrium states of many-body systems with deep learning. Science, 365(6457):eaaw1147, 2019.
  28. Transition path sampling with boltzmann generator-based mcmc moves. In NeurIPS 2023 AI for Science Workshop, 2023.
  29. Improved sampling via learned diffusions. arXiv preprint arXiv:2307.01198, 2023.
  30. Exponential convergence of langevin distributions and their discrete approximations. Bernoulli, pp.  341–363, 1996.
  31. A contrastive divergence for combining variational inference and mcmc. In International Conference on Machine Learning, pp.  5537–5545. PMLR, 2019.
  32. Markov chain monte carlo and variational inference: Bridging the gap. In International conference on machine learning, pp.  1218–1226. PMLR, 2015.
  33. De-randomizing mcmc dynamics with the diffusion stein operator. Advances in Neural Information Processing Systems, 34:17507–17517, 2021.
  34. Deep unsupervised learning using nonequilibrium thermodynamics. In International conference on machine learning, pp.  2256–2265. PMLR, 2015.
  35. A-nice-mc: Adversarial training for mcmc. Advances in Neural Information Processing Systems, 30, 2017.
  36. Score-based generative modeling through stochastic differential equations. arXiv preprint arXiv:2011.13456, 2020.
  37. Maximum likelihood training of score-based diffusion models. Advances in Neural Information Processing Systems, 34:1415–1428, 2021.
  38. Gradients should stay on path: better estimators of the reverse-and forward kl divergence for normalizing flows. Machine Learning: Science and Technology, 3(4):045006, 2022.
  39. A boltzmann generator for the isobaric-isothermal ensemble. arXiv preprint arXiv:2305.08483, 2023.
  40. Denoising diffusion samplers. arXiv preprint arXiv:2302.13834, 2023.
  41. Stochastic normalizing flows. Advances in Neural Information Processing Systems, 33:5933–5944, 2020.
  42. Path integral sampler: a stochastic control approach for sampling. arXiv preprint arXiv:2111.15141, 2021.
  43. Ergodic inference: Accelerate convergence by optimisation. arXiv preprint arXiv:1805.10377, 2018.
  44. Towards predicting equilibrium distributions for molecular systems with deep learning. arXiv preprint arXiv:2306.05445, 2023.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets