Generative Flow Networks: a Markov Chain Perspective (2307.01422v1)
Abstract: While Markov chain Monte Carlo methods (MCMC) provide a general framework to sample from a probability distribution defined up to normalization, they often suffer from slow convergence to the target distribution when the latter is highly multi-modal. Recently, Generative Flow Networks (GFlowNets) have been proposed as an alternative framework to mitigate this issue when samples have a clear compositional structure, by treating sampling as a sequential decision making problem. Although they were initially introduced from the perspective of flow networks, the recent advances of GFlowNets draw more and more inspiration from the Markov chain literature, bypassing completely the need for flows. In this paper, we formalize this connection and offer a new perspective for GFlowNets using Markov chains, showing a unifying view for GFlowNets regardless of the nature of the state space as recurrent Markov chains. Positioning GFlowNets under the same theoretical framework as MCMC methods also allows us to identify the similarities between both frameworks, and most importantly to highlight their
- Flow Network based Generative Models for Non-Iterative Diverse Candidate Generation. Neural Information Processing Systems, 2021a.
- GFlowNet Foundations. arXiv preprint, 2021b.
- Bayesian Structure Learning with Generative Flow Networks. Uncertainty in Artificial Intelligence, 2022.
- Joint Bayesian Inference of Graphical Structure and Parameters with a Single Generative Flow Network. arXiv preprint, 2023.
- Markov Chains. Springer, 2018.
- GFlowNet-EM for learning compositional latent variable models. International Conference on Machine Learning, 2023.
- Biological Sequence Design with GFlowNets. International Conference on Machine Learning, 2022.
- GFlowNets for AI-Driven Scientific Discovery. Digital Discovery, 2023.
- A Theory of Continuous Generative Flow Networks. International Conference on Machine Learning, 2023.
- CFlowNets: Continuous control with Generative Flow Networks. International Conference on Learning Representations, 2023.
- David JC MacKay. Information theory, inference and learning algorithms. Cambridge university press, 2003.
- Trajectory balance: Improved credit assignment in GFlowNets. Neural Information Processing Systems, 2022.
- Markov Chains and Stochastic Stability. Springer-Verlag, 1993.
- Radford M Neal. Probabilistic inference using Markov chain Monte Carlo methods. 1993.
- Esa Nummelin. A splitting technique for Harris recurrent Markov chains. Zeitschrift für Wahrscheinlichkeitstheorie und verwandte Gebiete, 1978.
- Unifying generative models with GFlowNets and beyond. arXiv preprint, 2023.
Collections
Sign up for free to add this paper to one or more collections.
