Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
153 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Stochastic Generative Flow Networks (2302.09465v3)

Published 19 Feb 2023 in cs.LG

Abstract: Generative Flow Networks (or GFlowNets for short) are a family of probabilistic agents that learn to sample complex combinatorial structures through the lens of "inference as control". They have shown great potential in generating high-quality and diverse candidates from a given energy landscape. However, existing GFlowNets can be applied only to deterministic environments, and fail in more general tasks with stochastic dynamics, which can limit their applicability. To overcome this challenge, this paper introduces Stochastic GFlowNets, a new algorithm that extends GFlowNets to stochastic environments. By decomposing state transitions into two steps, Stochastic GFlowNets isolate environmental stochasticity and learn a dynamics model to capture it. Extensive experimental results demonstrate that Stochastic GFlowNets offer significant advantages over standard GFlowNets as well as MCMC- and RL-based approaches, on a variety of standard benchmarks with stochastic dynamics.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (45)
  1. An introduction to mcmc for machine learning. Machine learning, 50(1):5–43, 2003.
  2. Planning in stochastic environments with a learned model. In International Conference on Learning Representations, 2021.
  3. Flow network based generative models for non-iterative diverse candidate generation. Advances in Neural Information Processing Systems, 34:27381–27394, 2021a.
  4. GFlowNet foundations. arXiv preprint 2111.09266, 2021b.
  5. Bayesian structure learning with generative flow networks. Uncertainty in Artificial Intelligence (UAI), 2022.
  6. Addressing function approximation error in actor-critic methods. In International conference on machine learning, pages 1587–1596. PMLR, 2018.
  7. Generative adversarial nets. Neural Information Processing Systems (NIPS), pages 2672–2680, 2014.
  8. Reinforcement learning with deep energy-based policies. International Conference on Machine Learning (ICML), 2017.
  9. Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor. International Conference on Machine Learning (ICML), 2018.
  10. Dream to control: Learning behaviors by latent imagination. arXiv preprint arXiv:1912.01603, 2019.
  11. W Keith Hastings. Monte carlo sampling methods using markov chains and their applications. 1970.
  12. Learning continuous control policies by stochastic value gradients. Advances in neural information processing systems, 28, 2015.
  13. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems, 33:6840–6851, 2020.
  14. Biological sequence design with GFlowNets. International Conference on Machine Learning (ICML), 2022a.
  15. Multi-objective gflownets. arXiv preprint arXiv:2210.12765, 2022b.
  16. Adam: A method for stochastic optimization. International Conference on Learning Representations (ICLR), 2015.
  17. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013.
  18. Diversity in recommender systems–a survey. Knowledge-based systems, 123:154–162, 2017.
  19. Continuous control with deep reinforcement learning. arXiv preprint arXiv:1509.02971, 2015.
  20. Gflowout: Dropout with generative flow networks. ArXiv, abs/2210.12928, 2022.
  21. Revisiting the arcade learning environment: Evaluation protocols and open problems for general agents. Journal of Artificial Intelligence Research, 61:523–562, 2018.
  22. Learning GFlowNets from partial episodes for improved convergence and stability. ICLR’2023; arXiv:2209.12782, 2022.
  23. Trajectory balance: Improved credit assignment in GFlowNets. Neural Information Processing Systems (NeurIPS), 2022a.
  24. Gflownets and variational inference. arXiv preprint arXiv:2210.00580, 2022b.
  25. Equation of state calculations by fast computing machines. The journal of chemical physics, 21(6):1087–1092, 1953.
  26. Human-level control through deep reinforcement learning. nature, 518(7540):529–533, 2015.
  27. Asynchronous methods for deep reinforcement learning. Neural Information Processing Systems (NIPS), 2016.
  28. Bayesian learning of causal structure and mechanisms with GFlowNets and variational bayes. arXiv preprint 2211.02763, 2022.
  29. Generative augmented flow networks. arXiv preprint 2210.03308, 2022.
  30. Better training of gflownets with local credit and incomplete trajectories. ArXiv, abs/2302.01687, 2023.
  31. You can’t count on luck: Why decision transformers fail in stochastic environments. arXiv preprint arXiv:2205.15967, 2022.
  32. Dbaasp v3: database of antimicrobial/cytotoxic activity and structure of peptides as a resource for development of new therapeutics. Nucleic acids research, 49(D1):D288–D297, 2021.
  33. Mastering atari, go, chess and shogi by planning with a learned model. Nature, 588(7839):604–609, 2020.
  34. Proximal policy optimization algorithms. arXiv preprint 1707.06347, 2017.
  35. Large-scale de novo oligonucleotide synthesis for whole-genome synthesis and data storage: Challenges and opportunities. Frontiers in bioengineering and biotechnology, 9:689797, 2021.
  36. Reinforcement learning: An introduction. MIT press, 2018.
  37. Design-bench: Benchmarks for data-driven offline model-based optimization. In International Conference on Machine Learning, pages 21658–21676. PMLR, 2022.
  38. Attention is all you need. Neural Information Processing Systems (NIPS), 2017.
  39. Improving multi-step prediction of learned time series models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 29, 2015.
  40. Mars: Markov molecular sampling for multi-objective drug discovery. arXiv preprint arXiv:2103.10432, 2021.
  41. Dichotomy of control: Separating what you can control from what you cannot. arXiv preprint arXiv:2210.13435, 2022.
  42. Robust scheduling with GFlowNets. International Conference on Learning Representations (ICLR), 2023a.
  43. Unifying generative models with GFlowNets. arXiv preprint 2209.02606, 2022a.
  44. Generative flow networks for discrete probabilistic modeling. International Conference on Machine Learning (ICML), 2022b.
  45. Distributional gflownets with quantile flows. arXiv preprint arXiv:2302.05793, 2023b.
Citations (26)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now