Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
157 tokens/sec
GPT-4o
43 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Local Search GFlowNets (2310.02710v2)

Published 4 Oct 2023 in cs.LG and stat.ML

Abstract: Generative Flow Networks (GFlowNets) are amortized sampling methods that learn a distribution over discrete objects proportional to their rewards. GFlowNets exhibit a remarkable ability to generate diverse samples, yet occasionally struggle to consistently produce samples with high rewards due to over-exploration on wide sample space. This paper proposes to train GFlowNets with local search, which focuses on exploiting high-rewarded sample space to resolve this issue. Our main idea is to explore the local neighborhood via backtracking and reconstruction guided by backward and forward policies, respectively. This allows biasing the samples toward high-reward solutions, which is not possible for a typical GFlowNet solution generation scheme, which uses the forward policy to generate the solution from scratch. Extensive experiments demonstrate a remarkable performance improvement in several biochemical tasks. Source code is available: \url{https://github.com/dbsxodud-11/ls_gfn}.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (37)
  1. Dyngfn: Bayesian dynamic causal discovery using generative flow networks. arXiv preprint arXiv:2302.04178, 2023.
  2. Flow network based generative models for non-iterative diverse candidate generation. Advances in Neural Information Processing Systems, 34:27381–27394, 2021.
  3. Gflownet foundations. Journal of Machine Learning Research, 24(210):1–55, 2023. URL http://jmlr.org/papers/v24/22-0364.html.
  4. Bayesian structure learning with generative flow networks. In Uncertainty in Artificial Intelligence, pp.  518–528. PMLR, 2022.
  5. Joint bayesian inference of graphical structure and parameters with a single generative flow network. arXiv preprint arXiv:2305.19366, 2023.
  6. Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor. In International conference on machine learning, pp. 1861–1870. PMLR, 2018.
  7. WK Hastings. Monte carlo sampling methods using markov chains and their applications. Biometrika, pp.  97–109, 1970.
  8. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840–6851, 2020.
  9. Biological sequence design with gflownets. In International Conference on Machine Learning, pp. 9786–9801. PMLR, 2022a.
  10. Multi-objective gflownets. In International Conference on Machine Learning, 2022b. URL https://api.semanticscholar.org/CorpusID:253097761.
  11. Gflownets for ai-driven scientific discovery. Digital Discovery, 2(3):557–577, 2023a.
  12. GFlowNets for AI-driven scientific discovery. Digital Discovery, 2023b.
  13. Multi-objective gflownets. In International Conference on Machine Learning, pp. 14631–14653. PMLR, 2023c.
  14. Adam: A method for stochastic optimization. In International Conference on Learning Representations (ICLR), San Diega, CA, USA, 2015.
  15. A tutorial on energy-based learning. Predicting structured data, 1(0), 2006.
  16. Batch multi-fidelity active learning with budget constraints. Advances in Neural Information Processing Systems, 35:995–1007, 2022.
  17. Learning gflownets from partial episodes for improved convergence and stability. In International Conference on Machine Learning, pp. 23467–23483. PMLR, 2023.
  18. Trajectory balance: Improved credit assignment in gflownets. Advances in Neural Information Processing Systems, 35:5955–5967, 2022.
  19. GFlowNets and variational inference. International Conference on Learning Representations (ICLR), 2023.
  20. Asynchronous methods for deep reinforcement learning. In International conference on machine learning, pp. 1928–1937. PMLR, 2016.
  21. Generative augmented flow networks. International Conference on Learning Representations (ICLR), 2022.
  22. Better training of GFlowNets with local credit and incomplete trajectories. International Conference on Machine Learning (ICML), 2023a.
  23. Stochastic generative flow networks. Conference on Uncertainty in Artificial Intelligence, 2023b.
  24. Hierarchical variational models. In International conference on machine learning, pp. 324–333. PMLR, 2016.
  25. Thompson sampling for improved exploration in gflownets. arXiv preprint arXiv:2306.17693, 2023.
  26. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017.
  27. Towards understanding and improving GFlowNet training. In International Conference on Machine Learning, pp. 30956–30975. PMLR, 2023.
  28. Adalead: A simple and robust adaptive greedy search algorithm for sequence design. arXiv preprint arXiv:2010.02141, 2020.
  29. Richard S Sutton. Learning to predict by the methods of temporal differences. Machine learning, 3:9–44, 1988.
  30. Design-bench: Benchmarks for data-driven offline model-based optimization. In International Conference on Machine Learning, pp. 21658–21676. PMLR, 2022.
  31. Mars: Markov molecular sampling for multi-objective drug discovery. In International Conference on Learning Representations, 2020.
  32. Robust scheduling with gflownets. In The Eleventh International Conference on Learning Representations, 2022a.
  33. Generative flow networks for discrete probabilistic modeling. In International Conference on Machine Learning, pp. 26412–26428. PMLR, 2022b.
  34. Unifying generative models with GFlowNets and beyond. International Conference on Machine Learning (ICML) workshop of Beyond Bayes:Paths Towards Universal Reasoning Systems, 2023a.
  35. Let the flows tell: Solving graph combinatorial optimization problems with gflownets. arXiv preprint arXiv:2305.17010, 2023b.
  36. Distributional GFlowNets with quantile flows. arXiv preprint 2302.05793, 2023c.
  37. Molecular mechanics-driven graph neural network with multiplex graph for molecular structures. arXiv preprint arXiv:2011.07457, 2020.
Citations (25)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now