Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
139 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
46 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Genetic-guided GFlowNets for Sample Efficient Molecular Optimization (2402.05961v4)

Published 5 Feb 2024 in q-bio.BM, cs.LG, and cs.NE

Abstract: The challenge of discovering new molecules with desired properties is crucial in domains like drug discovery and material design. Recent advances in deep learning-based generative methods have shown promise but face the issue of sample efficiency due to the computational expense of evaluating the reward function. This paper proposes a novel algorithm for sample-efficient molecular optimization by distilling a powerful genetic algorithm into deep generative policy using GFlowNets training, the off-policy method for amortized inference. This approach enables the deep generative policy to learn from domain knowledge, which has been explicitly integrated into the genetic algorithm. Our method achieves state-of-the-art performance in the official molecular optimization benchmark, significantly outperforming previous methods. It also demonstrates effectiveness in designing inhibitors against SARS-CoV-2 with substantially fewer reward calls.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (52)
  1. Guiding deep molecular optimization with genetic exploration. Advances in neural information processing systems, 33:12008–12021, 2020.
  2. Neural combinatorial optimization with reinforcement learning. arXiv preprint arXiv:1611.09940, 2016.
  3. Flow network based generative models for non-iterative diverse candidate generation. Advances in Neural Information Processing Systems, 34:27381–27394, 2021a.
  4. Gflownet foundations. arXiv preprint arXiv:2111.09266, 2021b.
  5. Quantifying the chemical beauty of drugs. Nature chemistry, 4(2):90–98, 2012.
  6. GuacaMol: benchmarking models for de novo molecular design. Journal of chemical information and modeling, 59(3):1096–1108, 2019.
  7. Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint arXiv:1412.3555, 2014.
  8. Sample efficiency matters: a benchmark for practical molecular optimization. Advances in Neural Information Processing Systems, 35:21342–21357, 2022a.
  9. Amortized tree generation for bottom-up synthesis planning and synthesizable molecular design. In International Conference on Learning Representations, 2022b. URL https://openreview.net/forum?id=FRxhHdnxt1.
  10. Automatic chemical design using a data-driven continuous representation of molecules. ACS central science, 4(2):268–276, 2018.
  11. Efficient active search for combinatorial optimization problems. arXiv preprint arXiv:2106.05126, 2021.
  12. Amortizing intractable inference in large language models, 2023.
  13. Therapeutics data commons: Machine learning datasets and tasks for drug discovery and development. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 1), 2021. URL https://openreview.net/forum?id=8nvgnORnoWr.
  14. Principles of early drug discovery. British journal of pharmacology, 162(6):1239–1249, 2011.
  15. Biological sequence design with gflownets. In International Conference on Machine Learning, pp.  9786–9801. PMLR, 2022.
  16. Learning energy decompositions for partial inference of gflownets, 2023.
  17. Jensen, J. H. A graph-based genetic algorithm and generative model/monte carlo tree search for the exploration of chemical space. Chemical science, 10(12):3567–3572, 2019.
  18. Enhancing sample efficiency in black-box combinatorial optimization via symmetric replay training, 2023a.
  19. Symmetric Replay Training: Enhancing sample efficiency in deep reinforcement learning for combinatorial optimization, 2023b.
  20. Learning collaborative policies to solve np-hard routing problems. Advances in Neural Information Processing Systems, 34:10418–10430, 2021.
  21. Sym-nco: Leveraging symmetricity for neural combinatorial optimization. Advances in Neural Information Processing Systems, 35:1936–1949, 2022.
  22. Learning to scale logits for temperature-conditional GFlowNets. arXiv preprint arXiv:2310.02823, 2023c.
  23. Local search gflownets. arXiv preprint arXiv:2310.02710, 2023d.
  24. Attention, learn to solve routing problems! In International Conference on Learning Representations, 2018.
  25. Self-referencing embedded strings (SELFIES): A 100% robust molecular string representation. Machine Learning: Science and Technology, 1(4):045024, 2020.
  26. POMO: Policy optimization with multiple optima for reinforcement learning. Advances in Neural Information Processing Systems, 33:21188–21198, 2020.
  27. Multi-objective de novo drug design with conditional graph generative model. Journal of cheminformatics, 10:1–24, 2018.
  28. Gflownets with human feedback, 2023.
  29. Baking symmetry into gflownets. In NeurIPS 2023 AI for Science Workshop, 2023.
  30. Learning gflownets from partial episodes for improved convergence and stability. In International Conference on Machine Learning, pp.  23467–23483. PMLR, 2023.
  31. Trajectory balance: Improved credit assignment in gflownets. Advances in Neural Information Processing Systems, 35:5955–5967, 2022.
  32. Augmenting genetic algorithms with deep neural networks for exploring the chemical space. In International Conference on Learning Representations, 2019.
  33. Beyond generative models: superfast traversal, optimization, novelty, exploration and discovery (STONED) algorithm for molecules using SELFIES. Chemical science, 12(20):7079–7090, 2021.
  34. Molecular de-novo design through deep reinforcement learning. Journal of cheminformatics, 9(1):1–14, 2017.
  35. Generative augmented flow networks. In The Eleventh International Conference on Learning Representations, 2022.
  36. Better training of gflownets with local credit and incomplete trajectories. arXiv preprint arXiv:2302.01687, 2023.
  37. Polishchuk, P. CReM: chemically reasonable mutations framework for structure generation. Journal of Cheminformatics, 12(1):1–18, 2020.
  38. Direct preference optimization: Your language model is secretly a reward model. arXiv preprint arXiv:2305.18290, 2023.
  39. Thompson sampling for improved exploration in gflownets. arXiv preprint arXiv:2306.17693, 2023.
  40. Towards understanding and improving GFlowNet training. In Proceedings of the 40th International Conference on Machine Learning, 2023.
  41. Meta-SAGE: Scale meta-learning scheduled adaptation with guided exploration for mitigating scale shift on combinatorial optimization. In Krause, A., Brunskill, E., Cho, K., Engelhardt, B., Sabato, S., and Scarlett, J. (eds.), Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pp.  32194–32210. PMLR, 23–29 Jul 2023. URL https://proceedings.mlr.press/v202/son23a.html.
  42. Zinc 15–ligand discovery for everyone. Journal of chemical information and modeling, 55(11):2324–2337, 2015.
  43. Genetic algorithms are strong baselines for molecule generation. arXiv preprint arXiv:2310.09267, 2023.
  44. Sample-efficient optimization in the latent space of deep generative models via weighted retraining. Advances in Neural Information Processing Systems, 33:11259–11272, 2020.
  45. A fresh look at de novo molecular design benchmarks. In NeurIPS 2021 AI for Science Workshop, 2021.
  46. A hybrid genetic algorithm for multidepot and periodic vehicle routing problems. Operations Research, 60(3):611–624, 2012.
  47. Weininger, D. SMILES, a chemical language and information system. 1. introduction to methodology and encoding rules. Journal of chemical information and computer sciences, 28(1):31–36, 1988.
  48. Non-fullerene acceptors for organic solar cells. Nature Reviews Materials, 3(3):1–19, 2018.
  49. Population-based de novo molecule generation, using grammatical evolution. Chemistry Letters, 47(11):1431–1434, 2018.
  50. Optimization of molecules via deep reinforcement learning. Scientific reports, 9(1):1–10, 2019.
  51. Fine-tuning language models from human preferences. arXiv preprint arXiv:1909.08593, 2019.
  52. A variational perspective on generative flow networks. Transactions on Machine Learning Research, 2023. ISSN 2835-8856. URL https://openreview.net/forum?id=AZ4GobeSLq.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now