Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
184 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Saturn: Sample-efficient Generative Molecular Design using Memory Manipulation (2405.17066v1)

Published 27 May 2024 in q-bio.BM and cs.LG

Abstract: Generative molecular design for drug discovery has very recently achieved a wave of experimental validation, with language-based backbones being the most common architectures employed. The most important factor for downstream success is whether an in silico oracle is well correlated with the desired end-point. To this end, current methods use cheaper proxy oracles with higher throughput before evaluating the most promising subset with high-fidelity oracles. The ability to directly optimize high-fidelity oracles would greatly enhance generative design and be expected to improve hit rates. However, current models are not efficient enough to consider such a prospect, exemplifying the sample efficiency problem. In this work, we introduce Saturn, which leverages the Augmented Memory algorithm and demonstrates the first application of the Mamba architecture for generative molecular design. We elucidate how experience replay with data augmentation improves sample efficiency and how Mamba synergistically exploits this mechanism. Saturn outperforms 22 models on multi-parameter optimization tasks relevant to drug discovery and may possess sufficient sample efficiency to consider the prospect of directly optimizing high-fidelity oracles.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (114)
  1. Ai-powered therapeutic target discovery. Trends in Pharmacological Sciences, 2023.
  2. AlphaFold accelerates artificial intelligence powered drug discovery: efficient discovery of a novel CDK20 small molecule inhibitor. Chem. Sci., 14(6):1443–1452, 2023.
  3. Discovery of novel and selective SIK2 inhibitors by the application of AlphaFold structures and generative models. Bioorg. Med. Chem., 91:117414, 2023.
  4. Discovery of potent, selective, and orally bioavailable small-molecule inhibitors of CDK8 for the treatment of cancer. J. Med. Chem., 2023.
  5. Discovery of 3-hydroxymethyl-azetidine derivatives as potent polymerase theta inhibitors. Bioorg. Med. Chem., page 117662, 2024.
  6. A small-molecule TNIK inhibitor targets fibrosis in preclinical and clinical models. Nat. Biotechnol., March 2024. ISSN 1546-1696. doi: 10.1038/s41587-024-02143-0.
  7. Ism9682a, a novel and potent kif18a inhibitor, shows robust antitumor effects against chromosomally unstable cancers. Cancer Research, 84(6_Supplement):5727–5727, 2024.
  8. Dockstream: a docking wrapper to enhance de novo molecular design. Journal of cheminformatics, 13:1–21, 2021.
  9. Augmented hill-climb increases reinforcement learning efficiency for language-based de novo molecule generation. Journal of cheminformatics, 14(1):68, 2022.
  10. Tacogfn: Target conditioned gflownet for drug design. In NeurIPS 2023 Generative AI and Biology (GenBio) Workshop, 2023.
  11. Hit and lead discovery with explorative rl and fragment-based molecule generation. Advances in Neural Information Processing Systems, 34:7924–7936, 2021.
  12. Exploring chemical space with score-based out-of-distribution generation. In International Conference on Machine Learning, pages 18872–18892. PMLR, 2023a.
  13. Drug discovery with dynamic goal-aware fragments. arXiv preprint arXiv:2310.00841, 2023b.
  14. Reinforced genetic algorithm for structure-based drug design. Advances in Neural Information Processing Systems, 35:12325–12338, 2022.
  15. Machine learning guided aqfep: A fast & efficient absolute free energy perturbation solution for virtual screening. 2023.
  16. Protein–ligand binding free energy calculations with fep+. Biomolecular simulations: methods and protocols, pages 201–232, 2019.
  17. Icolos: a workflow manager for structure-based post-processing of de novo generated small molecules. Bioinformatics, 38(21):4951–4952, 2022.
  18. Automated relative binding free energy calculations from smiles to δ𝛿\deltaitalic_δδ𝛿\deltaitalic_δg. Communications Chemistry, 6(1):82, 2023.
  19. Mfbind: a multi-fidelity approach for evaluating drug compounds in practical generative modeling. arXiv preprint arXiv:2402.10387, 2024.
  20. Sample efficiency matters: a benchmark for practical molecular optimization. Advances in neural information processing systems, 35:21342–21357, 2022.
  21. Augmented memory: Sample-efficient generative molecular design with reinforcement learning. JACS Au, 2024a.
  22. Beam enumeration: Probabilistic explainability for sample efficient self-conditioned molecular design. In Proc. 12th International Conference on Learning Representations, 2024b.
  23. Molecular de-novo design through deep reinforcement learning. Journal of cheminformatics, 9:1–14, 2017.
  24. Reinvent 2.0: an ai tool for de novo drug design. Journal of chemical information and modeling, 60(12):5918–5922, 2020a.
  25. Esben Jannik Bjerrum. Smiles enumeration as data augmentation for neural network modeling of molecules. arXiv preprint arXiv:1703.07076, 2017.
  26. Long short-term memory. Neural computation, 9(8):1735–1780, 1997.
  27. Attention is all you need. Advances in neural information processing systems, 30, 2017.
  28. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9, 2019.
  29. Mamba: Linear-time sequence modeling with selective state spaces. arXiv preprint arXiv:2312.00752, 2023.
  30. David Weininger. Smiles, a chemical language and information system. 1. introduction to methodology and encoding rules. Journal of chemical information and computer sciences, 28(1):31–36, 1988.
  31. Reinvent 4: Modern ai–driven generative molecule design. Journal of Cheminformatics, 16(1):20, 2024a.
  32. Generating focused molecule libraries for drug discovery with recurrent neural networks. ACS central science, 4(1):120–131, 2018.
  33. Exploring deep recurrent models with reinforcement learning for molecule design. In Proc. 6th International Conference on Learning Representations, 2018.
  34. Deep reinforcement learning for de novo drug design. Science advances, 4(7):eaap7885, 2018.
  35. Faster and more diverse de novo molecular optimization with double-loop reinforcement learning using augmented smiles. Journal of Computer-Aided Molecular Design, 37(8):373–394, 2023.
  36. Molgpt: molecular generation using a transformer-decoder model. Journal of Chemical Information and Modeling, 62(9):2064–2076, 2021.
  37. cmolgpt: A conditional generative pre-trained transformer for target-specific de novo molecular generation. Molecules, 28(11):4430, 2023.
  38. Molecular de novo design through transformer-based reinforcement learning. arXiv preprint arXiv:2310.05365, 2023.
  39. Molecule generation using transformers and policy gradient reinforcement learning. Scientific Reports, 13(1):8799, 2023.
  40. De novo drug design using reinforcement learning with multiple gpt agents. Advances in Neural Information Processing Systems, 36, 2024.
  41. Evaluation of reinforcement learning in transformer-based molecular design. 2024.
  42. Syntalinker: automatic fragment linking with deep conditional transformer neural networks. Chemical science, 11(31):8312–8322, 2020.
  43. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013.
  44. Automatic chemical design using a data-driven continuous representation of molecules. ACS central science, 4(2):268–276, 2018.
  45. Junction tree variational autoencoder for molecular graph generation. In International conference on machine learning, pages 2323–2332. PMLR, 2018.
  46. Deep learning enables rapid identification of potent ddr1 kinase inhibitors. Nature biotechnology, 37(9):1038–1040, 2019.
  47. Generative adversarial nets. Advances in neural information processing systems, 27, 2014.
  48. The cornucopia of meaningful leads: Applying deep adversarial autoencoders for new molecule development in oncology. Oncotarget, 8(7):10883, 2017.
  49. Objective-reinforced generative adversarial networks (organ) for sequence generation models. arXiv preprint arXiv:1705.10843, 2017.
  50. Optimizing distributions over molecular space. an objective-reinforced generative adversarial network for inverse-design chemistry (organic). 2017.
  51. Reinforced adversarial neural computer for de novo molecular design. Journal of chemical information and modeling, 58(6):1194–1204, 2018.
  52. Molgan: An implicit generative model for small molecular graphs. arXiv preprint arXiv:1805.11973, 2018.
  53. Chemistry42: an ai-driven platform for molecular design and optimization. Journal of Chemical Information and Modeling, 63(3):695–701, 2023.
  54. Graph convolutional policy network for goal-directed molecular graph generation. In Advances in neural information processing systems. NeurIPS, 2018.
  55. Multi-objective molecule generation using interpretable substructures. In International conference on machine learning, pages 4849–4859. PMLR, 2020a.
  56. Graph networks for molecular design. Machine Learning: Science and Technology, 2(2):025023, 2021.
  57. De novo drug design using reinforcement learning with graph-based deep generative models. Journal of Chemical Information and Modeling, 62(20):4863–4872, 2022.
  58. Learning to extend molecular scaffolds with structural motifs. In Proc. 10th International Conference on Learning Representations, 2022.
  59. DiGress: Discrete denoising diffusion for graph generation. In Proc. 11th International Conference on Learning Representations, 2023.
  60. Gflownet foundations. Journal of Machine Learning Research, 24(210):1–55, 2023.
  61. Flow network based generative models for non-iterative diverse candidate generation. Advances in Neural Information Processing Systems, 34:27381–27394, 2021.
  62. Melanie Mitchell. An introduction to genetic algorithms. MIT press, 1998.
  63. Jan H Jensen. A graph-based genetic algorithm and generative model/monte carlo tree search for the exploration of chemical space. Chemical science, 10(12):3567–3572, 2019.
  64. Equivariant 3d-conditional diffusion model for molecular linker design. Nature Machine Intelligence, pages 1–11, 2024.
  65. Flexible structure-based design of small molecules with equivariant diffusion models. In PROTEIN SCIENCE, volume 32. WILEY 111 RIVER ST, HOBOKEN 07030-5774, NJ USA, 2023.
  66. Self-referencing embedded strings (selfies): A 100% robust molecular string representation. Machine Learning: Science and Technology, 1(4):045024, 2020.
  67. Selfies and the future of molecular string representations. Patterns, 3(10), 2022.
  68. Michael A Skinnider. Invalid smiles are beneficial rather than detrimental to chemical language models. Nature Machine Intelligence, pages 1–12, 2024.
  69. Chemical language modeling with structured state spaces. 2024.
  70. Guacamol: benchmarking models for de novo molecular design. Journal of chemical information and modeling, 59(3):1096–1108, 2019.
  71. Molecular sets (moses): a benchmarking platform for molecular generation models. Frontiers in pharmacology, 11:565644, 2020.
  72. Randomized smiles strings improve the quality of molecular generative models. Journal of cheminformatics, 11:1–13, 2019.
  73. Generative molecular design in low data regimes. Nature Machine Intelligence, 2(3):171–180, 2020.
  74. Chemical language models enable navigation in sparsely populated chemical space. Nature Machine Intelligence, 3(9):759–770, 2021.
  75. De novo design of nurr1 agonists via fragment-augmented generative deep learning in low-data regime. Journal of Medicinal Chemistry, 66(12):8170–8177, 2023.
  76. Libinvent: reaction-based generative scaffold decoration for in silico library design. Journal of Chemical Information and Modeling, 62(9):2046–2063, 2021.
  77. Memory-assisted reinforcement learning for diverse molecular de novo design. Journal of cheminformatics, 12(1):68, 2020b.
  78. The properties of known drugs. 1. molecular frameworks. Journal of medicinal chemistry, 39(15):2887–2893, 1996.
  79. Chembl: a large-scale bioactivity database for drug discovery. Nucleic acids research, 40(D1):D1100–D1107, 2012.
  80. Drugex v2: de novo design of drug molecules by pareto-based multi-objective reinforcement learning in polypharmacology. Journal of cheminformatics, 13(1):85, 2021.
  81. Improving de novo molecular design with curriculum learning. Nature Machine Intelligence, 4(6):555–563, 2022.
  82. Roughness of molecular property landscapes and its impact on modellability. Journal of Chemical Information and Modeling, 62(19):4660–4671, 2022.
  83. Umap: Uniform manifold approximation and projection for dimension reduction. arXiv preprint arXiv:1802.03426, 2018.
  84. Structure of the d2 dopamine receptor bound to the atypical antipsychotic drug risperidone. Nature, 555(7695):269–273, 2018.
  85. Structure of acetylcholinesterase complexed with e2020 (aricept®): implications for the design of new anti-alzheimer drugs. Structure, 7(3):297–307, 1999.
  86. 2, 4-diaminopyrimidine mk2 inhibitors. part i: observation of an unexpected inhibitor binding mode. Bioorganic & medicinal chemistry letters, 20(1):330–333, 2010.
  87. Quantifying the chemical beauty of drugs. Nature chemistry, 4(2):90–98, 2012.
  88. Autodock vina: improving the speed and accuracy of docking with a new scoring function, efficient optimization, and multithreading. Journal of computational chemistry, 31(2):455–461, 2010.
  89. Zinc 15–ligand discovery for everyone. Journal of chemical information and modeling, 55(11):2324–2337, 2015.
  90. Fast, accurate, and reliable molecular docking with quickvina 2. Bioinformatics, 31(13):2214–2216, 2015.
  91. Estimation of synthetic accessibility score of drug-like molecules based on molecular complexity and fragment contributions. Journal of cheminformatics, 1:1–11, 2009.
  92. How much space has been explored? measuring the chemical space covered by databases and machine-generated molecules. In Proc. 11th International Conference on Learning Representations, 2023.
  93. Graphaf: a flow-based autoregressive model for molecular graph generation. In Proc. 8th International Conference on Learning Representations, 2020.
  94. Autonomous molecule generation using reinforcement learning and docking to develop potential novel inhibitors. Scientific reports, 10(1):22104, 2020.
  95. Hierarchical generation of molecular graphs using structural motifs. In International conference on machine learning, pages 4839–4848. PMLR, 2020b.
  96. Graphdf: A discrete flow model for molecular graph generation. In International conference on machine learning, pages 7192–7203. PMLR, 2021.
  97. Limo: Latent inceptionism for targeted molecule generation. In International conference on machine learning. PMLR, 2022.
  98. Score-based generative modeling of graphs via the system of stochastic differential equations. In International Conference on Machine Learning, pages 10362–10383. PMLR, 2022.
  99. Augmenting genetic algorithms with deep neural networks for exploring the chemical space. In Proc. 8th International Conference on Learning Representations, 2020.
  100. Mars: Markov molecular sampling for multi-objective drug discovery. In Proc. 9th International Conference on Learning Representations, 2021.
  101. Guiding deep molecular optimization with genetic exploration. volume 33, pages 12008–12021, 2020.
  102. Molecule generation by principal subgraph mining and assembling. Advances in Neural Information Processing Systems, 35:2550–2563, 2022.
  103. Optimal molecular design: Generative active learning combining reinvent with absolute binding free energy simulations. 2024b.
  104. Sample efficient reinforcement learning with active learning for molecular design. Chemical Science, 15(11):4146–4160, 2024.
  105. Link-invent: generative linker design with reinforcement learning. Digital Discovery, 2(2):392–408, 2023.
  106. Molecular optimization by capturing chemist’s intuition using deep neural networks. Journal of cheminformatics, 13:1–17, 2021.
  107. Exhaustive local chemical space exploration using a transformer model. 2023.
  108. Language models can learn complex molecular distributions. Nature Communications, 13(1):3293, 2022.
  109. Genetic algorithms are strong baselines for molecule generation. arXiv preprint arXiv:2310.09267, 2023.
  110. Importance resampling for off-policy prediction. Advances in Neural Information Processing Systems, 32, 2019.
  111. Protein and ligand preparation: parameters, protocols, and influence on virtual screening enrichments. Journal of computer-aided molecular design, 27:221–234, 2013.
  112. Opls3e: Extending force field coverage for drug-like small molecules. Journal of chemical theory and computation, 15(3):1863–1874, 2019.
  113. Uff, a full periodic table force field for molecular mechanics and molecular dynamics simulations. Journal of the American chemical society, 114(25):10024–10035, 1992.
  114. Emergent abilities of large language models. arXiv preprint arXiv:2206.07682, 2022.
Citations (3)
View on Semantic Scholar

Summary

  • The paper introduces an augmented memory and Mamba integration that boosts sample efficiency for generative molecular design.
  • The paper demonstrates superior performance by optimizing high-fidelity oracles and surpassing 22 models in multi-parameter optimization tasks.
  • The paper outlines an oracle caching strategy that reduces redundant evaluations, enhancing computational efficiency in drug discovery.

An Expert Analysis of "Saturn: Sample-efficient Generative Molecular Design using Memory Manipulation"

The paper presents Saturn, a novel framework aimed at enhancing the sample efficiency of generative molecular design, crucially influencing drug discovery processes. The cornerstone of this work is Saturn's ability to optimize high-fidelity oracles, which typically demand substantial computational resources, through the employment of memory manipulation techniques. The presentation leverages the augmented memory algorithm and utilizes the Mamba architecture, marking its first application in generative molecular design.

Key Contributions

  1. Augmented Memory and Mamba Integration: Saturn utilizes augmented memory to revisit and enhance the learning from high-reward samples, significantly improving sample efficiency. Mamba's architecture, known for its selective scanning capabilities, aligns seamlessly with this approach, facilitating a robust mechanism to synthesize molecular designs effectively.
  2. Performance Benchmarking: Saturn outperforms a range of models—22, to be specific—accomplishing tasks associated with multi-parameter optimization (MPO) in drug discovery. The promising results suggest Saturn's potential to directly optimize demanding high-fidelity oracles without the necessity of less precise surrogate models.
  3. Oracle Evaluation Strategy: The introduction of oracle caching during molecular generation mitigates redundancies in oracle calls by assuming near determinism in rewards, which optimizes computational load and expedites processes.

Insights into Mechanism and Efficiency

The paper delineates how augmented memory, in synergy with data augmentation, boosts Saturn's sample efficiency. Notably, the paper shows that data augmentation modifies sampling likelihoods in a fashion that biases generations towards the replay buffer's molecules, thus fostering effective local exploration—a method the authors refer to as "hop-and-locally-explore."

On the architectural front, Saturn's deployment of the Mamba framework enhances the model's ability to approximate optimal solutions quickly and accurately. With a focus on autoregressive LLMing, Saturn approximates molecular designs through experience-driven adjustments, reinforcing Saturn's efficacy in the face of MPO objectives.

Implications and Future Directions

The implications of Saturn's contributions resonate across both practical applications and theoretical advancements. Practically, Saturn enables substantial improvement in the efficiency of computational workflows involved in the drug discovery pipeline, effectively handling expensive oracle evaluations. Theoretically, the model’s lean toward high-fidelity oracle optimization opens discussions on evolving generative models toward even more complex simulations and expansive molecular spaces.

The prospect of employing Saturn in fields requiring intricate simulations, such as quantum mechanics/molecular mechanics (QM/MM) and free energy protocols, is a viable future avenue. Additionally, potential developments could explore curriculum learning strategies or modifications to adapt Saturn for optimization landscapes characterized by increased roughness, where the current "hop-and-locally-explore" strategy might need reevaluation.

Conclusion

The introduction of Saturn underscores significant advancements in sample-efficient molecular design. By amalgamating memory manipulation strategies with cutting-edge architectural frameworks, Saturn not only augments the precision of generative outputs but also sets a precedence in the direct optimization of high-fidelity oracles, crucially impacting drug discovery and related computational fields.

File Document Download Save Streamline Icon: https://streamlinehq.com PDF File Document Download Save Streamline Icon: https://streamlinehq.com Markdown