Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
143 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
46 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Revealing the Relationship Between Publication Bias and Chemical Reactivity with Contrastive Learning (2402.16882v2)

Published 19 Feb 2024 in physics.chem-ph, cs.AI, cs.LG, and q-bio.BM

Abstract: A synthetic method's substrate tolerance and generality are often showcased in a "substrate scope" table. However, substrate selection exhibits a frequently discussed publication bias: unsuccessful experiments or low-yielding results are rarely reported. In this work, we explore more deeply the relationship between such publication bias and chemical reactivity beyond the simple analysis of yield distributions using a novel neural network training strategy, substrate scope contrastive learning. By treating reported substrates as positive samples and non-reported substrates as negative samples, our contrastive learning strategy teaches a model to group molecules within a numerical embedding space, based on historical trends in published substrate scope tables. Training on 20,798 aryl halides in the CAS Content Collection${\text{TM}}$, spanning thousands of publications from 2010-2015, we demonstrate that the learned embeddings exhibit a correlation with physical organic reactivity descriptors through both intuitive visualizations and quantitative regression analyses. Additionally, these embeddings are applicable to various reaction modeling tasks like yield prediction and regioselectivity prediction, underscoring the potential to use historical reaction data as a pre-training task. This work not only presents a chemistry-specific machine learning training strategy to learn from literature data in a new way, but also represents a unique approach to uncover trends in chemical reactivity reflected by trends in substrate selection in publications.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (30)
  1. Huh, M.; Agrawal, P.; Efros, A. A. What makes ImageNet good for transfer learning? arXiv preprint arXiv:1608.08614 2016,
  2. Devlin, J.; Chang, M.-W.; Lee, K.; Toutanova, K. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 2018,
  3. Radford, A.; Narasimhan, K.; Salimans, T.; Sutskever, I., et al. Improving language understanding by generative pre-training. 2018,
  4. Bommasani, R.; Hudson, D. A.; Adeli, E.; Altman, R.; Arora, S.; von Arx, S.; Bernstein, M. S.; Bohg, J.; Bosselut, A.; Brunskill, E., et al. On the opportunities and risks of foundation models. arXiv preprint arXiv:2108.07258 2021,
  5. He, K.; Chen, X.; Xie, S.; Li, Y.; Dollár, P.; Girshick, R. Masked autoencoders are scalable vision learners. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2022; pp 16000–16009
  6. Chen, W.-Y.; Liu, Y.-C.; Kira, Z.; Wang, Y.-C. F.; Huang, J.-B. A closer look at few-shot classification. arXiv preprint arXiv:1904.04232 2019,
  7. Koch, G.; Zemel, R.; Salakhutdinov, R., et al. Siamese neural networks for one-shot image recognition. ICML deep learning workshop. 2015
  8. Snell, J.; Swersky, K.; Zemel, R. Prototypical networks for few-shot learning. Advances in neural information processing systems 2017, 30
  9. Sung, F.; Yang, Y.; Zhang, L.; Xiang, T.; Torr, P. H.; Hospedales, T. M. Learning to compare: Relation network for few-shot learning. Proceedings of the IEEE conference on computer vision and pattern recognition. 2018; pp 1199–1208
  10. Li, H.; Eigen, D.; Dodge, S.; Zeiler, M.; Wang, X. Finding task-relevant features for few-shot learning by category traversal. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2019; pp 1–10
  11. Rao, R.; Meier, J.; Sercu, T.; Ovchinnikov, S.; Rives, A. Transformer protein language models are unsupervised structure learners. International Conference on Learning Representations. 2020
  12. Hu, W.; Liu, B.; Gomes, J.; Zitnik, M.; Liang, P.; Pande, V.; Leskovec, J. Strategies For Pre-training Graph Neural Networks. International Conference on Learning Representations (ICLR). 2020
  13. Wang, H.; Li, W.; Jin, X.; Cho, K.; Ji, H.; Han, J.; Burke, M. D. Chemical-reaction-aware molecule representation learning. arXiv preprint arXiv:2109.09888 2021,
  14. Liu, S.; Wang, H.; Liu, W.; Lasenby, J.; Guo, H.; Tang, J. Pre-training molecular graph representation with 3d geometry. arXiv preprint arXiv:2110.07728 2021,
  15. Stärk, H.; Beaini, D.; Corso, G.; Tossou, P.; Dallago, C.; Günnemann, S.; Liò, P. 3d infomax improves gnns for molecular property prediction. International Conference on Machine Learning. 2022; pp 20479–20502
  16. Honda, S.; Shi, S.; Ueda, H. R. Smiles transformer: Pre-trained molecular fingerprint for low data drug discovery. arXiv preprint arXiv:1911.04738 2019,
  17. Chithrananda, S.; Grand, G.; Ramsundar, B. ChemBERTa: large-scale self-supervised pretraining for molecular property prediction. arXiv preprint arXiv:2010.09885 2020,
  18. Frey, N. C.; Soklaski, R.; Axelrod, S.; Samsi, S.; Gomez-Bombarelli, R.; Coley, C. W.; Gadepally, V. Neural scaling of deep chemical models. Nature Machine Intelligence 2023, 1–9
  19. Wang, X.; Zhao, H.; Tu, W.-w.; Yao, Q. Automated 3D pre-training for molecular property prediction. Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 2023; pp 2419–2430
  20. Schultz, M.; Joachims, T. Learning a distance metric from relative comparisons. Advances in neural information processing systems 2003, 16
  21. Kozlowski, M. C. On the Topic of Substrate Scope. 2022
  22. Xu, K.; Hu, W.; Leskovec, J.; Jegelka, S. How powerful are graph neural networks? arXiv preprint arXiv:1810.00826 2018,
  23. Hinton, G. E.; Roweis, S. Stochastic neighbor embedding. Advances in neural information processing systems 2002, 15
  24. Kohn, W.; Sham, L. Density functional theory. CONFERENCE PROCEEDINGS-ITALIAN PHYSICAL SOCIETY. 1996; pp 561–572
  25. Frisch, M. J. et al. Gaussian˜16 Revision C.01. 2016; Gaussian Inc. Wallingford CT
  26. Yuan, T.; Deng, W.; Tang, J.; Tang, Y.; Chen, B. Signal-to-noise ratio: A robust distance metric for deep metric learning. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2019; pp 4815–4824
  27. Kingma, D. P.; Ba, J. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 2014,
  28. Huang, K.; Fu, T.; Gao, W.; Zhao, Y.; Roohani, Y. H.; Leskovec, J.; Coley, C. W.; Xiao, C.; Sun, J.; Zitnik, M. Therapeutics Data Commons: Machine Learning Datasets and Tasks for Drug Discovery and Development. Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 1). 2021
  29. Wolf, T.; Debut, L.; Sanh, V.; Chaumond, J.; Delangue, C.; Moi, A.; Cistac, P.; Rault, T.; Louf, R.; Funtowicz, M., et al. Huggingface’s transformers: State-of-the-art natural language processing. arXiv preprint arXiv:1910.03771 2019,
  30. Noutahi, E.; Wognum, C.; Mary, H.; Hounwanou, H.; Kovary, K. M.; Gilmour, D.; Burns, J.; St-Laurent, J.; D.,; Maheshkar, S.; rbyrne momatx, datamol-io/molfeat: 0.9.4 (0.9.4). 2023; Zenodo. https://doi.org/10.5281/zenodo.8373019

Summary

We haven't generated a summary for this paper yet.