Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
110 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
44 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Structure-Aware E(3)-Invariant Molecular Conformer Aggregation Networks (2402.01975v3)

Published 3 Feb 2024 in cs.LG

Abstract: A molecule's 2D representation consists of its atoms, their attributes, and the molecule's covalent bonds. A 3D (geometric) representation of a molecule is called a conformer and consists of its atom types and Cartesian coordinates. Every conformer has a potential energy, and the lower this energy, the more likely it occurs in nature. Most existing machine learning methods for molecular property prediction consider either 2D molecular graphs or 3D conformer structure representations in isolation. Inspired by recent work on using ensembles of conformers in conjunction with 2D graph representations, we propose $\mathrm{E}$(3)-invariant molecular conformer aggregation networks. The method integrates a molecule's 2D representation with that of multiple of its conformers. Contrary to prior work, we propose a novel 2D-3D aggregation mechanism based on a differentiable solver for the Fused Gromov-Wasserstein Barycenter problem and the use of an efficient conformer generation method based on distance geometry. We show that the proposed aggregation mechanism is $\mathrm{E}$(3) invariant and propose an efficient GPU implementation. Moreover, we demonstrate that the aggregation mechanism helps to significantly outperform state-of-the-art molecule property prediction methods on established datasets.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (76)
  1. Chemberta-2: Towards chemical foundation models, 2022.
  2. Molecular machine learning with conformer ensembles. Mach. Learn.: Sci. Technol., 4(3):035025, September 2023. ISSN 2632-2153. doi: 10.1088/2632-2153/acefa7.
  3. Mace: Higher order equivariant message passing neural networks for fast and accurate force fields. In Koyejo, S., Mohamed, S., Agarwal, A., Belgrave, D., Cho, K., and Oh, A. (eds.), Advances in Neural Information Processing Systems, volume 35, pp.  11423–11436. Curran Associates, Inc., 2022.
  4. A foundation model for atomistic materials chemistry, 2023.
  5. E(3)-equivariant graph neural networks for data-efficient and accurate interatomic potentials. Nat. Commun., 13(1):2453, May 2022. ISSN 2041-1723.
  6. Geometric and physical quantities improve e(3) equivariant message passing. In International Conference on Learning Representations, 2022.
  7. Learning to Predict Graphs with Fused Gromov-Wasserstein Barycenters. In Chaudhuri, K., Jegelka, S., Song, L., Szepesvari, C., Niu, G., and Sabato, S. (eds.), Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedings of Machine Learning Research, pp.  2321–2335. PMLR, July 2022.
  8. Machine learning for molecular and materials science. Nature, 559(7715):547–555, Jul 2018. ISSN 1476-4687.
  9. Design of protein-binding proteins from the target structure alone. Nature, 605(7910):551–560, 2022.
  10. Bidirectional generation of structure and properties through a single molecular foundation model. arXiv preprint arXiv:2211.10590, 2023.
  11. Optimal transport graph neural networks. arXiv preprint arXiv:2006.04804, 2020.
  12. Recent advances and applications of deep learning methods in materials science. npj Comput. Mater., 8(1):59, Apr 2022. ISSN 2057-3960.
  13. Generalized spectral clustering via gromov-wasserstein learning. In International Conference on Artificial Intelligence and Statistics, pp.  712–720. PMLR, 2021.
  14. Cuturi, M. Sinkhorn distances: Lightspeed computation of optimal transport. Advances in neural information processing systems, 26, 2013.
  15. Fast computation of wasserstein barycenters. In International conference on machine learning, pp. 685–693. PMLR, 2014.
  16. Identification of inhibitors of sars-cov-2 in-vitro cellular toxicity in human (caco-2) cells using a large scale drug repurposing collection, 2020.
  17. Chemrl-gem: Geometry enhanced molecular representation learning for property prediction. Nature Machine Intelligence, 2021. doi: 10.48550/ARXIV.2106.06130.
  18. Geometry-enhanced molecular representation learning for property prediction. Nature Machine Intelligence, 4(2):127–134, 2022.
  19. Extending machine learning beyond interatomic potentials for predicting molecular properties. Nat. Rev. Chem., 6(9):653–672, Sep 2022. ISSN 2397-3358. doi: 10.1038/s41570-022-00416-3.
  20. Interpolating between optimal transport and mmd using sinkhorn divergences. In The 22nd International Conference on aRtIfIcIaL InTeLlIgEnCe and Statistics, pp.  2681–2690. PMLR, 2019.
  21. Se(3)-transformers: 3d roto-translation equivariant attention networks. In Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M., and Lin, H. (eds.), Advances in Neural Information Processing Systems, volume 33, pp.  1970–1981. Curran Associates, Inc., 2020.
  22. Directional message passing for molecular graphs. In International Conference on Learning Representations (ICLR), 2020.
  23. Gemnet: Universal directional graph neural networks for molecules. In Beygelzimer, A., Dauphin, Y., Liang, P., and Vaughan, J. W. (eds.), Advances in Neural Information Processing Systems, 2021.
  24. Neural message passing for quantum chemistry. In International Conference on Machine Learning, pp. 1263–1272, 2017a.
  25. Neural message passing for quantum chemistry. In Precup, D. and Teh, Y. W. (eds.), Proceedings of the 34th ICML, volume 70 of Proceedings of Machine Learning Research, pp. 1263–1272. PMLR, 06–11 Aug 2017b.
  26. Hawkins, P. C. Conformation generation: the state of the art. Journal of chemical information and modeling, 57(8):1747–1756, 2017.
  27. Strategies for pre-training graph neural networks. In International Conference on Learning Representations, 2020a.
  28. Strategies for pre-training graph neural networks. In International Conference on Learning Representations, 2020b.
  29. Semi-supervised classification with graph convolutional networks. In International Conference on Learning Representations, 2017.
  30. Landrum, G. Rdkit: open-source cheminformatics http://www. rdkit. org. 3(8), 2016.
  31. Entropic Gromov-Wasserstein between Gaussian distributions. In ICML, 2022.
  32. Fast convergence of empirical barycenters in Alexandrov spaces and the Wasserstein space. Journal of the European Mathematical Society, 25(6):2229–2250, May 2022. ISSN 1435-9855.
  33. Fixed-support Wasserstein barycenters: Computational hardness and fast algorithm. In NeurIPS, pp.  5368–5380, 2020.
  34. Pre-training molecular graph representation with 3d geometry. In International Conference on Learning Representations, 2022a.
  35. Spherical message passing for 3d molecular graphs. In International Conference on Learning Representations, 2022b.
  36. Fused gromov-wasserstein graph mixup for graph-level classifications. In Thirty-seventh Conference on Neural Information Processing Systems, 2023.
  37. Morgan, H. L. The generation of a unique machine description for chemical structures-a technique developed at chemical abstracts service. Journal of Chemical Documentation, 5(2):107–113, May 1965. ISSN 1541-5732. doi: 10.1021/c160017a018.
  38. Netal: a new graph-based method for global alignment of protein–protein interaction networks. Bioinformatics, 29(13):1654–1662, 2013.
  39. Automatic differentiation in pytorch. In NIPS 2017 Workshop on Autodiff, 2017.
  40. Peyré, G. Entropic approximation of wasserstein gradient flows. SIAM Journal on Imaging Sciences, 8(4):2323–2351, 2015.
  41. Gromov-wasserstein averaging of kernel and distance matrices. In International conference on machine learning, pp. 2664–2672. PMLR, 2016.
  42. Computational optimal transport: With applications to data science. Foundations and Trends® in Machine Learning, 11(5-6):355–607, 2019.
  43. Gromov-Wasserstein Averaging of Kernel and Distance Matrices. In Balcan, M. F. and Weinberger, K. Q. (eds.), Proceedings of The 33rd International Conference on Machine Learning, volume 48 of Proceedings of Machine Learning Research, pp.  2664–2672, New York, New York, USA, June 2016. PMLR.
  44. Entropic gromov-wasserstein distances: Stability, algorithms, and distributional limits. arXiv preprint arXiv:2306.00182, 2023.
  45. Self-supervised graph transformer on large-scale molecular data. Advances in Neural Information Processing Systems, 33:12559–12571, 2020.
  46. Large-scale chemical language representations capture molecular structure and properties. Nature Machine Intelligence, 4(12):1256–1264, 2022.
  47. E(n) equivariant graph neural networks. In Meila, M. and Zhang, T. (eds.), Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pp.  9323–9332. PMLR, 18–24 Jul 2021.
  48. The graph neural network model. IEEE Transactions on Neural Networks, 20(1):61–80, 2009.
  49. Schmitzer, B. Stabilized sparse scaling algorithms for entropy regularized transport problems. SIAM Journal on Scientific Computing, 41(3):A1443–A1481, 2019.
  50. Schnet: A continuous-filter convolutional neural network for modeling quantum interactions. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA, pp.  991–1001, 2017.
  51. Schnet: A continuous-filter convolutional neural network for modeling quantum interactions. In Guyon, I., Luxburg, U. V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., and Garnett, R. (eds.), Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017.
  52. Equivariant message passing for the prediction of tensorial properties and molecular spectra. ICML, pp.  1–13, 2021.
  53. Source, D. L. Main protease structure and xchem fragment screen, 2020.
  54. 3D infomax improves GNNs for molecular property prediction. In Chaudhuri, K., Jegelka, S., Song, L., Szepesvari, C., Niu, G., and Sabato, S. (eds.), Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedings of Machine Learning Research, pp.  20479–20502. PMLR, 17–23 Jul 2022.
  55. A fused gromov-wasserstein framework for unsupervised knowledge graph entity alignment. arXiv preprint arXiv:2305.06574, 2023.
  56. Tensor field networks: Rotation- and translation-equivariant neural networks for 3d point clouds, 2018.
  57. Optimal Transport for structured data with application on graphs. In Chaudhuri, K. and Salakhutdinov, R. (eds.), Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pp.  6275–6284. PMLR, June 2019.
  58. Fused Gromov-Wasserstein Distance for Structured Objects. Algorithms, 13(9):212, August 2020. ISSN 1999-4893. doi: 10.3390/a13090212.
  59. In vitro screening of a fda approved chemical library reveals potential inhibitors of sars-cov-2 replication. Sci Rep, 10:13093, 2020. doi: 10.1038/s41598-020-70143-6.
  60. Applications of machine learning in drug discovery and development. Nat. Rev. Drug Discov., 18(6):463–477, Jun 2019. ISSN 1474-1784.
  61. Graph attention networks. In International Conference on Learning Representations, 2018.
  62. Graph attention networks. In ICLR, 2018.
  63. Online graph dictionary learning. In International conference on machine learning, pp. 10564–10574. PMLR, 2021.
  64. Template based Graph Neural Network with Optimal Transport Distances. In Koyejo, S., Mohamed, S., Agarwal, A., Belgrave, D., Cho, K., and Oh, A. (eds.), Advances in Neural Information Processing Systems, volume 35, pp.  11800–11814. Curran Associates, Inc., 2022.
  65. Smiles-bert: Large scale unsupervised pre-training for molecular property prediction. In Proceedings of the 10th ACM International Conference on Bioinformatics, Computational Biology and Health Informatics, BCB ’19, pp. 429–436, New York, NY, USA, 2019. Association for Computing Machinery. ISBN 9781450366663. doi: 10.1145/3307339.3342186.
  66. Molecular contrastive learning of representations via graph neural networks. Nature Machine Intelligence, 4(3):279–287, 2022.
  67. Multi-modal representation learning for molecular property prediction: Sequence, graph, geometry. arXiv preprint arXiv:2401.03369, 2024.
  68. MoleculeNet: A benchmark for molecular machine learning. Chemical Science, pp.  513–530, 2018.
  69. Pushing the boundaries of molecular representation for drug discovery with the graph attention mechanism. Journal of medicinal chemistry, 63(16):8749–8760, 2019.
  70. Scalable gromov-wasserstein learning for graph partitioning and matching. Advances in neural information processing systems, 32, 2019a.
  71. Gromov-wasserstein learning for graph matching and node embedding. In International conference on machine learning, pp. 6932–6941. PMLR, 2019b.
  72. Representation learning on graphs with jumping knowledge networks. In Dy, J. and Krause, A. (eds.), Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Research, pp.  5453–5462. PMLR, 10–15 Jul 2018.
  73. Analyzing learned molecular representations for property prediction. Journal of Chemical Information and Modeling, 59(8):3370–3388, July 2019. ISSN 1549-960X. doi: 10.1021/acs.jcim.9b00237.
  74. Generative graph dictionary learning. In International Conference on Machine Learning, pp. 40749–40769. PMLR, 2023.
  75. Uni-mol: A universal 3d molecular representation learning framework. In The Eleventh International Conference on Learning Representations, 2023.
  76. Learning over molecular conformer ensembles: Datasets and benchmarks, 2023.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (10)
  1. Duy M. H. Nguyen (14 papers)
  2. Nina Lukashina (4 papers)
  3. Tai Nguyen (10 papers)
  4. An T. Le (15 papers)
  5. TrungTin Nguyen (17 papers)
  6. Nhat Ho (126 papers)
  7. Jan Peters (253 papers)
  8. Daniel Sonntag (55 papers)
  9. Viktor Zaverkin (18 papers)
  10. Mathias Niepert (85 papers)
Citations (3)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets