Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
36 tokens/sec
GPT-4o
12 tokens/sec
Gemini 2.5 Pro Pro
37 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
4 tokens/sec
DeepSeek R1 via Azure Pro
33 tokens/sec
2000 character limit reached

FreeCG: Free the Design Space of Clebsch-Gordan Transform for Machine Learning Force Fields (2407.02263v4)

Published 2 Jul 2024 in cs.LG, physics.chem-ph, q-bio.BM, and quant-ph

Abstract: Machine Learning Force Fields (MLFFs) are of great importance for chemistry, physics, materials science, and many other related fields. The Clebsch-Gordan Transform (CG transform) effectively encodes many-body interactions and is thus an important building block for many models of MLFFs. However, the permutation-equivariance requirement of MLFFs limits the design space of CG transform, that is, intensive CG transform has to be conducted for each neighboring edge and the operations should be performed in the same manner for all edges. This constraint results in reduced expressiveness of the model while simultaneously increasing computational demands. To overcome this challenge, we first implement the CG transform layer on the permutation-invariant abstract edges generated from real edge information. We show that this approach allows complete freedom in the design of the layer without compromising the crucial symmetry. Developing on this free design space, we further propose group CG transform with sparse path, abstract edges shuffling, and attention enhancer to form a powerful and efficient CG transform layer. Our method, known as FreeCG, achieves state-of-the-art (SOTA) results in force prediction for MD17, rMD17, MD22, and is well extended to property prediction in QM9 datasets with several improvements greater than 15% and the maximum beyond 20%. The extensive real-world applications showcase high practicality. FreeCG introduces a novel paradigm for carrying out efficient and expressive CG transform in future geometric neural network designs. To demonstrate this, the recent SOTA, QuinNet, is also enhanced under our paradigm. Code will be publicly available.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (55)
  1. Multiscale methods in drug design bridge chemical and biological complexity in the search for cures. Nature Reviews Chemistry 2, 0148 (2018).
  2. Das, P. et al. Accelerated antimicrobial discovery via deep generative models and molecular dynamics simulations. Nature Biomedical Engineering 5, 613–623 (2021).
  3. Chen, S. et al. Design of target specific peptide inhibitors using generative deep learning and molecular dynamics simulations. Nature Communications 15, 1611 (2024).
  4. Probing the limits of metal plasticity with molecular dynamics simulations. Nature 550, 492–495 (2017).
  5. Liu, M. et al. Layer-by-layer phase transformation in ti3o5 revealed by machine-learning molecular dynamics simulations. Nature Communications 15, 3079 (2024).
  6. Complex reaction processes in combustion unraveled by neural network-based molecular dynamics simulation. Nature communications 11, 5713 (2020).
  7. Meuwly, M. Machine learning for chemical reactions. Chemical Reviews 121, 10218–10239 (2021).
  8. Recent advances of molecular dynamics simulations in nanotribology. Journal of Molecular Liquids 335, 116154 (2021).
  9. Novel strategy for reducing the minimum miscible pressure in a co2–oil system using nonionic surfactant: Insights from molecular dynamics simulations. Applied Energy 352, 121966 (2023).
  10. Self-consistent equations including exchange and correlation effects. Physical review 140, A1133 (1965).
  11. Martin, R. M. Electronic structure: basic theory and practical methods (Cambridge university press, 2020).
  12. Ground state of the electron gas by a stochastic method. Physical review letters 45, 566 (1980).
  13. Coupled-cluster theory in quantum chemistry. Reviews of Modern Physics 79, 291 (2007).
  14. Burke, K. Perspective on density functional theory. The Journal of chemical physics 136 (2012).
  15. Jones, R. O. Density functional theory: Its origins, rise to prominence, and future. Reviews of modern physics 87, 897 (2015).
  16. Insights into current limitations of density functional theory. Science 321, 792–794 (2008).
  17. Lindorff-Larsen, K. et al. Improved side-chain torsion potentials for the amber ff99sb protein force field. Proteins: Structure, Function, and Bioinformatics 78, 1950–1958 (2010).
  18. Brooks, B. R. et al. Charmm: the biomolecular simulation program. Journal of computational chemistry 30, 1545–1614 (2009).
  19. Cui, T. et al. Geometry-enhanced pretraining on interatomic potentials. Nature Machine Intelligence 1–9 (2024).
  20. Quinnet: efficiently incorporating quintuple interactions into geometric deep learning force fields. In Proceedings of the 37th International Conference on Neural Information Processing Systems, 77043–77055 (2023).
  21. Wang, Y. et al. Enhancing geometric representations for molecules with equivariant vector-scalar interactive message passing. Nature Communications 15, 313 (2024).
  22. Musaelian, A. et al. Learning local equivariant representations for large-scale atomistic dynamics. Nature Communications 14, 579 (2023).
  23. Batzner, S. et al. E (3)-equivariant graph neural networks for data-efficient and accurate interatomic potentials. Nature communications 13, 2453 (2022).
  24. Drautz, R. Atomic cluster expansion for accurate and transferable interatomic potentials. Physical Review B 99, 014104 (2019).
  25. Mace: Higher order equivariant message passing neural networks for fast and accurate force fields. Advances in Neural Information Processing Systems 35, 11423–11436 (2022).
  26. Equivariant transformers for neural network based molecular potentials. In International Conference on Learning Representations (2021).
  27. Schnet–a deep learning architecture for molecules and materials. The Journal of Chemical Physics 148 (2018).
  28. Chmiela, S. et al. Machine learning of accurate energy-conserving molecular force fields. Science advances 3, e1603015 (2017).
  29. Equivariant message passing for the prediction of tensorial properties and molecular spectra. In International Conference on Machine Learning, 9377–9388 (PMLR, 2021).
  30. Thomas, N. et al. Tensor field networks: Rotation-and translation-equivariant neural networks for 3d point clouds. arXiv preprint arXiv:1802.08219 (2018).
  31. E (n) equivariant graph neural networks. In International conference on machine learning, 9323–9332 (PMLR, 2021).
  32. Directional message passing for molecular graphs. arXiv preprint arXiv:2003.03123 (2020).
  33. Vaswani, A. et al. Attention is all you need. Advances in neural information processing systems 30 (2017).
  34. Se (3)-transformers: 3d roto-translation equivariant attention networks. Advances in neural information processing systems 33, 1970–1981 (2020).
  35. Equiformer: Equivariant graph attention transformer for 3d atomistic graphs. arXiv preprint arXiv:2206.11990 (2022).
  36. Gemnet: Universal directional graph neural networks for molecules. Advances in Neural Information Processing Systems 34, 6790–6802 (2021).
  37. Shufflenet: An extremely efficient convolutional neural network for mobile devices. In Proceedings of the IEEE conference on computer vision and pattern recognition, 6848–6856 (2018).
  38. On the role of gradients for machine learning of molecular energies and forces. Machine Learning: Science and Technology 1, 045018 (2020).
  39. Chmiela, S. et al. Accurate global machine learning force fields for molecules with hundreds of atoms. Science Advances 9, eadf0873 (2023).
  40. Enumeration of 166 billion organic small molecules in the chemical universe database gdb-17. Journal of chemical information and modeling 52, 2864–2875 (2012).
  41. Quantum chemistry structures and properties of 134 kilo molecules. Scientific data 1, 1–7 (2014).
  42. Zee, A. Group theory in a nutshell for physicists, vol. 17 (Princeton University Press, 2016).
  43. Theory of group representations and applications (World Scientific Publishing Company, 1986).
  44. Jeevanjee, N. An introduction to tensors and group theory for physicists (Springer, 2011).
  45. Spherical cnns. arXiv preprint arXiv:1801.10130 (2018).
  46. Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems 25 (2012).
  47. Liu, Y. et al. Spherical message passing for 3d graph networks. arXiv preprint arXiv:2102.05013 (2021).
  48. Efficient and accurate physics-aware multiplex graph neural networks for 3d small molecules and macromolecule complexes. arXiv preprint arXiv:2206.02789 (2022).
  49. Unke, O. T. et al. Spookynet: Learning force fields with electronic degrees of freedom and nonlocal effects. Nature communications 12, 7273 (2021).
  50. Comenet: Towards complete and efficient message passing for 3d molecular graphs. Advances in Neural Information Processing Systems 35, 650–664 (2022).
  51. Qiao, Z. et al. Informing geometric deep learning with electronic interactions to accelerate quantum chemistry. Proceedings of the National Academy of Sciences 119, e2205221119 (2022).
  52. So3krates: Equivariant attention for interactions on arbitrary length-scales in molecular systems. Advances in Neural Information Processing Systems 35, 29400–29413 (2022).
  53. Batatia, I. et al. The design space of e (3)-equivariant atom-centered interatomic potentials. arXiv preprint arXiv:2205.06643 (2022).
  54. Li, Y. et al. Long-short-range message-passing: A physics-informed framework to capture non-local interaction for scalable molecular dynamics simulation. arXiv preprint arXiv:2304.13542 (2023).
  55. Aimd-chig: Exploring the conformational space of a 166-atom protein chignolin with ab initio molecular dynamics. Scientific Data 10, 549 (2023).

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
Dice Question Streamline Icon: https://streamlinehq.com

Follow-up Questions

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now