Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
110 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
44 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

From Peptides to Nanostructures: A Euclidean Transformer for Fast and Stable Machine Learned Force Fields (2309.15126v2)

Published 21 Sep 2023 in physics.chem-ph and cs.LG

Abstract: Recent years have seen vast progress in the development of machine learned force fields (MLFFs) based on ab-initio reference calculations. Despite achieving low test errors, the reliability of MLFFs in molecular dynamics (MD) simulations is facing growing scrutiny due to concerns about instability over extended simulation timescales. Our findings suggest a potential connection between robustness to cumulative inaccuracies and the use of equivariant representations in MLFFs, but the computational cost associated with these representations can limit this advantage in practice. To address this, we propose a transformer architecture called SO3krates that combines sparse equivariant representations (Euclidean variables) with a self-attention mechanism that separates invariant and equivariant information, eliminating the need for expensive tensor products. SO3krates achieves a unique combination of accuracy, stability, and speed that enables insightful analysis of quantum properties of matter on extended time and system size scales. To showcase this capability, we generate stable MD trajectories for flexible peptides and supra-molecular structures with hundreds of atoms. Furthermore, we investigate the PES topology for medium-sized chainlike molecules (e.g., small peptides) by exploring thousands of minima. Remarkably, SO3krates demonstrates the ability to strike a balance between the conflicting demands of stability and the emergence of new minimum-energy conformations beyond the training data, which is crucial for realistic exploration tasks in the field of biochemistry.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (95)
  1. Mark E Tuckerman. Ab initio molecular dynamics: basic concepts, current trends and novel applications. J. Phys. Condens. Matter, 14(50):R1297, 2002.
  2. Generalized neural-network representation of high-dimensional potential-energy surfaces. Phys. Rev. Lett., 98(14):146401, 2007.
  3. Gaussian Approximation Potentials: the accuracy of quantum mechanics, without the electrons. Phys. Rev. Lett., 104(13):136403, 2010.
  4. Jörg Behler. Atom-centered symmetry functions for constructing high-dimensional neural network potentials. J. Chem. Phys., 134(7):074106, 2011.
  5. Fast and accurate modeling of molecular atomization energies with machine learning. Phys. Rev. Lett., 108(5):058301, 2012.
  6. On representing chemical environments. Phys. Rev. B, 87(18):184115, 2013.
  7. Molecular dynamics with on-the-fly machine learning of quantum-mechanical forces. Phys. Rev. Lett., 114(9):096405, 2015.
  8. Machine learning of accurate energy-conserving molecular force fields. Sci. Adv., 3(5):e1603015, 2017.
  9. Quantum-chemical insights from deep tensor neural networks. Nat. Commun., 8:13890, 2017.
  10. Machine learning molecular dynamics for the simulation of infrared spectra. Chem. Sci., 8(10):6924–6935, 2017.
  11. Towards exact molecular dynamics simulations with machine-learned force fields. Nat. Commun., 9(1):3887, 2018. doi: 10.1038/s41467-018-06169-2.
  12. SchNet – a deep learning architecture for molecules and materials. J. Chem. Phys., 148(24):241722, 2018.
  13. ANI-1: an extensible neural network potential with DFT accuracy at force field computational cost. Chem. Sci., 8(4):3192–3203, 2017.
  14. Hierarchical modeling of molecular energies using a deep neural network. J. Chem. Phys., 148(24):241715, 2018.
  15. Accurate many-body repulsive potentials for density-functional tight-binding from deep tensor neural networks. J. Phys. Chem. Lett., 11:6835–6843, 2020.
  16. Alchemical and structural distribution based representation for universal quantum machine learning. J. Chem. Phys., 148(24):241717, 2018.
  17. PhysNet: A neural network for predicting energies, forces, dipole moments, and partial charges. J. Chem. Theory Comput., 15(6):3678–3693, 2019.
  18. FCHL revisited: faster and more accurate quantum machine learning. J. Chem. Phys., 152(4):044107, 2020.
  19. Embedded Atom Neural Network Potentials: efficient and accurate machine learning with a physically inspired representation. J. Phys. Chem. Lett., 10(17):4962–4967, 2019.
  20. Reactive dynamics and spectroscopy of hydrogen transfer from neural network-based reactive potential energy surfaces. New J. Phys., 22:55002, 2020.
  21. Machine learning for molecular simulation. Annu. Rev. Phys. Chem., 71:361–390, 2020.
  22. Exploring chemical compound space with quantum-based machine learning. Nat. Rev. Chem., 4(7):347–358, 2020.
  23. Machine learning force fields. Chem. Rev., 121(16):10142–10186, 2021a.
  24. SpookyNet: Learning force fields with electronic degrees of freedom and nonlocal effects. Nat. Commun., 12:7273, 2021b.
  25. Combining machine learning and computational chemistry for predictive insights into chemical systems. Chem. Rev., 121(16):9816–9872, 2021. URL https://pubs.acs.org/doi/abs/10.1021/acs.chemrev.1c00107.
  26. Accurate machine learned quantum-mechanical force fields for biomolecular simulations. arXiv preprint arXiv:2205.08306, 2022.
  27. Bigdml—towards accurate quantum machine learning force fields for materials. Nat. Commun., 13(1):3733, 2022.
  28. The ani-1ccx and ani-1x data sets, coupled-cluster and density functional theory properties for molecules. Sci. Data, 7(1):1–10, 2020.
  29. Qm7-x, a comprehensive dataset of quantum-mechanical properties spanning the chemical space of small organic molecules. Sci. Data, 8(1):1–11, 2021.
  30. Strategies for the construction of machine-learning potentials for accurate and efficient atomic-scale simulations. Mach. Learn. Sci. Technol., 2(3):031001, 2021.
  31. How robust are modern graph neural network potentials in long and hot molecular dynamics simulations? Mach. Learn. Sci. Technol., 3(4):045010, 2022.
  32. Forces are not enough: Benchmark and critical evaluation for machine learning force fields with molecular simulations. arXiv preprint arXiv:2210.07237, 2022.
  33. Improving machine learning force fields for molecular dynamics simulations with fine-grained force metrics. The Journal of chemical physics, 159(3), 2023.
  34. Neural message passing for quantum chemistry. In International Conference on Machine Learning, pages 1263–1272. Pmlr, 2017.
  35. sgdml: Constructing accurate and data efficient molecular force fields using machine learning. Comput. Phys. Commun., 240:38–45, 2019.
  36. E (3)-equivariant graph neural networks for data-efficient and accurate interatomic potentials. Nat. Commun., 13(1):2453, 2022.
  37. Equivariant message passing for the prediction of tensorial properties and molecular spectra. In International Conference on Machine Learning, pages 9377–9388. PMLR, 2021.
  38. Equivariant transformers for neural network based molecular potentials. In International Conference on Learning Representations, 2021.
  39. So3krates: Equivariant attention for interactions on arbitrary length-scales in molecular systems. Advances in Neural Information Processing Systems, 35:29400–29413, 2022.
  40. Importance of equivariant features in machine-learning interatomic potentials for reactive chemistry at metal surfaces. arXiv preprint arXiv:2305.10873, 2023.
  41. Incompleteness of atomic structure representations. Phys. Rev. Lett., 125(16):166001, 2020.
  42. Tensor field networks: rotation-and translation-equivariant neural networks for 3D point clouds. arXiv preprint arXiv:1802.08219, 2018.
  43. Gemnet: Universal directional graph neural networks for molecules. arXiv preprint arXiv:2106.08903, 2021.
  44. Attention is all you need. arXiv preprint arXiv:1706.03762, 2017.
  45. E (n) equivariant graph neural networks. In International Conference on Machine Learning, pages 9323–9332. PMLR, 2021.
  46. Stefan Goedecker. Minima hopping: An efficient search method for the global minimum of the potential energy surface of complex molecular systems. J. Chem. Phys., 120(21):9911–9917, 2004.
  47. Nonlinear component analysis as a kernel eigenvalue problem. Neural Comput., 10(5):1299–1319, 1998.
  48. Vladimir Vapnik. The nature of statistical learning theory. Springer science & business media, 1999.
  49. On relevant dimensions in kernel feature spaces. The Journal of Machine Learning Research, 9:1875–1908, 2008.
  50. Directional message passing for molecular graphs. arXiv preprint arXiv:2003.03123, 2020.
  51. Detect the interactions that matter in matter: Geometric attention for many-body systems. arXiv preprint arXiv:2106.02549, 2021.
  52. Mace: Higher order equivariant message passing neural networks for fast and accurate force fields. Advances in Neural Information Processing Systems, 35:11423–11436, 2022a.
  53. Learning local equivariant representations for large-scale atomistic dynamics. Nat. Commun., 14(1):579, 2023.
  54. Forcenet: A graph neural network for large-scale quantum calculations. arXiv preprint arXiv:2103.01436, 2021.
  55. Spherical message passing for 3d molecular graphs. In International Conference on Learning Representations, 2021.
  56. DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators. arXiv preprint arXiv:1910.03193, 2019.
  57. Kernel based quantum machine learning at record rate: Many-body distribution functionals as compact representations. The Journal of Chemical Physics, 159(3), 2023.
  58. JAX: composable transformations of Python+NumPy programs, 2018. URL http://github.com/google/jax.
  59. E3x: E(3)-equivariant deep learning made easy. arXiv preprint arXiv:2401.07595, 2024.
  60. Jax, md a framework for differentiable physics. Journal of Statistical Mechanics: Theory and Experiment, 2021(12):124016, 2021.
  61. Lammps-a flexible simulation tool for particle-based materials modeling at the atomic, meso, and continuum scales. Comput. Phys. Commun., 271:108171, 2022.
  62. Radius of gyration as an indicator of protein structure compactness. Molecular Biology, 42:623–628, 2008.
  63. Measuring the radius of gyration and intrinsic flexibility of viral proteins in buffer solution using small-angle x-ray scattering. ACS Measurement Science Au, 2(6):547–552, 2022.
  64. Universal relation between instantaneous diffusivity and radius of gyration of proteins in aqueous solution. Physical review letters, 126(12):128101, 2021.
  65. Accurate global machine learning force fields for molecules with hundreds of atoms. Sci. Adv., 9(2):eadf0873, 2023.
  66. Many-body van der waals interactions in molecules and condensed matter. J. Phys. Condens. Matter., 26(21):213202, 2014.
  67. Emil Roduner. Size matters: why nanomaterials are different. Chem. Soc. Rev., 35(7):583–592, 2006.
  68. Molecular dynamics study of double-walled carbon nanotubes for nano-mechanical manipulation. Jpn. J. Appl. Phys., 44(4R):1641, 2005.
  69. Molecular force fields with gradient-domain machine learning (gdml): Comparison and synergies with classical force fields. J. Chem. Phys., 153(12):124109, 2020.
  70. The topology of multidimensional potential energy surfaces: Theory and application to peptide structure and kinetics. J. Chem. Phys., 106(4):1495–1517, 1997.
  71. Continuous metadynamics in essential coordinates as a tool for free energy modelling of conformational changes. J. Mol. Model., 14:995–1002, 2008.
  72. Structure and dynamics of an unfolded protein examined by molecular dynamics simulation. Journal of the American Chemical Society, 134(8):3787–3791, 2012.
  73. Machine learning enables long time scale molecular photodynamics simulations. Chemical science, 10(35):8100–8107, 2019.
  74. Heat flux for semi-local machine-learning potentials. arXiv preprint arXiv:2303.14434, 2023a.
  75. Stress and heat flux via automatic differentiation. arXiv preprint arXiv:2305.01401, 2023b.
  76. Cormorant: covariant molecular neural networks. arXiv preprint arXiv:1906.04015, 2019.
  77. Algorithmic differentiation for automated modeling of machine learned force fields. J. Phys. Chem. Lett., 13(43):10183–10189, 2022.
  78. Reducing so (3) convolutions to so (2) for efficient equivariant gnns. arXiv preprint arXiv:2302.03655, 2023.
  79. Enabling efficient equivariant operations in the fourier basis via gaunt tensor products. arXiv preprint arXiv:2401.10216, 2024.
  80. Ralf Drautz. Atomic cluster expansion for accurate and transferable interatomic potentials. Phys. Rev. B, 99(1):014104, 2019.
  81. Atomic cluster expansion and wave function representations. arXiv preprint arXiv:2206.11375, 2022.
  82. The design space of e (3)-equivariant atom-centered interatomic potentials. arXiv preprint arXiv:2205.06643, 2022b.
  83. Reconstructing kernel-based machine learning force fields with superlinear convergence. J. Chem. Theory Comput., 19(14):4619–4630, 2023.
  84. Capturing long-range interaction with reciprocal space neural network. arXiv preprint arXiv:2211.16684, 2022.
  85. Machine learning force fields based on local parametrization of dispersion interactions: Application to the phase diagram of c 60. Physical Review B, 104(5):054106, 2021.
  86. Long-range dispersion-inclusive machine learning potentials for structure search and optimization of hybrid organic–inorganic interfaces. Digital Discovery, 1(4):463–475, 2022.
  87. High-dimensional neural-network potentials for multicomponent systems: Applications to zinc oxide. Physical Review B, 83(15):153101, 2011.
  88. A neural network potential-energy surface for the water dimer based on environment-dependent atomic energies and charges. The Journal of chemical physics, 136(6), 2012.
  89. A fourth-generation high-dimensional neural network potential with accurate electrostatics including non-local charge transfer. Nat. Commun., 12(1):1–11, 2021.
  90. Predicting the properties of salt water using neural network potentials and continuum solvent theory. 2022.
  91. Long-short-range message-passing: A physics-informed framework to capture non-local interaction for scalable molecular dynamics simulation. arXiv preprint arXiv:2304.13542, 2023.
  92. New langevin and gradient thermostats for rigid body dynamics. The Journal of chemical physics, 142(14), 2015.
  93. Timothy D Barfoot. State estimation for robotics. Cambridge University Press, 2017.
  94. Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
  95. Enhancing geometric representations for molecules with equivariant vector-scalar interactive message passing. Nature Communications, 15(1):313, 2024.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (4)
  1. J. Thorben Frank (6 papers)
  2. Oliver T. Unke (24 papers)
  3. Klaus-Robert Müller (167 papers)
  4. Stefan Chmiela (24 papers)
Citations (2)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now