Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
169 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Overcoming the Barrier of Orbital-Free Density Functional Theory for Molecular Systems Using Deep Learning (2309.16578v2)

Published 28 Sep 2023 in stat.ML, cs.LG, and physics.chem-ph

Abstract: Orbital-free density functional theory (OFDFT) is a quantum chemistry formulation that has a lower cost scaling than the prevailing Kohn-Sham DFT, which is increasingly desired for contemporary molecular research. However, its accuracy is limited by the kinetic energy density functional, which is notoriously hard to approximate for non-periodic molecular systems. Here we propose M-OFDFT, an OFDFT approach capable of solving molecular systems using a deep learning functional model. We build the essential non-locality into the model, which is made affordable by the concise density representation as expansion coefficients under an atomic basis. With techniques to address unconventional learning challenges therein, M-OFDFT achieves a comparable accuracy with Kohn-Sham DFT on a wide range of molecules untouched by OFDFT before. More attractively, M-OFDFT extrapolates well to molecules much larger than those seen in training, which unleashes the appealing scaling of OFDFT for studying large molecules including proteins, representing an advancement of the accuracy-efficiency trade-off frontier in quantum chemistry.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (127)
  1. Jorge M Seminario. Recent developments and applications of modern density functional theory. Elsevier, 1996.
  2. Commentary: The Materials Project: A materials genome approach to accelerating materials innovation. APL Materials, 1(1):011002, 07 2013. ISSN 2166-532X. doi: 10.1063/1.4812323. URL https://doi.org/10.1063/1.4812323.
  3. Self-consistent equations including exchange and correlation effects. Physical review, 140(4A):A1133, 1965.
  4. Llewellyn H Thomas. The calculation of atomic fields. In Mathematical proceedings of the Cambridge philosophical society, volume 23, pages 542–548. Cambridge University Press, 1927.
  5. Enrico Fermi. Eine statistische methode zur bestimmung einiger eigenschaften des atoms und ihre anwendung auf die theorie des periodischen systems der elemente. Zeitschrift für Physik, 48(1):73–79, 1928.
  6. John C Slater. A simplification of the Hartree-Fock method. Physical review, 81(3):385, 1951.
  7. Inhomogeneous electron gas. Physical review, 136(3B):B864, 1964.
  8. Orbital-free kinetic-energy density functional theory. Theoretical methods in condensed phase chemistry, 5:117–184, 2000.
  9. Progress on new approaches to old ideas: Orbital-free density functionals. In Volker Bach and Luigi Delle Site, editors, Many-electron approaches in physics, chemistry and mathematics: a multidisciplinary view, pages 113–134. Springer, 2014.
  10. The central role of density functional theory in the AI age. Science, 381(6654):170–175, 2023.
  11. C. H. Hodges. Quantum corrections to the Thomas–Fermi approximation — the Kirzhnits method. Canadian Journal of Physics, 51(13):1428–1437, 1973. doi: 10.1139/p73-189. URL https://doi.org/10.1139/p73-189.
  12. On the extended Thomas-Fermi approximation to the kinetic energy density. Physics Letters B, 65(1):1–4, 1976. ISSN 0370-2693. doi: https://doi.org/10.1016/0370-2693(76)90519-0. URL https://www.sciencedirect.com/science/article/pii/0370269376905190.
  13. Kinetic-energy functional of the electron density. Physical Review B, 45(23):13196, 1992.
  14. Orbital-free kinetic-energy density functionals with a density-dependent kernel. Physical Review B, 60(24):16350, 1999.
  15. Nonlocal orbital-free kinetic energy density functional for semiconductors. Physical Review B, 81(4):045206, 2010.
  16. Accurate simulations of metals at the mesoscale: Explicit treatment of 1 million atoms with quantum mechanics. Chemical Physics Letters, 475(4):163–170, 2009. ISSN 0009-2614. doi: https://doi.org/10.1016/j.cplett.2009.04.059. URL https://www.sciencedirect.com/science/article/pii/S0009261409005041.
  17. Orbital-free density functional theory for materials research. Journal of Materials Research, 33(7):777–795, 2018.
  18. David García-Aldea and JE Alvarellos. Kinetic energy density study of some representative semilocal kinetic energy functionals. The Journal of chemical physics, 127(14):144109, 2007.
  19. Can orbital-free density functional theory simulate molecules? The Journal of chemical physics, 136(8):084102, 2012.
  20. DFT exchange: sharing perspectives on the workhorse of quantum chemistry and materials science. Physical chemistry chemical physics, 24(47):28700–28781, 2022.
  21. Finding density functionals with machine learning. Physical review letters, 108(25):253002, 2012.
  22. Pure density functional for strong correlation and the thermodynamic limit from machine learning. Physical Review B, 94(24):245129, 2016.
  23. Bypassing the Kohn-Sham equations with machine learning. Nature communications, 8(872), 2017.
  24. Semi-local machine-learned kinetic energy density functional with third-order gradients of electron density. The Journal of Chemical Physics, 148(24):241705, 2018.
  25. Semi-local machine-learned kinetic energy density functional demonstrating smooth potential energy curves. Chemical Physics Letters, 734:136732, 2019.
  26. Order-N orbital-free density-functional calculations with machine learning of functional derivatives for semiconductors and metals. Physical Review Research, 3(3):033198, 2021.
  27. Kinetic energy of hydrocarbons as a function of electron density and convolutional neural networks. Journal of chemical theory and computation, 12(3):1139–1147, 2016.
  28. Machine learning approaches toward orbital-free density functional theory: Simultaneous training on the kinetic energy density functional and its functional derivative. Journal of chemical theory and computation, 16(9):5685–5694, 2020.
  29. KineticNet: Deep learning a transferable kinetic energy functional for orbital-free density functional theory. The Journal of Chemical Physics, 159(14):144113, 10 2023. ISSN 0021-9606. doi: 10.1063/5.0158275. URL https://doi.org/10.1063/5.0158275.
  30. Nonlocal kinetic-energy-density functionals. Phys. Rev. B, 53:9509–9512, Apr 1996. doi: 10.1103/PhysRevB.53.9509. URL https://link.aps.org/doi/10.1103/PhysRevB.53.9509.
  31. Nonlocal kinetic energy functionals by functional integration. The Journal of Chemical Physics, 148(18):184107, 05 2018. ISSN 0021-9606. doi: 10.1063/1.5023926. URL https://doi.org/10.1063/1.5023926.
  32. Tosio Kato. On the eigenfunctions of many-particle systems in quantum mechanics. Communications on Pure and Applied Mathematics, 10(2):151–177, 1957.
  33. Do Transformers really perform badly for graph representation? Advances in Neural Information Processing Systems, 34:28877–28888, 2021.
  34. Benchmarking Graphormer on large-scale molecular modeling datasets. arXiv preprint arXiv:2203.04810, 2022.
  35. Attention is all you need. Advances in neural information processing systems, 30:6000–6010, 2017.
  36. ANI-1: an extensible neural network potential with DFT accuracy at force field computational cost. Chemical science, 8(4):3192–3203, 2017.
  37. SchNet – a deep learning architecture for molecules and materials. The Journal of Chemical Physics, 148(24):241722, 2018.
  38. Equivariant transformers for neural network based molecular potentials. In International Conference on Learning Representations, 2021.
  39. Equiformer: Equivariant graph attention transformer for 3d atomistic graphs. In The Eleventh International Conference on Learning Representations, 2023. URL https://openreview.net/forum?id=KwmPfARgOTD.
  40. Machine learning of accurate energy-conserving molecular force fields. Science advances, 3(5):e1603015, 2017.
  41. sGDML: Constructing accurate and data efficient molecular force fields using machine learning. Computer Physics Communications, 240:38–45, 2019.
  42. Enumeration of 166 billion organic small molecules in the chemical universe database GDB-17. Journal of chemical information and modeling, 52(11):2864–2875, 2012.
  43. Quantum chemistry structures and properties of 134 kilo molecules. Scientific Data, 1(140022), 2014.
  44. Semiclassical neutral atom as a reference system in density functional theory. Physical review letters, 106(18):186406, 2011.
  45. Issues and challenges in orbital-free density functional calculations. Computer Physics Communications, 183(12):2519–2527, 2012.
  46. CF von Weizsäcker. Zur theorie der kernmassen. Zeitschrift für Physik, 96(7):431–458, 1935.
  47. F. L. Hirshfeld. Bonded-atom fragments for describing molecular charge densities. Theoretica chimica acta, 44(2):129–138, 1977. doi: 10.1007/BF00549096. URL https://doi.org/10.1007/BF00549096.
  48. PySCF: the Python-based simulations of chemistry framework. Wiley Interdisciplinary Reviews: Computational Molecular Science, 8(1):e1340, 2018.
  49. QMugs, quantum mechanical properties of drug-like molecules. Scientific Data, 9(273), 2022.
  50. Implicit generation and generalization in energy-based models. arXiv preprint arXiv:1903.08689, 2019.
  51. Learning iterative reasoning through energy minimization. In International Conference on Machine Learning, pages 5570–5582. PMLR, 2022.
  52. How fast-folding proteins fold. Science, 334(6055):517–520, 2011.
  53. The folding mechanism of BBL: Plasticity of transition-state structure observed within an ultrafast folding protein family. Journal of molecular biology, 390(5):1060–1073, 2009.
  54. Folding of a three-helix bundle at the folding speed limit. The Journal of Physical Chemistry B, 108(12):3694–3697, 2004.
  55. Rapid iterative method for electronic-structure eigenproblems using localised basis functions. Computer Physics Communications, 178(2):128–134, 2008.
  56. QUICKSTEP: Fast and accurate density functional calculations using a mixed Gaussian and plane waves approach. Computer Physics Communications, 167(2):103–128, 2005. ISSN 0010-4655. doi: https://doi.org/10.1016/j.cpc.2004.12.014. URL https://www.sciencedirect.com/science/article/pii/S0010465505000615.
  57. Fast periodic Gaussian density fitting by range separation. The Journal of Chemical Physics, 154(13):131104, 04 2021. doi: 10.1063/5.0046617.
  58. Pietro Cortona. Self-consistently determined properties of solids without band-structure calculations. Physical Review B, 44(16):8454, 1991.
  59. Accurate ab initio energetics of extended systems via explicit correlation embedded in a density functional environment. Chemical physics letters, 295(1-2):129–134, 1998.
  60. Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901, 2020.
  61. A survey of large language models. arXiv preprint arXiv:2303.18223, 2023.
  62. Towards predicting equilibrium distributions for molecular systems with deep learning. arXiv preprint arXiv:2306.05445, 2023.
  63. Active learning of uniformly accurate interatomic potentials for materials simulation. Physical Review Materials, 3(2):023804, 2019.
  64. Symmetry- and gradient-enhanced Gaussian process regression for the active learning of potential energy surfaces in porous materials. The Journal of Chemical Physics, 159(1):014115, 07 2023. ISSN 0021-9606. doi: 10.1063/5.0154989. URL https://doi.org/10.1063/5.0154989.
  65. Machine learning accurate exchange and correlation functionals of the electronic density. Nature communications, 11(1):3509, 2020.
  66. DeePKS: A comprehensive data-driven approach toward chemically accurate density functional theory. Journal of Chemical Theory and Computation, 17(1):170–181, 2021.
  67. Orbital-free bond breaking via machine learning. The Journal of chemical physics, 139(22):224104, 2013.
  68. Orbital-free density functional theory calculation applying semi-local machine-learned kinetic energy density functional and kinetic potential. Chemical Physics Letters, 748:137358, 2020.
  69. Pablo del Mazo-Sevillano and Jan Hermann. Variational principle to regularize machine-learned density functionals: the non-interacting kinetic-energy functional. arXiv preprint arXiv:2306.17587, 2023.
  70. Completing density functional theory by machine learning hidden messages from molecules. npj Computational Materials, 6(1):1–8, 2020.
  71. Pushing the frontiers of density functionals by solving the fractional electron problem. Science, 374(6573):1385–1389, 2021.
  72. Kohn-Sham equations as regularizer: Building prior knowledge into machine-learned physics. Physical review letters, 126(3):036401, 2021a.
  73. Brett I. Dunlap. Robust and variational fitting. Phys. Chem. Chem. Phys., 2:2113–2116, 2000. doi: 10.1039/B000027M. URL http://dx.doi.org/10.1039/B000027M.
  74. Even-tempered atomic orbitals. VI. optimal orbital exponents and optimal contractions of Gaussian primitives for hydrogen, carbon, and oxygen in molecules. The Journal of Chemical Physics, 60(3):918–931, 1974. doi: 10.1063/1.1681168. URL https://doi.org/10.1063/1.1681168.
  75. Deep Potential: A general representation of a many-body potential energy surface. Communications in Computational Physics, 23(3):629–639, 2018.
  76. Deep-learning density functional theory Hamiltonian for efficient ab initio electronic-structure calculation. Nature Computational Science, 2(6):367–377, 2022.
  77. A closer look at rotation-invariant deep point cloud analysis. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 16218–16227, 2021b.
  78. Frame averaging for invariant and equivariant network design. In International Conference on Learning Representations, 2022. URL https://openreview.net/forum?id=zIUyj55nXR.
  79. Equivariant message passing for the prediction of tensorial properties and molecular spectra. In International Conference on Machine Learning, pages 9377–9388. PMLR, 2021.
  80. E (3)-equivariant graph neural networks for data-efficient and accurate interatomic potentials. Nature communications, 13(1):2453, 2022.
  81. Efficient and accurate estimation of Lipschitz constants for deep neural networks. Advances in Neural Information Processing Systems, 32:11427–11438, 2019.
  82. Shun-Ichi Amari. Natural gradient works efficiently in learning. Neural computation, 10(2):251–276, 1998.
  83. Automatic differentiation for the direct minimization approach to the Hartree–Fock method. The Journal of Physical Chemistry A, 126(45):8487–8493, 2022.
  84. Roald Hoffmann. An Extended Hückel Theory. I. Hydrocarbons. The Journal of Chemical Physics, 39(6):1397–1412, 06 1963. ISSN 0021-9606. doi: 10.1063/1.1734456. URL https://doi.org/10.1063/1.1734456.
  85. Machine learning of coarse-grained molecular dynamics force fields. ACS central science, 5(5):755–767, 2019.
  86. Mdtraj: a modern open library for the analysis of molecular dynamics trajectories. Biophysical journal, 109(8):1528–1532, 2015.
  87. OpenMM 7: Rapid development of high performance algorithms for molecular dynamics. PLoS computational biology, 13(7):e1005659, 2017.
  88. Amber 10. Technical report, University of California, 2008.
  89. Introducing PROFESS: A new program for orbital-free density functional theory calculations. Computer physics communications, 179(11):839–854, 2008.
  90. Electronic structure calculations with GPAW: a real-space implementation of the projector augmented-wave method. Journal of physics: Condensed matter, 22(25):253202, 2010.
  91. ATLAS: A real-space finite-difference implementation of orbital-free density functional theory. Computer Physics Communications, 200:87–95, 2016.
  92. DFTpy: An efficient and object-oriented platform for orbital-free dft simulations. Wiley Interdisciplinary Reviews: Computational Molecular Science, 11(1):e1482, 2021.
  93. Overcoming the barrier of orbital-free density functional theory in molecular systems using deep learning, Feb 2024a. URL https://doi.org/10.6084/m9.figshare.c.6877432.
  94. Overcoming the Barrier of Orbital-Free Density Functional Theory in Molecular Systems Using Deep Learning, February 2024b. URL https://doi.org/10.5281/zenodo.10616893.
  95. Mel Levy. Universal variational functionals of electron densities, first-order density matrices, and natural spin-orbitals and solution of the v-representability problem. Proceedings of the National Academy of Sciences, 76(12):6062–6065, 1979.
  96. Elliott H Lieb. Density functionals for Coulomb systems. International Journal of Quantum Chemistry, 24(3):243–277, 1983.
  97. The Self-Consistent Field Equations for Generalized Valence Bond and Open-Shell Hartree-Fock Wave Functions, pages 79–127. Springer US, Boston, MA, 1977. ISBN 978-1-4757-0887-5. doi: 10.1007/978-1-4757-0887-5˙4. URL https://doi.org/10.1007/978-1-4757-0887-5_4.
  98. Quantum chemistry, volume 6. Pearson Prentice Hall Upper Saddle River, NJ, 2009.
  99. SM Blinder. Basic concepts of self-consistent-field theory. American journal of physics, 33(6):431–443, 1965.
  100. P. Pulay. Improved scf convergence acceleration. Journal of Computational Chemistry, 3(4):556–560, 1982. doi: https://doi.org/10.1002/jcc.540030413. URL https://onlinelibrary.wiley.com/doi/abs/10.1002/jcc.540030413.
  101. A black-box self-consistent field convergence algorithm: One step closer. The Journal of Chemical Physics, 116(19):8255–8261, 04 2002. ISSN 0021-9606. doi: 10.1063/1.1470195. URL https://doi.org/10.1063/1.1470195.
  102. Evaluation of molecular integrals over Gaussian basis functions. The Journal of Chemical Physics, 65(1):111–116, 07 1976. ISSN 0021-9606. doi: 10.1063/1.432807. URL https://doi.org/10.1063/1.432807.
  103. Computation of electron repulsion integrals using the rys quadrature method. Journal of Computational Chemistry, 4(2):154–157, 1983. doi: https://doi.org/10.1002/jcc.540040206. URL https://onlinelibrary.wiley.com/doi/abs/10.1002/jcc.540040206.
  104. Qiming Sun. Libcint: An efficient general integral library for Gaussian basis functions. Journal of computational chemistry, 36(22):1664–1671, 2015.
  105. Automatic differentiation in machine learning: a survey. Journal of Marchine Learning Research, 18:1–43, 2018.
  106. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems, 32, 2019.
  107. Generalized gradient approximation made simple. Physical review letters, 77(18):3865, 1996.
  108. Automatic differentiation for orbital-free density functional theory. The Journal of Chemical Physics, 158(12):124801, 03 2023. ISSN 0021-9606. doi: 10.1063/5.0138429. URL https://doi.org/10.1063/5.0138429.
  109. Hans Hellman. Einführung in die Quantenchemie. Franz Deuticke, Leipzig, 285, 1937.
  110. R. P. Feynman. Forces in molecules. Phys. Rev., 56:340–343, Aug 1939. doi: 10.1103/PhysRev.56.340. URL https://link.aps.org/doi/10.1103/PhysRev.56.340.
  111. Peter Pulay. Ab initio calculation of force constants and equilibrium geometries in polyatomic molecules: I. Theory. Molecular Physics, 17(2):197–204, 1969.
  112. Density-functional theory of atoms and molecules. 1989.
  113. Can exact conditions improve machine-learned density functionals? The Journal of chemical physics, 148(24):241743, 2018.
  114. Learning to approximate density functionals. Accounts of Chemical Research, 54(4):818–826, 2021.
  115. Multilayer feedforward networks are universal approximators. Neural networks, 2(5):359–366, 1989.
  116. Gaussian error linear units (GELUs). arXiv preprint arXiv:1606.08415, 2016.
  117. Layer normalization. arXiv preprint arXiv:1607.06450, 2016.
  118. Log-transformation and its implications for data analysis. Shanghai archives of psychiatry, 26(2):105, 2014.
  119. Learning local equivariant representations for large-scale atomistic dynamics. Nature Communications, 14(1):579, 2023.
  120. Susi Lehtola. Assessment of initial guesses for self-consistent field calculations. Superposition of atomic potentials: Simple yet efficient. Journal of chemical theory and computation, 15(3):1593–1604, 2019.
  121. Highly accurate machine learning model for kinetic energy density functional. Physics Letters A, 414:127621, 2021.
  122. Accurate parameterization of the kinetic energy functional. The Journal of Chemical Physics, 156(2):024110, 2022.
  123. Kinetic energy densities based on the fourth order gradient expansion: performance in different classes of materials and improvement via machine learning. Phys. Chem. Chem. Phys., 21:378–395, 2019. doi: 10.1039/C8CP06433D. URL http://dx.doi.org/10.1039/C8CP06433D.
  124. Orbital-free density functional theory with small datasets and deep learning. arXiv preprint arXiv:2104.05408, 2021.
  125. RDKit: Open-source cheminformatics. 2013. doi: 10.5281/zenodo.591637. URL https://www.rdkit.org.
  126. Longformer: The long-document transformer. arXiv preprint arXiv:2004.05150, 2020.
  127. Focal self-attention for local-global interactions in vision transformers. arXiv preprint arXiv:2107.00641, 2021.
Citations (14)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets