Optimizing Temperature Distributions for Training Neural Quantum States using Parallel Tempering (2410.23018v3)
Abstract: Parameterized artificial neural networks (ANNs) can be very expressive ansatzes for variational algorithms, reaching state-of-the-art energies on many quantum many-body Hamiltonians. Nevertheless, the training of the ANN can be slow and stymied by the presence of local minima in the parameter landscape. One approach to mitigate this issue is to use parallel tempering methods, and in this work we focus on the role played by the temperature distribution of the parallel tempering replicas. Using an adaptive method that adjusts the temperatures in order to equate the exchange probability between neighboring replicas, we show that this temperature optimization can significantly increase the success rate of the variational algorithm with negligible computational cost by eliminating bottlenecks in the replicas' random walk. We demonstrate this using two different neural networks, a restricted Boltzmann machine and a feedforward network, which we use to study a toy problem based on a permutation invariant Hamiltonian with a pernicious local minimum and the J1-J2 model on a rectangular lattice.
- J. Kempe and O. Regev, 3-local hamitonian is qma-complete, Quantum Info. Comput. 3, 258 (2003).
- J. Kempe, A. Kitaev, and O. Regev, The complexity of the local hamiltonian problem, SIAM Journal on Computing 35, 1070 (2006).
- Y. Huang, Two-dimensional local hamiltonian problem with area laws is qma-complete, Journal of Computational Physics , 110534 (2021).
- T. Zhao, J. Stokes, and S. Veerapaneni, Scalable neural quantum states architecture for quantum chemistry, Machine Learning: Science and Technology 4, 025034 (2023).
- Y.-T. Zhou, Z.-W. Zhou, and X. Liang, Solving fermi-hubbard-type models by tensor representations of backflow corrections, Phys. Rev. B 109, 245107 (2024).
- S. Sorella, Green function monte carlo with stochastic reconfiguration, Phys. Rev. Lett. 80, 4558 (1998).
- S. Sorella, Generalized lanczos algorithm for variational quantum monte carlo, Phys. Rev. B 64, 024512 (2001).
- A. Chen and M. Heyl, Empowering deep neural quantum states through efficient optimization, Nature Physics 20, 1476 (2024).
- G. Carleo and M. Troyer, Solving the quantum many-body problem with artificial neural networks, Science 355, 602 (2017).
- C. Roth and A. H. MacDonald, Group Convolutional Neural Networks Improve Quantum State Accuracy, arXiv e-prints , arXiv:2104.05085 (2021).
- C. Roth, A. Szabó, and A. H. MacDonald, High-accuracy variational monte carlo for frustrated magnets with deep neural networks, Phys. Rev. B 108, 054410 (2023).
- M. Medvidović and J. R. Moreno, Neural-network quantum states for many-body physics, The European Physical Journal Plus 139, 631 (2024).
- A. Szabó and C. Castelnovo, Neural network wave functions and the sign problem, Phys. Rev. Res. 2, 033075 (2020).
- M. Bukov, M. Schmitt, and M. Dupont, Learning the ground state of a non-stoquastic quantum Hamiltonian in a rugged neural network landscape, SciPost Phys. 10, 147 (2021).
- R. H. Swendsen and J.-S. Wang, Replica monte carlo simulation of spin-glasses, Phys. Rev. Lett. 57, 2607 (1986).
- C. J. Geyer, Parallel tempering, in Computing Science and Statistics Proceedings of the 23rd Symposium on the Interface, edited by E. M. Keramidas and S. M. Kaufman (American Statistical Association, New York, 1991) p. 156.
- K. Hukushima and K. Nemoto, Exchange monte carlo method and application to spin glass simulations, Journal of the Physical Society of Japan 65, 1604 (1996).
- P. Smolensky, Information Processing in Dynamical Systems: Foundations of Harmony Theory, in Parallel Distributed Processing, Volume 1: Explorations in the Microstructure of Cognition: Foundations (The MIT Press, 1986).
- G. E. Hinton, Training Products of Experts by Minimizing Contrastive Divergence, Neural Computation 14, 1771 (2002).
- K. Hornik, M. Stinchcombe, and H. White, Multilayer feedforward networks are universal approximators, Neural Networks 2, 359 (1989).
- D. Hendrycks and K. Gimpel, Gaussian Error Linear Units (GELUs), arXiv e-prints , arXiv:1606.08415 (2016).
- M. Hibat-Allah, R. G. Melko, and J. Carrasquilla, Supplementing Recurrent Neural Network Wave Functions with Symmetry and Annealing to Improve Accuracy, arXiv e-prints , arXiv:2207.14314 (2022).
- K. Hukushima, Domain-wall free energy of spin-glass models: Numerical method and boundary conditions, Phys. Rev. E 60, 3606 (1999).
- W. van Dam, M. Mosca, and U. Vazirani, How powerful is adiabatic quantum computation?, in Proceedings 42nd IEEE Symposium on Foundations of Computer Science (2001) pp. 279–287.
- A. Kone and D. A. Kofke, Selection of temperature intervals for parallel-tempering simulations, The Journal of Chemical Physics 122, 206101 (2005).
- S. R. White, Density matrix formulation for quantum renormalization groups, Phys. Rev. Lett. 69, 2863 (1992).
- S. R. White, Density-matrix algorithms for quantum renormalization groups, Phys. Rev. B 48, 10345 (1993).
- U. Schollwöck, The density-matrix renormalization group, Rev. Mod. Phys. 77, 259 (2005).
- U. Schollwöck, The density-matrix renormalization group in the age of matrix product states, Annals of Physics 326, 96 (2011), january 2011 Special Issue.
- S.-C. T. Choi and M. A. Saunders, Algorithm 937: Minres-qlp for symmetric and hermitian linear equations and least-squares problems, ACM Trans. Math. Softw. 40 (2014).
- M. Schmitt and M. Heyl, Quantum many-body dynamics in two dimensions with artificial neural networks, Phys. Rev. Lett. 125, 100503 (2020).
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.