Accelerated Sampling of Rare Events using a Neural Network Bias Potential (2401.06936v1)
Abstract: In the field of computational physics and material science, the efficient sampling of rare events occurring at atomic scale is crucial. It aids in understanding mechanisms behind a wide range of important phenomena, including protein folding, conformal changes, chemical reactions and materials diffusion and deformation. Traditional simulation methods, such as Molecular Dynamics and Monte Carlo, often prove inefficient in capturing the timescale of these rare events by brute force. In this paper, we introduce a practical approach by combining the idea of importance sampling with deep neural networks (DNNs) that enhance the sampling of these rare events. In particular, we approximate the variance-free bias potential function with DNNs which is trained to maximize the probability of rare event transition under the importance potential function. This method is easily scalable to high-dimensional problems and provides robust statistical guarantees on the accuracy of the estimated probability of rare event transition. Furthermore, our algorithm can actively generate and learn from any successful samples, which is a novel improvement over existing methods. Using a 2D system as a test bed, we provide comparisons between results obtained from different training strategies, traditional Monte Carlo sampling and numerically solved optimal bias potential function under different temperatures. Our numerical results demonstrate the efficacy of the DNN-based importance sampling of rare events.
- Y. Fu, L. Xiang, Y. Zahid, G. Ding, T. Mei, Q. Shen, and J. Han, “Long-tailed visual recognition with deep models: A methodological survey and evaluation,” Neurocomputing, vol. 509, pp. 290–309, 2022.
- Y. Zhang, B. Kang, B. Hooi, S. Yan, and J. Feng, “Deep long-tailed learning: A survey,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 45, no. 9, pp. 10795–10816, 2023.
- V. Feldman and C. Zhang, “What neural networks memorize and why: Discovering the long tail via influence estimation,” in Advances in Neural Information Processing Systems (H. Larochelle, M. Ranzato, R. Hadsell, M. Balcan, and H. Lin, eds.), vol. 33, pp. 2881–2891, Curran Associates, Inc., 2020.
- J. Ren, P. J. Liu, E. Fertig, J. Snoek, R. Poplin, M. Depristo, J. Dillon, and B. Lakshminarayanan, “Likelihood ratios for out-of-distribution detection,” in Advances in Neural Information Processing Systems (H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R. Garnett, eds.), vol. 32, Curran Associates, Inc., 2019.
- N. V. Chawla, K. W. Bowyer, L. O. Hall, and W. P. Kegelmeyer, “Smote: synthetic minority over-sampling technique,” Journal of artificial intelligence research, vol. 16, pp. 321–357, 2002.
- M. Buda, A. Maki, and M. A. Mazurowski, “A systematic study of the class imbalance problem in convolutional neural networks,” Neural networks, vol. 106, pp. 249–259, 2018.
- R. Harada, R. Morita, and Y. Shigeta, “Free-energy profiles for membrane permeation of compounds calculated using rare-event sampling methods,” Journal of Chemical Information and Modeling, vol. 63, pp. 259–269, 01 2023.
- C. W. Jang, J. W. Mullinax, and J. W. Lawson, “Mechanical properties and failure of aerospace-grade epoxy resins from reactive molecular dynamics simulations with nanoscale defects,” ACS Applied Polymer Materials, vol. 4, no. 8, pp. 5269–5274, 2022.
- M. F. C. N. A. Marks and C. Kocer, “The importance of rare events in thin film deposition: a molecular dynamics study of tetrahedral amorphous carbon,” Molecular Simulation, vol. 32, no. 15, pp. 1271–1277, 2006.
- R. Elber and M. Karplus, “Multiple conformational states of proteins: A molecular dynamics analysis of myoglobin,” Science, vol. 235, no. 4786, pp. 318–321, 1987.
- E. Vanden-Eijnden et al., “Towards a theory of transition paths,” Journal of statistical physics, vol. 123, no. 3, pp. 503–523, 2006.
- E. Vanden-Eijnden et al., “Transition-path theory and path-finding algorithms for the study of rare events.,” Annual review of physical chemistry, vol. 61, pp. 391–420, 2010.
- R. E. Gillilan and K. R. Wilson, “Shadowing, rare events, and rubber bands. a variational verlet algorithm for molecular dynamics,” The Journal of chemical physics, vol. 97, no. 3, pp. 1757–1772, 1992.
- Springer, 2010.
- J. N. Reddy, Introduction to the finite element method. McGraw-Hill Education, 2019.
- J. Alberty, C. Carstensen, and S. A. Funken, “Remarks around 50 lines of matlab: short finite element implementation,” Numerical algorithms, vol. 20, no. 2-3, pp. 117–137, 1999.
- M. de Koning, W. Cai, B. Sadigh, T. Oppelstrup, M. H. Kalos, and V. V. Bulatov, “Adaptive importance sampling monte carlo simulation of rare transition events,” The Journal of chemical physics, vol. 122, no. 7, 2005.
- W. Cai, M. H. Kalos, M. de Koning, and V. V. Bulatov, “Importance sampling of rare transition events in markov processes,” Physical Review E, vol. 66, no. 4, p. 046703, 2002.
- C. Hartmann, L. Richter, C. Schütte, and W. Zhang, “Variational characterization of free energy: theory and algorithms,” Entropy, vol. 19, no. 11, p. 626, 2017.
- M. Karplus and G. A. Petsko, “Molecular dynamics simulations in biology,” Nature, vol. 347, no. 6294, pp. 631–639, 1990.
- D. B. Korlepara, C. S. Vasavi, S. Jeurkar, P. K. Pal, S. Roy, S. Mehta, S. Sharma, V. Kumar, C. Muvva, B. Sridharan, A. Garg, R. Modee, A. P. Bhati, D. Nayar, and U. D. Priyakumar, “Plas-5k: Dataset of protein-ligand affinities from molecular dynamics for machine learning applications,” Scientific Data, vol. 9, no. 1, p. 548, 2022.
- Y. Gao, T. Li, X. Li, and J.-G. Liu, “Transition path theory for langevin dynamics on manifolds: Optimal control and data-driven solver,” Multiscale Modeling & Simulation, vol. 21, no. 1, pp. 1–33, 2023.
- C. Hartmann and C. Schütte, “Efficient rare event simulation by optimal nonequilibrium forcing,” Journal of Statistical Mechanics: Theory and Experiment, vol. 2012, no. 11, p. P11004, 2012.
- M. Chak, T. Lelièvre, G. Stoltz, and U. Vaes, “Optimal importance sampling for overdamped langevin dynamics,” 2023.
- L. Zhang, H. Wang, et al., “Reinforced dynamics for enhanced sampling in large atomic and molecular systems,” The Journal of chemical physics, vol. 148, no. 12, 2018.
- D. Passerone and M. Parrinello, “Action-derived molecular dynamics in the study of rare events,” Physical Review Letters, vol. 87, no. 10, p. 108302, 2001.
- W. Cai, M. H. Kalos, M. de Koning, and V. V. Bulatov, “Importance sampling of rare transition events in markov processes,” Phys. Rev. E, vol. 66, p. 046703, Oct 2002.
- Y. Khoo, J. Lu, and L. Ying, “Solving for high-dimensional committor functions using artificial neural networks,” Research in the Mathematical Sciences, vol. 6, pp. 1–13, 2019.
- Q. Li, B. Lin, and W. Ren, “Computing committor functions for the study of rare events using deep learning,” The Journal of Chemical Physics, vol. 151, no. 5, 2019.
- J. Yuan, A. Shah, C. Bentz, and M. Cameron, “Optimal control for sampling the transition path process and estimating rates,” 2023.
- H. Li, Y. Khoo, Y. Ren, and L. Ying, “A semigroup method for high dimensional committor functions based on neural network,” in Proceedings of the 2nd Mathematical and Scientific Machine Learning Conference (J. Bruna, J. Hesthaven, and L. Zdeborova, eds.), vol. 145 of Proceedings of Machine Learning Research, pp. 598–618, PMLR, 16–19 Aug 2022.
- D. Frenkel and B. Smit, Understanding molecular simulation: from algorithms to applications. Elsevier, 2023.
- S. Nangia, A. W. Jasper, T. F. Miller III, and D. G. Truhlar, “Army ants algorithm for rare event sampling of delocalized nonadiabatic transitions by trajectory surface hopping and the estimation of sampling errors by the bootstrap method,” The Journal of chemical physics, vol. 120, no. 8, pp. 3586–3597, 2004.
- John Wiley & Sons, 2009.
- R. Pastor, “Techniques and applications of langevin dynamics simulations,” in The Molecular Dynamics of Liquid Crystals, pp. 85–138, Springer, 1994.
- S. Ruder, “An overview of gradient descent optimization algorithms,” arXiv preprint arXiv:1609.04747, 2016.
- L. Martino, V. Elvira, and F. Louzada, “Effective sample size for importance sampling based on discrepancy measures,” Signal Processing, vol. 131, pp. 386–401, 2017.
- P.-O. Persson and G. Strang, “A simple mesh generator in matlab,” SIAM review, vol. 46, no. 2, pp. 329–345, 2004.
- A. F. Voter, “Hyperdynamics: Accelerated molecular dynamics of infrequent events,” Physical Review Letters, vol. 78, no. 20, p. 3908, 1997.
- M. Frassek, A. Arjun, and P. Bolhuis, “An extended autoencoder model for reaction coordinate discovery in rare event molecular dynamics datasets,” The Journal of Chemical Physics, vol. 155, no. 6, 2021.