Re-exploring Control Strategies in a Non-Markovian Open Quantum System by Reinforcement Learning (2312.11853v1)
Abstract: In this study, we reexamine a recent optimal control simulation targeting the preparation of a superposition of two excited electronic states in the UV range in a complex molecular system. We revisit this control from the perspective of reinforcement learning, offering an efficient alternative to conventional quantum control methods. The two excited states are addressable by orthogonal polarizations and their superposition corresponds to a right or left localization of the electronic density. The pulse duration spans tens of femtoseconds to prevent excitation of higher excited bright states what leads to a strong perturbation by the nuclear motions. We modify an open source software by L. Giannelli et al., Phys. Lett. A, 434, 128054 (2022) that implements reinforcement learning with Lindblad dynamics, to introduce non-Markovianity of the surrounding either by timedependent rates or more exactly by using the hierarchical equations of motion with the QuTiP-BoFiN package. This extension opens the way to wider applications for non-Markovian environments, in particular when the active system interacts with a highly structured noise.
- L. M. K. Vandersypen and I. L. Chuang, Nmr techniques for quantum control and computation, Rev. Mod. Phys. 76, 1037 (2005).
- D. Keefer and R. de Vivie-Riedle, Pathways to new applications for quantum control, Acc. Chem. Res. 51, 2279 (2018), pMID: 30152675.
- C.-P. Yang and S. Han, n𝑛nitalic_n-qubit-controlled phase gate with superconducting quantum-interference devices coupled to a resonator, Phys. Rev. A 72, 032311 (2005).
- D. J. Tannor and S. A. Rice, Control of selectivity of chemical reaction via control of wave packet evolution, J. Chem. Phys. 83, 5013 (1985).
- P. Brumer and M. Shapiro, Laser control of molecular processes, Ann. Rev. Phys. Chem. 43, 257 (1992), pMID: 18397166.
- V. Engel, C. Meier, and D. J. Tannor, Local control theory: Recent applications to energy and particle transfer processes in molecules, Adv. Chem. Phys. 141, 29 (2009).
- U. Boscain, M. Sigalotti, and D. Sugny, Introduction to the pontryagin maximum principle for quantum optimal control, PRX Quantum 2, 030203 (2021).
- W. Zhu and H. Rabitz, Quantum control design via adaptive tracking, J. Chem. Phys. 119, 3619 (2003).
- W. Zhu, J. Botina, and H. Rabitz, Rapidly convergent iteration methods for quantum optimal control of population, J. Chem. Phys. 108, 1953 (1998).
- Y. Maday and G. Turinici, New formulations of monotonically convergent quantum control algorithms, J. Chem. Phys. 118, 8191 (2003).
- J. P. Palao and R. Kosloff, Optimal control theory for unitary transformations, Phys. Rev. A 68, 062308 (2003).
- P. Doria, T. Calarco, and S. Montangero, Optimal control technique for many-body quantum dynamics, Phys. Rev. Lett. 106, 190501 (2011).
- B. Riaz, C. Shuang, and S. Qamar, Optimal control methods for quantum gate preparation: a comparative study, Quantum Inf. Process. 18, 100 (2019).
- Z. An and D. L. Zhou, Deep reinforcement learning for quantum gate control, Europhys. Lett. 126, 60002 (2019).
- I. Paparelle, L. Moro, and E. Prati, Digitally stimulated raman passage by deep reinforcement learning, Phys. Lett. A 384, 126266 (2020).
- Y. Tanimura, Numerically “exact” approach to open quantum dynamics: The hierarchical equations of motion (HEOM), J. Chem. Phys. 153, 10.1063/5.0011599 (2020), 020901.
- X. Dan and Q. Shi, Theoretical study of nonadiabatic hydrogen atom scattering dynamics on metal surfaces using the hierarchical equations of motion method, J. Chem. Phys. 159, 044101 (2023).
- R. Borrelli and S. Dolgov, Expanding the range of hierarchical equations of motion by tensor-train implementation, J. Phys. Chem. B 125, 5397 (2021).
- L. Giannelli, https://www.github.com/luigiannelli/threeLS_populationTransfer..
- J. Johansson, P. Nation, and F. Nori, Qutip: An open-source python framework for the dynamics of open quantum systems, Comput. Phys. Commun. 183, 1760 (2012).
- E. Mangaud, C. Meier, and M. Desouter-Lecomte, Analysis of the non-markovianity for electron transfer reactions in an oligothiophene-fullerene heterojunction, Chem. Phys. 494, 90 (2017).
- E. K.-L. Ho, T. Etienne, and B. Lasorne, Vibronic properties of para-polyphenylene ethynylenes: TD-DFT insights, J. Chem. Phys. 146, 164303 (2017).
- E. K.-L. Ho and B. Lasorne, Diabatic pseudofragmentation and nonadiabatic excitation-energy transfer in meta-substituted dendrimer building blocks, Comput. Theor. Chem. 1156, 25 (2019).
- J. Galiana and B. Lasorne, On the unusual Stokes shift in the smallest PPE dendrimer building block: Role of the vibronic symmetry on the band origin?, J. Chem. Phys. 158, 10.1063/5.0141352 (2023).
- K. Fujii, Introduction to the rotating wave approximation (rwa): Two coherent oscillations., J. Mod. Phys. 8, 2042 (2017).
- G. F. Thomas, Validity of the rosen-zener conjecture for gaussian-modulated pulses, Phys. Rev. A 27, 2744 (1983).
- H.-G. Duan and M. Thorwart, Quantum mechanical wave packet dynamics at a conical intersection with strong vibrational dissipation, J. Phys. Chem. Lett. 7, 382 (2016), pMID: 26751091.
- H.-P. Breuer and F. Petruccione, The Theory of Open Quantum Systems (Oxford University Press, 2002).
- I. de Vega and D. Alonso, Dynamics of non-Markovian open quantum systems, Rev. Mod. Phys. 89, 015001 (2017).
- G. Lindblad, On the generators of quantum dynamical semigroups, J. Chem. Phys. 48, 119 (1976).
- D. Manzano, A short introduction to the Lindblad master equation, AIP Adv. 10, 10.1063/1.5115323 (2020).
- G. Kimura, The Bloch vector for N-level systems, Phys. Lett. A 314, 339 (2003).
- D. Aerts and M. S. de Bianchi, The extended Bloch representation of quantum mechanics and the hidden-measurement solution to the measurement problem, Ann. Phys. 351, 975 (2014).
- E. Andersson, J. D. Cresser, and M. J. W. Hall, Finding the kraus decomposition from a master equation and vice versa, J. Mod. Opt. 54, 1695 (2007).
- A. Pomyalov, C. Meier, and D. J. Tannor, The importance of initial correlations in rate dynamics: A consistent non-Markovian master equation approach, Chem. Phys. 370, 98 (2010).
- C. Meier and D. J. Tannor, Non-Markovian evolution of the density operator in the presence of strong laser fields, J. Chem. Phys. 111, 3365 (1999).
- U. Kleinekathöfer, Non-Markovian theories based on a decomposition of the spectral density, J. Chem. Phys. 121, 2505 (2004).
- R. S. Sutton and A. G. Barto, Reinforcement Learning, Second Edition : An Introduction (The MIT Press, 2018).
- R. J. Williams, Simple statistical gradient-following algorithms for connectionist reinforcement learning, Mach. Learn. 8, 229 (1992).
- J. D. Martín-Guerrero and L. Lamata, Reinforcement learning and physics, Applied Sciences 11, 10.3390/app11188589 (2021).
- F. Marquardt, Machine learning and quantum devices, SciPost Phys. Lect. Notes , 29 (2021).
- Y. Ohtsuki, W. Zhu, and H. Rabitz, Monotonically convergent algorithm for quantum optimal control with dissipation, J. Chem. Phys. 110, 9825 (1999).
- T.-S. Ho and H. Rabitz, Accelerated monotonic convergence of optimal control over quantum dynamics, Phys. Rev. E 82, 026703 (2010).
- R. Marks, Handbook of Fourier Analysis & Its Applications (Oxford University Press, 2009).
- See supplemental material at [url will be inserted by publisher] for the modifications of the rl software to run non-markovian dynamics.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.