Deep-Q Learning with Hybrid Quantum Neural Network on Solving Maze Problems (2304.10159v3)
Abstract: Quantum computing holds great potential for advancing the limitations of machine learning algorithms to handle higher dimensions of data and reduce overall training parameters in deep learning (DL) models. This study uses a trainable variational quantum circuit (VQC) on a gate-based quantum computing model to investigate the potential for quantum benefit in a model-free reinforcement learning problem. Through a comprehensive investigation and evaluation of the current model and capabilities of quantum computers, we designed and trained a novel hybrid quantum neural network based on the latest Qiskit and PyTorch framework. We compared its performance with a full-classical CNN with and without an incorporated VQC. Our research provides insights into the potential of deep quantum learning to solve a maze problem and, potentially, other reinforcement learning problems. We conclude that reinforcement learning problems can be practical with reasonable training epochs. Moreover, a comparative study of full-classical and hybrid quantum neural networks is discussed to understand these two approaches' performance, advantages, and disadvantages to deep-Q learning problems, especially on larger-scale maze problems larger than 4x4.
- Quantum reinforcement learning: the maze problem. Quantum Machine Intelligence, 4:11, 2022. https://doi.org/10.1007/s42484-022-00068-y
- S. Y. -C. Chen, C. -H. H. Yang, J. Qi, P. -Y. Chen, X. Ma, and H. -S. Goan, “Variational Quantum Circuits for Deep Reinforcement Learning,” in IEEE Access, vol. 8, pp. 141007-141024, 2020, doi: 10.1109/ACCESS.2020.3010470.
- C. Zhao and X. S. Gao, “QDNN: DNN with quantum neural network layers.” arXiv preprint arXiv:1912.12660, 2019.
- D. Arthur, “A hybrid quantum-classical neural network architecture for binary classification.” arXiv preprint arXiv:2201.01820, 2022.
- M. Schuld, “Quantum machine learning models are kernel methods.” arXiv preprint arXiv:2101.11020, 2021.
- K. Beer, D. Bondarenko, T. Farrelly, T. J. Osborne, R. Salzmann, D. Scheiermann, R. Wolf, “Training deep quantum neural networks.” Nat Commun, 11(1):808, 2020. doi: 10.1038/s41467-020-14454-2.
- S. Lokes, C. S. J. Mahenthar, S. P. Kumaran, P. Sathyaprakash, V. Jayakumar, “Implementation of Quantum Deep Reinforcement Learning Using Variational Quantum Circuits,” 2022 International Conference on Trends in Quantum Computing and Emerging Business Technologies (TQCEBT), Pune, India, 2022, pp. 1-4, doi: 10.1109/TQCEBT54229.2022.10041479.
- D. Heimann, H. Hohenfeld, F. Wiebe, F. Kirchner, “Quantum deep reinforcement learning for robot navigation tasks.” arXiv preprint arXiv:2202.12180, 2022.
- L. Kunczik, “Quantum Reinforcement Learning—Connecting Reinforcement Learning and Quantum Computing.” In: Reinforcement Learning with Hybrid Quantum Approximation in the NISQ Context. Springer Vieweg, Wiesbaden, 2022.
- L. Kunczik, “Evaluating Quantum REINFORCE on IBM’s Quantum Hardware.” In: Reinforcement Learning with Hybrid Quantum Approximation in the NISQ Context. Springer Vieweg, Wiesbaden, 2022.
- L. Kunczik, “Future Steps in Quantum Reinforcement Learning for Complex Scenarios.” In: Reinforcement Learning with Hybrid Quantum Approximation in the NISQ Context. Springer Vieweg, Wiesbaden, 2022.
- O. Lockwood and M. Si, “Playing Atari with Hybrid Quantum-Classical Reinforcement Learning.” NeurIPS 2020 Workshop on Pre-registration in Machine Learning, in Proceedings of Machine Learning Research 148:285-301, 2021.
- D. Arthur and P. Date, “Hybrid Quantum-Classical Neural Networks,” 2022 IEEE International Conference on Quantum Computing and Engineering (QCE), Broomfield, CO, USA, 2022, pp. 49-55, doi: 10.1109/QCE53715.2022.00023.
- S. Park, D. K. Park, J. K. K. Rhee, “Variational quantum approximate support vector machine with inference transfer.” Sci Rep, 13, 3288 (2023).