Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
144 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
46 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Variational Quantum Circuit Design for Quantum Reinforcement Learning on Continuous Environments (2312.13798v1)

Published 21 Dec 2023 in quant-ph

Abstract: Quantum Reinforcement Learning (QRL) emerged as a branch of reinforcement learning (RL) that uses quantum submodules in the architecture of the algorithm. One branch of QRL focuses on the replacement of neural networks (NN) by variational quantum circuits (VQC) as function approximators. Initial works have shown promising results on classical environments with discrete action spaces, but many of the proposed architectural design choices of the VQC lack a detailed investigation. Hence, in this work we investigate the impact of VQC design choices such as angle embedding, encoding block architecture and postprocessesing on the training capabilities of QRL agents. We show that VQC design greatly influences training performance and heuristically derive enhancements for the analyzed components. Additionally, we show how to design a QRL agent in order to solve classical environments with continuous action spaces and benchmark our agents against classical feed-forward NNs.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (34)
  1. The power of quantum neural networks. Nature Computational Science, 1(6):403–409.
  2. Variational quantum soft actor-critic for robotic arm control. arXiv preprint arXiv:2212.11681.
  3. Quantum variational algorithms are swamped with traps. Nature Communications, 13(1):7760.
  4. Generalization in quantum machine learning: A quantum information standpoint. PRX Quantum, 2(4):040321.
  5. A generative modeling approach for benchmarking and training shallow quantum circuits. npj Quantum Information, 5(1).
  6. Openai gym. arXiv preprint arXiv:1606.01540.
  7. Generalization in quantum machine learning from few training data. Nature Communications, 13(1).
  8. Chen, S. Y.-C. (2023). Asynchronous training of quantum reinforcement learning.
  9. Minimalistic gridworld environment for openai gym.
  10. Quantifying generalization in reinforcement learning. In Chaudhuri, K. and Salakhutdinov, R., editors, Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pages 1282–1289. PMLR.
  11. Quantum algorithms: A survey of applications and end-to-end complexities.
  12. Quantum reinforcement learning for solving a stochastic frozen lake environment and the impact of quantum architecture choices. arXiv preprint arXiv:2212.07932.
  13. Expressive power of parametrized quantum circuits. Phys. Rev. Res., 2:033125.
  14. Soft actor-critic algorithms and applications.
  15. Parametrized quantum policies for reinforcement learning. Advances in Neural Information Processing Systems, 34:28362–28375.
  16. Lan, Q. (2021). Variational quantum soft actor-critic. arXiv preprint arXiv:2112.11921.
  17. Theory of overparametrization in quantum neural networks. Nature Computational Science, 3(6):542–551.
  18. Barren plateaus in quantum neural network training landscapes. Nature Communications, 9(1).
  19. Quantum policy gradient algorithm with optimized action decoding. In International Conference on Machine Learning, pages 24592–24613. PMLR.
  20. A survey on quantum reinforcement learning.
  21. Quantum multi-agent reinforcement learning for autonomous mobility cooperation. IEEE Communications Magazine.
  22. Data re-uploading for a universal quantum classifier. Quantum, 4:226.
  23. The dilemma of quantum neural networks. IEEE Transactions on Neural Networks and Learning Systems.
  24. Effect of data encoding on the expressive power of variational quantum-machine-learning models. Physical Review A, 103(3):032430.
  25. Trust region policy optimization. In International conference on machine learning, pages 1889–1897. PMLR.
  26. Proximal policy optimization algorithms.
  27. Quantum agents in the gym: a variational quantum algorithm for deep q-learning. Quantum, 6:720.
  28. Robustness of quantum reinforcement learning under hardware errors. EPJ Quantum Technology, 10(1):1–43.
  29. Smith, L. N. (2018). A disciplined approach to neural network hyper-parameters: Part 1–learning rate, batch size, momentum, and weight decay. arXiv preprint arXiv:1803.09820.
  30. Differentiable quantum architecture search for quantum reinforcement learning.
  31. Quantum reinforcement learning in continuous action space. arXiv preprint arXiv:2012.10711.
  32. Reinforcement learning in healthcare: A survey.
  33. A review of deep reinforcement learning for smart building energy management.
  34. Toward trainability of quantum neural networks.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets