Hybrid action Reinforcement Learning for quantum architecture search (2511.04967v1)

Abstract: Designing expressive yet trainable quantum circuit architectures remains a major challenge for variational quantum algorithms, where manual or heuristic designs often lead to suboptimal performance. We propose HyRLQAS (Hybrid-Action Reinforcement Learning for Quantum Architecture Search), a unified framework that couples discrete gate placement and continuous parameter generation within a hybrid action space. Unlike existing approaches that treat structure and parameter optimization separately, HyRLQAS jointly learns circuit topology and initialization while dynamically refining previously placed gates through a reinforcement learning process. Trained in a variational quantum eigensolver (VQE) environment, the agent constructs circuits that minimize molecular ground-state energy. Experiments show that HyRLQAS achieves consistently lower energy errors and shorter circuits than both discrete-only and continuous-only baselines. Furthermore, the hybrid action space not only leads to better circuit structures but also provides favorable parameter initializations, resulting in post-optimization energy distributions with consistently lower minima. These results suggest that hybrid action reinforcement learning provides a principled pathway toward automated, hardware-efficient quantum circuit design.