Layerwise learning for quantum neural networks (2006.14904v1)

Abstract: With the increased focus on quantum circuit learning for near-term applications on quantum devices, in conjunction with unique challenges presented by cost function landscapes of parametrized quantum circuits, strategies for effective training are becoming increasingly important. In order to ameliorate some of these challenges, we investigate a layerwise learning strategy for parametrized quantum circuits. The circuit depth is incrementally grown during optimization, and only subsets of parameters are updated in each training step. We show that when considering sampling noise, this strategy can help avoid the problem of barren plateaus of the error surface due to the low depth of circuits, low number of parameters trained in one step, and larger magnitude of gradients compared to training the full circuit. These properties make our algorithm preferable for execution on noisy intermediate-scale quantum devices. We demonstrate our approach on an image-classification task on handwritten digits, and show that layerwise learning attains an 8% lower generalization error on average in comparison to standard learning schemes for training quantum circuits of the same size. Additionally, the percentage of runs that reach lower test errors is up to 40% larger compared to training the full circuit, which is susceptible to creeping onto a plateau during training.

• The paper presents a layerwise strategy to mitigate barren plateaus in quantum neural networks by optimizing shallow circuits first.
• It incrementally builds parameterized quantum circuits by progressively adding and freezing layers to retain higher gradient values.
• Empirical results show an 8% lower generalization error and up to 40% higher chances of achieving lower test errors on practical tasks.

Layerwise Learning for Quantum Neural Networks: An In-Depth Analysis

The paper "Layerwise Learning for Quantum Neural Networks" presents a novel approach to training quantum neural networks (QNNs) on noisy intermediate-scale quantum (NISQ) devices. The authors propose a layerwise learning (LL) strategy to mitigate the challenges posed by barren plateaus, a phenomenon where gradients become exponentially small, hindering the effective optimization of parameterized quantum circuits (PQCs).

Background and Motivation

The concept of parameterized quantum circuits with variational objectives is a promising route toward achieving quantum advantage using NISQ devices. These devices, while not fully error-corrected, can outperform classical computers for certain tasks, making PQCs a focal point in quantum machine learning, optimization, and chemistry. However, the major obstacle in training such circuits lies in the barren plateaus of the cost function landscape, where the gradients vanish, making it difficult to find optimal solutions using gradient-based methods. The authors identify that layerwise training can effectively exploit larger gradient magnitudes present in shallower circuits and thus avoid these barren plateaus.

Methodology

The layerwise learning approach incrementally constructs the PQC by adding layers progressively during optimization. Initially, a circuit with few layers is optimized with all parameters set to zero. Subsequently, more layers are added incrementally while freezing the parameters from earlier ones. This staggered and sequential approach curtails the depths and breadth of optimization required at any one instance, which in turn sustains significant gradient values and circumvents plateau regions in the cost landscape.

The authors distinguish between two phases in their training strategy. In the first phase, the circuit's depth increases incrementally, adding new layers to the previously optimized structure. In the second phase, larger blocks of layers are optimized together, leveraging the well-initialized circuit structure to further refine the quantum model. The circuits are tested on the image classification task of handwritten digits, demonstrating that QNNs trained with the LL approach exhibit an 8% lower generalization error on average compared to standard techniques.

Numerical Insights

Empirical evaluations showcase significant improvements in the performance and efficiency of QNNs optimized with LCDLs (Complete-Depth Learning). The algorithm demonstrates a probability of achieving lower test errors up to 40% higher than standard approaches. The layerwise learning provides substantial gains in terms of achieving higher accuracy within fewer experiments, especially highlighting its robustness to stochastic noise.

Implications and Future Directions

The implications of this research span both practical execution on current quantum devices and the theoretical underpinnings of quantum machine learning. Practically, the LL strategy could enable more efficient usage of quantum resources, reducing the runtime and sampling requirements for learning tasks on NISQ processors. Theoretically, it serves as a basis for further developing hierarchical and modular learning strategies that better exploit the hardware-imposed constraints in quantum computing.

Future developments could include the exploration of adaptive layerwise addition, where the inclusion of additional layers is contingent on the convergence state of the preceding set. Furthermore, integrating this method with parameter initialization strategies that target specific regions of the parameter space could provide further improvements to circumvent the plateau phenomena more effectively.

In conclusion, this paper presents a compelling case for layerwise learning as an effective strategy for training PQCs on NISQ devices. By leveraging the utility of maintaining high gradient magnitudes in shallower circuits, this method proposes a significant advancement in overcoming one of the primary barriers in quantum neural network training. As the field progresses, the insights from this work could inspire additional methodologies that ensure efficient and practical implementation of quantum algorithms in real-world applications.