Quantum Annealing Formulation for Binary Neural Networks

Abstract: Quantum annealing is a promising paradigm for building practical quantum computers. Compared to other approaches, quantum annealing technology has been scaled up to a larger number of qubits. On the other hand, deep learning has been profoundly successful in pushing the boundaries of AI. It is thus natural to investigate potentially game changing technologies such as quantum annealers to augment the capabilities of deep learning. In this work, we explore binary neural networks, which are lightweight yet powerful models typically intended for resource constrained devices. Departing from current training regimes for binary networks that smooth/approximate the activation functions to make the network differentiable, we devise a quadratic unconstrained binary optimization formulation for the training problem. While the problem is intractable, i.e., the cost to estimate the binary weights scales exponentially with network size, we show how the problem can be optimized directly on a quantum annealer, thereby opening up to the potential gains of quantum computing. We experimentally validated our formulation via simulation and testing on an actual quantum annealer (D-Wave Advantage), the latter to the extent allowable by the capacity of current technology.

• The paper reframes BNN training as a QUBO problem, enabling direct optimization of binary weights using quantum annealing.
• It utilizes the D-Wave Advantage QPU to compare quantum and classical optimization methods, highlighting runtime benefits and accuracy challenges.
• Experimental results show promising runtime improvements with quantum annealing, though scaling to full-scale networks remains a significant hurdle.

Quantum Annealing Formulation for Binary Neural Networks

The paper "Quantum Annealing Formulation for Binary Neural Networks" (2107.02751) explores the intersection of quantum computing and machine learning by leveraging quantum annealers for training Binary Neural Networks (BNNs). This crossover represents a novel approach to utilizing the growing capabilities of quantum computers for machine learning tasks specifically focused on optimizing the resource-intensive operations within BNNs.

Introduction to Quantum Annealing and BNNs

Quantum annealing, a form of adiabatic quantum computing, is particularly suited for addressing specific types of optimization problems. The paper capitalizes on this feature by formulating the training of BNNs as a Quadratic Unconstrained Binary Optimization (QUBO) problem, which can be directly solved using quantum annealers like the D-Wave Advantage (Figure 1).

Figure 1: (a) Topology of the Pegasus QPU of D-Wave Advantage, where the yellow lines indicate qubit interconnections.

BNNs, known for their lightweight architecture, are ideal for deployment in resource-constrained environments. Traditional training techniques for BNNs employ smoothing techniques to approximate functions to maintain differentiability. This paper departs from these conventional approaches by directly tackling the optimization of binary weights using quantum annealing.

QUBO Formulation for BNN

The core contribution is reframing the BNN training as a QUBO problem. This involves mapping the traditional training objectives and constraints of neural networks onto a form that quantum annealers can process. The traditional approach of using stochastic gradient descent is replaced by a direct optimization of binary weights, bypassing the need for approximations in the activation functions.

The QUBO approach hinges on the concept that the energy minimization intrinsic to quantum annealers can effectively address the NP-hard problem of optimizing BNN weights. The QUBO formulation incorporates penalty terms to enforce neuron activation constraints and weight relationships, creating a complex graph that must be efficiently mapped onto the quantum processing unit.

Experimental Validation

The implementation of this formulation was empirically validated using both simulations and execution on an actual quantum annealer. The experiments compared various classical optimization algorithms with quantum annealing to solve the formulated QUBO problems.

Figure 2: Histograms of runtime (in logarithmic scale) and distance of solution of different algorithms on two-layer BNNs.

The results, depicted in Figure 2, show that quantum annealing provides solutions with a runtime that significantly benefits from the quantum annealer's parallelism and inherent tunneling abilities. However, the accuracy measured as distance to the optimal classical solution still posed challenges, such as thermal effects.

Scaling and Challenges

While the D-Wave Advantage's Pegasus architecture has the capacity to handle a notable number of qubits, embedding real-world problem instances remains a challenge due to the complex connectivity required by practical neural network models. The scaling analysis showed exponential growth in runtime with problem size when solved using classical algorithms, highlighting the potential advantage quantum annealers could provide (Figure 3).

Figure 3: Runtime of BnB as a function of number of variables of QCBO on several BNNs.

Despite the promising results, one critical limitation remains: the current generation of quantum annealers supports only small neural networks due to the connectivity and qubit limitations. Quantum annealing's real-world applications in training full-scale neural networks remain theoretical, pending future advancements in quantum hardware capabilities.

Conclusion

This paper demonstrates a significant step towards utilizing quantum computing for practical machine learning problems, specifically in training BNNs. The QUBO-based formulation opens new pathways for efficient optimization using quantum hardware, although challenges in embedding and scaling persist. Future work is directed towards optimizing the formulation further and exploring online training paradigms, potentially enhancing the applicability of quantum annealers in large-scale neural network training. As quantum technologies progress, the integration of such methods could potentially reshape how machine learning models are trained with respect to speed and resource efficiency.