Hybrid quantum programming with PennyLane Lightning on HPC platforms (2403.02512v1)
Abstract: We introduce PennyLane's Lightning suite, a collection of high-performance state-vector simulators targeting CPU, GPU, and HPC-native architectures and workloads. Quantum applications such as QAOA, VQE, and synthetic workloads are implemented to demonstrate the supported classical computing architectures and showcase the scale of problems that can be simulated using our tooling. We benchmark the performance of Lightning with backends supporting CPUs, as well as NVidia and AMD GPUs, and compare the results to other commonly used high-performance simulator packages, demonstrating where Lightning's implementations give performance leads. We show improved CPU performance by employing explicit SIMD intrinsics and multi-threading, batched task-based execution across multiple GPUs, and distributed forward and gradient-based quantum circuit executions across multiple nodes. Our data shows we can comfortably simulate a variety of circuits, giving examples with up to 30 qubits on a single device or node, and up to 41 qubits using multiple nodes.
- Pennylane: Automatic differentiation of hybrid quantum-classical computations, 2022.
- Autograd: Effortless gradients in numpy. In ICML 2015 AutoML Workshop, volume 238, page 5, 2015.
- Automatic differentiation in pytorch, 2017.
- TensorFlow: Large-scale machine learning on heterogeneous systems, 2015. URL https://www.tensorflow.org/. Software available from tensorflow.org.
- JAX: composable transformations of Python+NumPy programs, 2018. URL http://github.com/google/jax.
- Quantum circuit learning. Physical Review A, 98(3), September 2018. ISSN 2469-9934. doi: 10.1103/physreva.98.032309. URL http://dx.doi.org/10.1103/PhysRevA.98.032309.
- Evaluating analytic gradients on quantum hardware. Physical Review A, 99(3), March 2019. ISSN 2469-9934. doi: 10.1103/physreva.99.032331. URL http://dx.doi.org/10.1103/PhysRevA.99.032331.
- General parameter-shift rules for quantum gradients. Quantum, 6:677, March 2022. ISSN 2521-327X. doi: 10.22331/q-2022-03-30-677. URL http://dx.doi.org/10.22331/q-2022-03-30-677.
- Fast quantum circuit cutting with randomized measurements. Quantum, 7:934, March 2023. ISSN 2521-327X. doi: 10.22331/q-2023-03-02-934. URL https://doi.org/10.22331/q-2023-03-02-934.
- Johannes Meyer. Fisher information in noisy intermediate-scale quantum applications. Quantum, 5:539, 09 2021. doi: 10.22331/q-2021-09-09-539.
- PennyLane Lightning developers. PennyLane Lightning Plugins, 2024. URL https://github.com/PennyLaneAI/pennylane-lightning/.
- The cuQuantum development team. NVIDIA cuQuantum SDK. URL https://github.com/nvidia/cuquantum.
- NVIDIA. Cuda, release: 12.3, 2023. URL https://developer.nvidia.com/cuda-toolkit.
- Kokkos: Enabling manycore performance portability through polymorphic memory access patterns. Journal of Parallel and Distributed Computing, 74(12):3202 – 3216, 2014. ISSN 0743-7315. doi: https://doi.org/10.1016/j.jpdc.2014.07.003. URL http://www.sciencedirect.com/science/article/pii/S0743731514001257. Domain-Specific Languages and High-Level Frameworks for High-Performance Computing.
- Kokkos 3: Programming model extensions for the exascale era. IEEE Transactions on Parallel and Distributed Systems, 33(4):805–817, 2022. doi: 10.1109/TPDS.2021.3097283.
- AMD. ROCm, release: 5.7, 2023. URL https://rocm.docs.amd.com/en/docs-5.7.0/release.html.
- PennyLane Lightning developers. DockerHub – pennylaneai/pennylane, 2023. URL https://hub.docker.com/r/pennylaneai/pennylane.
- The spack package manager: bringing order to hpc software chaos. In SC15: International Conference for High-Performance Computing, Networking, Storage and Analysis, pages 1–12, Los Alamitos, CA, USA, nov 2015. IEEE Computer Society. doi: 10.1145/2807591.2807623. URL https://doi.ieeecomputersociety.org/10.1145/2807591.2807623.
- pybind11 – seamless operability between c++11 and python, 2017. https://github.com/pybind/pybind11.
- Parallel programming in OpenMP. Morgan kaufmann, 2001.
- Efficient calculation of gradients in classical simulations of variational quantum algorithms, 2020.
- Intel Xeon Phi Processor High Performance Programming, 2nd Edition. Morgan Kaufmann, 2016.
- Qulacs: a fast and versatile quantum circuit simulator for research purpose. Quantum, 5:559, October 2021. ISSN 2521-327X. doi: 10.22331/q-2021-10-06-559. URL http://dx.doi.org/10.22331/q-2021-10-06-559.
- QCLAB++: Simulating Quantum Circuits on GPUs, 2023.
- Quantum simulation with just-in-time compilation. Quantum, 6:814, September 2022. ISSN 2521-327X. doi: 10.22331/q-2022-09-22-814. URL https://doi.org/10.22331/q-2022-09-22-814.
- Quest and high performance simulation of quantum computers. Scientific Reports, 9(1), July 2019. ISSN 2045-2322. doi: 10.1038/s41598-019-47174-9. URL http://dx.doi.org/10.1038/s41598-019-47174-9.
- Lisandro Dalcin. mpi4py.futures – MPIPoolExecutor, 2023. URL https://mpi4py.readthedocs.io/en/stable/mpi4py.futures.html#mpipoolexecutor.
- Hybrid quantum programming with PennyLane Lightning on HPC platforms accompanying data repository, 2024. URL https://github.com/PennyLaneAI/lightning-on-hpc.
- Joe Mario. C2C - False Sharing Detection in Linux Perf. URL https://joemario.github.io/blog/2016/09/01/c2c-blog/.
- Christina Lee. PennyLane: Adjoint Differentiation, 2021. URL {https://pennylane.ai/qml/demos/tutorial_adjoint_diff/}.
- A variational eigenvalue solver on a photonic quantum processor. Nature Communications, 5(1):4213, Jul 2014. ISSN 2041-1723. doi: 10.1038/ncomms5213. URL https://doi.org/10.1038/ncomms5213.
- Utkarsh Azad. Pennylane quantum chemistry datasets. https://pennylane.ai/datasets/, 2023.
- A quantum approximate optimization algorithm, 2014.
- Quantum supremacy through the quantum approximate optimization algorithm, 2019.
- From the quantum approximate optimization algorithm to a quantum alternating operator ansatz. Algorithms, 12(2):34, February 2019. ISSN 1999-4893. doi: 10.3390/a12020034. URL http://dx.doi.org/10.3390/a12020034.
- Progress toward larger molecular simulation on a quantum computer: Simulating a system with up to 28 qubits accelerated by point-group symmetry. Phys. Rev. A, 105:062452, Jun 2022. doi: 10.1103/PhysRevA.105.062452. URL https://link.aps.org/doi/10.1103/PhysRevA.105.062452.
- Measuring all compatible operators in one series of single-qubit measurements using unitary transformations. Journal of Chemical Theory and Computation, 16(4):2400–2409, 2020. doi: 10.1021/acs.jctc.0c00008. URL https://doi.org/10.1021/acs.jctc.0c00008. PMID: 32150412.
- Measurement optimization in the variational quantum eigensolver using a minimum clique cover. The Journal of Chemical Physics, 152(12):124114, 03 2020. ISSN 0021-9606. doi: 10.1063/1.5141458. URL https://doi.org/10.1063/1.5141458.
- Circuit-centric quantum classifiers. Phys. Rev. A, 101:032308, Mar 2020. doi: 10.1103/PhysRevA.101.032308. URL https://link.aps.org/doi/10.1103/PhysRevA.101.032308.
- NERSC Documentation: Python, 2023a. URL https://docs.nersc.gov/development/languages/python/.
- HPE. HPE Cray Programming Environment User Guide, Dec 2023. URL {https://support.hpe.com/hpesc/public/docDisplay?docLocale=en_US&docId=dp00003823en_usl}.
- NERSC Documentation: Parallel Python – Using mpi4py.futures, 2023b. URL https://docs.nersc.gov/development/languages/python/parallel-python/#using-mpi4pyfutures.
- LUMI – Distribution and binding options, 2023. URL https://docs.lumi-supercomputer.eu/runjobs/scheduled-jobs/distribution-binding/.