Faster and Better Quantum Software Testing through Specification Reduction and Projective Measurements (2405.15450v2)
Abstract: Quantum computing promises polynomial and exponential speedups in many domains, such as unstructured search and prime number factoring. However, quantum programs yield probabilistic outputs from exponentially growing distributions and are vulnerable to quantum-specific faults. Existing quantum software testing (QST) approaches treat quantum superpositions as classical distributions. This leads to two major limitations when applied to quantum programs: (1) an exponentially growing sample space distribution and (2) failing to detect quantum-specific faults such as phase flips. To overcome these limitations, we introduce a QST approach, which applies a reduction algorithm to a quantum program specification. The reduced specification alleviates the limitations (1) by enabling faster sampling through quantum parallelism and (2) by performing projective measurements in the mixed Hadamard basis. Our evaluation of 143 quantum programs across four categories demonstrates significant improvements in test runtimes and fault detection with our reduction approach. Average test runtimes improved from 169.9s to 11.8s, with notable enhancements in programs with large circuit depths (383.1s to 33.4s) and large program specifications (464.8s to 7.7s). Furthermore, our approach increases mutation scores from 54.5% to 74.7%, effectively detecting phase flip faults that non-reduced specifications miss. These results underline our approach's importance to improve QST efficiency and effectiveness.
- Qiskit: An Open-source Framework for Quantum Computing. https://doi.org/10.5281/zenodo.2562111
- Shaukat Ali and Tao Yue. 2023. Quantum Software Testing: A Brief Introduction. In 2023 IEEE/ACM 45th International Conference on Software Engineering: Companion Proceedings (ICSE-Companion). 332–333. https://doi.org/10.1109/ICSE-Companion58688.2023.00093
- Simon Anders and Hans J. Briegel. 2006. Fast simulation of stabilizer circuits using a graph-state representation. Phys. Rev. A 73 (Feb 2006), 022334. Issue 2. https://doi.org/10.1103/PhysRevA.73.022334
- Bug Characteristics in Quantum Software Ecosystem. (2022). arXiv:2204.11965 [cs.SE]
- Andrea Arcuri and Lionel Briand. 2011. A Practical Guide for Using Statistical Tests to Assess Randomized Algorithms in Software Engineering. In Proceedings of the 33rd International Conference on Software Engineering (Waikiki, Honolulu, HI, USA) (ICSE ’11). Association for Computing Machinery, New York, NY, USA, 1–10. https://doi.org/10.1145/1985793.1985795
- Sebastian Baltes and Paul Ralph. 2022. Sampling in software engineering research: a critical review and guidelines. Empirical Softw. Engg. 27, 4 (jul 2022), 31 pages. https://doi.org/10.1007/s10664-021-10072-8
- A scalable maximum likelihood method for quantum state tomography. New Journal of Physics 15, 12 (Dec. 2013), 125004. https://doi.org/10.1088/1367-2630/15/12/125004
- Quantum error mitigation. Reviews of Modern Physics 95, 4 (Dec. 2023). https://doi.org/10.1103/revmodphys.95.045005
- J. Campos and A. Souto. 2021. QBugs: A Collection of Reproducible Bugs in Quantum Algorithms and a Supporting Infrastructure to Enable Controlled Quantum Software Testing and Debugging Experiments. In 2021 IEEE/ACM 2nd International Workshop on Quantum Software Engineering (Q-SE). IEEE Computer Society, Los Alamitos, CA, USA, 28–32. https://doi.org/10.1109/Q-SE52541.2021.00013
- Forest Benchmarking: QCVV using PyQuil. https://doi.org/10.5281/zenodo.3455848
- Efficient quantum state tomography. Nature Communications 1, 1 (dec 2010). https://doi.org/10.1038/ncomms1147
- How many qubits are needed for quantum computational supremacy? Quantum 4 (May 2020), 264. https://doi.org/10.22331/q-2020-05-11-264
- Hints on Test Data Selection: Help for the Practicing Programmer. Computer 11, 4 (1978), 34–41. https://doi.org/10.1109/C-M.1978.218136
- Cirq Developers. 2023. Cirq. https://doi.org/10.5281/zenodo.10247207
- Quantum Computing for Finance: State-of-the-Art and Future Prospects. IEEE Transactions on Quantum Engineering 1 (2020), 1–24. https://doi.org/10.1109/tqe.2020.3030314
- Quantum certification and benchmarking. Nature Reviews Physics 2, 7 (June 2020), 382–390. https://doi.org/10.1038/s42254-020-0186-4
- Pablo Fernández and Miguel A. Martin-Delgado. 2024. Implementing the Grover Algorithm in Homomorphic Encryption Schemes. arXiv:2403.04922 [quant-ph]
- Richard P. Feynman. 1982. Simulating physics with computers. International Journal of Theoretical Physics 21, 6 (June 1982), 467–488. https://doi.org/10.1007/BF02650179
- QMutPy: a mutation testing tool for Quantum algorithms and applications in Qiskit. In Proceedings of the 31st ACM SIGSOFT International Symposium on Software Testing and Analysis (Virtual, South Korea,) (ISSTA 2022). Association for Computing Machinery, New York, NY, USA, 797–800. https://doi.org/10.1145/3533767.3543296
- Learning to Measure: Adaptive Informationally Complete Generalized Measurements for Quantum Algorithms. PRX Quantum 2, 4 (Nov. 2021). https://doi.org/10.1103/prxquantum.2.040342
- Heath Gerhardt and John Watrous. 2003. Continuous-Time Quantum Walks on the Symmetric Group. In Approximation, Randomization, and Combinatorial Optimization.. Algorithms and Techniques, Sanjeev Arora, Klaus Jansen, José D. P. Rolim, and Amit Sahai (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 290–301. https://doi.org/10.1007/978-3-540-45198-3_25
- Research note: Does PLS have advantages for small sample size or non-normal data? MIS Quarterly 36 (09 2012), 981–1001. https://doi.org/10.2307/41703490
- Gaussian Entanglement Measure: Applications to Multipartite Entanglement of Graph States and Bosonic Field Theory. arXiv:2401.17938 [quant-ph]
- Lov K. Grover. 1996. A Fast Quantum Mechanical Algorithm for Database Search. In Proceedings of the Twenty-Eighth Annual ACM Symposium on Theory of Computing (Philadelphia, Pennsylvania, USA) (STOC ’96). Association for Computing Machinery, New York, NY, USA, 212–219. https://doi.org/10.1145/237814.237866
- Leonid Gurvits. 2003. Classical deterministic complexity of Edmonds’ Problem and quantum entanglement. In Proceedings of the Thirty-Fifth Annual ACM Symposium on Theory of Computing (San Diego, CA, USA) (STOC ’03). Association for Computing Machinery, New York, NY, USA, 10–19. https://doi.org/10.1145/780542.780545
- Multiparty entanglement in graph states. Physical Review A 69, 6 (jun 2004). https://doi.org/10.1103/physreva.69.062311
- Melinda Hess and Jeffrey Kromrey. 2004. Robust Confidence Intervals for Effect Sizes: A Comparative Study of Cohen’s d and Cliff’s Delta Under Non-normality and Heterogeneous Variances. Paper Presented at the Annual Meeting of the American Educational Research Association (01 2004).
- Property-Based Testing of Quantum Programs in Q#. In Proceedings of the IEEE/ACM 42nd International Conference on Software Engineering Workshops (Seoul, Republic of Korea) (ICSEW’20). Association for Computing Machinery, New York, NY, USA, 430–435. https://doi.org/10.1145/3387940.3391459
- Quantum advantage in learning from experiments. Science 376, 6598 (2022), 1182–1186. https://doi.org/10.1126/science.abn7293
- A comprehensive survey on quantum computer usage: How many qubits are employed for what purposes? arXiv:2307.16130 [quant-ph]
- Christian M. Ringle Joe F. Hair and Marko Sarstedt. 2011. PLS-SEM: Indeed a Silver Bullet. Journal of Marketing Theory and Practice 19, 2 (2011), 139–152. https://doi.org/10.2753/MTP1069-6679190202 arXiv:https://doi.org/10.2753/MTP1069-6679190202
- Programming Quantum Computers: Essential Algorithms and Code Samples. O’Reilly Media, Incorporated.
- Thierry Nicolas Kaldenbach and Matthias Heller. 2023. Mapping quantum circuits to shallow-depth measurement patterns based on graph states. arXiv:2311.16223 [quant-ph]
- K. Khadiev and E. Krendeleva. 2023. Quantum Algorithm for Searching of Two Sets Intersection. Russian Microelectronics 52, S1 (Dec. 2023), S379–S383. https://doi.org/10.1134/s106373972360084x
- William H. Kruskal and W. Allen Wallis. 1952. Use of Ranks in One-Criterion Variance Analysis. J. Amer. Statist. Assoc. 47, 260 (1952), 583–621. https://doi.org/10.1080/01621459.1952.10483441
- Evaluating Search-Based Software Microbenchmark Prioritization. IEEE Transactions on Software Engineering (March 2024), 1–16. https://doi.org/10.1109/TSE.2024.3380836
- Projection-Based Runtime Assertions for Testing and Debugging Quantum Programs. Proc. ACM Program. Lang. 4, OOPSLA, Article 150 (Nov. 2020), 29 pages. https://doi.org/10.1145/3428218
- Generation of complete graph states in a spin-1/2 Heisenberg chain with a globally optimized magnetic field. Physical Review A 109, 4 (April 2024). https://doi.org/10.1103/physreva.109.042604
- R. B. MARIMONT and M. B. SHAPIRO. 1979. Nearest Neighbour Searches and the Curse of Dimensionality. IMA Journal of Applied Mathematics 24, 1 (08 1979), 59–70. https://doi.org/10.1093/imamat/24.1.59
- Benchmarking the algorithmic performance of near-term neutral atom processors. arXiv:2402.02127 [quant-ph]
- Muskit: A Mutation Analysis Tool for Quantum Software Testing. In 2021 36th IEEE/ACM International Conference on Automated Software Engineering (ASE). 1266–1270. https://doi.org/10.1109/ASE51524.2021.9678563
- Is Your Quantum Program Bug-Free?. In Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering: New Ideas and Emerging Results (Seoul, South Korea) (ICSE-NIER ’20). Association for Computing Machinery, New York, NY, USA, 29–32. https://doi.org/10.1145/3377816.3381731
- Quantum optimization using variational algorithms on near-term quantum devices. Quantum Science and Technology 3, 3 (2018), 030503. https://doi.org/10.1088/2058-9565/aab822
- Efficient algorithm for optimizing data-pattern tomography. Physical Review A 89, 5 (May 2014). https://doi.org/10.1103/physreva.89.054102
- Quantum symbolic execution. Quantum Information Processing 22, 10 (20 Oct. 2023), 389. https://doi.org/10.1007/s11128-023-04144-5
- Michael A. Nielsen and Isaac L. Chuang. 2011. Quantum Computation and Quantum Information: 10th Anniversary Edition (10th ed.). Cambridge University Press, USA.
- Replication package for our paper entitled "Faster and Better Quantum Software Testing through Specification Reduction and Projective Measurements". https://doi.org/10.5281/zenodo.11191215
- Matteo Paltenghi and Michael Pradel. 2022. Bugs in Quantum computing platforms: an empirical study. Proc. ACM Program. Lang. 6, OOPSLA1, Article 86 (apr 2022), 27 pages. https://doi.org/10.1145/3527330
- Matteo Paltenghi and Michael Pradel. 2023. MorphQ: Metamorphic Testing of the Qiskit Quantum Computing Platform. In Proceedings of the 45th International Conference on Software Engineering (Melbourne, Victoria, Australia) (ICSE ’23). IEEE Press, 2413–2424. https://doi.org/10.1109/ICSE48619.2023.00202
- Karl Pearson. 1900. X. On the criterion that a given system of deviations from the probable in the case of a correlated system of variables is such that it can be reasonably supposed to have arisen from random sampling. The London, Edinburgh, and Dublin Philosophical Magazine and Journal of Science 50, 302 (July 1900), 157–175. https://doi.org/10.1080/14786440009463897
- Hypercube quantum search: exact computation of the probability of success in polynomial time. Quantum Information Processing 22, 3 (March 2023). https://doi.org/10.1007/s11128-023-03883-9
- John Preskill. 2018. Quantum Computing in the NISQ era and beyond. Quantum 2 (Aug. 2018), 79. https://doi.org/10.22331/q-2018-08-06-79
- MQT Bench: Benchmarking Software and Design Automation Tools for Quantum Computing. Quantum (2023). MQT Bench is available at https://www.cda.cit.tum.de/mqtbench/.
- Eleanor Rieffel and Wolfgang Polak. 2011. Quantum Computing: A Gentle Introduction (1st ed.). The MIT Press.
- Jun John Sakurai. 1994. Modern quantum mechanics; rev. ed. Addison-Wesley, Reading, MA. https://cds.cern.ch/record/1167961
- Raqueline A. M. Santos. 2016. Szegedy’s quantum walk with queries. Quantum Information Processing 15, 11 (1 Nov. 2016), 4461–4475. https://doi.org/10.1007/s11128-016-1427-4
- Efficient quantum state tomography with convolutional neural networks. npj Quantum Information 8, 1 (Sept. 2022). https://doi.org/10.1038/s41534-022-00621-4
- Correlation coefficients: appropriate use and interpretation. Anesthesia & analgesia 126, 5 (2018), 1763–1768. https://doi.org/10.1213/ANE.0000000000002864
- Automatic generation of Grover quantum oracles for arbitrary data structures. Quantum Science and Technology 8, 2 (2023), 025003. https://doi.org/10.1088/2058-9565/acaf9d
- Peter W. Shor. 1995. Scheme for reducing decoherence in quantum computer memory. Phys. Rev. A 52 (Oct 1995), R2493–R2496. Issue 4. https://doi.org/10.1103/PhysRevA.52.R2493
- Peter W. Shor. 1997. Polynomial-Time Algorithms for Prime Factorization and Discrete Logarithms on a Quantum Computer. SIAM J. Comput. 26, 5 (oct 1997), 1484–1509. https://doi.org/10.1137/s0097539795293172
- Nikolai A. Sinitsyn and Bin Yan. 2023. Topologically protected Grover’s oracle for the partition problem. Physical Review A 108, 2 (Aug. 2023). https://doi.org/10.1103/physreva.108.022412
- Dag I. K. Sjøberg and Gunnar Rye Bergersen. 2023. Construct Validity in Software Engineering. IEEE Transactions on Software Engineering 49, 3 (2023), 1374–1396. https://doi.org/10.1109/TSE.2022.3176725
- Klaas-Jan Stol and Brian Fitzgerald. 2018. The ABC of Software Engineering Research. ACM Trans. Softw. Eng. Methodol. 27, 3, Article 11 (sep 2018), 51 pages. https://doi.org/10.1145/3241743
- Neural-network quantum state tomography. Nature Physics 14, 5 (Feb. 2018), 447–450. https://doi.org/10.1038/s41567-018-0048-5
- Entangling single atoms over 33 km telecom fibre. Nature 607, 7917 (jul 2022), 69–73. https://doi.org/10.1038/s41586-022-04764-4
- András Vargha and Harold D. Delaney. 2000. A Critique and Improvement of the CL Common Language Effect Size Statistics of McGraw and Wong. Journal of Educational and Behavioral Statistics 25, 2 (2000), 101–132. https://doi.org/10.3102/10769986025002101
- Salvador Elías Venegas-Andraca. 2012. Quantum walks: a comprehensive review. Quantum Information Processing 11, 5 (July 2012), 1015–1106. https://doi.org/10.1007/s11128-012-0432-5
- Breaking the Curse of Dimensionality Using Decompositions of Incomplete Tensors: Tensor-based scientific computing in big data analysis. IEEE Signal Processing Magazine 31, 5 (2014), 71–79. https://doi.org/10.1109/MSP.2014.2329429
- David J. Wales and Jonathan P. K. Doye. 1997. Global Optimization by Basin-Hopping and the Lowest Energy Structures of Lennard-Jones Clusters Containing up to 110 Atoms. The Journal of Physical Chemistry A 101, 28 (1997), 5111–5116. https://doi.org/10.1021/jp970984n
- QDiff: Differential Testing of Quantum Software Stacks. In 2021 36th IEEE/ACM International Conference on Automated Software Engineering (ASE). IEEE/ACM, Australia, 692–704. https://doi.org/10.1109/ASE51524.2021.9678792
- Quito: a Coverage-Guided Test Generator for Quantum Programs. In 2021 36th IEEE/ACM International Conference on Automated Software Engineering (ASE). 1237–1241. https://doi.org/10.1109/ASE51524.2021.9678798
- QuSBT: Search-Based Testing of Quantum Programs. In Proceedings of the ACM/IEEE 44th International Conference on Software Engineering: Companion Proceedings (Pittsburgh, Pennsylvania) (ICSE ’22). Association for Computing Machinery, New York, NY, USA, 173–177. https://doi.org/10.1145/3510454.3516839
- Decision diagrams for quantum computing. In Design Automation of Quantum Computers. Springer, 1–23.
- Mingyou Wu. 2024. Efficiency of k-Local Quantum Search and its Adiabatic Variant on Random k-SAT. arXiv:2403.03237 [quant-ph]
- Jianjun Zhao. 2021. Quantum Software Engineering: Landscapes and Horizons. arXiv:2007.07047 [cs.SE]
- Bugs4Q: A Benchmark of Real Bugs for Quantum Programs. In 2021 36th IEEE/ACM International Conference on Automated Software Engineering (ASE). IEEE Computer Society, Los Alamitos, CA, USA, 1373–1376. https://doi.org/10.1109/ASE51524.2021.9678908
- An applied quantum Hoare logic. In Proceedings of the 40th ACM SIGPLAN Conference on Programming Language Design and Implementation (Phoenix, AZ, USA) (PLDI 2019). Association for Computing Machinery, New York, NY, USA, 1149–1162. https://doi.org/10.1145/3314221.3314584