MQT Bench: Benchmarking Software and Design Automation Tools for Quantum Computing (2204.13719v3)

Abstract: Quantum software tools for a wide variety of design tasks on and across different levels of abstraction are crucial in order to eventually realize useful quantum applications. This requires practical and relevant benchmarks for new software tools to be empirically evaluated and compared to the current state of the art. Although benchmarks for specific design tasks are commonly available, the demand for an overarching cross-level benchmark suite has not yet been fully met and there is no mutual consolidation in how quantum software tools are evaluated thus far. In this work, we propose the MQT Bench benchmark suite (as part of the Munich Quantum Toolkit, MQT) based on four core traits: (1) cross-level support for different abstraction levels, (2) accessibility via an easy-to-use web interface (https://www.cda.cit.tum.de/mqtbench) and a Python package, (3) provision of a broad selection of benchmarks to facilitate generalizability, as well as (4) extendability to future algorithms, gate-sets, and hardware architectures. By comprising more than 70,000 benchmark circuits ranging from 2 to 130 qubits on four abstraction levels, MQT Bench presents a first step towards benchmarking different abstraction levels with a single benchmark suite to increase comparability, reproducibility, and transparency.

• The paper introduces MQT Bench, providing over 70,000 benchmark circuits spanning four quantum abstraction levels.
• It offers dual accessibility via a user-friendly web interface and Python package, promoting practical evaluations for researchers and developers.
• The suite streamlines fragmented testing approaches by ensuring comparability and reproducibility in quantum software tool analysis.

Overview of MQT Bench: A Benchmarking Suite for Quantum Software Tools

This paper presents MQT Bench, a benchmark suite explicitly designed for quantum computing software tools and design automation processes. The authors identify the need for comprehensive benchmarking across different abstraction levels in quantum computing due to the burgeoning interest and application of quantum processors. The suite is part of the Munich Quantum Toolkit (MQT), encompassing a wide range of benchmarks that address the functionalities and performance of quantum algorithms and their compilation flows.

Key Contributions

The primary contribution of MQT Bench is the provision of more than 70,000 benchmark circuits. These circuits span four abstraction levels: algorithmic, target-independent, target-dependent native gates, and target-dependent mapped levels. Notably, these circuits vary between 2 to 130 qubits. This approach ensures that benchmarks cater to multiple stages of quantum algorithm development, from high-level descriptions to machine-executable mappings.

The authors emphasize the suite’s accessibility and usability through an intuitive web interface and a Python package. This duality facilitates user engagement by offering both an easy-to-access online platform and tools for local generation of benchmarks. It fosters broad adoption by both researchers and developers, thus encouraging practical, transparent empirical evaluations of quantum software tools.

Technical Implications

MQT Bench is significant in addressing a critical gap in quantum computing infrastructure—the benchmarking of software tools across various abstraction levels. The paper argues that existing benchmarks are fragmented, often aligned with specific levels of the quantum compilation process. MQT Bench addresses this by offering a consolidated suite, effectively enabling evaluations that are comprehensive and consistent. This consolidation supports the comparability and reproducibility of research findings, two pillars essential for the progression of quantum software research and development.

Results and Discussion

The paper discusses several numerical results, elaborating on distribution characteristics such as qubit usage across benchmarks, compiler performance, and device deployment. These analyses illustrate the suite’s robustness and adaptability to various user needs and configurations, highlighting the potential for significant user-driven insights into software tool performance. The four core traits of the suite—cross-level support, accessibility, algorithm selection, and extendability—form the foundation upon which these results are built.

Future Directions

The authors introduce speculative possibilities for future enhancements in the benchmarking of quantum software. Given the rapid evolution of quantum computing architectures and algorithms, MQT Bench is conceptualized with inherent flexibility to extend its capabilities, adapting to emerging paradigms within the field. This includes integration of novel algorithms, native gate-sets, and architectures as quantum computing technology progresses.

In conclusion, MQT Bench stands as a seminal tool, facilitating a structured approach to benchmarking quantum software, with implications both practical for current application scenarios and theoretical for future advancements in quantum computing infrastructure. By fostering an ecosystem that promotes comparability, reproducibility, and transparency, it potentially accelerates progress in the field, supporting both existing and emerging quantum technologies.