A comment on comparing optimization on D-Wave and IBM quantum processors (2406.19351v1)

Abstract: Recent work [Sachdeva et al.] presented an iterative hybrid quantum variational optimization algorithm designed by Q-CTRL and executed on IBM gate-based quantum processing units (QPUs), claiming a significant performance advantage against a D-Wave quantum annealer. Here we point out major methodological problems with this comparison. Using a simple unoptimized workflow for quantum annealing, we show success probabilities multiple orders of magnitude higher than those reported by [Sachdeva et al.]. These results, which can be reproduced using open-source code and free trial access to a D-Wave quantum annealer, contradict Q-CTRL's claims of superior performance. We also provide a direct comparison between quantum annealing and a recent demonstration of digitized quantum annealing on an IBM processor, showing that analog quantum annealing on a D-Wave QPU reaches far lower energies than digitized quantum annealing on an IBM QPU.

• The paper challenges previous claims by revealing that averaging poor parameter settings misrepresented quantum annealing's success probabilities.
• The analysis emphasizes the need to account for time-to-solution in addition to raw success probabilities for fair comparisons between quantum processors.
• Experimental reevaluation on a 133-qubit spin glass problem demonstrates that D-Wave’s performance remains competitive when benchmarked equitably.

Comparative Analysis of Quantum Optimization: D-Wave vs. IBM Processors

In this paper, the authors critically evaluate the claims made by a paper that suggested a hybrid variational algorithm executed on IBM gate-based quantum processors outperformed D-Wave's quantum annealers in specific optimization tasks. The focus is on the methodological shortcomings of this comparison and presenting data that refutes the original paper's claims of superiority.

Critique of Methodology

The authors identify several methodological weaknesses in the original comparison by Sachdeva et al., specifically targeting:

1. Parametric Grid Search Misrepresentation: The success probabilities for quantum annealing (QA) were computed by averaging across all runs of a grid search, which included poor parameter settings, leading to a skewed perception of QA performance. The so-called exploratory process was not mirrored in the results reported for the IBM processor's performance.
2. Neglect of Run Time in Success Probabilities: An exclusive focus on success probability without accounting for solution time renders comparisons inequitable. Unlike time to solution (TTS), which is a comprehensive metric incorporating both time and success probability, raw success probabilities do not reflect computational efficiency adequately.
3. Asymmetric Postprocessing: Classical postprocessing was selectively applied to the IBM processor results but not to the D-Wave results, which obscures the actual computational contribution and efficiency of the quantum annealing process.
4. Biased Problem Set Selection: Problem instances were chosen in a seemingly biased manner favoring gate-based processors by highlighting cases where QA performed poorly and omitting favorable test cases for quantum annealers.

Results and Experimental Reevaluation

Through re-examination, the paper delivers an improved understanding of the benchmarks. Utilizing analogous benchmarking principles, the researchers attained considerably higher success probabilities with quantum annealing than those initially reported. For instance, quantum annealing exhibited better TTS metrics across various problem instances, exemplifying its competitive advantage when executed optimally and compared on equal footing with IBM's approach.

Further demonstrative comparisons between analog quantum annealing and digitized quantum annealing were conducted on a 133-qubit spin glass problem. In these trials, D-Wave's devices produced superior residual energies compared to IBM's processors, especially when scaling the energy levels appropriately.

Implications and Future Directions

The research highlights the necessity for standardized, equitable methodological frameworks while benchmarking quantum optimization algorithms across different QPU modalities. The demonstrated discrepancies in methodology spotlight the need for caution, clarity, and rigor in reporting comparative studies, to avoid misconceptions regarding quantum processors' capabilities.

The findings suggest that quantum annealers remain a viable approach for certain optimization problems, achieving competitive, if not superior, performance against digital quantum simulators. Future work may involve broader testbeds of problems and further exploration of quantum error mitigation strategies to enhance performance robustness in scalable quantum computing contexts.

The implications of this analysis are twofold: methodologically, by establishing a precedent for comprehensive and fair comparison practices; and practically, by reaffirming the potential of quantum annealing as an enduring tool in optimizing computationally intensive tasks.