Representativeness of the density-controlled random circuit generator for benchmarking compilers

Determine whether a gate-density-controlled random circuit generator that allows explicit control of width, depth, and gate density is an effective surrogate for recognized benchmark suites when profiling quantum compiler performance and cost. Ascertain whether the observed discrepancies in average and maximum speedup and in overhead metrics between results on generated circuits and on benchmark-suite circuits arise primarily from differences in circuit scale (width, depth, and density distributions) or from structural characteristics of benchmark circuits that are not captured by the generator.

Background

The paper introduces a random circuit generator that, unlike common tools, provides control over gate density as well as width and depth—three factors shown to have significant influence on compilation time. This capability is used to generate thousands of large-scale circuits for evaluating a parallel compilation strategy alongside recognized benchmark suites.

When comparing results obtained on circuits produced by the generator to those from benchmark suites, the authors observe noticeable differences in average and maximum speedup and in extreme values for gate, SWAP, and depth overheads. They partially control for scale by filtering to similar width and density, which narrows but does not eliminate the discrepancies. Benchmark suites also include circuits with algorithm-specific structure (e.g., Shor and Grover), which the current generator does not explicitly reproduce, motivating the question of representativeness and the root cause of the observed differences.