Metriq Score: Composite Quantum Benchmark Index

• Metriq Score is a composite index that aggregates normalized outcomes from diverse quantum benchmarks through a three-stage process of width aggregation, baseline normalization, and weighted averaging.
• It facilitates cross-platform, leaderboard-style comparisons while preserving benchmarking details and addressing missing data by assigning zero scores when needed.
• The score synthesizes both system-level and application-inspired benchmarks, offering transparent, actionable insights into the practical performance envelope of quantum devices.

Searching arXiv for papers on Metriq Score and closely related benchmarking context. The Metriq Score is a composite index for summarizing the performance of a quantum device across the benchmark suite of the Metriq platform, an open-source collaborative system for reproducible cross-platform quantum benchmarking (Cosentino et al., 9 Mar 2026). In the formulation presented by Metriq, the score assigns a single scalar value to each device by aggregating normalized outcomes from multiple heterogeneous quantum benchmarks under an explicit weighting scheme. It is designed as a practical summary for leaderboard-style comparison across vendors and architectures, while preserving the underlying benchmark-level results as the primary evidence. The authors state explicitly that it is not a physical fidelity and not an estimate of one latent hardware property; rather, it is a transparent, baseline-normalized composite over diverse benchmark outcomes (Cosentino et al., 9 Mar 2026).

1. Origin within the Metriq platform

Metriq was introduced as a response to a fragmented benchmarking landscape in quantum computing, characterized by system-specific tools and inconsistent evaluation methodologies that impede reliable cross-platform assessment (Cosentino et al., 9 Mar 2026). The platform integrates benchmark definition and execution, data collection, and public presentation into a unified workflow. Its benchmark suite spans both system-level metrics, such as entanglement quality, gate performance, and circuit speed, and application-inspired protocols drawn from quantum machine learning, optimization, and quantum simulation (Cosentino et al., 9 Mar 2026).

Within that framework, the Metriq Score appears as the platform’s aggregate summary statistic. The paper situates it as a pragmatic device for synthesizing a device’s behavior across a heterogeneous suite rather than as a replacement for component benchmarks. This distinction is central: the score exists because isolated benchmark results are difficult to compare across platforms, yet no single benchmark captures “quality, speed, and scale” together (Cosentino et al., 9 Mar 2026). A plausible implication is that the score is intended to facilitate operational comparison while leaving methodological pluralism intact.

The score is defined per device and per suite series, written as $\mathrm{MS}(d,s)$, where $d$ denotes a device and $s$ the benchmark-suite series or version (Cosentino et al., 9 Mar 2026). This versioning matters because the suite can evolve, benchmarks can be added or removed, and weights can be changed. The paper therefore treats the score as a versioned index rather than a timeless scalar.

2. Benchmark suite included in the score

For the suite used in the paper’s main results, the Metriq Score aggregates eight benchmarks (Cosentino et al., 9 Mar 2026). These include four system-level benchmarks and four application-inspired benchmarks.

Category	Benchmark	Role in suite
System-level	BSEQ	Bell State Effective Qubits
System-level	EPLG	Error Per Layered Gate
System-level	Mirror Circuits	Multi-width circuit benchmark
System-level	CLOPS	Circuit Layer Operations Per Second
Application-inspired	QML Kernel	Quantum machine learning kernel
Application-inspired	LR-QAOA	Linear-ramp QAOA
Application-inspired	WIT	Wormhole-Inspired Teleportation
Application-inspired	QFT	Quantum Fourier Transform

The score does not include provider pricing or cost estimates, runtime cost accounting from Section 6, uncertainty bars, calibration metadata directly, or all possible future Metriq-supported benchmarks beyond the chosen suite version (Cosentino et al., 9 Mar 2026). Runtime enters only indirectly through CLOPS, which is itself a throughput benchmark based on reported execution time. Processor size does not appear as an explicit multiplier either, but scale enters structurally because benchmarks are run at different widths and larger widths receive more weight (Cosentino et al., 9 Mar 2026).

This benchmark selection makes the score intentionally heterogeneous. BSEQ emphasizes connectivity and effective entangling scale, EPLG measures layered-gate error behavior, Mirror Circuits capture compiled circuit fidelity via polarization, CLOPS measures throughput, and the application-inspired protocols probe performance on structured workloads. This suggests that the score is meant to summarize a device’s practical performance envelope rather than any single fault model.

3. Formal definition and aggregation procedure

The Metriq Score is defined through a three-stage aggregation process: within-benchmark aggregation across widths, baseline normalization of benchmark-level results, and weighted averaging across benchmarks (Cosentino et al., 9 Mar 2026).

The paper introduces a component index set $\mathcal{C}=\{1,\dots,K\}$ for all recorded benchmark components in a suite series and defines the set of distinct benchmarks as

$$\mathcal{B} := \{\, b_i : i\in\mathcal{C} \,\}, \qquad \mathcal{C}_b := \{\, i\in\mathcal{C} : b_i=b \,\} \ \ (b\in\mathcal{B}).$$

Here, component $i$ corresponds to a particular benchmark, metric, and run configuration; $n_i$ denotes circuit width, $v_i(d,s)$ the measured value on device $d$ in series $s$, and $d$0 the designated baseline device for that series (Cosentino et al., 9 Mar 2026).

Within-benchmark aggregation across widths

Many benchmarks are evaluated at multiple widths. The within-benchmark weight for component $d$1 is

$d$2

so larger-width instances contribute more (Cosentino et al., 9 Mar 2026). The width-aggregated raw value for benchmark $d$3 is then

$d$4

This is a linear weighted average over raw values. The paper explicitly does not define the Metriq Score via geometric means, percentile scoring, z-scores, PCA, learned weights, or nonlinear aggregation (Cosentino et al., 9 Mar 2026).

Baseline normalization

After obtaining one raw benchmark value per device and benchmark, the paper normalizes against a designated baseline device. If higher is better for benchmark $d$5,

$d$6

If lower is better,

$d$7

Thus, a benchmark subscore $d$8 means parity with the baseline, values above 100 indicate better-than-baseline performance, and values below 100 indicate worse-than-baseline performance (Cosentino et al., 9 Mar 2026).

Across-benchmark weighting and final score

To combine benchmark subscores, the paper defines an “effective width”

$d$9

For BSEQ, which does not have a natural width sweep, the suite assigns a reference width; in the present series, $s$0 (Cosentino et al., 9 Mar 2026). Benchmark weights are then

$s$1

The final Metriq Score is

$s$2

The final score is therefore a weighted arithmetic mean of dimensionless benchmark ratios. This is the paper’s central definition of the Metriq Score (Cosentino et al., 9 Mar 2026).

4. Benchmark-specific raw scores entering the composite

Although the composite formula is uniform, the raw benchmark values entering it are benchmark-specific. The score therefore aggregates heterogeneous submetrics rather than repeated instances of a common observable (Cosentino et al., 9 Mar 2026).

For BSEQ, the platform uses the largest connected component size and connection fraction. The paper defines

$s$3

then combines them as

$s$4

This already embeds a benchmark-internal weighted average before entry into the global score (Cosentino et al., 9 Mar 2026).

For EPLG, lower values are better, so the benchmark is normalized inversely against the baseline. For lengths $s$5,

$s$6

The benchmark-level score is then described as a weighted harmonic mean with a coverage penalty, intended to penalize missing larger chain lengths (Cosentino et al., 9 Mar 2026).

For Mirror Circuits, polarization at panel point $s$7 is

$s$8

The raw benchmark value is then aggregated as

$s$9

That raw value is normalized against the baseline at the next stage (Cosentino et al., 9 Mar 2026).

For CLOPS, the raw benchmark is

$\mathcal{C}=\{1,\dots,K\}$0

Because CLOPS is evaluated at a single width in the paper’s series, width aggregation is trivial (Cosentino et al., 9 Mar 2026).

For QML Kernel, the raw aggregate is

$\mathcal{C}=\{1,\dots,K\}$1

with weights proportional to width (Cosentino et al., 9 Mar 2026).

For LR-QAOA, the effective approximation ratio is

$\mathcal{C}=\{1,\dots,K\}$2

and the benchmark-level aggregate over $\mathcal{C}=\{1,\dots,K\}$3 is

$\mathcal{C}=\{1,\dots,K\}$4

For QFT, the raw score is

$\mathcal{C}=\{1,\dots,K\}$5

For WIT, the benchmark is evaluated at a single width $\mathcal{C}=\{1,\dots,K\}$6, so width aggregation is again trivial (Cosentino et al., 9 Mar 2026).

A key implication is that the final Metriq Score combines observables with distinct semantics—polarization, approximation ratio, throughput, connectivity, and others—only after each has been normalized against a common baseline.

5. Weighting scheme, missing-data policy, and interpretation

The benchmark weights used in the paper are derived from effective widths $\mathcal{C}=\{1,\dots,K\}$7 reported in the appendix (Cosentino et al., 9 Mar 2026). The values given are:

Benchmark	Effective width $\mathcal{C}=\{1,\dots,K\}$8	Relative role
BSEQ	100	Largest default weight
EPLG	$\mathcal{C}=\{1,\dots,K\}$9	Large
Mirror Circuits	$$\mathcal{B} := \{\, b_i : i\in\mathcal{C} \,\}, \qquad \mathcal{C}_b := \{\, i\in\mathcal{C} : b_i=b \,\} \ \ (b\in\mathcal{B}).$$0	Large
CLOPS	100	Largest default weight
QML Kernel	$$\mathcal{B} := \{\, b_i : i\in\mathcal{C} \,\}, \qquad \mathcal{C}_b := \{\, i\in\mathcal{C} : b_i=b \,\} \ \ (b\in\mathcal{B}).$$1	Moderate
WIT	7	Small
LR-QAOA	$$\mathcal{B} := \{\, b_i : i\in\mathcal{C} \,\}, \qquad \mathcal{C}_b := \{\, i\in\mathcal{C} : b_i=b \,\} \ \ (b\in\mathcal{B}).$$2	Large
QFT	$$\mathcal{B} := \{\, b_i : i\in\mathcal{C} \,\}, \qquad \mathcal{C}_b := \{\, i\in\mathcal{C} : b_i=b \,\} \ \ (b\in\mathcal{B}).$$3	Small

The appendix gives $$\mathcal{B} := \{\, b_i : i\in\mathcal{C} \,\}, \qquad \mathcal{C}_b := \{\, i\in\mathcal{C} : b_i=b \,\} \ \ (b\in\mathcal{B}).$$4, so $$\mathcal{B} := \{\, b_i : i\in\mathcal{C} \,\}, \qquad \mathcal{C}_b := \{\, i\in\mathcal{C} : b_i=b \,\} \ \ (b\in\mathcal{B}).$$5 (Cosentino et al., 9 Mar 2026). The largest default weights therefore go to BSEQ and CLOPS, followed by Mirror Circuits, then EPLG and LR-QAOA. This means the score is intentionally scale-aware: larger and broader-width benchmarks contribute more.

The paper also adopts a strong missing-data policy. If benchmark $$\mathcal{B} := \{\, b_i : i\in\mathcal{C} \,\}, \qquad \mathcal{C}_b := \{\, i\in\mathcal{C} : b_i=b \,\} \ \ (b\in\mathcal{B}).$$6 is missing for device $$\mathcal{B} := \{\, b_i : i\in\mathcal{C} \,\}, \qquad \mathcal{C}_b := \{\, i\in\mathcal{C} : b_i=b \,\} \ \ (b\in\mathcal{B}).$$7 in series $$\mathcal{B} := \{\, b_i : i\in\mathcal{C} \,\}, \qquad \mathcal{C}_b := \{\, i\in\mathcal{C} : b_i=b \,\} \ \ (b\in\mathcal{B}).$$8, it sets $$\mathcal{B} := \{\, b_i : i\in\mathcal{C} \,\}, \qquad \mathcal{C}_b := \{\, i\in\mathcal{C} : b_i=b \,\} \ \ (b\in\mathcal{B}).$$9 (Cosentino et al., 9 Mar 2026). Missing widths within benchmark-specific aggregates are also set to zero for several benchmarks, including QML Kernel, LR-QAOA, and Mirror Circuits. The table caption further states that the CLOPS score contributes a zero value to the composite score for devices that do not provide the needed timing information (Cosentino et al., 9 Mar 2026).

This has an important interpretive consequence. The score rewards breadth of demonstrated capability and penalizes incomplete observability. The authors present this as a deliberate design choice. A plausible implication is that the score captures not only measured performance but also benchmark coverage and metadata availability. The paper treats this as a feature, though it also identifies it as a limitation.

In practical interpretation, a score of 100 corresponds to equality with the baseline device on the weighted composite. A score above 100 means better-than-baseline aggregate performance under the chosen suite and weights, while a score below 100 indicates worse-than-baseline performance (Cosentino et al., 9 Mar 2026). The authors caution that this does not mean “twice as good” in any physical sense when the score is 200; it is only a relative composite index over normalized benchmark subscores.

6. Reported results, correlations, and caveats

The baseline device in the paper’s main table is ibm_torino, set to 100.00 for every normalized component and therefore for the Metriq Score as well (Cosentino et al., 9 Mar 2026). The reported aggregate values are:

Device	Metriq Score
ibm_boston	252.61
quantinuum_h2_2	188.05
ibm_pittsburgh	174.51
ibm_kingston	174.23
ibm_marrakesh	156.82
ibm_fez	116.77
ibm_torino	100.00
iqm_emerald	23.76
iqm_garnet	14.34
rigetti_ankaa_3	4.54
wukong_72	3.38

The paper notes that ibm_boston leads the table by a substantial margin, with strong benchmark values in BSEQ, EPLG, Mirror Circuits, and QML, plus CLOPS support (Cosentino et al., 9 Mar 2026). quantinuum_h2_2 performs strongly on QML, WIT, QFT, and Mirror Circuits, but is penalized by missing CLOPS timing support and smaller-scale availability in some benchmarks. Devices lacking timing data lose CLOPS contribution entirely, while smaller devices or those unable to execute large-width instances are penalized through zero-valued widths or missing benchmark subscores (Cosentino et al., 9 Mar 2026).

The paper explicitly warns that these values were collected at different times between March 2025 and March 2026 and should not be interpreted as a definitive or current ranking (Cosentino et al., 9 Mar 2026). That caution is consistent with the platform’s longitudinal design: scores are snapshots computed from available results at a particular time.

Metriq also reports cross-benchmark analyses enabled by the shared dataset. The Metriq Score is reported to correlate most tightly with Mirror Circuits, with Spearman $i$0 (Cosentino et al., 9 Mar 2026). Across devices with available public calibration data, the score correlates with two-qubit gate fidelity at Spearman $i$1 (Cosentino et al., 9 Mar 2026). Additional correlations include Mirror Circuits with QML Kernel at $i$2, QML Kernel with QFT at $i$3, and BSEQ with LR-QAOA at $i$4 (Cosentino et al., 9 Mar 2026). The paper also reports that the first principal component of z-scored log-scores on complete-data devices explains 88% of the variance, suggesting a strong common performance axis across benchmarks (Cosentino et al., 9 Mar 2026).

The authors nonetheless caution against overinterpretation. The score is not a physical quantity, weighting choices are normative, baseline choice can affect ranking, missing-data penalties may confound performance with observability, compilation differences can materially affect outcomes, and uncertainty is not propagated through the composite (Cosentino et al., 9 Mar 2026). Benchmarks are heterogeneous and architecture-sensitive, and timing support is not uniformly available across providers. The score is therefore best understood as a transparent, reproducible composite index under explicit methodological choices rather than as a universal ranking of quantum computer quality.

The paper’s broader significance lies in making those methodological choices explicit. By defining the Metriq Score as

$i$5

with width-weighted aggregation, baseline-ratio normalization, and zero assignment for missing benchmarks, Metriq offers a reproducible template for cross-platform composite benchmarking in quantum computing (Cosentino et al., 9 Mar 2026). At the same time, the authors emphasize that alternative weighting schemes can be substituted transparently, community recomputation is encouraged, and future suite versions may change the score as benchmarking practice evolves (Cosentino et al., 9 Mar 2026).