---
title: 'Metriq Score: Composite Quantum Benchmark Index'
url: https://www.emergentmind.com/topics/metriq-score
type: topic
---

# Metriq Score: Composite Quantum Benchmark Index

Searching arXiv for recent papers on Metriq Score and closely related benchmarking context.
The **Metriq Score** is a composite index for summarizing the performance of a quantum device across the benchmark suite of the Metriq platform, an open-source collaborative system for reproducible cross-platform quantum benchmarking [2603.08680]. In the formulation presented by Metriq, the score assigns a single scalar value to each device by aggregating normalized outcomes from multiple heterogeneous quantum benchmarks under an explicit weighting scheme. It is designed as a practical summary for leaderboard-style comparison across vendors and architectures, while preserving the underlying benchmark-level results as the primary evidence. The authors state explicitly that it is **not** a physical fidelity and **not** an estimate of one latent hardware property; rather, it is a transparent, baseline-normalized composite over diverse benchmark outcomes [2603.08680].

## 1. Origin within the Metriq platform

Metriq was introduced as a response to a fragmented benchmarking landscape in quantum computing, characterized by system-specific tools and inconsistent evaluation methodologies that impede reliable cross-platform assessment [2603.08680]. The platform integrates benchmark definition and execution, data collection, and public presentation into a unified workflow. Its benchmark suite spans both system-level metrics, such as entanglement quality, gate performance, and circuit speed, and application-inspired protocols drawn from quantum machine learning, optimization, and quantum simulation [2603.08680].

Within that framework, the Metriq Score appears as the platform’s aggregate summary statistic. The paper situates it as a pragmatic device for synthesizing a device’s behavior across a heterogeneous suite rather than as a replacement for component benchmarks. This distinction is central: the score exists because isolated benchmark results are difficult to compare across platforms, yet no single benchmark captures “quality, speed, and scale” together [2603.08680]. A plausible implication is that the score is intended to facilitate operational comparison while leaving methodological pluralism intact.

The score is defined per device and per suite series, written as $\mathrm{MS}(d,s)$, where $d$ denotes a device and $s$ the benchmark-suite series or version [2603.08680]. This versioning matters because the suite can evolve, benchmarks can be added or removed, and weights can be changed. The paper therefore treats the score as a versioned index rather than a timeless scalar.

## 2. Benchmark suite included in the score

For the suite used in the paper’s main results, the Metriq Score aggregates eight benchmarks [2603.08680]. These include four system-level benchmarks and four application-inspired benchmarks.

| Category | Benchmark | Role in suite |
|---|---|---|
| System-level | BSEQ | Bell State Effective Qubits |
| System-level | EPLG | Error Per Layered Gate |
| System-level | Mirror Circuits | Multi-width circuit benchmark |
| System-level | CLOPS | Circuit Layer Operations Per Second |
| Application-inspired | QML Kernel | Quantum machine learning kernel |
| Application-inspired | LR-QAOA | Linear-ramp QAOA |
| Application-inspired | WIT | Wormhole-Inspired Teleportation |
| Application-inspired | QFT | Quantum Fourier Transform |

The score does **not** include provider pricing or cost estimates, runtime cost accounting from Section 6, uncertainty bars, calibration metadata directly, or all possible future Metriq-supported benchmarks beyond the chosen suite version [2603.08680]. Runtime enters only indirectly through CLOPS, which is itself a throughput benchmark based on reported execution time. Processor size does not appear as an explicit multiplier either, but scale enters structurally because benchmarks are run at different widths and larger widths receive more weight [2603.08680].

This benchmark selection makes the score intentionally heterogeneous. BSEQ emphasizes connectivity and effective entangling scale, EPLG measures layered-gate error behavior, Mirror Circuits capture compiled circuit fidelity via polarization, CLOPS measures throughput, and the application-inspired protocols probe performance on structured workloads. This suggests that the score is meant to summarize a device’s practical performance envelope rather than any single fault model.

## 3. Formal definition and aggregation procedure

The Metriq Score is defined through a three-stage aggregation process: within-benchmark aggregation across widths, baseline normalization of benchmark-level results, and weighted averaging across benchmarks [2603.08680].

The paper introduces a component index set $\mathcal{C}=\{1,\dots,K\}$ for all recorded benchmark components in a suite series and defines the set of distinct benchmarks as
$$
\mathcal{B} := \{\, b_i : i\in\mathcal{C} \,\}, \qquad \mathcal{C}_b := \{\, i\in\mathcal{C} : b_i=b \,\} \ \ (b\in\mathcal{B}).
$$
Here, component $i$ corresponds to a particular benchmark, metric, and run configuration; $n_i$ denotes circuit width, $v_i(d,s)$ the measured value on device $d$ in series $s$, and $d^{\mathrm{base},s}$ the designated baseline device for that series [2603.08680].

### Within-benchmark aggregation across widths

Many benchmarks are evaluated at multiple widths. The within-benchmark weight for component $i$ is
$$
\alpha_i := \frac{n_i}{\sum_{j\in\mathcal{C}_{b_i}} n_j}, \qquad i\in\mathcal{C},
$$
so larger-width instances contribute more [2603.08680]. The width-aggregated raw value for benchmark $b$ is then
$$
\bar{v}_b(d,s) := \sum_{i\in\mathcal{C}_b} \alpha_i\, v_i(d,s), \qquad b\in\mathcal{B}.
$$

This is a linear weighted average over raw values. The paper explicitly does **not** define the Metriq Score via geometric means, percentile scoring, z-scores, PCA, learned weights, or nonlinear aggregation [2603.08680].

### Baseline normalization

After obtaining one raw benchmark value per device and benchmark, the paper normalizes against a designated baseline device. If higher is better for benchmark $b$,
$$
\mathrm{BS}_b(d,s) := 100 \cdot \dfrac{\bar{v}_b(d,s)}{\bar{v}_b(d^{\mathrm{base},s})}.
$$
If lower is better,
$$
\mathrm{BS}_b(d,s) := 100 \cdot \dfrac{\bar{v}_b(d^{\mathrm{base},s})}{\bar{v}_b(d,s)}.
$$
Thus, a benchmark subscore $\mathrm{BS}_b(d,s)=100$ means parity with the baseline, values above 100 indicate better-than-baseline performance, and values below 100 indicate worse-than-baseline performance [2603.08680].

### Across-benchmark weighting and final score

To combine benchmark subscores, the paper defines an “effective width”
$$
\mu_b := \sum_{i\in\mathcal{C}_b} \alpha_i\, n_i = \dfrac{\sum_{i\in\mathcal{C}_b} n_i^2}{\sum_{i\in\mathcal{C}_b} n_i}, \qquad b\in\mathcal{B}.
$$
For BSEQ, which does not have a natural width sweep, the suite assigns a reference width; in the present series, $n_{\mathrm{BSEQ}}^{\mathrm{ref}}=100$ [2603.08680]. Benchmark weights are then
$$
w_b := \dfrac{\mu_b}{\sum_{c\in\mathcal{B}} \mu_c}, \qquad b\in\mathcal{B}.
$$
The final Metriq Score is
$$
\mathrm{MS}(d,s) := \sum_{b\in\mathcal{B}} w_b \, \mathrm{BS}_b(d,s).
$$

The final score is therefore a weighted arithmetic mean of dimensionless benchmark ratios. This is the paper’s central definition of the Metriq Score [2603.08680].

## 4. Benchmark-specific raw scores entering the composite

Although the composite formula is uniform, the raw benchmark values entering it are benchmark-specific. The score therefore aggregates heterogeneous submetrics rather than repeated instances of a common observable [2603.08680].

For **BSEQ**, the platform uses the largest connected component size and connection fraction. The paper defines
$$
S_{\text{LCCS}}(d) = 100 \frac{\text{LCCS}(d)}{\text{LCCS}(d^{\mathrm{base}})} \quad \text{and} \quad S_{f_{\text{conn}}}(d) = 100 \frac{f_{\text{conn}}(d)}{f_{\text{conn}}(d^{\mathrm{base}})},
$$
then combines them as
$$
\mathrm{BSEQ~Score}(d) = \frac{7}{8} S_{\text{LCCS}}(d) + \frac{1}{8} S_{f_{\text{conn}}}(d).
$$
This already embeds a benchmark-internal weighted average before entry into the global score [2603.08680].

For **EPLG**, lower values are better, so the benchmark is normalized inversely against the baseline. For lengths $\ell\in\{10,20,50,100\}$,
$$
S_{\mathrm{EPLG}(\ell)}(d) = 100 \frac{\mathrm{EPLG}(\ell, d^{\mathrm{base}})}{\mathrm{EPLG}(\ell, d)}.
$$
The benchmark-level score is then described as a weighted harmonic mean with a coverage penalty, intended to penalize missing larger chain lengths [2603.08680].

For **Mirror Circuits**, polarization at panel point $i$ is
$$
\pi_i = \max\left\{0, \frac{\hat{p}_i - 2^{-w_i}}{1 - 2^{-w_i}}\right\}
= \max\left\{0,\frac{2^{w_i} \hat{p}_i - 1}{2^{w_i} - 1}\right\}.
$$
The raw benchmark value is then aggregated as
$$
MC~Score := \sum_{i=1}^{M} \alpha_i\, \pi_i, \qquad \alpha_i := \frac{n_i}{\sum_{j=1}^{M} n_j}.
$$
That raw value is normalized against the baseline at the next stage [2603.08680].

For **CLOPS**, the raw benchmark is
$$
\mathrm{CLOPS} = \frac{L M S}{T_{tot}}.
$$
Because CLOPS is evaluated at a single width in the paper’s series, width aggregation is trivial [2603.08680].

For **QML Kernel**, the raw aggregate is
$$
QMLK~Score := \sum_{n\in\{10,20,30,50\}} \frac{n}{110}\, \text{QMLK-n},
$$
with weights proportional to width [2603.08680].

For **LR-QAOA**, the effective approximation ratio is
$$
r_{\mathrm{eff}} = \frac{r - r_{\mathrm{random}}}{1 - r_{\mathrm{random}}},
$$
and the benchmark-level aggregate over $N\in\{10,20,50,100\}$ is
$$
LR\text{-}QAOA~Score = \frac{ 10\,\text{LR-QAOA-10} + 20\,\text{LR-QAOA-20} + 50\,\text{LR-QAOA-50} + 100\,\text{LR-QAOA-100} }{180}.
$$

For **QFT**, the raw score is
$$
QFT~Score := \sum_{n\in\{4,8,12,20\}} \frac{n}{44}\, \text{QFT-N}.
$$

For **WIT**, the benchmark is evaluated at a single width $n=7$, so width aggregation is again trivial [2603.08680].

A key implication is that the final Metriq Score combines observables with distinct semantics—polarization, approximation ratio, throughput, connectivity, and others—only after each has been normalized against a common baseline.

## 5. Weighting scheme, missing-data policy, and interpretation

The benchmark weights used in the paper are derived from effective widths $\mu_b$ reported in the appendix [2603.08680]. The values given are:

| Benchmark | Effective width $\mu_b$ | Relative role |
|---|---:|---|
| BSEQ | 100 | Largest default weight |
| EPLG | $\approx 72.2$ | Large |
| Mirror Circuits | $\approx 82.4$ | Large |
| CLOPS | 100 | Largest default weight |
| QML Kernel | $\approx 35.5$ | Moderate |
| WIT | 7 | Small |
| LR-QAOA | $\approx 72.2$ | Large |
| QFT | $\approx 14.2$ | Small |

The appendix gives $\sum_{b\in B}\mu_b \approx 483.5$, so $w_b=\mu_b/483.5$ [2603.08680]. The largest default weights therefore go to BSEQ and CLOPS, followed by Mirror Circuits, then EPLG and LR-QAOA. This means the score is intentionally scale-aware: larger and broader-width benchmarks contribute more.

The paper also adopts a strong missing-data policy. If benchmark $b$ is missing for device $d$ in series $s$, it sets $\mathrm{BS}_b(d,s):=0$ [2603.08680]. Missing widths within benchmark-specific aggregates are also set to zero for several benchmarks, including QML Kernel, LR-QAOA, and Mirror Circuits. The table caption further states that the CLOPS score contributes a zero value to the composite score for devices that do not provide the needed timing information [2603.08680].

This has an important interpretive consequence. The score rewards breadth of demonstrated capability and penalizes incomplete observability. The authors present this as a deliberate design choice. A plausible implication is that the score captures not only measured performance but also benchmark coverage and metadata availability. The paper treats this as a feature, though it also identifies it as a limitation.

In practical interpretation, a score of 100 corresponds to equality with the baseline device on the weighted composite. A score above 100 means better-than-baseline aggregate performance under the chosen suite and weights, while a score below 100 indicates worse-than-baseline performance [2603.08680]. The authors caution that this does **not** mean “twice as good” in any physical sense when the score is 200; it is only a relative composite index over normalized benchmark subscores.

## 6. Reported results, correlations, and caveats

The baseline device in the paper’s main table is **ibm_torino**, set to **100.00** for every normalized component and therefore for the Metriq Score as well [2603.08680]. The reported aggregate values are:

| Device | Metriq Score |
|---|---:|
| ibm_boston | 252.61 |
| quantinuum_h2_2 | 188.05 |
| ibm_pittsburgh | 174.51 |
| ibm_kingston | 174.23 |
| ibm_marrakesh | 156.82 |
| ibm_fez | 116.77 |
| ibm_torino | 100.00 |
| iqm_emerald | 23.76 |
| iqm_garnet | 14.34 |
| rigetti_ankaa_3 | 4.54 |
| wukong_72 | 3.38 |

The paper notes that **ibm_boston** leads the table by a substantial margin, with strong benchmark values in BSEQ, EPLG, Mirror Circuits, and QML, plus CLOPS support [2603.08680]. **quantinuum_h2_2** performs strongly on QML, WIT, QFT, and Mirror Circuits, but is penalized by missing CLOPS timing support and smaller-scale availability in some benchmarks. Devices lacking timing data lose CLOPS contribution entirely, while smaller devices or those unable to execute large-width instances are penalized through zero-valued widths or missing benchmark subscores [2603.08680].

The paper explicitly warns that these values were collected at different times between March 2025 and March 2026 and should not be interpreted as a definitive or current ranking [2603.08680]. That caution is consistent with the platform’s longitudinal design: scores are snapshots computed from available results at a particular time.

Metriq also reports cross-benchmark analyses enabled by the shared dataset. The Metriq Score is reported to correlate most tightly with **Mirror Circuits**, with Spearman $\rho = 0.991$ [2603.08680]. Across devices with available public calibration data, the score correlates with two-qubit gate fidelity at Spearman $\rho = 0.982$ [2603.08680]. Additional correlations include Mirror Circuits with QML Kernel at $\rho = 0.991$, QML Kernel with QFT at $\rho = 0.918$, and BSEQ with LR-QAOA at $\rho = 0.936$ [2603.08680]. The paper also reports that the first principal component of z-scored log-scores on complete-data devices explains **88% of the variance**, suggesting a strong common performance axis across benchmarks [2603.08680].

The authors nonetheless caution against overinterpretation. The score is not a physical quantity, weighting choices are normative, baseline choice can affect ranking, missing-data penalties may confound performance with observability, compilation differences can materially affect outcomes, and uncertainty is not propagated through the composite [2603.08680]. Benchmarks are heterogeneous and architecture-sensitive, and timing support is not uniformly available across providers. The score is therefore best understood as a transparent, reproducible composite index under explicit methodological choices rather than as a universal ranking of quantum computer quality.

The paper’s broader significance lies in making those methodological choices explicit. By defining the Metriq Score as
$$
\mathrm{MS}(d,s) := \sum_{b\in\mathcal{B}} w_b \, \mathrm{BS}_b(d,s),
$$
with width-weighted aggregation, baseline-ratio normalization, and zero assignment for missing benchmarks, Metriq offers a reproducible template for cross-platform composite benchmarking in quantum computing [2603.08680]. At the same time, the authors emphasize that alternative weighting schemes can be substituted transparently, community recomputation is encouraged, and future suite versions may change the score as benchmarking practice evolves [2603.08680].

Source: https://www.emergentmind.com/topics/metriq-score