Graph Reinforcement Learning for Calibration-Aware Quantum Circuit Routing

Abstract: Quantum circuit routing is a key step in compiling programs for noisy intermediate-scale quantum processors. Routes that appear efficient by standard overhead metrics can still lose fidelity when they pass through poorly calibrated couplers. We study a calibration-aware graph reinforcement-learning router that uses same-day IBM Heron r2 calibration data to choose hardware-edge SWAPs. We train the policy with proximal policy optimization and evaluate it with exact simulated fidelity across nine Munich Quantum Toolkit (MQT) Bench circuits and three calibration snapshots. Across these evaluations, pooled mean exact fidelity is $0.727$, compared with $0.440$ for SABRE-best20 and $0.481$ for target-aware SABRE. Fidelity gains come with higher routed two-qubit counts and are concentrated in the 5q and 8q circuit families; under the fixed tree action graph, all 10q families favor SABRE-best20. Overall, our results show that calibration-aware learned routing can improve fidelity beyond gate-count-driven compilation.

• The paper introduces a calibration-aware graph reinforcement learning policy that optimizes quantum circuit routing by focusing on fidelity improvements rather than just minimizing gate counts.
• It employs a two-layer message-passing neural network with shaped intermediate rewards using PPO to integrate real-time hardware calibration data into routing decisions.
• Results demonstrate significantly higher simulated fidelities (mean 0.727) compared to SABRE baselines, particularly benefiting 5-qubit and 8-qubit circuits with flexible routing options.

Calibration-Aware Graph RL for Quantum Circuit Routing

Background and Motivation

Quantum circuit routing is a crucial compilation step for NISQ processors, where circuits must be mapped to hardware with sparse and error-prone connectivity. Traditional routing metrics (e.g., SWAP count, routed two-qubit count, or depth) provide hardware-agnostic overhead estimates, but these are often misaligned with actual circuit fidelity on real devices due to highly variable, time-dependent calibration properties of couplers and qubits. Most classical approaches, such as token swapping and heuristic search (e.g., SABRE), optimize for minimized gate overheads, disregarding backend noise heterogeneity. However, recent work demonstrates that calibration-aware routing can deliver substantial fidelity improvements by actively avoiding less reliable or recently degraded hardware paths. Incorporating such calibration adaptivity into the routing decision process is currently a challenging and largely unsolved problem for quantum compilers.

Methodology: Graph RL Policy with Calibration Context

The authors address this challenge by proposing a calibration-aware routing framework leveraging graph reinforcement learning (RL). Specifically, the routing problem is modeled as a finite-horizon sequential decision process, where at each step the agent chooses a legal SWAP on a calibrated hardware graph. Observations are graph-structured, encoding node-level and edge-level calibration attributes, remaining circuit structure, current logical-to-physical placement, and upcoming gate demands (Figure 1).

Figure 1: Routing-state graph construction encoding calibration data, circuit state, and placements for message-passing policy scoring of legal SWAP actions.

Node features include readout error, coherence, local error rates, hosted logical index, demand distances, and gate flags; edge features provide calibrated two-qubit error and legality masks. The policy utilizes a two-layer message-passing neural network (MPNN) to produce contextual node embeddings. Each physical SWAP action is scored via a multilayer perceptron over the concatenated embeddings and edge attributes, and a masked softmax yields a policy distribution over legal SWAPs.

Training is performed with PPO, using shaped intermediate rewards that reflect progress toward making front blocking gates executable, route completion, and a calibration-aligned proxy for fidelity (expected success probability product over routed gates). The final terminal reward incorporates exact density-matrix fidelity improvement over a strong SABRE baseline, penalizes large gate overheads, and discourages illegal or inefficient routing sequences.

Benchmark Protocol

For empirical benchmarking, the routing agent is trained and evaluated on nine circuits from the MQT Bench suite mapped to tree-structured 10-qubit subgraphs of the IBM Heron r2 hardware, using three distinct same-day calibration snapshots. All experiments fix the topology to isolate calibration effects. The evaluation uses exact density-matrix simulation parameterized with matched calibration data and considers gate overheads and fidelity. Each condition is evaluated over matched seed-specific randomized calibration jitters, controlling for variability and ensuring equitable baseline comparisons.

Two SABRE-family baselines are used for reference: SABRE-best20, which minimizes two-qubit count (plus tie-breaking on depth), and target-aware SABRE, which leverages calibration-weighted Qiskit Target routing and selects by proxy fidelity.

Results

The graph RL router produces markedly higher simulated fidelities compared to both SABRE-best20 and target-aware SABRE. Pooled across circuits and calibration snapshots, the learned policy achieves mean fidelity 0.727, compared to 0.440 (SABRE-best20) and 0.481 (target-aware), a strong numerical improvement across all snapshot and condition means.

Detailed analysis reveals that these fidelity gains are most pronounced in 5-qubit (5q) and 8-qubit (8q) circuit families, where the RL agent expends additional two-qubit gates (on average +8.63) and depth (+4.61) to avoid unreliable couplers, thus mitigating the influence of hardware noise. The effective error per two-qubit gate, computed as $$\bar{\epsilon}_{\mathrm{eff}}=-\ln(\bar F)/\bar g_{2Q}$$, is reduced for the learned agent (0.011) compared to the baselines (0.041/0.036), reinforcing the value of calibration-aware routing.

Conversely, in 10-qubit circuits, the fixed action graph topology provides insufficient alternative routing paths so that added gate count introduces overhead without commensurate fidelity gain. Therefore, in these cases, SABRE-best20 routing remains preferable.

Figure 2: Comparison of exact fidelity and routed 2Q count across benchmarks; fidelity gains for RL coincide with higher gate overhead where calibration-aware routing is effective.

Wilcoxon signed-rank tests confirm the statistical significance (p ≪ 0.05) of the fidelity differences between the learned policy and each baseline.

Implications and Future Directions

This work clearly demonstrates that calibration-aware RL-driven routing can outperform traditional gate-count-driven compilation in terms of output state fidelity, especially under hardware calibration heterogeneity when the routing action space is rich enough to supply useful alternatives. However, it also highlights the structural limitation: in topologies with constrained options (e.g., tree subgraphs with few alternative paths), calibration awareness cannot compensate for restricted routing flexibility, and excess gates become detrimental.

From a practical perspective, these results suggest quantum compilers should integrate real-time calibration context directly into their mapping engines, and that RL-based policies offer a tractable and automatable route for doing so. Theoretically, the work strengthens the argument that fidelity metrics—not just gate or depth counts—should be standard for evaluation, especially as devices scale and calibration errors become a dominant performance bottleneck. This approach may generalize to more sophisticated topologies, variable device calibrations, or even real hardware runs.

Future developments could extend the agent’s action space to richer (cyclic) subgraphs, apply the framework to held-out circuits and live device calibration data, and implement more fine-grained hardware-aware objectives (e.g., minimizing correlated error accumulation or decoherence-sensitive routing). Large-scale benchmarking and real-system validation are necessary next steps to fully realize the benefit of RL-based calibration-aware routers in contemporary and next-generation quantum computing architectures.

Conclusion

The authors present a graph RL quantum circuit router that dynamically incorporates same-day device calibration during compilation, achieving significantly higher exact fidelities than standard routing baselines across diverse quantum circuits where the action graph permits alternate paths. Calibration-awareness enables the policy to trade off additional gate overhead for greater noise resilience, but only in topologies with sufficient flexibility. These findings support a future where RL-driven, calibration-adaptive policies become integral to quantum program compilation, provided the underlying hardware landscape supports such adaptivity.