FIDDLE: Reinforcement Learning for Quantum Fidelity Enhancement

Abstract: Quantum computing has the potential to revolutionize fields like quantum optimization and quantum machine learning. However, current quantum devices are hindered by noise, reducing their reliability. A key challenge in gate-based quantum computing is improving the reliability of quantum circuits, measured by process fidelity, during the transpilation process, particularly in the routing stage. In this paper, we address the Fidelity Maximization in Routing Stage (FMRS) problem by introducing FIDDLE, a novel learning framework comprising two modules: a Gaussian Process-based surrogate model to estimate process fidelity with limited training samples and a reinforcement learning module to optimize routing. Our approach is the first to directly maximize process fidelity, outperforming traditional methods that rely on indirect metrics such as circuit depth or gate count. We rigorously evaluate FIDDLE by comparing it with state-of-the-art fidelity estimation techniques and routing optimization methods. The results demonstrate that our proposed surrogate model is able to provide a better estimation on the process fidelity compared to existing learning techniques, and our end-to-end framework significantly improves the process fidelity of quantum circuits across various noise models.

• The paper introduces the FIDDLE framework, which combines a Gaussian Process surrogate model with reinforcement learning to directly optimize quantum process fidelity during circuit routing.
• The paper utilizes a TGCN autoencoder and GP regression to achieve RMSE losses 1.5–2.5× lower than previous GNN-based methods under various noise models.
• The paper demonstrates fidelity improvements of 1.5–9% on average, with up to 12.8% gains over state-of-the-art methods in QAOA and QML circuits.

Reinforcement Learning for Quantum Fidelity Enhancement: The FIDDLE Framework

Introduction and Motivation

Quantum computing in the NISQ era is fundamentally constrained by noise and decoherence, which severely limit the reliability of quantum circuits. The process fidelity (PF) metric, which quantifies the overlap between the ideal and actual quantum processes, is the most comprehensive measure of circuit reliability. However, PF is QSZK-hard to compute, making direct optimization intractable for practical circuit sizes. Most existing transpilation and routing methods optimize indirect proxies such as circuit depth or gate count, which do not always correlate with fidelity, especially under realistic noise models.

The FIDDLE framework addresses the Fidelity Maximization in Routing Stage (FMRS) problem by directly optimizing PF during the routing stage of quantum circuit transpilation. The approach is characterized by two key innovations: (1) a Gaussian Process (GP)-based surrogate model for efficient PF estimation with limited data, and (2) a reinforcement learning (RL) module that leverages the surrogate model to optimize gate sequences for maximal fidelity.

Transpilation and Routing: Problem Formalization

The transpilation process for quantum circuits consists of three stages: layout, routing, and translation. The routing stage is responsible for ordering gates and inserting SWAPs to satisfy hardware connectivity and gate dependencies, directly impacting the circuit's noise sensitivity and, consequently, its fidelity.

Figure 1: The transpilation process: layout maps logical to physical qubits, routing inserts SWAPs to satisfy connectivity, and translation decomposes gates into native hardware operations.

The FMRS problem is formally defined as finding a valid gate sequence (including SWAPs) that, when mapped to a circuit under a given noise model, maximizes PF. The constraints include inclusion of all target gates, respect for dependency order, and satisfaction of hardware connectivity at each step.

Surrogate Modeling of Process Fidelity

Circuit Embedding via TGCN Autoencoder

To enable efficient learning, FIDDLE embeds discrete circuit representations into a continuous latent space using a Temporal Graph Convolutional Network (TGCN)-based autoencoder. This architecture captures both the temporal structure of gate execution and the spatial structure of hardware connectivity. The encoder maps the circuit table and hardware graph to a latent vector, while the decoder reconstructs fidelity-related traits (e.g., depth, gate counts).

Gaussian Process Regression for Fidelity Estimation

In the latent space, PF estimation is cast as a regression problem. GP regression is chosen for its ability to provide accurate predictions and uncertainty estimates with limited training data, a critical property given the high cost of generating PF labels via direct fidelity estimation.

Figure 2: Training flow for the surrogate model: circuits are embedded via TGCN, and a GP is trained on latent vectors and measured fidelities.

The GP surrogate is trained on a carefully selected subset of circuits, maximizing informativeness via submodular optimization. The kernel choice (e.g., Exponential, MLP) is empirically shown to be critical, with non-RBF kernels outperforming RBF in this context.

RL-Based Routing Optimization

MDP Formulation and Reward Structure

The FMRS problem is modeled as an MDP, where states encode the current partial gate sequence, dependency graph, and hardware graph. Actions correspond to selecting the next gate (target or SWAP) and its placement. The reward function penalizes unnecessary SWAPs, rewards inclusion of target gates, and, at episode termination, assigns a reward proportional to the surrogate-estimated PF if the solution is feasible.

Actor-Critic Architecture

Both actor and critic networks utilize TGCN and GCN modules to process physical (circuit, hardware) and logical (dependency graph, target gates) information. The actor outputs action probabilities, while the critic estimates state values. Training proceeds via standard actor-critic updates using experience replay.

Experimental Evaluation

Datasets and Noise Models

FIDDLE is evaluated on QAOA and QML circuits for 5- and 7-qubit systems, under five noise models: depolarizing, bit-flip, phase-flip, mixed, and real (Google Rainbow processor). The focus on QAOA and QML is motivated by their commutation properties and practical relevance.

Surrogate Model Performance

The latent space learned by the TGCN autoencoder exhibits strong correlation between Euclidean distance and PF gap, supporting the smoothness assumption required for effective GP regression. The GP surrogate achieves RMSE losses 1.5–2.5× lower than GNN-based estimators, with Exponential and MLP kernels performing best. Notably, the RBF kernel is suboptimal, especially under bit-flip noise.

RL Training and Inference

FIDDLE achieves high feasibility rates (>0.9) in generating valid gate sequences. The RL policy yields fidelity improvements of 1.5–9% over random routing, with the largest gains observed under real noise models. The improvement is less pronounced for bit-flip noise, attributed to the surrogate model's higher estimation error in this regime.

Comparison with State-of-the-Art

FIDDLE is compared against Qiskit (sabre, lookahead, stochastic), VIC (QAOA-specific), and GNN-RL (GNN-based fidelity estimator with RL). Across all datasets and noise models, FIDDLE consistently achieves higher PF, with improvements up to 12.8% over GNN-RL and 7.6% over Qiskit in the most challenging scenarios.

Figure 3: Fidelity comparison for QAOA_5 circuits across methods and noise models, demonstrating FIDDLE's consistent advantage.

Figure 4: Fidelity comparison for QAOA_7 circuits, highlighting FIDDLE's scalability and robustness to increased circuit complexity.

Figure 5: Fidelity comparison for QML_5 circuits, showing FIDDLE's effectiveness in circuits with complex dependency graphs.

Figure 6: Fidelity comparison for QML_7 circuits, with FIDDLE achieving the largest improvements in the most complex setting.

Implications and Future Directions

FIDDLE demonstrates that direct optimization of process fidelity during routing is feasible and yields substantial improvements over indirect metrics. The modular design—separating layout, routing, and translation—enables integration with existing transpilation pipelines. The GP-based surrogate model is particularly effective in low-data regimes, a critical consideration for scaling to larger circuits where PF estimation is prohibitive.

The results suggest several avenues for future research:

• Scalability: Extending the surrogate modeling and RL framework to circuits with more qubits, potentially via transfer learning or active learning to minimize PF labeling cost.
• Joint Optimization: Integrating layout and translation optimization with fidelity-aware routing for holistic transpilation.
• Hardware Adaptation: Incorporating hardware-specific noise profiles and calibration data to further tailor routing strategies.
• Quantum RL: Exploring quantum-native RL agents for routing, leveraging quantum policy representations.

Conclusion

FIDDLE provides a principled, learning-based approach to fidelity-aware quantum circuit routing, combining efficient surrogate modeling with RL-driven optimization. The framework achieves significant fidelity gains over state-of-the-art methods across diverse noise models and circuit types, establishing a new standard for reliability-driven transpilation in the NISQ era and laying the groundwork for scalable, noise-adaptive quantum compilation.