Hierarchical Qubit-Merging Transformer (HQMT)

• The paper introduces an HQMT decoder that integrates local syndrome patches using a hierarchical transformer to achieve significantly lower logical error rates.
• It employs qubit fusion techniques to transform paired qubits into qudit representations, facilitating non-Clifford gate synthesis and enhanced fault tolerance.
• The design optimizes resource allocation via sensitivity-based mapping and maintains constant latency, making it ideal for real-time quantum error correction.

A Hierarchical Qubit-Merging Transformer (HQMT) is a quantum error correction (QEC) decoding framework that utilizes a multi-scale transformer architecture to exploit the hierarchical structure of stabilizer codes, notably surface codes, by merging local syndrome information into qubit-centric, globally processed representations. HQMT incorporates advances in deep learning fused with quantum code theory, qubit fusion protocols, and hierarchical resource allocation strategies, providing reliable, scalable decoding with significantly lower logical error rates compared to previous methods (Park et al., 13 Oct 2025).

1. Architectural Principles and Workflow

The HQMT decoder operates in two key stages aligned with the physical structure of the underlying stabilizer code. The first stage processes syndrome data locally: for each physical qubit $q(i)$, two sets of syndrome patches are constructed—$p_Z(i)$ from adjacent $Z$-type stabilizers and $p_X(i)$ from $X$-type stabilizers:

• For $Z$ patches: $v_{Z,j}(i) = 1-2s_{Z,j}$ if $s_{Z,j}$ neighbors $q(i)$, zero otherwise.
• Analogously for $X$ patches.

These $m$-dimensional patches are projected into a $d_{\text{model}}$-dimensional token space through fully connected layers, forming $2n$ initial tokens (for $n$ qubits), yielding $X_1 \in \mathbb{R}^{2n \times d_{\text{model}}}$. $N$ transformer blocks compute local self-attention, using projection matrices $W^\text{Q}, W^\text{K}, W^\text{V}$ and

$$\text{Attention}(Q, K, V) = \text{softmax}\left(\frac{Q K^\top}{\sqrt{d_h}}\right) V$$

A dedicated qubit-merging layer concatenates the $Z$-type and $X$-type tokens for each qubit, projects them into unified $d_{\text{model}}$-dimensional tokens—integrating fine-grained local information into holistic error representations.

Stage two processes the merged sequence $X_2 \in \mathbb{R}^{n \times d_{\text{model}}}$ through further transformer blocks to learn global correlations—culminating in mean-pooling and classification into one of four logical error classes: $\{\bar{I}, \bar{X}, \bar{Y}, \bar{Z}\}$.

2. Hierarchical Qubit Fusion and Transformer Integration

HQMT leverages the group-theoretic framework of qubit fusion (Moussa, 2015), where pairs of qubits are merged to form four-dimensional qudits. The fusion operation: $$F : |x\rangle \otimes |y\rangle \rightarrow |2y + x\rangle \ (\text{mod}\ 4)$$ and its inverse $F^\dagger$ enable embedding of lower-level qubit operators into the higher dimensional Clifford hierarchy ($C_1^2 \subset C_2^4 \subset C_3^2$). In practice, this permits efficient synthesis of non-Clifford gates and complex logical operations by “lifting” qubit operators to qudit space, processing them, and “projecting” them back, which is highly suitable for transformer architectures that manipulate information across multiple scales.

The fusion and fission gates are realized using stabilizer circuits, consuming distilled resource states $$|F\rangle = \frac{1}{\sqrt{2}} (|0\rangle + |1\rangle)$$, with error detection and suppression protocols analogous to magic state distillation. This ensures robust, fault-tolerant merging and splitting of qubits that can be operationalized hierarchically within HQMT layers.

3. Hierarchical Qubit Maps and Error Correction Optimization

HQMT design incorporates hierarchical qubit maps and Hierarchical Quantum Error Correction (HI-QEC) (Klco et al., 2021), wherein physical-to-logical qubit allocation is differentially optimized based on the sensitivity $\gamma_q$ of observables to specific logical qubits.

For quantum simulations mapping physical degrees of freedom onto quantum registers, hierarchical encoding organizes qubits by effective energy scales (infrared/ultraviolet bands). Error sensitivities (e.g., for $\langle\phi^2\rangle$) inform code distance allocation via: $$\gamma_q P_L^{(q)} \leq \frac{\varepsilon}{n N_\text{cycles}} \quad \Rightarrow \quad d_q \gtrsim 2\left\lceil \frac{\log(\varepsilon/(n \bar{c}_0 \gamma_q))}{\log(p/p_\text{th})} \right\rceil - 1$$ allowing for up to $\sim60\%$ reduction in qubit resources in early error-corrected simulations.

HQMT’s merging operations and attention blocks can be made “fidelity-aware,” assigning more error protection to IR (low-frequency) qubits crucial to observable accuracy, and fewer resources to UV (high-frequency, less critical) qubits, optimizing both resource usage and precision.

4. Logical Error Rate Performance and Scalability

The HQMT framework delivers substantially lower logical error rates (LER) compared to both neural and classical decoders (Park et al., 13 Oct 2025):

• Outperforms Feedforward and Convolutional Neural Network decoders at code distances $d=3,5,7,9$; LER advantage increases with larger codes.
• Surpasses BP+OSD (belief propagation with ordered statistics decoding), maintaining lower LERs across the entire range of physical error rates for $d=5,7,9,11$.
• Higher pseudothresholds in HQMT than comparators for $d=5$ and $d=7$ (Table I), supporting scalability and robustness.

The model’s architecture maintains constant output dimension regardless of code size (classification into four logical error classes), yielding predictable, fixed decoding latency—a critical requirement for real-time quantum error correction.

5. Fault-Tolerant Implementation and Resource State Synthesis

HQMT integrates fault-tolerant merging by relying on stabilizer circuits for both fusion and fission operations. Error-detection circuits based on stabilizer measurements, e.g.,

$$W_X(\rho) = \frac{1}{2}(\rho + X H^2 \rho H^2 X^\dagger)$$

and

$$W_Z(\rho) = \frac{1}{2}(\rho + Z^\dagger S^2 \rho S^2 Z)$$

effectively suppress errors in the resource states and ensure distillation of high-fidelity $|F\rangle$ states. These protocols, while resource-intensive, provide error floor reductions competitive with magic state distillation schemes, and can be tuned within HQMT to balance error suppression with circuit cost, especially when the number of required non-Clifford operations is reduced by recasting them as Clifford operations on fused qudits.

6. Future Developments and Extensions

HQMT’s modular, hierarchical transformer architecture admits further optimizations:

• Increasing transformer block depth per stage (notably $N=3$, as observed in ablation studies) improves decoding performance up to saturation.
• Advanced merging algorithms might further exploit inter-stabilizer correlations, via multi-head cross-attention or more sophisticated fusion protocols.
• Extensions beyond surface codes to other stabilizer code classes are suggested by the generality of HQMT’s hierarchical principles and architecture.
• Integration with adaptive resource allocation schemes, as discussed in hierarchical QEC co-design, may yield dynamic HQMT implementations, concentrating error correction on critical regions in real-time computations.
• Real-time integration into quantum devices remains an open direction, enabled by HQMT’s constant latency and scalability.

7. Context and Significance in Quantum Error Correction

HQMT embodies the convergence of quantum code theory, fusion-based gate synthesis, and hierarchical deep learning architectures. Its explicit leveraging of code structure and multi-scale modeling enables more effective decoding in large-scale quantum computation, supporting lower error rates, resource efficiencies, and robust, fault-tolerant extraction of logical information.

HQMT’s hierarchical design parallels physical and computational hierarchies in quantum simulations, aligning quantum hardware resource allocation, error sensitivity, and data representation. This synergy positions HQMT as a scalable, high-fidelity decoder framework with potential impact across quantum computing and quantum simulation domains (Moussa, 2015, Klco et al., 2021, Park et al., 13 Oct 2025).