Vectorized Quantum Transformer (VQT)

• Vectorized Quantum Transformer (VQT) is a hybrid architecture that integrates quantum circuit simulation with transformer models using vectorized encoding and quantum-inspired attention.
• It leverages novel quantum dot-product circuits and nonlinear quantum encoders to process both quantum and classical data with improved efficiency on NISQ hardware.
• VQT demonstrates near-perfect quantum state classification and scalable performance with low-depth circuits, paving the way for advanced quantum machine learning applications.

The Vectorized Quantum Transformer (VQT) is a machine learning architecture that integrates quantum circuit simulation with transformer-based models through vectorized block encoding and quantum-inspired attention mechanisms. VQT provides efficient structure-aware processing for both quantum and classical data, enabling end-to-end learning on noise intermediate-scale quantum (NISQ) devices, and offers empirical and algorithmic advances in both quantum state analysis and classical machine learning tasks (Sekuła et al., 28 Feb 2025, Guo et al., 25 Aug 2025).

1. Vectorized Input Encoding and Embedding

VQT employs two broad mechanisms for tokenization and encoding, tailored to the data domain:

• Quantum State Vectorization: For quantum state analysis, a density matrix $\rho \in \mathbb{C}^{n \times n}$ is flattened into a real vector of length $2n^2$ using the column-stacking operator:

$$\mathrm{vec}(\rho) = [\,\mathrm{Re}\,\rho_{11},\,\mathrm{Im}\,\rho_{11},\, \mathrm{Re}\,\rho_{21},\, \mathrm{Im}\,\rho_{21},\,\dots,\, \mathrm{Re}\,\rho_{nn},\, \mathrm{Im}\,\rho_{nn}\,]^T.$$

All matrix entries, including Hermitian-redundant ones, are mapped into tokens. Each token $x_i$ is embedded into $\mathbb{R}^d$ via a learned linear projection, optionally stacking real-imaginary pairs.

• Classical Data Embedding: For natural language processing or classical tasks, each token embedding $t_i \in \mathbb{R}^d$ is combined with a positional encoding $p_i$, followed by a Tanh projection to limit the range to $x_i = \tanh(z_i) \in [-1,1]^d$.

Learned positional encodings are critical, providing the model with explicit matrix structural information or token order (Sekuła et al., 28 Feb 2025, Guo et al., 25 Aug 2025).

2. Quantum-Inspired and Quantum-Native Attention Mechanisms

VQT introduces a novel quantum multi-head attention module:

• Vectorized Quantum Dot-Product (VQDP) Circuit: The standard attention operation—computing $QK^T$—is replaced by a VQDP quantum circuit. Batched address states are encoded in superposition, with "data" qubits loaded with real pairs $(Q_{b,i,k}, K_{b,j,k})$. A shallow entangling layer with $R_y$ and CNOT gates produces a quantum state $|\psi_{\Pi}\rangle$ where the expectation value of an observable $O_z$ yields estimates of $x_\ell y_\ell$. This mechanism supports masked attention by quantum approximation (Guo et al., 25 Aug 2025).
• Vectorized Nonlinear Quantum Encoder: Classical input vectors $x = (x_0, \dots, x_{d-1}) \in [-1,1]^d$ are mapped via rotation-angle encoding:

$$|\psi(x)\rangle = \bigotimes_{i=0}^{d-1} R_y(\arccos(x_i)) |0^d\rangle,$$

optionally permuted and compressed for single-shot loading. This supports quantum processing of high-dimensional data in logarithmic depth.

No trainable quantum gates are employed; all trainable weights are in a classical AngleMLP ("expressive quantum head") generating rotation angles for the quantum encoder. This design eliminates the need for parameter-shift rules in quantum gradient computation and circumvents barren plateau issues typical of deep parameterized quantum circuits (Guo et al., 25 Aug 2025).

3. Training Strategies and Objectives

VQT supports both unsupervised and supervised training protocols:

• Masked Autoencoding for Quantum States: A fraction (e.g., 15%) of matrix tokens is masked and replaced with a learned embedding. The model is trained to reconstruct masked entries via a linear decoder, optimizing mean squared error (MSE) loss:

$$\mathcal{L}_{\mathrm{pre}} = \mathbb{E}_{\rho,\,M}\left[\, \| \hat{v}_M - v_M \|_2^2 \,\right],$$

where $M$ indexes masked positions. The "Hermitian distance" metric ($h = \frac{1}{B}\sum_k \|A_k - A_k^\dagger\|_F$) is monitored to ensure automatic restoration of Hermiticity (Sekuła et al., 28 Feb 2025).

• Quantum Attention Simulation: The attention matrix is computed via Monte Carlo sampling on the VQDP circuit. The per-feature quantum inner product is estimated by repeated measurement, with error bounds scaling as $1/\sqrt{M}$, where $M$ is the number of shots. Stochastic regularization introduced by quantum shot noise serves to regularize training and helps prevent overfitting (Guo et al., 25 Aug 2025).
• Supervised Fine-tuning for Classification: For entanglement or token classification, a prepended [CLS] token representation feeds into a linear + softmax classifier, with standard cross-entropy loss.

In all VQT variants, only a small classical neural network head is trained; quantum circuits remain fixed, and angle loading allows conventional backpropagation (Guo et al., 25 Aug 2025).

4. Computational Cost and Experimental Benchmarks

VQT enables shot-efficient and gradient-free quantum circuit simulation, making it compatible with NISQ-era hardware. Quantum circuit depth scales as $O(\log N)$ (where $N$ is the total number of query-key dot-products per batch), allowing low-depth execution and improved robustness to hardware noise.

A comparative summary of circuit resource costs and root mean squared error (RMSE) for quantum dot-product estimation is provided below ((Guo et al., 25 Aug 2025), Table 2):

Batch Size (B)	Qubits ($n_q$)	Shots (M)	CX Gates (IBM)	CX Depth	RMSE (Ideal)	RMSE (IBM)	RMSE (IonQ)
4	4	10,000	18	14	0.017	0.022	0.057
8	5	20,000	29	25	0.016	0.023	0.155
16	6	40,000	78	69	0.017	0.037	0.215
32	7	80,000	164	121	0.017	0.059	--

The VQT achieves maximum attention matrix deviation below 1.2% on batches with $B = T = d = 10$ and $3 \times 10^6$ shots (Guo et al., 25 Aug 2025). For natural language tasks on the Brown Corpus, VQT (default and large encoder) achieves perplexity (QPL) values of 105.4 and 108.2, respectively, closely approaching the 92.5 of NanoGPT, and outperforming previous QT models at the same qubit count.

5. Performance in Quantum State Classification

The VQT framework for quantum state analysis achieves near-perfect entanglement detection. For bipartite systems:

• $C^2 \otimes C^2$ (two qubits): 99.995%
• $C^2 \otimes C^3$ (qubit-qutrit): 99.998%
• $C^3 \otimes C^3$ (two qutrits): 100%

Ablation studies show that for $3 \times 3$ systems, end-to-end fine-tuning of the full transformer backbone is necessary to maintain optimal accuracy; otherwise, performance drops to approximately 85% when only the classification head is trained (Sekuła et al., 28 Feb 2025). A single set of hyperparameters and architecture handles various bipartite dimensions and state classes, including separable, bound entangled, Werner, and maximally entangled states, substantiating the architecture's generalization.

6. Scalability, Hardware Compatibility, and NISQ Friendliness

VQT circumvents the limitations of deep parameterized quantum circuits by employing shallow, address-based superposition; all operations on the QPU are fixed and resource-efficient. All-to-all classical-to-quantum translation for the attention matrix is achieved in $O(\log N)$ circuit depth, facilitating execution on current superconducting (IBM Kingston) and trapped-ion (IonQ Aria-1) QPUs. The architecture is NISQ-friendly: no trainable quantum gates ensure absence of vanishing gradients or barren plateaus, and stochastic quantum approximation injects beneficial regularization (Guo et al., 25 Aug 2025).

7. Context and Implications for Quantum Machine Learning

The VQT establishes a practical paradigm for quantum-classical hybrid transformers that can process vectorized quantum states, classical sequence data, and potentially unified data modalities. Vectorized attention with learnable encoding supports efficient bridging between classical and quantum computation in machine learning workflows. Empirical evidence demonstrates the feasibility of end-to-end training on near-term quantum devices, setting the stage for more advanced fault-tolerant models and quantum-native tokenization strategies in the future (Guo et al., 25 Aug 2025). A plausible implication is that VQT architectures may accelerate the development of scalable quantum NLP and state characterization pipelines on available quantum hardware.

---

References:

(Sekuła et al., 28 Feb 2025) Quantum-aware Transformer model for state classification (Guo et al., 25 Aug 2025) Vectorized Attention with Learnable Encoding for Quantum Transformer