Quantum Vision Transformers (QViT) Overview

• Quantum Vision Transformers (QViTs) are hybrid models that fuse classical ViT structures with quantum circuits to enhance attention mechanisms and reduce computational complexity.
• They vary from replacing self-attention components with quantum modules to using transformer outputs as variational quantum states in many-body physics.
• Empirical results indicate that QViTs achieve competitive performance in image classification and quantum state approximations despite current hardware and simulation challenges.

Quantum Vision Transformers (QViTs) are a family of architectures that combine Vision Transformer (ViT) structure with quantum computation, but the term is used in two distinct ways in the recent literature. In hybrid image models, QViT usually denotes a ViT in which self-attention projections, attention-score computation, or related blocks are replaced by parameterized quantum circuits, quantum orthogonal layers, or broader quantum linear-algebra subroutines (Cherrat et al., 2022). In quantum many-body physics, the same term is also used for ViT-based variational ansätze in which the transformer outputs a wave-function amplitude or log-amplitude rather than a class label (Viteritti et al., 2022). Across both usages, the central idea is to preserve the long-range interaction structure associated with transformer attention while shifting part of the representation or update rule into a quantum model.

1. Terminology and scope

In the supplied literature, “QViT” does not refer to a single canonical architecture. Instead, it names a research area defined by where quantum computation is inserted into a transformer pipeline and by what objective the model optimizes.

Usage of “QViT”	Representative papers	Defining feature
Hybrid vision classifier	(Tesi et al., 2024, Cara et al., 2024, Unlu et al., 2024, Boucher et al., 10 Mar 2025, Zhang et al., 3 Apr 2025)	Quantum modules replace all or part of self-attention, projections, or MLPs
End-to-end quantum transformer	(Xue et al., 2024)	Transformer layer built from QLAM, QAM, DTM, and DCD
ViT-based neural quantum state	(Roca-Jerat et al., 2024, Cao et al., 2024)	ViT outputs a wave-function log-amplitude for variational quantum many-body problems

This terminological split is consequential. In image classification, the goal is usually to preserve ViT behavior while reducing parameters, modifying attention geometry, or moving expensive subroutines to a quantum device. In many-body physics, the transformer is not processing images in the conventional sense; it is acting as a neural quantum state over spin or fermionic configurations. A plausible implication is that discussions of “QViT performance” are only meaningful when the architectural role of the quantum component and the task class are specified explicitly.

A second source of ambiguity concerns the degree of quantization. Some models quantize only query/key projections or attention scoring (Tesi et al., 2024); others replace query, key, value, output projection, and even the feed-forward block with variational quantum circuits (Cara et al., 2024); still others construct a more thoroughly quantum transformer layer using block-encoding, quantum arithmetic, and state compression between layers (Xue et al., 2024). For this reason, QViT is better understood as an architectural umbrella than as a fixed model class.

2. Architectural patterns in hybrid QViTs

A recurrent design principle is to keep the outer ViT scaffold classical—patch extraction, tokenization, positional information, residual paths, and classification head—while replacing selected linear maps inside attention with quantum transformations.

In the high-energy-physics QViT based on quantum orthogonal neural networks (QONNs), the model is explicitly presented as a drop-in “quantum module” inside a classical ViT pipeline (Tesi et al., 2024). After patch embedding, positional encoding, and class-token preparation, the self-attention block replaces the classical query and key projections with parameterized quantum orthogonal networks. Each patch embedding $z_i \in \mathbb{R}^D$ is split into two halves $q_i, k_i \in \mathbb{R}^{D/2}$, and the unnormalized attention coefficient is obtained from a single-qubit measurement probability,

$$A_{ij} \propto \bigl|\langle 1|\,U(\boldsymbol\theta)\,|q_i\rangle \otimes |k_j\rangle\bigr|^2.$$

The resulting scores are then passed through the usual $\operatorname{softmax}(A/\sqrt{d_k})$ and multiplied by the classical value matrix $V$, leaving the remainder of the transformer unchanged. This architecture therefore preserves the standard attention flow while changing the parameterization of the query-key interaction.

A broader hybridization strategy appears in the quark–gluon classification model that replaces all attention projections by variational quantum circuits (VQCs) and also uses VQCs in the MLP block (Cara et al., 2024). In that design, each encoder block contains Quantum Multi-Head Attention and a Quantum MLP. The query, key, and value maps,

$$Q^h = VQC_Q^h(X), \quad K^h = VQC_K^h(X), \quad V^h = VQC_V^h(X),$$

are all quantum-generated, and the attention score $s^h_{ij}$ is itself computed by a VQC acting on $(Q_i^h, K_j^h)$. The post-attention feed-forward stage is likewise quantumized through two VQCs with a classical GELU between them. This is a more aggressive replacement strategy than the QONN-based approach.

Another family uses scalar query and key summaries extracted from expectation values and then performs a classical Gaussian-style attention. In one event-classification model, each head loads a token into a quantum circuit, extracts $Q_r$ and $K_r$ from $q_i, k_i \in \mathbb{R}^{D/2}$0, and defines

$q_i, k_i \in \mathbb{R}^{D/2}$1

with the final head output computed by classical $q_i, k_i \in \mathbb{R}^{D/2}$2 (Unlu et al., 2024). A closely related Quantum Self-Attention Layer is used in a later comparative study, where

$q_i, k_i \in \mathbb{R}^{D/2}$3

is normalized classically and combined with value vectors extracted from Pauli-string expectation values (Tran et al., 28 Apr 2026).

A distinct line of work replaces the dense self-attention projections $q_i, k_i \in \mathbb{R}^{D/2}$4 by separate quantum neural networks and emphasizes parameter scaling. In the biomedical image-classification QViT, each projection is implemented by an $q_i, k_i \in \mathbb{R}^{D/2}$5-qubit QNN with angle encoding, trainable $q_i, k_i \in \mathbb{R}^{D/2}$6 rotations, pairwise CNOT entanglers, and Pauli-$q_i, k_i \in \mathbb{R}^{D/2}$7 measurement, yielding a linear parameter law $q_i, k_i \in \mathbb{R}^{D/2}$8 rather than the classical $q_i, k_i \in \mathbb{R}^{D/2}$9 (Boucher et al., 10 Mar 2025). Here the primary architectural claim is not merely replacement of linear maps, but replacement with a parameterization whose scaling is explicitly different.

3. Quantum encodings, circuit constructions, and resource models

QViT proposals differ sharply in how classical features are loaded into quantum states and how the attention map is represented.

Orthogonal-layer QViTs based on reconfigurable beam splitter (RBS) gates are among the most explicit resource constructions. In the QONN attention model, unary amplitude encoding maps a normalized real vector $$A_{ij} \propto \bigl|\langle 1|\,U(\boldsymbol\theta)\,|q_i\rangle \otimes |k_j\rangle\bigr|^2.$$0 to

$$A_{ij} \propto \bigl|\langle 1|\,U(\boldsymbol\theta)\,|q_i\rangle \otimes |k_j\rangle\bigr|^2.$$1

For $$A_{ij} \propto \bigl|\langle 1|\,U(\boldsymbol\theta)\,|q_i\rangle \otimes |k_j\rangle\bigr|^2.$$2, the model allocates eight qubits. The amplitude loader uses a cascade of $$A_{ij} \propto \bigl|\langle 1|\,U(\boldsymbol\theta)\,|q_i\rangle \otimes |k_j\rangle\bigr|^2.$$3 RBS gates with depth-linear, $$A_{ij} \propto \bigl|\langle 1|\,U(\boldsymbol\theta)\,|q_i\rangle \otimes |k_j\rangle\bigr|^2.$$4-size complexity, and the orthogonal layer circuit uses exactly $$A_{ij} \propto \bigl|\langle 1|\,U(\boldsymbol\theta)\,|q_i\rangle \otimes |k_j\rangle\bigr|^2.$$5 RBS gates, matching the degrees of freedom of a real $$A_{ij} \propto \bigl|\langle 1|\,U(\boldsymbol\theta)\,|q_i\rangle \otimes |k_j\rangle\bigr|^2.$$6 orthogonal matrix (Tesi et al., 2024). Because the global unitary satisfies $$A_{ij} \propto \bigl|\langle 1|\,U(\boldsymbol\theta)\,|q_i\rangle \otimes |k_j\rangle\bigr|^2.$$7, the attention mechanism inherits an explicitly norm-preserving parameterization.

The earlier “Quantum Vision Transformers” paper develops the same RBS-based toolbox much more generally and introduces vector loaders, matrix loaders, and three trainable orthogonal-layer layouts: pyramid, butterfly, and X circuits (Cherrat et al., 2022). In that formulation, variant B computes pairwise attention coefficients by loading $$A_{ij} \propto \bigl|\langle 1|\,U(\boldsymbol\theta)\,|q_i\rangle \otimes |k_j\rangle\bigr|^2.$$8, applying an orthogonal map $$A_{ij} \propto \bigl|\langle 1|\,U(\boldsymbol\theta)\,|q_i\rangle \otimes |k_j\rangle\bigr|^2.$$9, un-loading $\operatorname{softmax}(A/\sqrt{d_k})$0, and measuring the first qubit so that the measurement probability yields $\operatorname{softmax}(A/\sqrt{d_k})$1. Variants C and D go further: direct quantum attention computes a patch update in a single circuit once the attention row is classically available, whereas the Compound Transformer loads the full patch matrix in a Hamming-weight-two subspace and applies a compound-matrix orthogonal layer so that all $\operatorname{softmax}(A/\sqrt{d_k})$2 are generated simultaneously by one circuit per layer. The resource comparison in that paper is therefore not simply “quantum versus classical attention,” but a spectrum from patch-wise orthogonal mapping to natively quantum compound-matrix attention.

Other QViTs pursue lower qubit counts through whole-image amplitude encoding. HQViT encodes the full patchified image as

$\operatorname{softmax}(A/\sqrt{d_k})$3

so that the qubit requirement is $\operatorname{softmax}(A/\sqrt{d_k})$4 (Zhang et al., 3 Apr 2025). It then applies parameterized circuits $\operatorname{softmax}(A/\sqrt{d_k})$5 and a swap test on the patch subsystems to estimate $\operatorname{softmax}(A/\sqrt{d_k})$6 for all patch pairs. The paper states that this offloads the classical $\operatorname{softmax}(A/\sqrt{d_k})$7 self-attention coefficient calculation to the quantum side and that the number of parameterized quantum gates scales as $\operatorname{softmax}(A/\sqrt{d_k})$8.

The end-to-end qTransformer adopts a more algorithmic quantum linear algebra perspective (Xue et al., 2024). Its transformer layer is built from Quantum Linear Algebra Modules, Quantum Arithmetic Modules, and Data Transfer Modules. Patch embeddings are loaded into address-indexed quantum registers; index-conditioned matrix multiplications prepare $\operatorname{softmax}(A/\sqrt{d_k})$9; quantum dot products generate unnormalized scores $V$0; and after classical softmax, a block-encoding unitary $V$1 applies $V$2. A distinctive element is DCD, which extracts a small set of Chebyshev coefficients and re-prepares the state for the next layer. The paper’s complexity claims are therefore contingent on state compression and block-encoding infrastructure rather than on VQC expressivity alone.

These heterogeneous constructions show that there is no single resource model for QViTs. Some papers optimize for shallow few-qubit circuits (Cara et al., 2024); some for orthogonal parameter efficiency (Tesi et al., 2024); some for asymptotic scaling under QRAM and block-encoding assumptions (Xue et al., 2024); and some for logarithmic qubit counts via amplitude encoding (Zhang et al., 3 Apr 2025). This suggests that “QViT complexity” is architecture-specific, not a universal property of the label.

4. Optimization regimes and learning objectives

Training protocols for QViTs separate cleanly into supervised image learning and variational optimization for quantum states.

In supervised settings, losses are standard classification objectives. The QONN-based jet-classification QViT trains on a balanced subset of 50,000 CMS Open Data jet images with a 70%/15%/15% train/validation/test split, uses Adam with learning rate $V$3, batch size 32, binary cross-entropy loss, 15 epochs, and dropout $V$4 in the MLP (Tesi et al., 2024). The VQC-based quark–gluon model uses binary cross-entropy, AdamW with weight decay, gradient-norm clipping at 1, a linear warmup to $V$5 over 5,000 steps followed by cosine decay, batch size 256, and 25 epochs; quantum gradients are computed by the parameter-shift rule in TensorCircuit while classical gradients are handled in JAX/Flax (Cara et al., 2024). The comparative QViT robustness study likewise uses parameter-shift with $V$6, Adam with cosine decay, and 1024 shots per observable (Tran et al., 28 Apr 2026).

In the many-body literature, the transformer is optimized as a variational ansatz for a wave function rather than as a classifier. In the spin-system QViT, the amplitude is written

$V$7

with $V$8 produced by a small complex-valued ViT over spin patches (Viteritti et al., 2022). Training minimizes the variational energy

$V$9

using Metropolis sampling and stochastic reconfiguration, where the update is $$Q^h = VQC_Q^h(X), \quad K^h = VQC_K^h(X), \quad V^h = VQC_V^h(X),$$0. In the long-range-model transformer wave function, the amplitude is similarly

$$Q^h = VQC_Q^h(X), \quad K^h = VQC_K^h(X), \quad V^h = VQC_V^h(X),$$1

but translational invariance is enforced by restricting attention weights to circulant kernels and symmetrizing over cyclic shifts; training again uses stochastic reconfiguration combined with SGD and Metropolis–Hastings sampling with single-spin flips and random flips of the total magnetization (Roca-Jerat et al., 2024).

A third optimization mode appears in impurity models, where conventional variational Monte Carlo is replaced by deterministic subspace expansion (Cao et al., 2024). There the QViT outputs

$$Q^h = VQC_Q^h(X), \quad K^h = VQC_K^h(X), \quad V^h = VQC_V^h(X),$$2

and the working subspace $$Q^h = VQC_Q^h(X), \quad K^h = VQC_K^h(X), \quad V^h = VQC_V^h(X),$$3 is iteratively expanded by Hamiltonian-connected neighbors before stochastic reconfiguration updates are computed exactly over the truncated support. This is not a standard minibatch-learning workflow; it is a domain-specific variational optimizer that exploits a highly skewed wave-function support.

The resulting picture is that QViT training is not defined by a uniquely quantum optimizer. Instead, the objective and training loop are inherited from the target domain: cross-entropy with hybrid autodiff in supervised vision, or variational-energy minimization with stochastic reconfiguration or subspace expansion in many-body problems.

5. Empirical performance across domains

The empirical record of QViTs is broad but non-uniform. The strongest consistent pattern is not categorical superiority over classical ViTs, but task-dependent competitiveness under different parameter, hardware, and simulation constraints.

In high-energy physics, several hybrid QViTs report near-parity with classical baselines. The QONN-based jet-image model reaches a validation AUC of about 0.675 for both classical and quantum-enhanced variants over fifteen epochs, and on the held-out test set the classical ViT records loss $$Q^h = VQC_Q^h(X), \quad K^h = VQC_K^h(X), \quad V^h = VQC_V^h(X),$$4, accuracy $$Q^h = VQC_Q^h(X), \quad K^h = VQC_K^h(X), \quad V^h = VQC_V^h(X),$$5, AUC $$Q^h = VQC_Q^h(X), \quad K^h = VQC_K^h(X), \quad V^h = VQC_V^h(X),$$6, while the QViT records loss $$Q^h = VQC_Q^h(X), \quad K^h = VQC_K^h(X), \quad V^h = VQC_V^h(X),$$7, accuracy $$Q^h = VQC_Q^h(X), \quad K^h = VQC_K^h(X), \quad V^h = VQC_V^h(X),$$8, AUC $$Q^h = VQC_Q^h(X), \quad K^h = VQC_K^h(X), \quad V^h = VQC_V^h(X),$$9 (Tesi et al., 2024). In the earlier quark–gluon classifier, a classical ViT with 5,178 trainable parameters achieves test AUC $s^h_{ij}$0, while the hybrid QViT with 4,170 parameters achieves $s^h_{ij}$1 (Cara et al., 2024). For photon-versus-electron event classification, the best classical and hybrid models are again closely matched: accuracy $s^h_{ij}$2 versus $s^h_{ij}$3, BCE loss $s^h_{ij}$4 versus $s^h_{ij}$5, and AUC $s^h_{ij}$6 versus $s^h_{ij}$7, with total parameter counts 4,801 and 4,601 respectively (Unlu et al., 2024).

In biomedical and medical-image classification, the reported gains are stronger. The QSA-based QViT for biomedical imaging states that replacing classical self-attention with quantum self-attention can match or exceed matched ViTs with linear rather than quadratic parameter scaling (Boucher et al., 10 Mar 2025). On RetinaMNIST, the reported QViT_28 obtains AUC $s^h_{ij}$8 and accuracy $s^h_{ij}$9, compared with ViT_28 at AUC $(Q_i^h, K_j^h)$0 and accuracy $(Q_i^h, K_j^h)$1; the abstract further states that the model is just $(Q_i^h, K_j^h)$2 below the top MedMamba model while using $(Q_i^h, K_j^h)$3 fewer parameters and $(Q_i^h, K_j^h)$4 fewer GFLOPs relative to the teacher. The same paper reports average ROC AUC $(Q_i^h, K_j^h)$5 versus $(Q_i^h, K_j^h)$6 and average accuracy $(Q_i^h, K_j^h)$7 versus $(Q_i^h, K_j^h)$8 for 4-qubit QViTs against matched ViTs across nine tasks. The earlier orthogonal-layer QViTs on MedMNIST likewise report competitive AUC/ACC with fewer trainable attention parameters, and in 7/12 tasks they outperform the classical transformer (Cherrat et al., 2022).

General image-classification results are mixed but often favorable to quantum attention variants. HQViT reports $(Q_i^h, K_j^h)$9 on MNIST $Q_r$0-$Q_r$1 versus QSAM’s $Q_r$2, $Q_r$3 on odd/even versus $Q_r$4, $Q_r$5 on BreastMNIST versus QOViT’s $Q_r$6, and modest gains on CIFAR-10 and Mini-ImageNet over both classical ViT and QSAM baselines (Zhang et al., 3 Apr 2025). A much lighter hybrid, which places a single-qubit quantum embedding after a pretrained ViT and a scalar reduction layer, reports median F1 $Q_r$7 on BirdCLEF-2021, compared with $Q_r$8 for CNN-Embed and $Q_r$9 for a classical CNN (Chen et al., 2024).

In quantum many-body applications, the reported advantages concern energy accuracy, phase-diagram fidelity, and parameter efficiency rather than classification accuracy. The transformer wave function for long-range models attains relative energy errors $K_r$0 across $K_r$1, with V-score $K_r$2 uniformly in $K_r$3 up to $K_r$4, and under a fixed 3 min wall time yields V-scores up to two orders of magnitude better than an RBM-like ansatz (Roca-Jerat et al., 2024). For the $K_r$5-$K_r$6 Heisenberg chain at $K_r$7, the shallow ViT wave function reports relative ground-state energy errors of approximately $K_r$8 at $K_r$9, $q_i, k_i \in \mathbb{R}^{D/2}$00 at $q_i, k_i \in \mathbb{R}^{D/2}$01, and $q_i, k_i \in \mathbb{R}^{D/2}$02 at $q_i, k_i \in \mathbb{R}^{D/2}$03, while deeper variants reduce errors below $q_i, k_i \in \mathbb{R}^{D/2}$04 on $q_i, k_i \in \mathbb{R}^{D/2}$05 (Viteritti et al., 2022). In impurity models, subspace-expanded QViT reaches $q_i, k_i \in \mathbb{R}^{D/2}$06 at $q_i, k_i \in \mathbb{R}^{D/2}$07 in the single-orbital benchmark and, in a three-orbital setting, a model with roughly 3,000 parameters matches or slightly outperforms an MPS DMRG result with bond dimension $q_i, k_i \in \mathbb{R}^{D/2}$08 (Cao et al., 2024).

6. Limitations, robustness, and open questions

A common misconception is that QViTs already deliver uniform practical speedups. The current literature does not support such a blanket statement. The end-to-end qTransformer reports a theoretically exponential and empirically polynomial speedup under its DCD-based cost model, with per-layer overhead

$q_i, k_i \in \mathbb{R}^{D/2}$09

and interprets the result as an exponential speedup in $q_i, k_i \in \mathbb{R}^{D/2}$10 up to the polylog factor when $q_i, k_i \in \mathbb{R}^{D/2}$11 (Xue et al., 2024). By contrast, the QONN-based HEP QViT states that training and inference are roughly $q_i, k_i \in \mathbb{R}^{D/2}$12–$q_i, k_i \in \mathbb{R}^{D/2}$13 slower than the classical ViT because quantum-circuit simulation dominates execution, and the event-classification hybrid reports about 5 hours on a GPU simulator versus about 10 minutes for the classical ViT (Tesi et al., 2024). These results are not contradictory; they operate under different hardware and cost assumptions. They do, however, show that asymptotic speedup claims and present-day wall-clock measurements should not be conflated.

Hardware assumptions remain a major constraint. The end-to-end qTransformer requires QRAM, block-encoding, QAE, and T-depths of order $q_i, k_i \in \mathbb{R}^{D/2}$14 for representative layer estimates, with error-rate conditions such as below $q_i, k_i \in \mathbb{R}^{D/2}$15 per QRAM call or $q_i, k_i \in \mathbb{R}^{D/2}$16 gate error to maintain $q_i, k_i \in \mathbb{R}^{D/2}$17 (Xue et al., 2024). More NISQ-oriented hybrids keep circuits shallow and qubit counts small—4 qubits per VQC in one quark–gluon model (Cara et al., 2024), or 4–6 qubit hardware demonstrations for orthogonal QViTs on RetinaMNIST (Cherrat et al., 2022)—but then the models typically rely on repeated circuit calls, classical post-processing, or simulator-based training.

Input constraints are another recurring limitation. Amplitude encoding requires normalized inputs in the QONN attention model (Tesi et al., 2024), and whole-image amplitude encoding in HQViT introduces a nontrivial measurement overhead $q_i, k_i \in \mathbb{R}^{D/2}$18 even while reducing qubit counts to $q_i, k_i \in \mathbb{R}^{D/2}$19 (Zhang et al., 3 Apr 2025). In the biomedical QSA work, classical emulation restricts experiments to $q_i, k_i \in \mathbb{R}^{D/2}$20 qubits, which bounds the scale at which the purported $q_i, k_i \in \mathbb{R}^{D/2}$21 parameter law can currently be tested directly (Boucher et al., 10 Mar 2025).

Robustness results are also mixed. The comparative analysis of hybrid quantum models finds that QViT maintains high robustness against measurement noise, channel noise, and finite-shot effects, with accuracy effectively unchanged up to noise levels $q_i, k_i \in \mathbb{R}^{D/2}$22 under several single-qubit noise channels (Tran et al., 28 Apr 2026). The same study, however, reports that under APGD adversarial perturbations QViT accuracy collapses to $q_i, k_i \in \mathbb{R}^{D/2}$23 even at $q_i, k_i \in \mathbb{R}^{D/2}$24, while its quantum-state fidelity drops from $q_i, k_i \in \mathbb{R}^{D/2}$25 to $q_i, k_i \in \mathbb{R}^{D/2}$26 as $q_i, k_i \in \mathbb{R}^{D/2}$27 increases. This indicates that robustness to intrinsic quantum noise does not imply robustness to classical adversarial perturbations.

Open questions follow directly from these tensions. One line of work emphasizes parameter efficiency and knowledge distillation, noting that higher-qubit QViTs benefit more from KD pre-training (Boucher et al., 10 Mar 2025). Another seeks more coherent quantum processing across blocks by avoiding repeated quantum–classical conversion boundaries (Zhang et al., 3 Apr 2025). In many-body physics, current limitations include finite-size effects, the need for phase-MLPs for complex or fermionic states, and possible extension to graph-attention or hybrid convolution-transformer variants in less symmetric settings (Roca-Jerat et al., 2024). Taken together, the literature suggests that QViTs are not a settled architecture but an active design space whose future depends on circuit compression, hardware maturity, and sharper problem-to-architecture matching.