CLAQS: Compact Quantum Token Mixer

• CLAQS is a compact quantum token mixer for text classification that uses a parameterized quantum circuit to replicate Transformer self-attention and feed-forward operations.
• It addresses NISQ hardware constraints by operating on just 8 qubits, stabilizing training with ℓ1 normalization and reducing the quadratic cost of classical attention.
• Empirical results on benchmarks like SST-2 and IMDB show that CLAQS outperforms classical and hybrid baselines with significantly fewer parameters.

Searching arXiv for the CLAQS paper and closely related quantum NLP work. CLAQS is a compact, fully quantum token mixer for text classification that is designed as an analogue of a Transformer layer’s self-attention plus feed-forward block, but realized almost entirely as a parameterized quantum circuit. It was introduced to address three constraints simultaneously: limited qubits and shallow depth in the NISQ regime, training instability in quantum circuits due to amplitude and post-selection effects, and the quadratic time and memory cost of classical attention (Chen et al., 8 Oct 2025). The model replaces dot-product self-attention with a learnable linear combination of unitaries (LCU), applies Quantum Singular Value Transformation (QSVT) as a learnable nonlinearity, and then uses a quantum feed-forward parameterized quantum circuit, all under a sliding-window regime with document-level aggregation. In the reported experiments, CLAQS operates with only eight data qubits and achieves 91.64% accuracy on SST-2 and 87.08% on IMDB, outperforming both a classical Transformer baseline and hybrid quantum-classical baselines under a tighter qubit and parameter budget (Chen et al., 8 Oct 2025).

1. Definition and architectural role

CLAQS, short for Compact Learnable All-Quantum Token Mixer with Shared-ansatz, is a quantum token-mixing module for text classification. Its architectural role is explicitly analogous to the combination of self-attention and feed-forward computation in a Transformer layer, but the token interaction and nonlinear transformation are carried out in Hilbert space rather than by a classical attention module (Chen et al., 8 Oct 2025).

The model targets three stated pain points. First, it is designed for quantum hardware constraints, specifically limited qubits and shallow depth in the NISQ regime. Second, it addresses training instability in quantum circuits, including exploding or vanishing amplitudes and unstable post-selection probabilities. Third, it is motivated by the cost of classical attention, whose time and memory usage scale quadratically in sequence length and whose parameterization can be large (Chen et al., 8 Oct 2025).

The central design choice is to replace attention-style token interaction with a learnable operator built from token-dependent unitaries. Instead of computing pairwise query-key similarities, CLAQS forms a global token-mixing operator by summing token unitaries with trainable complex coefficients, then subjects that operator to a learnable polynomial transformation via QSVT, and finally refines the resulting state with a shallow quantum feed-forward circuit (Chen et al., 8 Oct 2025). This yields an “all-quantum” mixer in the precise sense that the core mixing and nonlinearity are implemented quantumly rather than delegated to a classical submodule.

2. Core pipeline and mathematical formulation

The CLAQS pipeline has five stages. Each token is first mapped from a classical embedding vector to rotation angles that parameterize a shared $q$-qubit ansatz circuit $U(\boldsymbol\theta_w)$. For a window of tokens, the corresponding token unitaries $\{U_j\}$ are then combined through an LCU operator

$$M(\boldsymbol b)=\sum_{j=0}^{n-1} b_j\, U_j,\qquad \boldsymbol b\in\mathbb{C}^n .$$

The coefficients are complex and trainable, so the model can learn both magnitude and phase in the token-mixing pattern (Chen et al., 8 Oct 2025).

To stabilize this operator, CLAQS applies $\ell_1$ normalization to the mixing coefficients at every forward pass: $$\tilde b_j=\frac{b_j}{\sum_{k=0}^{n-1} |b_k|},\qquad \sum_{j=0}^{n-1} |\tilde b_j|=1.$$ This normalization is used to regulate amplitude scaling, keep the operator in a norm regime compatible with block-encoding assumptions, and prevent pathological post-selection behavior (Chen et al., 8 Oct 2025).

After token mixing, the model applies a learnable QSVT polynomial

$P_{\boldsymbol c}(M)=\sum_{k=0}^{d} c_k\, M^k,$

where both the degree $d$ and the coefficients $\boldsymbol c$ are trainable. This makes the nonlinearity itself part of end-to-end learning. The resulting operator acts on the initialized data register and is followed by a second parameterized quantum circuit,

$$\ket{\psi} \;=\; U_{\mathrm{FF}}\, P_{\boldsymbol c}\!\big(M(\tilde{\boldsymbol b})\big)\, \ket{0^q},$$

which serves as a quantum feed-forward module (Chen et al., 8 Oct 2025).

Readout is performed by measuring $U(\boldsymbol\theta_w)$0, $U(\boldsymbol\theta_w)$1, and $U(\boldsymbol\theta_w)$2 expectation values on each data qubit, producing a real feature vector $U(\boldsymbol\theta_w)$3. This vector is passed to a small classical MLP head that outputs window-level logits, which are then aggregated into a document-level prediction (Chen et al., 8 Oct 2025).

3. Shared-ansatz design and token representation

The “shared-ansatz” designation refers to the use of a fixed token embedding circuit structure across all tokens. CLAQS uses a shared ansatz-14 pattern, while allowing token-specific rotation angles derived from the classical embedding of each token (Chen et al., 8 Oct 2025). In this scheme, the circuit topology is fixed and only the angles vary from token to token.

Given a classical embedding $U(\boldsymbol\theta_w)$4, the model first applies a linear map

$U(\boldsymbol\theta_w)$5

with $U(\boldsymbol\theta_w)$6 initialized using Xavier initialization. These angles parameterize an ansatz-14 parameterized quantum circuit on $U(\boldsymbol\theta_w)$7 data qubits. Per layer, the ansatz uses alternating $U(\boldsymbol\theta_w)$8 rotations on each qubit and controlled-$U(\boldsymbol\theta_w)$9 gates between qubits; with $\{U_j\}$0 qubits and $\{U_j\}$1 repeated layers, the number of trainable angles is

$\{U_j\}$2

This construction is an angle encoding of the token embedding into a unitary $\{U_j\}$3 (Chen et al., 8 Oct 2025).

The architecture is explicitly two-stage. The first stage is the token-level shared embedding ansatz $\{U_j\}$4, which constructs token-dependent unitaries. The second stage is the window-level feed-forward PQC $\{U_j\}$5, applied after LCU mixing and QSVT. The parameters $\{U_j\}$6 are shared across windows and examples, while the LCU vector $\{U_j\}$7 and QSVT coefficients $\{U_j\}$8 are shared globally as well (Chen et al., 8 Oct 2025). This decouples representation construction from representation consumption and avoids a parameter count that grows with the number of tokens.

Position and order information are not introduced through explicit sinusoidal positional encodings. Instead, the model handles order implicitly through the window index and through the learned complex coefficients $\{U_j\}$9, which allow the system to weight positions differently within each window (Chen et al., 8 Oct 2025). This suggests that CLAQS encodes positional structure through the mixer itself rather than through a separate positional embedding mechanism.

4. Token mixing, nonlinearity, and stabilization

The LCU mixer is the core token interaction mechanism. For a window of $$M(\boldsymbol b)=\sum_{j=0}^{n-1} b_j\, U_j,\qquad \boldsymbol b\in\mathbb{C}^n .$$0 tokens, the operator

$$M(\boldsymbol b)=\sum_{j=0}^{n-1} b_j\, U_j,\qquad \boldsymbol b\in\mathbb{C}^n .$$1

combines token-dependent unitaries with global, trainable complex coefficients. On real quantum hardware, this would be implemented with a preparation unitary $$M(\boldsymbol b)=\sum_{j=0}^{n-1} b_j\, U_j,\qquad \boldsymbol b\in\mathbb{C}^n .$$2, a selection unitary $$M(\boldsymbol b)=\sum_{j=0}^{n-1} b_j\, U_j,\qquad \boldsymbol b\in\mathbb{C}^n .$$3, and post-selection on the control register; in classical simulation, the paper directly evaluates the linear combination, following Quixer’s simulation strategy (Chen et al., 8 Oct 2025).

The QSVT stage then applies a polynomial transformation to the singular values of the block-encoded operator. In CLAQS, this is used as a learnable nonlinearity. Operationally, in simulation the polynomial is computed through repeated multiplication by $$M(\boldsymbol b)=\sum_{j=0}^{n-1} b_j\, U_j,\qquad \boldsymbol b\in\mathbb{C}^n .$$4, rather than by explicit QSVT block-encoding circuits (Chen et al., 8 Oct 2025). The intended effect is that LCU provides a superposition-like linear mixture of token unitaries, while QSVT supplies higher-order operator-level interactions.

Training stability is treated as a first-class architectural issue. In addition to explicit $$M(\boldsymbol b)=\sum_{j=0}^{n-1} b_j\, U_j,\qquad \boldsymbol b\in\mathbb{C}^n .$$5 normalization, the model defines an $$M(\boldsymbol b)=\sum_{j=0}^{n-1} b_j\, U_j,\qquad \boldsymbol b\in\mathbb{C}^n .$$6 regularizer,

$$M(\boldsymbol b)=\sum_{j=0}^{n-1} b_j\, U_j,\qquad \boldsymbol b\in\mathbb{C}^n .$$7

though the reported setup typically sets $$M(\boldsymbol b)=\sum_{j=0}^{n-1} b_j\, U_j,\qquad \boldsymbol b\in\mathbb{C}^n .$$8 because the normalization is already enforced in the forward pass (Chen et al., 8 Oct 2025).

A second regularizer, the post-selection regularizer (PSR), targets the mean state norm interpreted as post-selection success probability: $$M(\boldsymbol b)=\sum_{j=0}^{n-1} b_j\, U_j,\qquad \boldsymbol b\in\mathbb{C}^n .$$9 The overall loss is

$\ell_1$0

where $\ell_1$1 is cross-entropy between logits and labels (Chen et al., 8 Oct 2025).

The paper emphasizes that the primary stabilization comes from explicit $\ell_1$2 normalization, while PSR is used more for logging or mild regularization. It reports that $\ell_1$3 normalization prevents coefficient explosion or collapse and keeps post-selection success probabilities well behaved, empirically stabilizing gradient-based optimization. The text further states that unconstrained complex weights can lead to pathological norm regimes and barren-plateau-like behavior, though no rigorous gradient-scaling analysis is provided (Chen et al., 8 Oct 2025).

5. Sliding-window computation and computational profile

CLAQS is designed to process long sequences with a fixed small quantum register. Because the number of data qubits is fixed at $\ell_1$4, long documents are split into windows of length $\ell_1$5, such as 128 for SST-2 and 256 for IMDB. Each window is processed independently by the same quantum mixer, producing a window-level quantum state and then a feature vector derived from $\ell_1$6, $\ell_1$7, and $\ell_1$8 expectation values on each qubit (Chen et al., 8 Oct 2025).

A shared MLP head maps each window feature vector to window logits $\ell_1$9, and a document-level aggregation mechanism combines these into a final prediction. The paper reports that the best-performing setup is the full CLAQS mix, and compares it in ablations to simple mean_logits averaging and attention pooling over window logits (Chen et al., 8 Oct 2025).

The resource profile is one of the defining features of the model. The experiments use 8 data qubits throughout. The control register requires $$\tilde b_j=\frac{b_j}{\sum_{k=0}^{n-1} |b_k|},\qquad \sum_{j=0}^{n-1} |\tilde b_j|=1.$$0 qubits for a window of length $$\tilde b_j=\frac{b_j}{\sum_{k=0}^{n-1} |b_k|},\qquad \sum_{j=0}^{n-1} |\tilde b_j|=1.$$1, while ancilla usage is constant for QSVT and parity combination, yielding total qubit scaling

$$\tilde b_j=\frac{b_j}{\sum_{k=0}^{n-1} |b_k|},\qquad \sum_{j=0}^{n-1} |\tilde b_j|=1.$$2

Gate complexity is reported as

$$\tilde b_j=\frac{b_j}{\sum_{k=0}^{n-1} |b_k|},\qquad \sum_{j=0}^{n-1} |\tilde b_j|=1.$$3

where $$\tilde b_j=\frac{b_j}{\sum_{k=0}^{n-1} |b_k|},\qquad \sum_{j=0}^{n-1} |\tilde b_j|=1.$$4 is polynomial degree, $$\tilde b_j=\frac{b_j}{\sum_{k=0}^{n-1} |b_k|},\qquad \sum_{j=0}^{n-1} |\tilde b_j|=1.$$5 is window length, $$\tilde b_j=\frac{b_j}{\sum_{k=0}^{n-1} |b_k|},\qquad \sum_{j=0}^{n-1} |\tilde b_j|=1.$$6 is the number of data qubits, and $$\tilde b_j=\frac{b_j}{\sum_{k=0}^{n-1} |b_k|},\qquad \sum_{j=0}^{n-1} |\tilde b_j|=1.$$7 is embedding depth (Chen et al., 8 Oct 2025). The model is therefore linear in window length and polynomial degree, in contrast to classical self-attention, whose time and memory scale quadratically in sequence length.

Because the architecture uses shallow circuits, small $$\tilde b_j=\frac{b_j}{\sum_{k=0}^{n-1} |b_k|},\qquad \sum_{j=0}^{n-1} |\tilde b_j|=1.$$8, and fixed $$\tilde b_j=\frac{b_j}{\sum_{k=0}^{n-1} |b_k|},\qquad \sum_{j=0}^{n-1} |\tilde b_j|=1.$$9, the paper presents it as realistic both for NISQ-oriented prototyping and for GPU-based simulation. The reported implementation uses TorchQuantum on an NVIDIA RTX A5000, with LCU and QSVT implemented as differentiable layers and optimization performed with AdamW under cosine annealing (Chen et al., 8 Oct 2025).

6. Empirical performance and ablation results

CLAQS is evaluated on two binary sentiment classification benchmarks: SST-2 and IMDB. The paper reports accuracy, precision, recall, and macro F1, and gives the following headline accuracies: 91.64% on SST-2 and 87.08% on IMDB (Chen et al., 8 Oct 2025).

The main comparison includes classical and quantum or hybrid baselines.

Method	Qubits	SST-2 Acc (%)
Classical Transformer	–	79.36
AQCF (hybrid)	20	81.88
CLAQS	8	91.64

Method	Qubits	IMDB Acc (%)
Classical Transformer	–	82.18
AQCF (hybrid)	20	86.30
CLAQS	8	87.08

Relative to the classical Transformer baseline, CLAQS improves accuracy by 12.28 points on SST-2 and 4.90 points on IMDB. Relative to AQCF, it improves by 9.76 points on SST-2 and 0.78 points on IMDB, while using 8 qubits instead of 20 (Chen et al., 8 Oct 2025).

The parameter-budget comparison is equally central. In the IMDB configuration, the paper reports 454 “attention parameters” for CLAQS versus 263,168 for a classical Transformer block and 8,268 for AQCF. For SST-2, the reported CLAQS count is 326. The paper states that CLAQS uses approximately 580× fewer attention parameters than a standard Transformer block in the IMDB case, and approximately 18×–25× fewer than AQCF (Chen et al., 8 Oct 2025). These “attention parameters” include LCU coefficients, QSVT coefficients, and the quantum feed-forward module, but exclude token embeddings.

The ablation studies isolate the contribution of the quantum mixer and aggregation strategy. In the aggregation ablation, full CLAQS reaches 91.64% on SST-2 versus 78.90% for mean_logits and 79.82% for attention pooling; on IMDB, full CLAQS reaches 87.08% versus 84.39% and 83.63%, respectively (Chen et al., 8 Oct 2025). The paper concludes that the LCU + QSVT + feed-forward quantum mixer yields substantial gains over purely classical window pooling, especially on shorter sequences.

A second ablation examines measurement masking and additional quantum-side regularization. On SST-2, full CLAQS attains 91.64%, mean_logits reaches 80.39%, and mean_logits + measurement masking + additional regularizers drops to 70.24%. On IMDB, the corresponding values are 87.08%, 85.33%, and 85.83% (Chen et al., 8 Oct 2025). The reported interpretation is that strong regularization and masking can reduce expressivity, particularly on the smaller and shorter SST-2 dataset.

7. Position in quantum NLP, limitations, and proposed directions

CLAQS is presented as part of an emerging class of quantum-native token mixers. The paper characterizes it as building directly on Quixer’s LCU-QSVT idea, but making both the LCU mixing coefficients and the QSVT polynomial coefficients fully learnable (Chen et al., 8 Oct 2025). In that framing, the model is not a quantum implementation of classical pairwise attention; rather, it learns a global spectral transformation over a token-mixing operator.

The comparison with Transformer self-attention is therefore deliberately partial. The paper notes analogies: the LCU coefficients $P_{\boldsymbol c}(M)=\sum_{k=0}^{d} c_k\, M^k,$0 play a role similar to attention weights over tokens, the QSVT polynomial acts analogously to a nonlinearity, the quantum feed-forward PQC corresponds to the Transformer feed-forward sublayer, and the shared token embedding ansatz resembles shared linear projections (Chen et al., 8 Oct 2025). It also stresses differences: the coefficients are complex rather than real and probability-like, they are global rather than dynamically computed from query-key similarities, and the nonlinearity is an operator-level polynomial rather than an elementwise activation.

Several limitations are explicitly stated. The evaluation is simulator-only, using TorchQuantum rather than noisy hardware execution. The sliding-window scheme restricts direct quantum interactions to tokens within a window, so cross-window dependencies are handled only through classical aggregation. The study examines only a single mixer layer with modest depth and polynomial degree, does not include realistic noise and error mitigation experiments, and offers no rigorous theoretical treatment of barren plateaus or gradient scaling (Chen et al., 8 Oct 2025).

The proposed future directions follow directly from these constraints. They include execution on actual QPUs with shot-based readout and real noise, extension beyond sentiment analysis to natural language inference, topic classification, and token-level tasks, exploration of multi-head and multi-layer CLAQS mixers, study of constrained or Chebyshev-based QSVT polynomials, and more formal analyses of post-selection success probability and optimization stability in compact LCU-QSVT mixers (Chen et al., 8 Oct 2025).

A plausible implication is that CLAQS is best understood not as a drop-in reimplementation of attention, but as a compact quantum alternative for token interaction under severe qubit and depth constraints. The reported results suggest that, at least in simulator-based sentiment classification, a shared shallow ansatz combined with learnable LCU mixing, learnable QSVT nonlinearity, and explicit $P_{\boldsymbol c}(M)=\sum_{k=0}^{d} c_k\, M^k,$1 amplitude control can form an effective text-classification module with unusually low qubit and parameter requirements (Chen et al., 8 Oct 2025).