---
title: 'CLAQS: Compact Quantum Token Mixer'
url: https://www.emergentmind.com/topics/claqs
type: topic
---

# CLAQS: Compact Quantum Token Mixer

Searching arXiv for the CLAQS paper and closely related quantum NLP work.
CLAQS is a compact, fully quantum token mixer for text classification that is designed as an analogue of a Transformer layer’s self-attention plus feed-forward block, but realized almost entirely as a parameterized quantum circuit. It was introduced to address three constraints simultaneously: limited qubits and shallow depth in the NISQ regime, training instability in quantum circuits due to amplitude and post-selection effects, and the quadratic time and memory cost of classical attention [2510.06532]. The model replaces dot-product self-attention with a learnable linear combination of unitaries (LCU), applies Quantum Singular Value Transformation (QSVT) as a learnable nonlinearity, and then uses a quantum feed-forward parameterized quantum circuit, all under a sliding-window regime with document-level aggregation. In the reported experiments, CLAQS operates with only eight data qubits and achieves 91.64% accuracy on SST-2 and 87.08% on IMDB, outperforming both a classical Transformer baseline and hybrid quantum-classical baselines under a tighter qubit and parameter budget [2510.06532].

## 1. Definition and architectural role

CLAQS, short for **Compact Learnable All-Quantum Token Mixer with Shared-ansatz**, is a quantum token-mixing module for text classification. Its architectural role is explicitly analogous to the combination of self-attention and feed-forward computation in a Transformer layer, but the token interaction and nonlinear transformation are carried out in Hilbert space rather than by a classical attention module [2510.06532].

The model targets three stated pain points. First, it is designed for **quantum hardware constraints**, specifically limited qubits and shallow depth in the NISQ regime. Second, it addresses **training instability in quantum circuits**, including exploding or vanishing amplitudes and unstable post-selection probabilities. Third, it is motivated by the **cost of classical attention**, whose time and memory usage scale quadratically in sequence length and whose parameterization can be large [2510.06532].

The central design choice is to replace attention-style token interaction with a learnable operator built from token-dependent unitaries. Instead of computing pairwise query-key similarities, CLAQS forms a global token-mixing operator by summing token unitaries with trainable complex coefficients, then subjects that operator to a learnable polynomial transformation via QSVT, and finally refines the resulting state with a shallow quantum feed-forward circuit [2510.06532]. This yields an “all-quantum” mixer in the precise sense that the core mixing and nonlinearity are implemented quantumly rather than delegated to a classical submodule.

## 2. Core pipeline and mathematical formulation

The CLAQS pipeline has five stages. Each token is first mapped from a classical embedding vector to rotation angles that parameterize a shared \(q\)-qubit ansatz circuit \(U(\boldsymbol\theta_w)\). For a window of tokens, the corresponding token unitaries \(\{U_j\}\) are then combined through an LCU operator
\[
M(\boldsymbol b)=\sum_{j=0}^{n-1} b_j\, U_j,\qquad \boldsymbol b\in\mathbb{C}^n .
\]
The coefficients are complex and trainable, so the model can learn both magnitude and phase in the token-mixing pattern [2510.06532].

To stabilize this operator, CLAQS applies \(\ell_1\) normalization to the mixing coefficients at every forward pass:
\[
\tilde b_j=\frac{b_j}{\sum_{k=0}^{n-1} |b_k|},\qquad \sum_{j=0}^{n-1} |\tilde b_j|=1.
\]
This normalization is used to regulate amplitude scaling, keep the operator in a norm regime compatible with block-encoding assumptions, and prevent pathological post-selection behavior [2510.06532].

After token mixing, the model applies a learnable QSVT polynomial
\[
P_{\boldsymbol c}(M)=\sum_{k=0}^{d} c_k\, M^k,
\]
where both the degree \(d\) and the coefficients \(\boldsymbol c\) are trainable. This makes the nonlinearity itself part of end-to-end learning. The resulting operator acts on the initialized data register and is followed by a second parameterized quantum circuit,
\[
\ket{\psi} \;=\; U_{\mathrm{FF}}\, P_{\boldsymbol c}\!\big(M(\tilde{\boldsymbol b})\big)\, \ket{0^q},
\]
which serves as a quantum feed-forward module [2510.06532].

Readout is performed by measuring \(X\), \(Y\), and \(Z\) expectation values on each data qubit, producing a real feature vector \(\mathbf o \in \mathbb{R}^{3q}\). This vector is passed to a small classical MLP head that outputs window-level logits, which are then aggregated into a document-level prediction [2510.06532].

## 3. Shared-ansatz design and token representation

The “shared-ansatz” designation refers to the use of a fixed token embedding circuit structure across all tokens. CLAQS uses a shared ansatz-14 pattern, while allowing token-specific rotation angles derived from the classical embedding of each token [2510.06532]. In this scheme, the circuit topology is fixed and only the angles vary from token to token.

Given a classical embedding \(\mathbf e_w\in\mathbb{R}^{d_e}\), the model first applies a linear map
\[
\boldsymbol\theta_w = W_E\,\mathbf e_w,
\]
with \(W_E\) initialized using Xavier initialization. These angles parameterize an ansatz-14 parameterized quantum circuit on \(q=8\) data qubits. Per layer, the ansatz uses alternating \(RY\) rotations on each qubit and controlled-\(RX\) gates between qubits; with \(q\) qubits and \(\ell\) repeated layers, the number of trainable angles is
\[
\#\mathrm{params}=4\ell q .
\]
This construction is an angle encoding of the token embedding into a unitary \(U_j = U(\boldsymbol\theta_{w_j})\) [2510.06532].

The architecture is explicitly two-stage. The first stage is the token-level shared embedding ansatz \(U(\boldsymbol\theta_w)\), which constructs token-dependent unitaries. The second stage is the window-level feed-forward PQC \(U_{\mathrm{FF}}(\boldsymbol\phi)\), applied after LCU mixing and QSVT. The parameters \(\boldsymbol\phi\) are shared across windows and examples, while the LCU vector \(\boldsymbol b\) and QSVT coefficients \(\boldsymbol c\) are shared globally as well [2510.06532]. This decouples **representation construction** from **representation consumption** and avoids a parameter count that grows with the number of tokens.

Position and order information are not introduced through explicit sinusoidal positional encodings. Instead, the model handles order implicitly through the window index and through the learned complex coefficients \(b_j\), which allow the system to weight positions differently within each window [2510.06532]. This suggests that CLAQS encodes positional structure through the mixer itself rather than through a separate positional embedding mechanism.

## 4. Token mixing, nonlinearity, and stabilization

The LCU mixer is the core token interaction mechanism. For a window of \(n\) tokens, the operator
\[
M(\boldsymbol b)=\sum_{j=0}^{n-1} b_j U_j
\]
combines token-dependent unitaries with global, trainable complex coefficients. On real quantum hardware, this would be implemented with a preparation unitary \(U_{\text{PREP}}\), a selection unitary \(U_{\text{SEL}}\), and post-selection on the control register; in classical simulation, the paper directly evaluates the linear combination, following Quixer’s simulation strategy [2510.06532].

The QSVT stage then applies a polynomial transformation to the singular values of the block-encoded operator. In CLAQS, this is used as a learnable nonlinearity. Operationally, in simulation the polynomial is computed through repeated multiplication by \(M\), rather than by explicit QSVT block-encoding circuits [2510.06532]. The intended effect is that LCU provides a superposition-like linear mixture of token unitaries, while QSVT supplies higher-order operator-level interactions.

Training stability is treated as a first-class architectural issue. In addition to explicit \(\ell_1\) normalization, the model defines an \(\mathrm{L1C}\) regularizer,
\[
\mathrm{L1C} = \lambda_{\ell_1}\big(\|\tilde{\mathbf b}\|_{1} - 1\big)^2,
\]
though the reported setup typically sets \(\lambda_{\ell_1}=0\) because the normalization is already enforced in the forward pass [2510.06532].

A second regularizer, the post-selection regularizer (PSR), targets the mean state norm interpreted as post-selection success probability:
\[
\mathrm{PSR} = \lambda_{\mathrm{ps}}\Big(\overline{\big\|P_{\mathbf c}(M)\lvert 0^q\rangle\big\|^2} - \tau\Big)^2.
\]
The overall loss is
\[
\mathcal{L} = \mathrm{CE}(z,y) + \mathrm{PSR} + \mathrm{L1C},
\]
where \(\mathrm{CE}(z,y)\) is cross-entropy between logits and labels [2510.06532].

The paper emphasizes that the primary stabilization comes from explicit \(\ell_1\) normalization, while PSR is used more for logging or mild regularization. It reports that \(\ell_1\) normalization prevents coefficient explosion or collapse and keeps post-selection success probabilities well behaved, empirically stabilizing gradient-based optimization. The text further states that unconstrained complex weights can lead to pathological norm regimes and barren-plateau-like behavior, though no rigorous gradient-scaling analysis is provided [2510.06532].

## 5. Sliding-window computation and computational profile

CLAQS is designed to process long sequences with a fixed small quantum register. Because the number of data qubits is fixed at \(q=8\), long documents are split into windows of length \(n\), such as 128 for SST-2 and 256 for IMDB. Each window is processed independently by the same quantum mixer, producing a window-level quantum state and then a feature vector derived from \(X\), \(Y\), and \(Z\) expectation values on each qubit [2510.06532].

A shared MLP head maps each window feature vector to window logits \(\mathbf z^{(w)}\in\mathbb{R}^C\), and a document-level aggregation mechanism combines these into a final prediction. The paper reports that the best-performing setup is the full CLAQS mix, and compares it in ablations to simple **mean\_logits** averaging and **attention pooling** over window logits [2510.06532].

The resource profile is one of the defining features of the model. The experiments use **8 data qubits** throughout. The control register requires \(\lceil \log_2 n\rceil\) qubits for a window of length \(n\), while ancilla usage is constant for QSVT and parity combination, yielding total qubit scaling
\[
\mathcal O(q+\log n).
\]
Gate complexity is reported as
\[
\mathcal O(d\,n\,q\,\ell),
\]
where \(d\) is polynomial degree, \(n\) is window length, \(q\) is the number of data qubits, and \(\ell\) is embedding depth [2510.06532]. The model is therefore linear in window length and polynomial degree, in contrast to classical self-attention, whose time and memory scale quadratically in sequence length.

Because the architecture uses shallow circuits, small \(d\), and fixed \(q=8\), the paper presents it as realistic both for NISQ-oriented prototyping and for GPU-based simulation. The reported implementation uses TorchQuantum on an NVIDIA RTX A5000, with LCU and QSVT implemented as differentiable layers and optimization performed with AdamW under cosine annealing [2510.06532].

## 6. Empirical performance and ablation results

CLAQS is evaluated on two binary sentiment classification benchmarks: SST-2 and IMDB. The paper reports accuracy, precision, recall, and macro F1, and gives the following headline accuracies: **91.64%** on SST-2 and **87.08%** on IMDB [2510.06532].

The main comparison includes classical and quantum or hybrid baselines.

| Method | Qubits | SST-2 Acc (%) |
|---|---:|---:|
| Classical Transformer | – | 79.36 |
| AQCF (hybrid) | 20 | 81.88 |
| CLAQS | 8 | 91.64 |

| Method | Qubits | IMDB Acc (%) |
|---|---:|---:|
| Classical Transformer | – | 82.18 |
| AQCF (hybrid) | 20 | 86.30 |
| CLAQS | 8 | 87.08 |

Relative to the classical Transformer baseline, CLAQS improves accuracy by **12.28** points on SST-2 and **4.90** points on IMDB. Relative to AQCF, it improves by **9.76** points on SST-2 and **0.78** points on IMDB, while using **8 qubits instead of 20** [2510.06532].

The parameter-budget comparison is equally central. In the IMDB configuration, the paper reports **454** “attention parameters” for CLAQS versus **263,168** for a classical Transformer block and **8,268** for AQCF. For SST-2, the reported CLAQS count is **326**. The paper states that CLAQS uses approximately **580× fewer attention parameters** than a standard Transformer block in the IMDB case, and approximately **18×–25× fewer** than AQCF [2510.06532]. These “attention parameters” include LCU coefficients, QSVT coefficients, and the quantum feed-forward module, but exclude token embeddings.

The ablation studies isolate the contribution of the quantum mixer and aggregation strategy. In the aggregation ablation, full CLAQS reaches **91.64%** on SST-2 versus **78.90%** for **mean\_logits** and **79.82%** for **attention pooling**; on IMDB, full CLAQS reaches **87.08%** versus **84.39%** and **83.63%**, respectively [2510.06532]. The paper concludes that the LCU + QSVT + feed-forward quantum mixer yields substantial gains over purely classical window pooling, especially on shorter sequences.

A second ablation examines measurement masking and additional quantum-side regularization. On SST-2, full CLAQS attains **91.64%**, **mean\_logits** reaches **80.39%**, and **mean\_logits + measurement masking + additional regularizers** drops to **70.24%**. On IMDB, the corresponding values are **87.08%**, **85.33%**, and **85.83%** [2510.06532]. The reported interpretation is that strong regularization and masking can reduce expressivity, particularly on the smaller and shorter SST-2 dataset.

## 7. Position in quantum NLP, limitations, and proposed directions

CLAQS is presented as part of an emerging class of **quantum-native token mixers**. The paper characterizes it as building directly on Quixer’s LCU-QSVT idea, but making both the LCU mixing coefficients and the QSVT polynomial coefficients fully learnable [2510.06532]. In that framing, the model is not a quantum implementation of classical pairwise attention; rather, it learns a global spectral transformation over a token-mixing operator.

The comparison with Transformer self-attention is therefore deliberately partial. The paper notes analogies: the LCU coefficients \(b_j\) play a role similar to attention weights over tokens, the QSVT polynomial acts analogously to a nonlinearity, the quantum feed-forward PQC corresponds to the Transformer feed-forward sublayer, and the shared token embedding ansatz resembles shared linear projections [2510.06532]. It also stresses differences: the coefficients are complex rather than real and probability-like, they are global rather than dynamically computed from query-key similarities, and the nonlinearity is an operator-level polynomial rather than an elementwise activation.

Several limitations are explicitly stated. The evaluation is **simulator-only**, using TorchQuantum rather than noisy hardware execution. The **sliding-window scheme** restricts direct quantum interactions to tokens within a window, so cross-window dependencies are handled only through classical aggregation. The study examines only a **single mixer layer** with modest depth and polynomial degree, does not include realistic **noise and error mitigation** experiments, and offers no rigorous theoretical treatment of **barren plateaus or gradient scaling** [2510.06532].

The proposed future directions follow directly from these constraints. They include execution on actual QPUs with shot-based readout and real noise, extension beyond sentiment analysis to natural language inference, topic classification, and token-level tasks, exploration of **multi-head** and **multi-layer** CLAQS mixers, study of constrained or Chebyshev-based QSVT polynomials, and more formal analyses of post-selection success probability and optimization stability in compact LCU-QSVT mixers [2510.06532].

A plausible implication is that CLAQS is best understood not as a drop-in reimplementation of attention, but as a compact quantum alternative for token interaction under severe qubit and depth constraints. The reported results suggest that, at least in simulator-based sentiment classification, a shared shallow ansatz combined with learnable LCU mixing, learnable QSVT nonlinearity, and explicit \(\ell_1\) amplitude control can form an effective text-classification module with unusually low qubit and parameter requirements [2510.06532].

Source: https://www.emergentmind.com/topics/claqs