Quantum Temporal Fusion Transformer

• Quantum Temporal Fusion Transformer (QTFT) is a hybrid model that integrates variational quantum circuits into the established Temporal Fusion Transformer architecture for time-series forecasting.
• It replaces key modules like the Gated Residual Network and multi-head self-attention with quantum-enhanced variants to boost expressivity and capture long-range dependencies.
• QTFT maintains classical components such as seq2seq layers and dense quantile forecasting, and uses parameter-shift gradients to optimize under NISQ constraints.

Quantum Temporal Fusion Transformer (QTFT) is a hybrid quantum-classical extension of the Temporal Fusion Transformer (TFT) for multi-horizon time-series forecasting. It preserves the core TFT pipeline—variable selection, static covariate conditioning, sequence-to-sequence locality modeling, interpretable multi-head self-attention, gating, and quantile forecasting—while replacing selected high-impact classical submodules with variational quantum circuits (Barik et al., 6 Aug 2025). In the reported formulation, QTFT replaces the Gated Residual Network (GRN) and all subroutines that depend on it, together with the interpretable multi-head attention mechanism, by quantum-enhanced variants that use shallow parameterized circuits, Pauli-$Z$ expectation measurements, and parameter-shift gradients, making the design explicitly NISQ-oriented (Barik et al., 6 Aug 2025). Within the broader taxonomy of quantum Transformers, QTFT belongs to the parameterized-quantum-circuit, hybrid-integration regime emphasized in the survey literature, where only selected Transformer components are quantumized and the remaining residual, normalization, and output pathways remain classical (Zhang et al., 4 Apr 2025).

1. Architectural lineage and conceptual position

The classical TFT, as summarized in the QTFT formulation, is a purpose-built architecture for multi-horizon forecasting that integrates six elements: variable selection networks, gated residual networks, a sequence-to-sequence encoder-decoder typically based on LSTM, static covariate encoders, an interpretable multi-head self-attention layer, and a quantile forecasting head (Barik et al., 6 Aug 2025). GRNs provide nonlinear transforms with gating and residual connections; variable selection uses GRNs and a softmax to generate time-varying feature weights; and the attention block uses standard scaled dot-product attention with a specialization in which value projections are shared across heads (Barik et al., 6 Aug 2025).

QTFT retains that overall organization but replaces the GRN family and the interpretable attention projections with variational quantum modules. The stated motivations are threefold: increasing expressivity through quantum feature maps and entangling circuits, leveraging variational quantum algorithm optimization, and injecting inductive biases such as shared value vectors and entanglement structures that may better capture cross-variable correlations and long-range temporal dependencies (Barik et al., 6 Aug 2025). The resulting model is therefore not a fully quantum Transformer. It is a selective quantumization of TFT blocks judged to be “parameter-intensive and influential” in the original architecture (Barik et al., 6 Aug 2025).

From the perspective of the survey taxonomy, the QTFT attention mechanism is closest to a PQC-based, QKV-only quantum mapping: quantum circuits generate query, key, and value representations, but attention is still computed classically through the standard softmax rule (Zhang et al., 4 Apr 2025). The survey also identifies quantumized variable selection and gating as natural hybrid insertion points for time-series models, which aligns directly with QTFT’s use of quantum GRNs and a Quantum GLU (Zhang et al., 4 Apr 2025).

TFT component	QTFT replacement	Role
GRN	QGRN	Nonlinear transform, conditioning, residual subroutines
Variable selection networks	QGRN-based variable selection	Time-varying feature weighting
Static covariate encoders	QGRN-based static encoders	Context vectors $c_s, c_e, c_c, c_h$
Interpretable multi-head attention	Quantum projections for $Q,K,V$ with classical attention	Long-range temporal dependency modeling
GLU gating blocks	QGLU	Gating and residual fusion
Seq2seq layer	Classical LSTM or QLSTM variant	Locality modeling
Forecast head	Classical dense quantile head	Multi-horizon outputs

2. Internal organization of the QTFT pipeline

The reported data flow begins with inputs at time $t$: observed features $z_t$, known features $x_t$ including future $x_{t+1:t+\tau_{\max}}$, and static features $s$ (Barik et al., 6 Aug 2025). Categorical variables are entity-embedded to $d_{\text{model}}$, and continuous variables are linearly projected to $d_{\text{model}}$ (Barik et al., 6 Aug 2025). Past inputs are flattened per time step as

$c_s, c_e, c_c, c_h$0

then passed through quantum variable selection, while static inputs undergo quantum variable selection followed by quantum static encoders that generate context vectors $c_s, c_e, c_c, c_h$1 (Barik et al., 6 Aug 2025).

After preprocessing, QTFT applies a sequence-to-sequence encoder-decoder. In the main configuration, this layer remains a classical LSTM in order to isolate the effects of quantum GRN and quantum attention, although a QLSTM variant is also reported (Barik et al., 6 Aug 2025). The seq2seq stage is followed by GLU, residual connection, and layer normalization; then a static enrichment step applies $c_s, c_e, c_c, c_h$2 (Barik et al., 6 Aug 2025).

The attention block is “quantum interpretable multi-head attention.” Query and key projections are produced by quantum circuits per head, while the value representation is shared across heads and is also produced by a quantum circuit (Barik et al., 6 Aug 2025). Attention itself remains classical: $c_s, c_e, c_c, c_h$3 The output of the multi-head block is then passed through a quantumized gating stage, residual connection, and layer normalization, followed by a position-wise feed-forward stage implemented as QGRN plus QGLU plus residual normalization (Barik et al., 6 Aug 2025). A classical dense head then outputs quantiles for horizons $c_s, c_e, c_c, c_h$4 (Barik et al., 6 Aug 2025).

A distinctive architectural decision is that QTFT omits the final linear projection $c_s, c_e, c_c, c_h$5 that ordinarily follows interpretable multi-head aggregation in TFT. The paper states that the omission is motivated by the fact that the quantum projections already provide sufficient parametrization (Barik et al., 6 Aug 2025). This is a specific example of the hybrid principle highlighted in the survey: quantum modules are inserted where they provide feature transformation capacity, while the surrounding classical scaffolding is retained for stability and scalability (Zhang et al., 4 Apr 2025).

3. Quantum submodules and mathematical formulation

QTFT uses several related quantum submodules. The central primitive is a variational quantum circuit that accepts a normalized real-valued input vector, prepares an $c_s, c_e, c_c, c_h$6-qubit state, applies a shallow trainable ansatz, and returns a real vector of Pauli-$c_s, c_e, c_c, c_h$7 expectation values (Barik et al., 6 Aug 2025). For a given encoded state $c_s, c_e, c_c, c_h$8, the output is

$c_s, c_e, c_c, c_h$9

These expectation values serve as the feature embedding for subsequent classical or quantum layers (Barik et al., 6 Aug 2025).

Two encoding schemes are specified. The generic QTFT design uses a ZZ feature map: $Q,K,V$0 with default choices $Q,K,V$1 and $Q,K,V$2 (Barik et al., 6 Aug 2025). The experiments also test AngleEmbedding for simplicity and hardware-friendliness, using single-qubit rotations consistent with PennyLane defaults: $Q,K,V$3 The paper states that features are normalized so that angle encodings map into $Q,K,V$4 (Barik et al., 6 Aug 2025).

For the variational ansatz, QTFT uses N-local, hardware-efficient layers composed of one-parameter single-qubit rotations followed by a ring of entanglers (Barik et al., 6 Aug 2025). Two variants are described: Basic Entangler Layers with $Q,K,V$5 rotations, and NLocal-style layers with $Q,K,V$6 rotations. In the experimental setting, the depth is $Q,K,V$7, explicitly to keep the design NISQ-friendly (Barik et al., 6 Aug 2025).

The GRN replacement is the Quantum Gated Residual Network (QGRN). In variable selection, the classical rule

$Q,K,V$8

becomes a QGRN-based computation, and the processed features

$Q,K,V$9

are likewise replaced by QGRN outputs before weighted aggregation (Barik et al., 6 Aug 2025). Gating blocks are replaced by Quantum GLU: $t$0 in the classical case, and

$t$1

in QTFT, where $t$2 and $t$3 are quantum-derived embeddings produced by two variational circuits from the same input (Barik et al., 6 Aug 2025).

Forecasting is performed through a dense quantile head: $t$4 and training uses the quantile pinball loss

$t$5

The batch objective is

$t$6

In the reported experiment, $t$7, the learning rate is $t$8, and training runs for 100 epochs (Barik et al., 6 Aug 2025).

4. Optimization, resource profile, and NISQ constraints

QTFT is trained end-to-end as a hybrid model. Classical loss gradients propagate through the classical parts of the network, while gradients of quantum parameters are evaluated analytically by the parameter-shift rule: $t$9 The hybrid training loop therefore updates both classical parameters $z_t$0 and quantum parameters $z_t$1 through standard gradient-based optimization (Barik et al., 6 Aug 2025).

The reported training procedure uses sliding windows with $z_t$2 past steps and $z_t$3 future steps. In the AXIS BANK experiment, the configuration is past steps $z_t$4, forecast steps $z_t$5, and overlapping windows that advance by one time step (Barik et al., 6 Aug 2025). Each batch passes through preprocessing, quantum variable selection and static encoders, seq2seq encoding and decoding, static enrichment, quantum attention with classical softmax aggregation, QGLU and normalization, position-wise QGRN and QGLU, and finally the dense quantile head and loss computation (Barik et al., 6 Aug 2025).

The hardware-software stack is explicitly hybrid: PyTorch is used for the TFT backbone and training loop, while PennyLane is used for variational quantum circuit simulation and parameter-shift differentiation (Barik et al., 6 Aug 2025). The ansatz design is described as Qiskit-inspired, specifically Basic Entangler Layers and NLocal patterns (Barik et al., 6 Aug 2025). The reported experiments use a simulator with exact expectation values. For real devices, the paper recommends shots $z_t$6, for example $z_t$7–$z_t$8, and grouping commuting Pauli-$z_t$9 observables to reduce overhead (Barik et al., 6 Aug 2025).

The computational bottleneck is determined by quantum parameter count and shot budget. If $x_t$0 denotes the total number of quantum parameters across all submodules in a forward pass, then parameter-shift requires $x_t$1 circuit evaluations per gradient step per batch; with $x_t$2 shots, the quantum cost per batch is $x_t$3 (Barik et al., 6 Aug 2025). With $x_t$4 heads, each head requires separate query and key circuits, while the value circuit is shared across heads (Barik et al., 6 Aug 2025). QGRN instances in variable selection and static encoders add further circuit calls per time step (Barik et al., 6 Aug 2025).

The survey’s broader resource analysis provides the rationale for the design choice made in QTFT. Angle encoding uses $x_t$5 qubits with shallow encoding depth, whereas amplitude encoding uses $x_t$6 qubits but requires deep, noise-sensitive loader circuits and can incur large shot counts (Zhang et al., 4 Apr 2025). QTFT’s experimental emphasis on small $x_t$7, shallow $x_t$8, ring entanglers, and Pauli-$x_t$9 measurements is therefore consistent with the survey’s recommendation to prefer low-depth PQC designs and limited hybrid quantumization under NISQ constraints (Zhang et al., 4 Apr 2025).

5. Empirical evaluation and interpretability

The reported benchmark is based on AXIS BANK stock market data from NSE India, NIFTY-50 symbol AXISBANK, covering 2000–2021 with 5306 rows and 15 columns (Barik et al., 6 Aug 2025). Because of NISQ constraints, the QTFT training setup uses a minimal subset: the first 10 rows from 2000, four input features—Open, High, Low, Last—and target Close (Barik et al., 6 Aug 2025). The train indices are 0–19 and the test indices are 20–26 (Barik et al., 6 Aug 2025). Evaluation uses quantile pinball loss at $x_{t+1:t+\tau_{\max}}$0 (Barik et al., 6 Aug 2025).

Model	Train / test loss	Trainable params
Classical TFT	0.2630 / 0.9856	190
QTFT without QLSTM	0.2028 / 0.8381	158
QTFT with QLSTM	0.1711 / 0.8007	174

These results show that, in the reported small-data regime, QTFT can achieve lower training and test loss than the classical TFT baseline, while the paper also notes that in other runs performance is comparable rather than uniformly superior (Barik et al., 6 Aug 2025). The same source explicitly cautions that the constrained dataset size and limited parameter counts preclude strong statistical claims, so the results should be interpreted as evidence of viability and competitiveness rather than established quantum advantage (Barik et al., 6 Aug 2025).

Interpretability is largely inherited from TFT. Variable selection weights still arise from a softmax over GRN-derived—here quantum-derived—outputs, so per-time-step feature importance analysis remains available (Barik et al., 6 Aug 2025). Attention patterns are computed classically from quantum-derived $x_{t+1:t+\tau_{\max}}$1, which means standard attention visualization continues to apply (Barik et al., 6 Aug 2025). A plausible implication is that QTFT alters the feature geometry that feeds those interpretable mechanisms without discarding the interpretability interfaces that make TFT attractive in applied forecasting.

The broader survey places these findings in context. PQC-based hybrids often show slight improvements over matched classical baselines on small tasks, while evidence for scalability and generalization remains limited (Zhang et al., 4 Apr 2025). The survey also notes that advantages can be neutral when encoding and measurement overheads dominate, which is directly relevant to any interpretation of QTFT’s small-scale empirical gains (Zhang et al., 4 Apr 2025).

6. Limitations, controversies, and prospective development

The most important limitation is epistemic rather than architectural: the existence of a general quantum advantage for time-series forecasting remains open (Barik et al., 6 Aug 2025). The QTFT experiments use a very small slice of one stock dataset, results may be sensitive to initialization and optimizer settings, and only a subset of TFT components is quantumized in the main experiments (Barik et al., 6 Aug 2025). The forecasting head is also restricted to a single quantile, $x_{t+1:t+\tau_{\max}}$2, rather than a fuller probabilistic output set (Barik et al., 6 Aug 2025). These constraints make it inappropriate to interpret QTFT as a settled replacement for classical TFT.

A common misconception is that QTFT is a fully quantum Transformer. It is not. The seq2seq component may remain classical, attention scores are computed classically, and residual, normalization, and dense output layers remain classical (Barik et al., 6 Aug 2025). This selective hybridization is deliberate and consistent with the survey’s conclusion that near-term quantum Transformers should usually quantumize only specific submodules because repeated encoding-measurement cycles, qubit limits, trainability issues, and noise sensitivity constrain end-to-end quantumization on NISQ hardware (Zhang et al., 4 Apr 2025).

Another misconception is that all quantumized attention mechanisms are semantically equivalent. The survey distinguishes QKV-only quantum mapping, quantum pairwise attention, quantum holistic attention, and quantum-assisted optimization (Zhang et al., 4 Apr 2025). QTFT, as currently instantiated, uses the first of these for attention: quantum circuits generate projections, but the attention matrix itself is classical (Barik et al., 6 Aug 2025). The survey also identifies more radical alternatives, such as QFT-based token mixing, sparse-attention acceleration via Grover-inspired search, quantum kernel self-attention, compound orthogonal layers, swap-test attention over mixed states, and Hadamard-test attention, as potential designs for future sequential models (Zhang et al., 4 Apr 2025). This suggests that “QTFT” names a broader design space than the single architecture currently reported.

Future work in the QTFT paper centers on improved ansätze, stronger error mitigation, broader placement of quantum modules, probabilistic forecasting with multiple quantiles, and larger-scale evaluation on electricity, traffic, retail, and clinical datasets (Barik et al., 6 Aug 2025). The survey points in the same direction but adds a sharper methodological distinction: near-term progress is likely to come from selective, resource-conscious hybrid quantumization, whereas more ambitious fully quantum attention and feed-forward blocks may depend on more mature hardware and, potentially, quantum linear algebra methods that are not yet practical for training-intensive forecasting settings (Zhang et al., 4 Apr 2025).

In that sense, QTFT occupies a specific historical and technical position. It is neither a demonstration of universal quantum superiority nor a merely symbolic “quantum-inspired” modification. It is a concrete NISQ-era hybrid architecture in which quantum GRNs, quantum gating, and quantum projection layers are integrated into a proven forecasting backbone, and its significance lies in establishing that such integration can be trained end-to-end and can remain competitive on forecasting tasks under severe resource constraints (Barik et al., 6 Aug 2025).