Holographic Quantum Transformer: A Generalist Neuro-Symbolic Architecture for Solving Frustrated Systems via Generative Attention

Abstract: Simulating two-dimensional frustrated quantum matter is a grand challenge due to the sign problem and exponential Hilbert space complexity. In this work, we introduce the Holographic Quantum Transformer (HQT), a physics-inspired generative architecture that leverages global self-attention to resolve non-local entanglement patterns. We validate HQT on the square lattice $J_1-J_2$ Heisenberg model. On the heavily frustrated $8 \times 8$ lattice at the quantum critical point ($J_2=0.5$), HQT reaches a ground-state energy per site ($E/N$) of $\mathbf{-0.5001(1)}$, consistent with the expected finite-size scaling trend. Beyond numerical accuracy, HQT exhibits intrinsic physical awareness, autonomously recovering the underlying $J_2$ interaction geometry through interpretable attention maps. Our central contribution is ``Holographic Transfer", a zero-shot size-extrapolation protocol with rapid alignment: a model trained on $8 \times 8$ systems is directly projected onto larger $10 \times 10$ lattices via continuous positional-embedding interpolation and head re-initialization, achieving high-fidelity initialization and rapid convergence. This zero-shot protocol yields an energy of $E/N = \mathbf{-0.49782(3)}$, statistically consistent with the variational state of the art while requiring no from-scratch training on the target lattice. Our results establish generative attention as a scalable paradigm for transferable quantum simulation.

• The paper introduces the Holographic Quantum Transformer that leverages holographic embedding and autoregressive attention to efficiently simulate frustrated quantum systems.
• It demonstrates high numerical precision on the square lattice J1-J2 Heisenberg antiferromagnet, matching state-of-the-art benchmarks with low energy variance.
• The model features zero-shot size extrapolation from 8x8 to 10x10 lattices, enabling rapid scalability and robust transfer of physical priors.

Holographic Quantum Transformer: Neuro-Symbolic Generative Attention for Frustrated Quantum Systems

Introduction

The exponential growth of Hilbert space in simulating quantum many-body systems, particularly frustrated spin models such as the square lattice $J_1$-$J_2$ Heisenberg antiferromagnet, presents significant challenges for both exact and stochastic numerical solvers. Traditional methodologies (ED, PEPS, QMC) confront hard computational barriers including the sign problem and area law constraints. Neural Quantum States (NQS), specifically deep neural architectures, have shown increased prowess in representing complex entangled states, yet limitations persist in their inductive biases, sampling efficiency, and scalability.

The Holographic Quantum Transformer (HQT) (2607.00398) presents an autoregressive, physics-aware generative model leveraging global attention for variational ground-state search in frustrated systems. The central innovation is the holographic encoding of geometry and interaction syntax, supporting both precise inference and scalable zero-shot transfer to larger lattice sizes via continuous positional embedding interpolation and policy head re-initialization.

Holographic Quantum Transformer Architecture

The HQT architecture is structured around three core modules: Holographic Embedding, Quantum Encoder with global self-attention, and an autoregressive head for wavefunction generation.

Figure 1: Schematic of HQT: (A) Holographic Embedding injects explicit lattice geometry via 2D positional encodings; (B) Quantum Encoder employs masked global attention to resolve long-range frustration; (C) Autoregressive Head outputs conditional probabilities for $\Psi(\mathbf{x})$.

Input spin configurations are mapped into continuous latent spaces augmented with 2D learnable positional embeddings, enforcing geometric locality critical for correct entanglement scaling. The Quantum Encoder’s multi-head causal self-attention, masked to enforce autoregressive normalization, acts as a trainable non-local correlator—essential for encoding both local ($J_1$) and frustrated, long-range ($J_2$) terms. The final output layer splits into amplitude and phase heads: the amplitude strictly parametrizes a normalized probability via the autoregressive chain rule, while the phase is initialized by the Marshall sign rule with neural residual correction.

Normalization and Sampling

By construction, the generative chain guarantees strict normalization ($\langle \Psi |\Psi\rangle=1$), allowing for i.i.d. perfect sampling. This yields zero autocorrelation in evaluation ($\tau=0$), outperforming traditional MCMC-VMC methods especially in highly frustrated regimes plagued by slow mixing.

Precision and Phase Resolution in the $J_1$-$J_2$ Model

Empirical evaluation on the $8\times8$ square lattice ($J_2$0) near maximal frustration ($J_2$1) demonstrates high numerical precision:

• HQT attains $J_2$2, matching finite-size scaling between exact diagonalization for $J_2$3 and state-of-the-art ResNets for $J_2$4 lattices.
• Energy variance remains low ($J_2$5), and effective sample size saturates ($J_2$6).
• The model identifies quantum criticality and order transitions without explicit order parameter supervision: the energy variance peaks at $J_2$7, coinciding with the crossing of N\'eel and Stripe structure factors.

Figure 2: (a) Energy per site and variance as a function of $J_2$8 reveal critical frustration and landscape ruggedness at $J_2$9. (b) Crossing of $\Psi(\mathbf{x})$0 (N\'eel) and $\Psi(\mathbf{x})$1 (Stripe) marks the quantum phase transition.

Zero-Shot Size Extrapolation with Holographic Transfer

A critical advancement is HQT’s zero-shot transfer protocol. By interpreting the Transformer’s backbone as encoding universal Hamiltonian syntax, the learned model on $\Psi(\mathbf{x})$2 is projected onto $\Psi(\mathbf{x})$3 by continuous interpolation of positional embeddings and re-initialization of the policy head, followed by rapid fine-tuning.

• Holographic transfer achieves immediate alignment with the physically relevant energy basin and converges to $\Psi(\mathbf{x})$4 after minimal adaptation.
• This result matches, within error bars, the best reported from-scratch variational benchmarks ($\Psi(\mathbf{x})$5).
• The projection protocol provides a structured and robust initialization, bypassing the costly optimization "burn-in" required by random initialization.

Figure 3: Holographic transfer (red) from $\Psi(\mathbf{x})$6 to $\Psi(\mathbf{x})$7 rapidly aligns and surpasses cold start (blue), matching the best reported from-scratch performance after policy head unfreezing.

Physical Interpretability via Attention Geometry

Analysis of learned attention patterns reveals that the model autonomously reconstructs the geometry of the frustrated interactions:

• The attention matrix displays off-diagonal dominance, demonstrating recognition of next-nearest-neighbor ($\Psi(\mathbf{x})$8) correlations.
• Intensity maxima in the attention map correspond spatially to the diagonal interaction partners, quantitatively supporting the hypothesis that the attention weights encode a holographic map of entanglement and interaction topology.

Figure 4: (a) Full $\Psi(\mathbf{x})$9 attention map exhibits off-diagonal stripes corresponding to $J_1$0 frustration. (b) Site-level analysis shows maximal attention assigned to diagonal next-nearest neighbors, validating geometric physical priors.

Theoretical and Practical Implications

HQT substantiates the generalization of generative attention mechanisms to quantum many-body wavefunctions, with several implications:

• The autoregressive, normalized-by-design framework obviates the sampling bottlenecks of MCMC in frustrated landscapes.
• Holographic projection opens a pathway to scalable neural solvers, where bulk physical laws (rather than specific microstates) are learned and transferred, thus supporting system size extrapolation for realistic models, a long-standing limitation for NQS approaches.
• Attention as entanglement mapping offers both interpretability and direct access to correlated physics, making it plausible to leverage HQT as an ansatz for near-term quantum devices (VQE), initializers for quantum hardware, and interpretable benchmarks for condensed matter theory.

Future extensions include explicit treatment of fermionic statistics, adaptation to time-dependent dynamics, and integration with efficient attention mechanisms (e.g., FlashAttention) to resolve quadratic complexity at scale.

Limitations

HQT’s limitations include $J_1$1 attention complexity, geometric prior dependence (requiring architectural adaptations for novel lattices), and potential breakdown in regimes with unknown, non-local sign structure, where presumed physical priors (like the Marshall rule) may fail.

Conclusion

The Holographic Quantum Transformer constitutes a robust, interpretable framework for high-precision quantum simulation in frustrated systems. By jointly leveraging geometric inductive bias, autoregressive normalization, and transformer-based global attention, HQT achieves precision, scalability, and interpretability not simultaneously realized in previous NQS architectures. Its success in zero-shot size extrapolation substantiates generative attention as a structurally transferable paradigm for many-body physics, suggesting potential cross-fertilization with quantum hardware, dynamical simulation, and exploration of exotic quantum phases.