Scaling Laws for Neural-Network Quantum States

Abstract: Scaling laws, the power-law relations between loss, architecture size, and compute observed in modern neural networks, offer a quantitative way to characterize the complexity of a learning problem, with the exponent governing the decay of the loss reflecting how rapidly additional resources translate into improved accuracy, and thus how hard the target is to learn. Whether an analogous framework can characterize the complexity of physical problems remains open. We address this question for Neural-Network Quantum States, a leading variational approach for strongly correlated quantum many-body systems. Using transformer wave functions to approximate ground states of the $J_1$-$J_2$ Heisenberg model on triangular and square lattices with up to $20\times 20$ sites, we find that the $V$-score, a measure of accuracy of a variational state, decays as a power law in training compute. Under an appropriate rescaling of compute, results for different system sizes collapse onto a single curve, analogous to scaling collapse in critical phenomena. The resulting power law is, to a good approximation, independent of the number of sites, showing that the transformer Ansatz is size-consistent for the systems considered. The exponent decreases systematically with frustration, identifying it as a quantitative measure of representational difficulty of the ground state and establishing scaling laws as a general framework for benchmarking variational ansätze.

• The paper establishes a power-law scaling of the V-score with training compute and system size across various quantum spin models, linking variational accuracy to computational resources.
• The transformer Ansatz proves size consistency as data from different lattice sizes collapse onto a universal curve, confirming the approach's predictive reliability.
• Architectural variations and differing levels of system frustration significantly affect the scaling exponent, providing quantitative benchmarks for variational representational complexity.

Scaling Laws for Neural-Network Quantum States

Introduction and Problem Setting

The development of scaling laws in deep learning has enabled predictive and diagnostic tools for understanding model performance with respect to model size, training data, and computational resources. This empirical framework, originally quantified in autoregressive transformers for language modeling, has been extended to modalities such as vision and reinforced in large-scale architectures. The analysis by Rende et al., "Scaling Laws for Neural-Network Quantum States" (2606.02794), investigates the extent to which analogous scaling behavior governs the representational power of neural-network quantum states (NQS) designed for many-body quantum systems.

The focus is on the transformer Ansatz as a scalable, size-consistent variational representation for quantum ground states, probing prototypical frustrated and unfrustrated spin Hamiltonians on two-dimensional lattices. The primary metric under consideration is the $V$-score, a normalized, variance-based measure which correlates strongly with variational energy errors. This work systematically studies the power-law relationship between $V$-score, training compute (measured in FLOPs), and system size for several model Hamiltonians of varying complexity.

Scaling Laws and Universality in Quantum Spin Models

A central result is the verification of a power-law scaling for the $V$-score as a function of training compute, with the exponent $\alpha$ encapsulating the efficiency with which variational accuracy improves with additional computational effort. The models studied include the unfrustrated square Heisenberg, frustrated square and triangular $J_1$-$J_2$ Heisenberg models, and the triangular Heisenberg model. For all cases, training transformer-based NQS on clusters up to $20\times 20$ sites, the $V$-score follows the scaling relation:

$V\text{-score} = A\, f^{-\alpha}\, N^{\beta}$

where $f$ is the cumulative number of FLOPs and $V$0 the number of sites.

Rescaling $V$1 by $V$2 enables data from different system sizes to collapse onto a single universal curve for each Hamiltonian, establishing that the transformer Ansatz is size consistent on these benchmarks. This is visually demonstrated for all four Hamiltonians:

Figure 1: Scaling collapse of the $V$3-score for the transformer wave function on four two-dimensional spin Hamiltonians, revealing universal behavior and a well-defined power-law exponent for each model.

The exponent $V$4 varies systematically with frustration: $V$5 for the unfrustrated square Heisenberg, dropping to $V$6 for the strongly frustrated triangular $V$7-$V$8 model, thus directly linking the physical representational complexity of the ground state to scaling-law behavior.

Interpretability, Size Consistency, and Architectural Robustness

The size consistency of the scaling law arises from the cost scaling of the transformer architecture. In the regime relevant to NQS (moderate sequence lengths), the linear term in sequence length dominates FLOPs count, leading to an approximate relation $V$9. Thus, $V$0-score is essentially independent of system size for fixed compute per site. This result implies that simulation outcomes for smaller lattices can be reliably extrapolated to much larger system sizes—a critical practical implication for computational studies of quantum materials.

Figure 2: Illustration of the scaling collapse procedure for the square $V$1-$V$2 Heisenberg model; rescaled data from varying system sizes collapse onto a single power-law curve.

The scaling exponent $V$3 is stable under variations of the neural network architecture, including changes in patch size and symmetry enforcement strategies.

Figure 3: Architectural variations (patch size, symmetry enforcement) do not alter the extracted scaling exponent $V$4, confirming robustness across key architectural choices.

Computational Regime and Limits of Scaling

Training compute is decomposed into FLOPs per forward pass, number of Monte Carlo samples per optimization step, and number of optimization steps. In the practical regime considered, the compute requirements scale linearly with system size, but as model size or depth increases, sampling and optimization limitations lead to the saturation of $V$5-score improvements, which bounds the validity of the scaling regime.

Figure 4: Computational cost in FLOPs as a function of input sequence length (patch numbers), exposing linear and quadratic regimes in the cost scaling.

Figure 5: Comparison of $V$6-score convergence for different sample sizes; saturation sets in at high model depth, indicating limits of scaling law applicability due to training and sampling effects.

Physical Implications and Outlook

The identification of a model-dependent, size-consistent scaling exponent offers a succinct, quantitative characterization of variational representational difficulty for ground states of quantum many-body systems. This exponent provides a metric for benchmarking neural variational ansätze across distinct Hamiltonians and for guiding architecture and resource allocation. The observed correlation between frustration and diminished $V$7 is consistent with increased physical complexity and challenges in representing critical and quantum spin liquid states.

Future work should:

• Relate the scaling exponent $V$8 to quantum state complexity measures (e.g., entanglement entropy, tensor network bond dimension, non-stabilizer monotones);
• Investigate the behavior of $V$9 near quantum critical points, hoping to develop new diagnostics for quantum phase transitions detectable via scaling law crossovers or anomalies;
• Extend analysis to fermionic and ab initio electronic Hamiltonians, further generalizing the size-consistent scaling framework.

Conclusion

This work demonstrates that scaling law phenomenology, a cornerstone of modern deep learning, governs the performance of transformer-based neural quantum states applied to complex quantum many-body systems. The practical and theoretical implications of universal, size-consistent scaling—from benchmarking variational accuracy to informing model selection and resource allocation—anchor the transformer Ansatz as a robust tool for studying strongly correlated quantum systems and provide new avenues for the quantitative classification of quantum state complexity.