CASK: A Gauge Covariant Transformer for Lattice Gauge Theory (2501.16955v1)

Abstract: We propose a Transformer neural network architecture specifically designed for lattice QCD, focusing on preserving the fundamental symmetries required in lattice gauge theory. The proposed architecture is gauge covariant/equivariant, ensuring it respects gauge symmetry on the lattice, and is also equivariant under spacetime symmetries such as rotations and translations on the lattice. A key feature of our approach lies in the attention matrix, which forms the core of the Transformer architecture. To preserve symmetries, we define the attention matrix using a Frobenius inner product between link variables and extended staples. This construction ensures that the attention matrix remains invariant under gauge transformations, thereby making the entire Transformer architecture covariant. We evaluated the performance of the gauge covariant Transformer in the context of self-learning HMC. Numerical experiments show that the proposed architecture achieves higher performance compared to the gauge covariant neural networks, demonstrating its potential to improve lattice QCD calculations.

• The paper introduces the CASK architecture, a gauge covariant Transformer that maintains essential lattice QCD symmetries during simulations.
• It redefines the attention mechanism using a Frobenius inner product between link variables and extended staples to ensure gauge invariance.
• Experimental results in SU(2) lattice gauge theory show improved simulation acceptance rates and enhanced modeling of complex gauge dynamics.

CASK: A Gauge Covariant Transformer for Lattice Gauge Theory

The paper "CASK: A Gauge Covariant Transformer for Lattice Gauge Theory" introduces an innovative Transformer architecture tailored for lattice Quantum Chromodynamics (QCD), ensuring compliance with the inherent symmetries of lattice gauge theories. This work emerges against the backdrop of increasing interest in leveraging machine learning techniques for computationally intensive physics tasks, such as those encountered in lattice QCD.

Motivation and Context

Traditional lattice QCD simulations can be highly resource-intensive, motivating the use of machine learning models to approximate computationally expensive parts of the simulation. Recently, flow-based and Transformer models have shown promise in this domain. However, maintaining gauge symmetry—a fundamental property of QCD—within these models has posed a significant challenge. The introduction of the gauge covariant Transformer architecture, dubbed CASK (Covariant Attention with Stout Kernel), addresses this challenge by ensuring symmetry properties are preserved.

Architecture and Methodology

At the core of CASK is the attention mechanism, a fundamental concept in Transformer models. The attention mechanism is redefined in terms of a Frobenius inner product between link variables and extended staples to maintain gauge covariance and equivariance under spacetime symmetries. This ensures that the attention matrix remains invariant under gauge transformations, a crucial requirement for physical authenticity in QCD simulations.

CASK is constructed upon a foundation of existing gauge covariant networks but integrates with the dynamic properties of Transformers. This integration allows CASK to capture non-local correlations effectively, a vital characteristic when dealing with fermion fields on a lattice. Moreover, the network supports differentiability, making it compatible with gradient-based learning methods commonly employed in machine learning.

Application and Results

The performance of the CASK architecture was evaluated in the context of self-learning Hybrid Monte Carlo (SLHMC) simulations. The study employed numerical experiments on SU(2) lattice gauge theory with dynamical fermions. Notably, CASK demonstrated an enhanced acceptance rate in these simulations compared to traditional gauge covariant neural networks, indicative of its superior expressive capabilities and potential for effectively modeling the intricate dynamics of gauge fields.

Implications and Future Directions

The research signifies a step forward in integrating machine learning with lattice gauge theories, opening up avenues for more efficient simulations that conserve physical symmetries. The implications of this work extend to various practical applications, including enhancing computational efficiency in large-scale QCD simulations, potentially reducing the barriers to exploring high-energy physics phenomena through lattice approaches.

Future work may involve scaling the CASK architecture to larger lattice volumes and refining the attention block to incorporate more complex loop structures. Further exploration into optimizing the training process of such models could also bolster their applicability across more complex systems within the lattice QCD framework.

In summary, this paper lays the groundwork for future innovations at the intersection of machine learning and lattice gauge theory, illustrating the feasibility and benefits of embedding physical constraints such as gauge covariance directly into the architecture of neural networks. The transition from gauge covariant neural networks to gauge covariant Transformers exemplifies the progress towards more expressive and computationally viable models for high-energy physics research.