Gauge-Covariant Markov Embedding

• Gauge-Covariant Markov Embedding is a framework where stochastic kernels transport features covariantly under local gauge symmetries.
• The approach employs gauge-invariant scalars for normalized weights and message passing to build discrete-time Markov evolutions on lattice features.
• It enables efficient gauge-consistent updates in models like GNNs, Transformers, and HMC proposals, validated on non-Abelian SU(2) and SU(3) lattices.

Gauge-Covariant Markov Embedding denotes a class of constructions in which a stochastic kernel, message-passing operator, or proposal mechanism acts on variables that transform covariantly under local gauge symmetry. In lattice gauge theory and symmetry-preserving machine learning, the common structure is to transport features with the lattice connection, combine them with normalized weights built from gauge-invariant scalars, and apply nonlinearities that preserve the prescribed representation law. This yields either a discrete-time Markov evolution on covariant features or a gauge-consistent proposal mechanism inside HMC and self-learning HMC, while maintaining exact local symmetry (Rayat et al., 22 Apr 2026, Nagai et al., 28 Jan 2025, Jin, 2024).

1. Local gauge symmetry, covariance, and transport

In the lattice formulation, one fixes a compact Lie group $G$ such as $SU(2)$ or $SU(3)$ and a $d$-dimensional lattice with sites $x$ and oriented nearest-neighbor links $(x,x+\hat\mu)$. The link variable $U_{x,\mu}\in G$ is the discrete parallel transporter from $x$ to $x+\hat\mu$. A local gauge transformation is a site-dependent choice $\{g_x\in G\}$. Under such a transformation, link variables obey the bi-fundamental rule

$SU(2)$0

Site features in a representation $SU(2)$1 transform as $SU(2)$2, while adjoint matrix-valued site features transform as $SU(2)$3. Oriented edge features satisfy

$SU(2)$4

These transformation laws ensure that observables built from closed products of links, together with contracted matter fields, remain gauge invariant (Rayat et al., 22 Apr 2026).

Gauge equivariance is the requirement that an operator $SU(2)$5 commute with the local action:

$SU(2)$6

For node covariants in the fundamental representation at site $SU(2)$7, this means $SU(2)$8; for adjoint node features, $SU(2)$9; for edge features on $SU(3)$0, $SU(3)$1. The central technical device is covariant transport. For a fundamental feature $SU(3)$2, the transported message

$SU(3)$3

picks up exactly the gauge factor at the destination. For an adjoint feature $SU(3)$4, the transported quantity

$SU(3)$5

transforms adjointly at $SU(3)$6 (Rayat et al., 22 Apr 2026).

The same transport law extends from one hop to arbitrary paths. For a path $SU(3)$7, the Wilson line is

$SU(3)$8

with transformation $SU(3)$9. When the path is closed, the Wilson loop

$d$0

is gauge invariant. This separation between covariant endpoint objects and invariant closed-loop observables is the structural basis of gauge-covariant embeddings (Rayat et al., 22 Apr 2026).

2. Message passing as a gauge-covariant Markov kernel

A concrete realization appears in gauge-equivariant graph neural networks for lattice gauge theories. The architecture operates on matrix-valued covariant features: node features $d$1, each an $d$2 matrix transforming adjointly, and edge features $d$3 transforming bi-fundamentally with $d$4. Initialization is

$d$5

while $d$6 are untraced Wilson loops built from nearby plaquettes adjacent to $d$7, with optional Polyakov loops anchored at $d$8 as additional gauge-covariant inputs (Rayat et al., 22 Apr 2026).

At layer $d$9, node aggregation is defined by

$x$0

where channel maps act on channels and matrix products act on gauge indices. Every term transforms adjointly at site $x$1. Channelwise normalization uses the gauge-invariant Frobenius norm,

$x$2

followed by an invariant gate,

$x$3

Because $x$4 transforms adjointly and $x$5 is invariant, the update is adjoint-equivariant. Edge updates are defined analogously through

$x$6

with

$x$7

which preserves bi-fundamental covariance (Rayat et al., 22 Apr 2026).

The Markov interpretation is explicit. For fundamental node features,

$x$8

with $x$9 and either column-stochastic or row-stochastic normalization depending on convention. A convenient construction is to compute unnormalized attention scores from gauge-invariant scalars and set

$(x,x+\hat\mu)$0

producing a row-stochastic kernel. Since $(x,x+\hat\mu)$1 is built from invariants, it does not transform, and therefore

$(x,x+\hat\mu)$2

For adjoint features,

$(x,x+\hat\mu)$3

which transforms as $(x,x+\hat\mu)$4. Stacking layers amounts to iterating $(x,x+\hat\mu)$5, and the paper identifies this construction as a Gauge-Covariant Markov Embedding: the layer implements a Markov kernel on the graph that transports features covariantly and aggregates with invariant, normalized weights (Rayat et al., 22 Apr 2026).

3. Locality, emergent nonlocality, and empirical scope

Although the operations are one-hop and strictly local, repeated covariant transport concatenates path-ordered products. After $(x,x+\hat\mu)$6 layers, compositions of $(x,x+\hat\mu)$7 or $(x,x+\hat\mu)$8 along multiple hops implicitly build Wilson lines of length up to $(x,x+\hat\mu)$9. Invariant readouts such as $U_{x,\mu}\in G$0 of latent loops therefore recover Wilson-loop information without explicit enumeration of all paths. This is the mechanism by which loop-like structures and nonlocal correlations emerge from local operations (Rayat et al., 22 Apr 2026).

Readout may return covariant outputs, such as $U_{x,\mu}\in G$1 or $U_{x,\mu}\in G$2, or invariant outputs formed from traces of node and edge covariants and of loops generated by message passing. Polyakov-loop invariants can be added as global features, after which a shared MLP produces local or global predictions. The construction is implemented and validated on non-Abelian $U_{x,\mu}\in G$3 and $U_{x,\mu}\in G$4, and the underlying operations are group-agnostic beyond matrix arithmetic (Rayat et al., 22 Apr 2026).

The reported validation covers pure gauge, gauge–matter, and dynamical settings:

Regime	Prediction target	Reported outcome
Pure gauge $U_{x,\mu}\in G$5, $U_{x,\mu}\in G$6D	$U_{x,\mu}\in G$7, local topological charge density $U_{x,\mu}\in G$8	MSE $U_{x,\mu}\in G$9, $x$0 for $x$1; MSE $x$2, $x$3 for $x$4
Gauge–matter $x$5 on $x$6 lattices with $x$7	total energy $x$8, local fermion density $x$9	$x+\hat\mu$0: $x+\hat\mu$1 MSE $x+\hat\mu$2, $x+\hat\mu$3; $x+\hat\mu$4 MSE $x+\hat\mu$5, $x+\hat\mu$6. $x+\hat\mu$7: $x+\hat\mu$8 MSE $x+\hat\mu$9, $\{g_x\in G\}$0; $\{g_x\in G\}$1 MSE $\{g_x\in G\}$2, $\{g_x\in G\}$3
Dynamical $\{g_x\in G\}$4 quantum link model (semiclassical, adiabatic)	$\{g_x\in G\}$5 and forces $\{g_x\in G\}$6	force MSE $\{g_x\in G\}$7, $\{g_x\in G\}$8; ML-driven dynamics accurately reproduces short-time Wilson-loop evolution and autocorrelation statistics

These results support the claim that exact site-wise symmetry handling and covariant transport are sufficient to learn both invariant observables and gauge-covariant responses from local updates alone (Rayat et al., 22 Apr 2026).

4. Transformer and HMC realizations

The same concept appears in Transformer-based and invertible-transformation-based constructions, but with a different operational meaning. In CASK, links are treated as tokens, and structured context is built from extended staples. The attention coefficient for each link and rectangle shape is

$\{g_x\in G\}$9

with

$SU(2)$00

Because the Frobenius inner product closes into rectangular Wilson loops, these attention scores are gauge invariant. The update is a covariant stout-type transformation,

$SU(2)$01

so the output transforms again as a link. In self-learning HMC, the exact accept/reject Hamiltonian is $SU(2)$02, while the MD proposal uses the effective action $SU(2)$03 with $SU(2)$04 and $SU(2)$05. On an $SU(2)$06 $SU(2)$07 lattice at $SU(2)$08, without training the acceptance is near zero due to the surrogate mismatch; with training, acceptance rises, CASK continues improving across epochs and surpasses a gauge-covariant neural-network baseline, and plaquette histograms remain consistent with expected physics (Nagai et al., 28 Jan 2025).

A related HMC realization uses a smooth, invertible, gauge-covariant neural field transformation. There the single-layer update is

$SU(2)$09

with bounded local coefficients

$SU(2)$10

The transformed action is

$SU(2)$11

and HMC is run in the transformed coordinates $SU(2)$12. The construction is gauge-covariant by design and is intended as a diffeomorphic reparameterization of the gauge-field manifold. On $SU(2)$13 $SU(2)$14 lattices, one Omelyan 2MN step was evaluated at $SU(2)$15; both GC and NN transformations produced smaller $SU(2)$16 and larger $SU(2)$17 at the same $SU(2)$18, and the effect persisted when models trained on $SU(2)$19 were applied to $SU(2)$20 (Jin, 2024).

A concise comparison is useful:

Realization	Embedded object	Markov role
Gauge-equivariant GNN	Covariant node and edge features	Row- or column-stochastic kernel on features
CASK	Gauge-invariant attention scalars and covariant links $SU(2)$21	Effective MD proposal inside SLHMC
Invertible gauge-field transform	Smooth gauge-covariant coordinates $SU(2)$22 with $SU(2)$23	HMC in transformed coordinates with $SU(2)$24

These realizations share the same principle: the Markov mechanism is built from quantities that either remain invariant or transform in a controlled representation under the local gauge group.

5. Abstract Markov-process gauge theory and the embedding viewpoint

The phrase also has a formal meaning in the theory of continuous-time Markov processes on denumerable state spaces. Let $SU(2)$25 be a column probability vector and $SU(2)$26 a generator satisfying

$SU(2)$27

with nonnegative off-diagonals and vanishing column sums. A time-dependent invertible linear map $SU(2)$28 acts on state weights by

$SU(2)$29

and the transformed generator is

$SU(2)$30

For diagonal positive $SU(2)$31, off-diagonals transform as $SU(2)$32 for $SU(2)$33, while the diagonal correction enforces stochasticity. Under full $SU(2)$34, any two differentiable generators on the same state space are equivalent through such a transformation; under the diagonal subgroup, invariants include the support of allowed transitions and products of rates along directed cycles (Caruso et al., 2016).

The embedding step consists of enlarging the state space by zero-padding the generator,

$SU(2)$35

and lifting the gauge transformation to

$SU(2)$36

With inclusion $SU(2)$37 and projection $SU(2)$38, the embedding is gauge-covariant in the precise sense

$SU(2)$39

This is the formal content of “gauge-covariant Markov embedding” in the Markov-process literature (Caruso et al., 2016).

This suggests that the contemporary lattice-gauge usage extends an older stochastic notion into a setting where the embedded variables are not probability vectors but gauge-covariant features, links, or transformed coordinates. The shared point is not the physical interpretation of the gauge group but the requirement that embedding, evolution, and projection commute with the relevant local symmetry.

6. Significance, misconceptions, and open problems

A recurrent misconception is that gauge covariance is interchangeable with gauge invariance. In these constructions, invariant scalars such as $SU(2)$40, $SU(2)$41, and Wilson loops serve as gates, weights, or readout features, but the internal states themselves may transform nontrivially as fundamental, adjoint, or bi-fundamental objects. Gauge-covariant embeddings therefore preserve symmetry without collapsing all intermediate information to invariants (Rayat et al., 22 Apr 2026).

A second misconception is that nonlocal observables require explicitly nonlocal operators. The graph-based construction shows the opposite: one-hop covariant transport, iterated across depth, naturally assembles long Wilson lines and loops. The Transformer variant reaches a similar goal by building attention from rectangular loop closures, while the HMC coordinate-transform approach uses local loop derivatives and Jacobian-controlled updates to improve proposal geometry. This suggests that locality of computation and nonlocality of represented physics are compatible when transport respects the connection (Rayat et al., 22 Apr 2026, Nagai et al., 28 Jan 2025, Jin, 2024).

The reported advantages are correspondingly specific. Gauge-equivariant message passing handles independent site-wise symmetry operations exactly; global-equivariant models cannot represent $SU(2)$42 that vary across sites. It also narrows the hypothesis space to symmetry-consistent functions, which the paper identifies as improving data efficiency and physical consistency. In proposal-based settings, strict covariance avoids gauge-choice-dependent drift in MD forces and preserves the legitimacy of the surrogate action (Rayat et al., 22 Apr 2026, Nagai et al., 28 Jan 2025).

Open problems are also explicit. The graph-based construction is discrete-time and layered; extending it to continuous-time equivariant flows or neural ODEs on groups is identified as an open direction. The same work notes that fermion-mediated observables are learned through gauge-covariant surrogates such as $SU(2)$43 rather than explicit Grassmann fields, and that non-contractible loops are not generated by purely local updates, so Polyakov loops are added as global inputs. CASK is currently reported for $SU(2)$44 on a small $SU(2)$45 volume, with sparse attention restricted to $SU(2)$46, $SU(2)$47, and $SU(2)$48 rectangles and no multihead attention. The invertible field-transformation approach is substantially more expensive than the bare action: on $SU(2)$49, force evaluation with GC is $SU(2)$50 s and about $SU(2)$51 GB, with NN $SU(2)$52 s and about $SU(2)$53 GB, versus less than $SU(2)$54 ms and about $SU(2)$55 MB for the baseline DBW2 force; on $SU(2)$56, GPU memory was exhausted in the reported implementation (Rayat et al., 22 Apr 2026, Nagai et al., 28 Jan 2025, Jin, 2024).

Taken together, these results define Gauge-Covariant Markov Embedding as a unifying pattern rather than a single architecture: a Markovian update or proposal is made symmetry-faithful by parallel transport with the gauge connection, by invariant normalization and weighting, and by representation-compatible nonlinear control. In graph neural networks this yields a stochastic kernel on covariant features; in Transformer-based SLHMC it yields a gauge-consistent effective proposal; in invertible field transformations it yields a covariant reparameterization of the gauge-field manifold.