Self-Attention Fermi Neural Network

• Self-Attention Fermi Neural Network is a determinant-based neural quantum state that leverages transformer-like self-attention to capture many-body electron correlations.
• It constructs configuration-dependent orbitals via multi-head self-attention while enforcing fermionic antisymmetry through determinant summation.
• Demonstrated in molecules, periodic solids, chiral superconductivity, and 2D electron gases, it achieves lower variational energies compared to traditional FermiNet architectures.

“Self-Attention Fermi Neural Network” is a broad category label for determinant-based fermionic neural-network wavefunctions in which electron or particle correlations are generated primarily by self-attention over particle embeddings, while fermionic antisymmetry is enforced by Slater determinants rather than by making the entire network antisymmetric directly. In the cited literature, the label is informal rather than universal: the molecular architecture is named Psiformer (Glehn et al., 2022), the periodic-solid adaptation is Psi-Solid (Geier et al., 7 Feb 2025), one study explicitly uses the phrase self-attention Fermi neural network for chiral superconductivity (Li et al., 3 Sep 2025), and a later large-scale 2D electron-gas study presents an attention-based determinant ansatz of the same general type (Gaggioli et al., 9 Jul 2026). Across these works, the recurring theme is that self-attention serves as the correlation engine inside a variational Monte Carlo wavefunction for continuum fermions.

1. Definition and formal structure

In this research line, the many-body wavefunction is written as a sum of determinants of configuration-dependent orbitals. The common structural idea is that the orbital assigned to particle $j$ is not a fixed one-body orbital $\phi_\mu(\mathbf x_j)$, but a generalized orbital depending on the full configuration, such as $\Phi^k_\mu(\mathbf x_j;\{\mathbf x_{/j}\})$ or $\phi_i^{(k)}(\mathbf r_j,\{\mathbf r_{\neq j}\})$ (Li et al., 3 Sep 2025, Gaggioli et al., 9 Jul 2026). This places the ansatz in the same broad determinant-based family as FermiNet and related neural quantum states, but with self-attention replacing the more hand-structured interaction backbone (Glehn et al., 2022, Geier et al., 7 Feb 2025).

A representative form appears in several papers: $$\Psi(\mathbf X)=\sum_{k=1}^{N_{\rm det}} \det\!\bigl[\Phi^k_\mu(\mathbf x_j;\{\mathbf x_{/j}\})\bigr],$$ with variants that optionally include a Jastrow factor,

$$\Psi_\theta(x)=\exp\!\big(\mathcal J_\theta(x)\big)\sum_{k=1}^{N_{\mathrm{det}}}\det\!\big[\mathbf \Phi_\theta^k(x)\big]$$

(Glehn et al., 2022, Li et al., 3 Sep 2025, Teng et al., 2024). Psi-Solid uses a determinant sum without a Jastrow factor in its final reported architecture, after the authors found little gain for their moiré systems (Geier et al., 7 Feb 2025). The large-scale 2D homogeneous electron gas study likewise writes the wavefunction directly as a determinant sum and explicitly notes that it is not built from a prescribed orbital basis (Gaggioli et al., 9 Jul 2026).

The symmetry logic is consistent across the fermionic papers. The neural network mapping from the set of particle coordinates to per-particle latent states is permutation equivariant, because identical transformations are applied to every particle and attention acts over the set of particles (Geier et al., 7 Feb 2025, Li et al., 3 Sep 2025). Antisymmetry is then imposed by the determinant. This is the same conceptual strategy emphasized in comparisons with FermiNet and PsiFormer: equivariant orbitals first, determinant antisymmetrization second (Geier et al., 7 Feb 2025).

The phrase itself should be used carefully. Psi-Solid is described as “very close in spirit to what one might informally call a ‘self-attention Fermi neural network,’” but that paper does not introduce the phrase as the formal model name (Geier et al., 7 Feb 2025). By contrast, the chiral-superconductivity paper explicitly frames its architecture as “a general-purpose self-attention Fermi neural network” (Li et al., 3 Sep 2025). A plausible editorial interpretation is that the term denotes a family of transformer-like fermionic neural wavefunctions rather than a single canonical architecture.

2. Architecture and self-attention as a correlation mechanism

The architectural core is multi-head self-attention over particle tokens. In Psiformer, each electron is represented by a hidden state $\mathbf h_i^\ell$, and each layer applies multi-head self-attention followed by a residual $\tanh$-MLP update: $$\mathbf f_i^{\ell+1}=\mathbf h_i^\ell+\mathbf W_o^\ell \,\mathrm{concat}_h\!\Big[\mathrm{SelfAttn}_i(\mathbf h_1^\ell,\dots,\mathbf h_N^\ell)\Big],$$

$$\mathbf h_i^{\ell+1}=\mathbf f_i^{\ell+1}+\tanh\!\left(\mathbf W^{\ell+1}\mathbf f_i^{\ell+1}+\mathbf b^{\ell+1}\right)$$

(Glehn et al., 2022). The attention is standard scaled dot-product attention over electrons, with learned query, key, and value projections (Glehn et al., 2022). The chiral-superconductivity architecture uses the same pattern on periodic coordinate embeddings of particles in continuous space (Li et al., 3 Sep 2025).

Psi-Solid adapts this design to periodic solids. Raw coordinates are first mapped to periodic features,

$\phi_\mu(\mathbf x_j)$0

with $\phi_\mu(\mathbf x_j)$1, and self-attention is then inserted between residual per-electron blocks (Geier et al., 7 Feb 2025). Its particlewise attention rule is written directly as

$\phi_\mu(\mathbf x_j)$2

with a nonstandard normalization $\phi_\mu(\mathbf x_j)$3 (Geier et al., 7 Feb 2025). The authors explicitly interpret the resulting correlated orbitals $\phi_\mu(\mathbf x_j)$4 as backflow-like, because the orbital assigned to one electron depends on the coordinates of all the others (Geier et al., 7 Feb 2025).

The FQH study also adapts Psiformer into a complex-valued fermionic ansatz. Each spin-polarized electron is treated as a token with coordinate-derived features, attention generates many-body orbitals, and real and imaginary parts are created in separate streams before determinant evaluation (Teng et al., 2024). The chiral-superconductivity paper follows the same basic template: periodic coordinate embedding, $\phi_\mu(\mathbf x_j)$5 layers of multi-head self-attention and MLP, projection to complex $\phi_\mu(\mathbf x_j)$6 orbital matrices, and determinant summation (Li et al., 3 Sep 2025). In that work, the hyperparameter table lists $\phi_\mu(\mathbf x_j)$7 network layers, $\phi_\mu(\mathbf x_j)$8 attention heads per layer, attention dimensions $\phi_\mu(\mathbf x_j)$9 for queries/keys and values, perceptron dimension $\Phi^k_\mu(\mathbf x_j;\{\mathbf x_{/j}\})$0, one perceptron per layer, and $\Phi^k_\mu(\mathbf x_j;\{\mathbf x_{/j}\})$1 determinants (Li et al., 3 Sep 2025).

The 2D homogeneous electron-gas paper introduces an additional physically motivated modification: spatially weighted particle attention,

$\Phi^k_\mu(\mathbf x_j;\{\mathbf x_{/j}\})$2

where $\Phi^k_\mu(\mathbf x_j;\{\mathbf x_{/j}\})$3 is the interparticle distance and $\Phi^k_\mu(\mathbf x_j;\{\mathbf x_{/j}\})$4 is a trainable length scale initialized from the average interparticle distance (Gaggioli et al., 9 Jul 2026). This preserves dense all-to-all attention while biasing attention toward locality. The paper presents the model as a stack of self-attention and perceptron layers, repeated $\Phi^k_\mu(\mathbf x_j;\{\mathbf x_{/j}\})$5 times, followed by projection into generalized orbitals and determinant evaluation (Gaggioli et al., 9 Jul 2026).

A recurrent architectural consequence is that self-attention functions as a learned many-body backflow or interaction-gating mechanism. Psiformer motivates attention by arguing that prior FermiNet- and PauliNet-style models “did not include an attention-like mechanism for gating interactions between electrons” (Glehn et al., 2022). Psi-Solid makes the stronger claim that electron correlation is produced entirely by self-attention in the core model (Geier et al., 7 Feb 2025). This suggests that, within this family, self-attention is treated not as an auxiliary feature mixer but as the main representation of fermionic correlation.

3. Variational Monte Carlo formulation and optimization

All of the fermionic self-attention wavefunctions in the cited literature are trained within variational Monte Carlo. The objective is the variational energy

$\Phi^k_\mu(\mathbf x_j;\{\mathbf x_{/j}\})$6

estimated by Monte Carlo sampling from $\Phi^k_\mu(\mathbf x_j;\{\mathbf x_{/j}\})$7 using the local energy

$\Phi^k_\mu(\mathbf x_j;\{\mathbf x_{/j}\})$8

(Geier et al., 7 Feb 2025, Glehn et al., 2022, Li et al., 3 Sep 2025, Teng et al., 2024). Sampling is performed with Metropolis–Hastings in the cited studies (Geier et al., 7 Feb 2025, Glehn et al., 2022, Li et al., 3 Sep 2025).

A common optimization strategy is natural-gradient descent or stochastic reconfiguration approximated by KFAC. Psi-Solid writes the wavefunction-space metric as the quantum geometric tensor

$\Phi^k_\mu(\mathbf x_j;\{\mathbf x_{/j}\})$9

and uses KFAC because exact inversion is too expensive (Geier et al., 7 Feb 2025). It notes an important caveat: the current KFAC implementation acts only on the wavefunction magnitude and neglects the phase, effectively reducing the metric to the Fisher information matrix (Geier et al., 7 Feb 2025). The chiral-superconductivity paper gives the analogous stochastic-reconfiguration matrix

$\phi_i^{(k)}(\mathbf r_j,\{\mathbf r_{\neq j}\})$0

and then uses KFAC as an efficient approximation (Li et al., 3 Sep 2025).

The training setups are concrete and differ by application. Psiformer uses KFAC for main training, LAMB for pretraining to Hartree–Fock orbitals computed by PySCF, $\phi_i^{(k)}(\mathbf r_j,\{\mathbf r_{\neq j}\})$1 optimization iterations, batch size $\phi_i^{(k)}(\mathbf r_j,\{\mathbf r_{\neq j}\})$2, $\phi_i^{(k)}(\mathbf r_j,\{\mathbf r_{\neq j}\})$3 MCMC decorrelation steps, KFAC norm constraint $\phi_i^{(k)}(\mathbf r_j,\{\mathbf r_{\neq j}\})$4, damping $\phi_i^{(k)}(\mathbf r_j,\{\mathbf r_{\neq j}\})$5, local-energy clipping parameter $\phi_i^{(k)}(\mathbf r_j,\{\mathbf r_{\neq j}\})$6, and learning rate

$\phi_i^{(k)}(\mathbf r_j,\{\mathbf r_{\neq j}\})$7

(Glehn et al., 2022). Psi-Solid uses $\phi_i^{(k)}(\mathbf r_j,\{\mathbf r_{\neq j}\})$8 training iterations, $\phi_i^{(k)}(\mathbf r_j,\{\mathbf r_{\neq j}\})$9 network layers, $$\Psi(\mathbf X)=\sum_{k=1}^{N_{\rm det}} \det\!\bigl[\Phi^k_\mu(\mathbf x_j;\{\mathbf x_{/j}\})\bigr],$$0 attention heads per layer, attention dimension $$\Psi(\mathbf X)=\sum_{k=1}^{N_{\rm det}} \det\!\bigl[\Phi^k_\mu(\mathbf x_j;\{\mathbf x_{/j}\})\bigr],$$1, perceptron dimension $$\Psi(\mathbf X)=\sum_{k=1}^{N_{\rm det}} \det\!\bigl[\Phi^k_\mu(\mathbf x_j;\{\mathbf x_{/j}\})\bigr],$$2, $$\Psi(\mathbf X)=\sum_{k=1}^{N_{\rm det}} \det\!\bigl[\Phi^k_\mu(\mathbf x_j;\{\mathbf x_{/j}\})\bigr],$$3, batch size $$\Psi(\mathbf X)=\sum_{k=1}^{N_{\rm det}} \det\!\bigl[\Phi^k_\mu(\mathbf x_j;\{\mathbf x_{/j}\})\bigr],$$4, KFAC norm constraint $$\Psi(\mathbf X)=\sum_{k=1}^{N_{\rm det}} \det\!\bigl[\Phi^k_\mu(\mathbf x_j;\{\mathbf x_{/j}\})\bigr],$$5, damping $$\Psi(\mathbf X)=\sum_{k=1}^{N_{\rm det}} \det\!\bigl[\Phi^k_\mu(\mathbf x_j;\{\mathbf x_{/j}\})\bigr],$$6, local-energy clipping $$\Psi(\mathbf X)=\sum_{k=1}^{N_{\rm det}} \det\!\bigl[\Phi^k_\mu(\mathbf x_j;\{\mathbf x_{/j}\})\bigr],$$7, and

$$\Psi(\mathbf X)=\sum_{k=1}^{N_{\rm det}} \det\!\bigl[\Phi^k_\mu(\mathbf x_j;\{\mathbf x_{/j}\})\bigr],$$8

(Geier et al., 7 Feb 2025). The chiral-superconductivity study uses random initialization with no pretraining, batch size $$\Psi(\mathbf X)=\sum_{k=1}^{N_{\rm det}} \det\!\bigl[\Phi^k_\mu(\mathbf x_j;\{\mathbf x_{/j}\})\bigr],$$9, burn-in steps $$\Psi_\theta(x)=\exp\!\big(\mathcal J_\theta(x)\big)\sum_{k=1}^{N_{\mathrm{det}}}\det\!\big[\mathbf \Phi_\theta^k(x)\big]$$0, sample move width $$\Psi_\theta(x)=\exp\!\big(\mathcal J_\theta(x)\big)\sum_{k=1}^{N_{\mathrm{det}}}\det\!\big[\mathbf \Phi_\theta^k(x)\big]$$1, $$\Psi_\theta(x)=\exp\!\big(\mathcal J_\theta(x)\big)\sum_{k=1}^{N_{\mathrm{det}}}\det\!\big[\mathbf \Phi_\theta^k(x)\big]$$2 training iterations, initial learning rate $$\Psi_\theta(x)=\exp\!\big(\mathcal J_\theta(x)\big)\sum_{k=1}^{N_{\mathrm{det}}}\det\!\big[\mathbf \Phi_\theta^k(x)\big]$$3, delay $$\Psi_\theta(x)=\exp\!\big(\mathcal J_\theta(x)\big)\sum_{k=1}^{N_{\mathrm{det}}}\det\!\big[\mathbf \Phi_\theta^k(x)\big]$$4, local-energy clipping $$\Psi_\theta(x)=\exp\!\big(\mathcal J_\theta(x)\big)\sum_{k=1}^{N_{\mathrm{det}}}\det\!\big[\mathbf \Phi_\theta^k(x)\big]$$5, KFAC norm constraint $$\Psi_\theta(x)=\exp\!\big(\mathcal J_\theta(x)\big)\sum_{k=1}^{N_{\mathrm{det}}}\det\!\big[\mathbf \Phi_\theta^k(x)\big]$$6, and KFAC damping $$\Psi_\theta(x)=\exp\!\big(\mathcal J_\theta(x)\big)\sum_{k=1}^{N_{\mathrm{det}}}\det\!\big[\mathbf \Phi_\theta^k(x)\big]$$7 (Li et al., 3 Sep 2025).

The role of auxiliary physical factors differs across works. Psiformer includes an explicit electron-electron Jastrow because its attention network does not receive electron-electron distances directly, and ablations show that removing the Jastrow makes energies very noisy (Glehn et al., 2022). The FQH paper also includes a Jastrow factor to reproduce the Coulomb cusp and short-distance nonanalyticity beyond lowest-Landau-level holomorphic forms (Teng et al., 2024). By contrast, Psi-Solid reports that a simple cusp-enforcing Jastrow gave no significant improvement in the spin-polarized moiré systems while increasing GPU time per step by about $$\Psi_\theta(x)=\exp\!\big(\mathcal J_\theta(x)\big)\sum_{k=1}^{N_{\mathrm{det}}}\det\!\big[\mathbf \Phi_\theta^k(x)\big]$$8–$$\Psi_\theta(x)=\exp\!\big(\mathcal J_\theta(x)\big)\sum_{k=1}^{N_{\mathrm{det}}}\det\!\big[\mathbf \Phi_\theta^k(x)\big]$$9, and therefore omits it in the final architecture (Geier et al., 7 Feb 2025).

4. Major realizations and physical applications

The earliest direct realization in the supplied corpus is Psiformer, introduced for ab initio quantum chemistry as a self-attention-based drop-in replacement for the neural-network component of FermiNet- and PauliNet-style wavefunctions (Glehn et al., 2022). It retains determinant antisymmetrization, envelope functions, and a Jastrow factor, but replaces the FermiNet interaction backbone with a self-attention network over electrons (Glehn et al., 2022). The paper emphasizes that the gains become larger on larger molecules, with improvements over FermiNet by dozens of kcal/mol and up to about $\mathbf h_i^\ell$0 mHa $\mathbf h_i^\ell$1 kcal/mol on CCl$\mathbf h_i^\ell$2 (Glehn et al., 2022).

A second realization appears in fractional quantum Hall physics, where an attention-based fermionic neural network adapts Psiformer to $\mathbf h_i^\ell$3 spin-polarized electrons in 2D under magnetic field, working directly with the full Hilbert space in disk geometry (Teng et al., 2024). There the architecture is a complex, determinant-based real-space ansatz with self-attention-generated many-body orbitals and a Jastrow factor for cusp physics (Teng et al., 2024). The paper reports energies consistently lower than lowest-Landau-level projected exact diagonalization and uses the learned wavefunction to visualize microscopic zero structures, Landau-level mixing effects, and a transition from FQH liquid to crystal state at strong LL mixing (Teng et al., 2024).

Psi-Solid extends the approach to periodic continuum solids, specifically a continuum model of electrons in the WSe$\mathbf h_i^\ell$4/WS$\mathbf h_i^\ell$5 moiré semiconductor heterobilayer (Geier et al., 7 Feb 2025). The Hamiltonian is

$\mathbf h_i^\ell$6

with periodic boundary conditions and Ewald-summed Coulomb interactions (Geier et al., 7 Feb 2025). The benchmark systems are $\mathbf h_i^\ell$7-, $\mathbf h_i^\ell$8-, and $\mathbf h_i^\ell$9-site supercells at filling $\tanh$0, corresponding to $\tanh$1, $\tanh$2, and $\tanh$3 spin-polarized electrons (Geier et al., 7 Feb 2025). The work frames Psi-Solid as a self-attention NN wavefunction inspired by PsiFormer rather than as a formal FermiNet variant (Geier et al., 7 Feb 2025).

The chiral-superconductivity study applies a general-purpose self-attention Fermi neural network to a spin-polarized two-dimensional attractive Fermi gas with periodic boundary conditions and Gaussian attraction,

$\tanh$4

(Li et al., 3 Sep 2025). The paper’s central methodological point is that the architecture has no explicit pairing structure, unlike Pfaffian, geminal, or BCS-like forms, and therefore asks whether a general-purpose antisymmetric network can discover chiral superconductivity by energy minimization alone (Li et al., 3 Sep 2025).

The most recent large-scale realization in the supplied material studies the fully spin-polarized 2D homogeneous electron gas up to $\tanh$5 particles in a periodic triangular cell with Ewald interaction (Gaggioli et al., 9 Jul 2026). The paper presents the model as an alternative to canonical FermiNet-style architectures, still determinant-based but driven by self-attention and not by a prescribed orbital basis or backflow on fixed plane-wave orbitals (Gaggioli et al., 9 Jul 2026).

5. Empirical behavior, observables, and scaling claims

The empirical record across these papers is centered on lower variational energies and on direct access to wavefunction-dependent observables.

In quantum chemistry, Psiformer consistently outperforms FermiNet on the standard small-molecule benchmarks from the original FermiNet paper (Glehn et al., 2022). Representative energies from the appendix include ozone,

$\tanh$6

and bicyclobutane,

$\tanh$7

(Glehn et al., 2022). On larger molecules the reported differences are larger: for benzene, Psiformer gives $\tanh$8 Ha versus $\tanh$9 Ha for FermiNet; for naphthalene, $$\mathbf f_i^{\ell+1}=\mathbf h_i^\ell+\mathbf W_o^\ell \,\mathrm{concat}_h\!\Big[\mathrm{SelfAttn}_i(\mathbf h_1^\ell,\dots,\mathbf h_N^\ell)\Big],$$0 Ha versus $$\mathbf f_i^{\ell+1}=\mathbf h_i^\ell+\mathbf W_o^\ell \,\mathrm{concat}_h\!\Big[\mathrm{SelfAttn}_i(\mathbf h_1^\ell,\dots,\mathbf h_N^\ell)\Big],$$1 Ha; and for CCl$$\mathbf f_i^{\ell+1}=\mathbf h_i^\ell+\mathbf W_o^\ell \,\mathrm{concat}_h\!\Big[\mathrm{SelfAttn}_i(\mathbf h_1^\ell,\dots,\mathbf h_N^\ell)\Big],$$2, $$\mathbf f_i^{\ell+1}=\mathbf h_i^\ell+\mathbf W_o^\ell \,\mathrm{concat}_h\!\Big[\mathrm{SelfAttn}_i(\mathbf h_1^\ell,\dots,\mathbf h_N^\ell)\Big],$$3 Ha versus $$\mathbf f_i^{\ell+1}=\mathbf h_i^\ell+\mathbf W_o^\ell \,\mathrm{concat}_h\!\Big[\mathrm{SelfAttn}_i(\mathbf h_1^\ell,\dots,\mathbf h_N^\ell)\Big],$$4 Ha, a gain of about $$\mathbf f_i^{\ell+1}=\mathbf h_i^\ell+\mathbf W_o^\ell \,\mathrm{concat}_h\!\Big[\mathrm{SelfAttn}_i(\mathbf h_1^\ell,\dots,\mathbf h_N^\ell)\Big],$$5 mHa (Glehn et al., 2022). The same paper notes that absolute-energy improvements do not automatically imply uniformly improved relative quantities such as binding energies or ionization potentials (Glehn et al., 2022).

In periodic moiré solids, Psi-Solid outperforms both self-consistent Hartree–Fock and band-projected exact diagonalization in the tested systems (Geier et al., 7 Feb 2025). For the $$\mathbf f_i^{\ell+1}=\mathbf h_i^\ell+\mathbf W_o^\ell \,\mathrm{concat}_h\!\Big[\mathrm{SelfAttn}_i(\mathbf h_1^\ell,\dots,\mathbf h_N^\ell)\Big],$$6-site system at $$\mathbf f_i^{\ell+1}=\mathbf h_i^\ell+\mathbf W_o^\ell \,\mathrm{concat}_h\!\Big[\mathrm{SelfAttn}_i(\mathbf h_1^\ell,\dots,\mathbf h_N^\ell)\Big],$$7 and $$\mathbf f_i^{\ell+1}=\mathbf h_i^\ell+\mathbf W_o^\ell \,\mathrm{concat}_h\!\Big[\mathrm{SelfAttn}_i(\mathbf h_1^\ell,\dots,\mathbf h_N^\ell)\Big],$$8, the reported total ground-state energies are

$$\mathbf f_i^{\ell+1}=\mathbf h_i^\ell+\mathbf W_o^\ell \,\mathrm{concat}_h\!\Big[\mathrm{SelfAttn}_i(\mathbf h_1^\ell,\dots,\mathbf h_N^\ell)\Big],$$9

For $$\mathbf h_i^{\ell+1}=\mathbf f_i^{\ell+1}+\tanh\!\left(\mathbf W^{\ell+1}\mathbf f_i^{\ell+1}+\mathbf b^{\ell+1}\right)$$0,

$$\mathbf h_i^{\ell+1}=\mathbf f_i^{\ell+1}+\tanh\!\left(\mathbf W^{\ell+1}\mathbf f_i^{\ell+1}+\mathbf b^{\ell+1}\right)$$1

For this larger system, BP-ED is restricted to a single band, and the neural ansatz is about $$\mathbf h_i^{\ell+1}=\mathbf f_i^{\ell+1}+\tanh\!\left(\mathbf W^{\ell+1}\mathbf f_i^{\ell+1}+\mathbf b^{\ell+1}\right)$$2 lower in energy than BP-ED (Geier et al., 7 Feb 2025). The same optimized wavefunctions yield charge density and pair correlation functions showing a Fermi-liquid-like state at weaker interactions and a generalized Wigner crystal at stronger interactions (Geier et al., 7 Feb 2025).

In chiral superconductivity, the evidence is not only variational. The paper defines the pair-binding energy

$$\mathbf h_i^{\ell+1}=\mathbf f_i^{\ell+1}+\tanh\!\left(\mathbf W^{\ell+1}\mathbf f_i^{\ell+1}+\mathbf b^{\ell+1}\right)$$3

and reports the reversed odd-even pattern expected for a topological spinless $$\mathbf h_i^{\ell+1}=\mathbf f_i^{\ell+1}+\tanh\!\left(\mathbf W^{\ell+1}\mathbf f_i^{\ell+1}+\mathbf b^{\ell+1}\right)$$4 superconductor for $$\mathbf h_i^{\ell+1}=\mathbf f_i^{\ell+1}+\tanh\!\left(\mathbf W^{\ell+1}\mathbf f_i^{\ell+1}+\mathbf b^{\ell+1}\right)$$5: $$\mathbf h_i^{\ell+1}=\mathbf f_i^{\ell+1}+\tanh\!\left(\mathbf W^{\ell+1}\mathbf f_i^{\ell+1}+\mathbf b^{\ell+1}\right)$$6 for odd $$\mathbf h_i^{\ell+1}=\mathbf f_i^{\ell+1}+\tanh\!\left(\mathbf W^{\ell+1}\mathbf f_i^{\ell+1}+\mathbf b^{\ell+1}\right)$$7 and $$\mathbf h_i^{\ell+1}=\mathbf f_i^{\ell+1}+\tanh\!\left(\mathbf W^{\ell+1}\mathbf f_i^{\ell+1}+\mathbf b^{\ell+1}\right)$$8 for even $$\mathbf h_i^{\ell+1}=\mathbf f_i^{\ell+1}+\tanh\!\left(\mathbf W^{\ell+1}\mathbf f_i^{\ell+1}+\mathbf b^{\ell+1}\right)$$9 (Li et al., 3 Sep 2025). It further performs $\phi_\mu(\mathbf x_j)$00 projection,

$\phi_\mu(\mathbf x_j)$01

finding the lowest-energy sector consistent with

$\phi_\mu(\mathbf x_j)$02

(Li et al., 3 Sep 2025). The strongest diagnostic is the two-body reduced density matrix, whose leading detached eigenvalue indicates ODLRO and whose leading eigenvector has a $\phi_\mu(\mathbf x_j)$03 phase winding around the origin, directly identifying chiral $\phi_\mu(\mathbf x_j)$04 symmetry (Li et al., 3 Sep 2025).

In the 2D homogeneous electron gas, the large-scale self-attention wavefunction reaches energies below state-of-the-art fixed-node DMC based on optimized Slater–Jastrow–backflow trial states (Gaggioli et al., 9 Jul 2026). For $\phi_\mu(\mathbf x_j)$05, the reported energies are

$\phi_\mu(\mathbf x_j)$06

For $\phi_\mu(\mathbf x_j)$07,

$\phi_\mu(\mathbf x_j)$08

and for $\phi_\mu(\mathbf x_j)$09,

$\phi_\mu(\mathbf x_j)$10

(Gaggioli et al., 9 Jul 2026). The paper quantifies the gain over SJB-VMC in recovered correlation energy as $\phi_\mu(\mathbf x_j)$11 for $\phi_\mu(\mathbf x_j)$12, $\phi_\mu(\mathbf x_j)$13 for $\phi_\mu(\mathbf x_j)$14, and $\phi_\mu(\mathbf x_j)$15 for $\phi_\mu(\mathbf x_j)$16, emphasizing that the gain does not deteriorate with increasing system size (Gaggioli et al., 9 Jul 2026). It also computes

$\phi_\mu(\mathbf x_j)$17

and reconstructs a Feynman upper bound

$\phi_\mu(\mathbf x_j)$18

recovering the small-$\phi_\mu(\mathbf x_j)$19 plasmon branch and a roton-like minimum near $\phi_\mu(\mathbf x_j)$20 (Gaggioli et al., 9 Jul 2026).

The most explicit scaling claim in this corpus comes from Psi-Solid. It does not claim that training cost scales as $\phi_\mu(\mathbf x_j)$21; rather, it defines a saturation threshold $\phi_\mu(\mathbf x_j)$22 for the smallest parameter count beyond which converged energies remain within one standard deviation of the best energy found, and fits

$\phi_\mu(\mathbf x_j)$23

across $\phi_\mu(\mathbf x_j)$24-, $\phi_\mu(\mathbf x_j)$25-, and $\phi_\mu(\mathbf x_j)$26-site systems at $\phi_\mu(\mathbf x_j)$27 (Geier et al., 7 Feb 2025). The same study cautions that parameter count alone is not enough, since roughly $\phi_\mu(\mathbf x_j)$28–$\phi_\mu(\mathbf x_j)$29 attention layers and a sufficient number of heads were necessary to reach the best energies (Geier et al., 7 Feb 2025). A plausible implication is that, within the reported architecture family, depth and head count are part of the effective scaling law, not merely width.

6. Relation to FermiNet, terminology, and limitations

Self-attention Fermi neural networks are best understood as transformer-like alternatives to FermiNet-family architectures rather than as a complete break with determinant-based neural quantum states. The commonality with FermiNet is determinant antisymmetrization, many-body-dependent orbitals, and VMC optimization (Glehn et al., 2022, Geier et al., 7 Feb 2025, Gaggioli et al., 9 Jul 2026). The architectural difference is that FermiNet traditionally uses specially designed one-electron and two-electron streams with permutation-equivariant mixing, whereas Psiformer and Psi-Solid use self-attention as the central correlation mechanism (Glehn et al., 2022, Geier et al., 7 Feb 2025). The 2020 FermiNet engineering paper is therefore an important baseline for accuracy, cost, determinant structure, and envelope design, but it does not introduce self-attention (Spencer et al., 2020).

The limitations are also consistent across the literature. Attention is dense and remains $\phi_\mu(\mathbf x_j)$30 in particle number at the attention-map level in naive implementations, while determinant evaluation remains a major cost (Geier et al., 7 Feb 2025, Gaggioli et al., 9 Jul 2026). The large-scale HEG paper explicitly frames scalability as a central question because self-attention wavefunctions have increased computational complexity relative to traditional variational ansätze (Gaggioli et al., 9 Jul 2026). Psi-Solid does not present a formal operation-count analysis and emphasizes that its favorable $\phi_\mu(\mathbf x_j)$31 law concerns required parameter count for energy saturation, not runtime or memory (Geier et al., 7 Feb 2025). Psiformer also notes that attention has the usual quadratic dependence on number of tokens within a layer, though overall VMC cost involves more than just attention (Glehn et al., 2022).

The physical scope of published demonstrations remains narrower than the phrase might suggest. Psi-Solid treats a specific continuum moiré model with spin-polarized electrons and relatively simple band structure rather than generic ab initio solids (Geier et al., 7 Feb 2025). The chiral-superconductivity work studies a spinless attractive Fermi gas with periodic boundary conditions (Li et al., 3 Sep 2025). The large-scale HEG study is fully spin-polarized (Gaggioli et al., 9 Jul 2026). Psiformer is molecular and not periodic (Glehn et al., 2022). This suggests that the phrase denotes a methodological class with successful demonstrations across multiple fermionic domains, but not yet a single universal architecture validated on all realistic multiorbital, spinful, material-specific Hamiltonians.

A related terminological confusion arises with non-fermionic attention models inspired by physics. “Fractional neural attention” replaces dot-product self-attention by a fractional-diffusion kernel and is explicitly not a fermionic or Fermi–Dirac attention model (Qu et al., 13 Nov 2025). Its relevance to self-attention Fermi neural networks is only conceptual, in the sense that both are physics-inspired modifications of attention (Qu et al., 13 Nov 2025).

Taken together, the literature supports a precise characterization. A self-attention Fermi neural network is a determinant-based neural quantum state for fermions in which particles are treated as tokens, correlations are learned through self-attention across particles, permutation equivariance is preserved through the internal network, and antisymmetry is enforced by determinant structure. In published realizations, this design has been used to lower molecular VMC energies below FermiNet on many benchmarks (Glehn et al., 2022), to outperform Hartree–Fock and band-projected exact diagonalization in a moiré solid (Geier et al., 7 Feb 2025), to identify chiral $\phi_\mu(\mathbf x_j)$32 superconductivity without explicit pairing bias (Li et al., 3 Sep 2025), to solve fractional quantum Hall states in real space (Teng et al., 2024), and to reach large-scale 2D electron-gas calculations with energies below fixed-node DMC (Gaggioli et al., 9 Jul 2026).