--- title: Sample-Based Neural Diagonalization (SND) url: https://www.emergentmind.com/topics/sample-based-neural-diagonalization-snd type: topic --- # Sample-Based Neural Diagonalization (SND) Sample-based Neural Diagonalization (SND) denotes a class of neural-network-assisted strategies for reducing the cost of diagonalization in quantum many-body computation. In the literature represented by Stratis et al., SND is a feed-forward eigenvalue surrogate used inside Monte Carlo sample generation for the spin–fermion, or double-exchange, model, replacing repeated exact diagonalization of a fermionic matrix by direct prediction of its spectrum [2206.07753]. In later work on sample-based diagonalization (SBD), SND instead denotes an autoregressive neural sampler that selects basis configurations for projected diagonalization, while adaptive-basis SND (AB-SND) augments this with a learnable basis transformation [2508.12724]. This suggests that the unifying idea is not a single canonical architecture, but the use of learned samples or learned spectra to make spectrally informed computation tractable. ## 1. Scope of the term and problem classes The term SND is used in two distinct settings. In the spin–fermion setting, the computational bottleneck is the repeated evaluation of all single-particle eigenvalues for a Hamiltonian that depends on a classical spin configuration. In the SBD setting, the bottleneck is the selection of a small but effective basis subspace whose projected Hamiltonian yields an accurate ground-state estimate. In both cases, the learned model is coupled directly to a diagonalization-related step rather than being used only as a generic regressor or classifier. | Setting | Learned object | Computational role | |---|---|---| | Spin–fermion model | Full spectrum $\{\hat\lambda_i(S)\}$ or free energy $\hat F(S)$ | Replace exact diagonalization during Monte Carlo proposals | | SBD for quantum Ising models | Autoregressive sampler $p_\theta(x)$ | Select basis states for projected diagonalization | | AB-SND | Sampler plus basis transformation | Concentrate the ground state before projected diagonalization | A common source of confusion is to assume that SND names a single fixed algorithm. In the cited literature, it instead names two related but non-identical constructions: one predicts eigenvalues of a configuration-dependent Hamiltonian, and the other learns how to sample basis states that make a projected Hamiltonian accurate. The overlap lies in the reliance on neural networks to extract spectrally relevant information from samples. ## 2. Spin–fermion SND as neural eigenvalue prediction In the spin–fermion formulation, the system consists of $N$ classical unit spins $S=\{S_1,\dots,S_N\}$ with $S_i\in\mathbb{R}^3$ and $|S_i|=1$, coupled to spinful fermions on the same sites. The single-orbital spin–fermion, or double-exchange, Hamiltonian is $$ H(S) = -t \sum_{\langle i,j\rangle,\sigma} (c_{i\sigma}^\dagger c_{j\sigma} + h.c.) + \frac{J}{2} \sum_{i,\alpha,\beta,\gamma} S_i^\gamma c_{i\alpha}^\dagger \sigma^\gamma_{\alpha\beta} c_{i\beta}, $$ with $t=1$, Hund coupling $J$, Pauli matrices $\sigma^\gamma$, and nearest-neighbor pairs $\langle i,j\rangle$. For each classical-spin update in a Monte Carlo sweep, one must recompute all single-particle eigenvalues $\epsilon_i(S)$ of the $2N\times 2N$ fermion matrix, and standard dense diagonalization scales as $\mathcal{O}((2N)^3)\sim\mathcal{O}(N^3)$ [2206.07753]. Three surrogate models are compared in this formulation: a linear Heisenberg model fitting $F(S)$ as $\sum_{|i-j|=r} J_r S_i\!\cdot\!S_j$, an $N1$ feed-forward network predicting free energy directly, and an $N1Eigenvalues$ model predicting the full spectrum. The SND construction is the eigenvalue model. Its input is the flattened real vector $$ x\in\mathbb{R}^{3N}: x=[S_1^x,S_1^y,S_1^z,\dots,S_N^z]. $$ The architecture is fully connected with one hidden layer of size $h$, Softplus activation $f(y)=\ln(1+e^y)$, and a linear output layer of size $2N$ interpreted as predicted eigenvalues. After inference, the $2N$ outputs are sorted so that $\hat\lambda_1\le \dots \le \hat\lambda_{2N}$. Training uses exact-diagonalization data generated at each temperature $T=1/\beta$ from three independent Metropolis–Hastings chains. After a warm-up of $1000\cdot N$ proposals, the procedure generates $N_s=10^4$ samples for training and $N_s=10^3$ for validation and test, with thinning by saving one in every $N/r$ proposals, where $r$ is the acceptance ratio. Each saved spin configuration is labeled by its full eigenvalue set and free energy. Data augmentation exploits translation invariance and global $SO(3)$ rotation invariance using a discrete Euler-angle grid, $$ \alpha\in\{0,\pi\},\quad \beta\in\{0,\pi/2,\pi,3\pi/2\},\quad \gamma\in\{0,\pi\}, $$ excluding the identity, which augments each raw sample by up to 23 non-trivial variants. The training loss for the eigenvalue model is the mean absolute error per eigenvalue, $$ L = \frac{1}{2N}\sum_{i=1}^{2N} |\hat\lambda_i-\lambda_i^{exact}|, $$ while mean squared error on a held-out test set is monitored as a secondary metric. Optimization uses Adam with initial learning rate $10^{-3}$ and exponential decay $$ lr(t)=lr(0)\cdot 0.9995^t, $$ for $2\cdot 10^4$ epochs. Recommended hyperparameters are specified for both one and two dimensions: for 1D with $N=20$, $N1$ uses $h=60, n_b=20$ and $N1Eigenvalues$ uses $h=80, n_b=50$; for 2D with $N=6\times 6=36$, $N1$ uses $h=108, n_b=20$ and $N1Eigenvalues$ uses $h=144, n_b=20$. ## 3. SND as learned basis selection in sample-based diagonalization In the later SBD formulation, the Hilbert space is expressed in the computational $(\sigma^z)$ basis $\{|x\rangle\equiv|x_1x_2\dots x_N\rangle\}$ with $x_i\in\{0,1\}$. A standard testbed is the transverse-field Ising model $$ H \;=\; -\sum_{\langle i,j\rangle}J_{ij}\,Z_iZ_j \;-\; h\sum_i X_i, $$ whose full Hilbert dimension is $2^N$. One selects a subset of $S$ basis states $$ \mathcal{S}=\{|x^{(1)}\rangle,\dots,|x^{(S)}\rangle\}, $$ defines the projector $$ P_{\mathcal{S}}=\sum_{l=1}^S |x^{(l)}\rangle\langle x^{(l)}|, $$ and forms the projected Hamiltonian $$ H_{\mathcal{S}} = P_{\mathcal{S}} H P_{\mathcal{S}}, \qquad [H_{\mathcal{S}}]_{lm}=\langle x^{(l)}|H|x^{(m)}\rangle. $$ Diagonalizing the $S\times S$ matrix $H_{\mathcal{S}}$ yields eigenvalues $E_{\mathcal{S}}^{(0)}\le E_{\mathcal{S}}^{(1)}\le \dots$, with $E_{\mathcal{S}}^{(0)}$ a variational upper bound to the true ground-state energy $E_0$ and $E_{\mathcal{S}}^{(0)}\to E_0$ as $S\to 2^N$ [2508.12724]. The central issue is how to choose the $S$ basis states so that $E_{\mathcal{S}}^{(0)}$ is accurate while $S\ll 2^N$. Standard SBD samples configurations from a heuristic distribution, such as squared amplitudes of a variational neural-quantum-state, but this fails when the true ground state is delocalized in the computational basis. SND replaces the fixed heuristic sampler by an autoregressive neural network $p_\theta(x)$ trained to focus on bitstrings with large ground-state weight, with the explicit target $$ p_\theta(x)\approx |\psi_0(x)|^2, \qquad \psi_0(x)=\langle x|\psi_0\rangle. $$ Training proceeds in $K$ batches of $S$ sampled bitstrings each. For batch $k$, the sampled set $\mathcal{S}^{(k)}=\{x^{(k,1)},\dots,x^{(k,S)}\}$ defines a subspace Hamiltonian whose lowest eigenvalue is $E^{(k)}$. The loss is $$ L(\theta) = \frac{1}{K}\sum_{k=1}^K \Bigl(E^{(k)}-\overline{E}\Bigr) \sum_{l=1}^S \log p_\theta\bigl(x^{(k,l)}\bigr), $$ with $\overline{E}=\tfrac{1}{K}\sum_k E^{(k)}$. The corresponding REINFORCE-style gradient is $$ \nabla_\theta L \approx \frac{1}{K}\sum_{k=1}^K \bigl(E^{(k)}-\overline{E}\bigr) \sum_{l=1}^S \nabla_\theta \log p_\theta\bigl(x^{(k,l)}\bigr). $$ Minimizing this objective shifts probability mass toward sample sets that yield low projected energies. AB-SND extends this by introducing a parameterized basis transformation optimized jointly with the sampler. The paper considers single-spin and non-overlapping two-spin rotations, which are tractable on classical computers, and also more expressive global unitaries that can be implemented using quantum circuits. The stated purpose is to make the ground-state wave function more concentrated, thereby enlarging the regime in which sample-based diagonalization remains effective. ## 4. Neural architectures, observables, and algorithmic use The two SND formulations differ sharply in architecture. In the spin–fermion case, the network is a shallow fully connected regressor with a single hidden layer and linear spectral output. In the SBD case, the network is a Transformer encoder with causal masking, using 2 layers, 4 heads, and embedding size 64 to realize the autoregressive factorization $$ p_\theta(x_1,\dots,x_N)=\prod_{i=1}^N p_\theta(x_i\mid x_{