---
title: Reciprocal-Space Attention in Atomistic MLIPs
url: https://www.emergentmind.com/topics/reciprocal-space-attention-rsa
type: topic
---

# Reciprocal-Space Attention in Atomistic MLIPs

Searching arXiv for Reciprocal-Space Attention and closely related reciprocal-space atomistic long-range modeling papers.
Reciprocal-Space Attention (RSA) is a long-range augmentation for machine-learning interatomic potentials (MLIPs) that moves the attention mechanism from real space into reciprocal/Fourier space so that global, periodic, slowly varying interactions can be learned directly from data. In the formulation introduced for atomistic modeling, RSA is designed to be attached to an existing local or semi-local backbone such as MACE, with the specific goal of capturing long-range interactions such as electrostatics and dispersion without relying on predefined charges or explicit analytical \(1/r\) or \(R^{-6}\) terms [2510.13055]. The central construction uses Fourier phase factors \(e^{i\mathbf{k}\cdot \mathbf r}\), reciprocal-space aggregation over low-frequency modes, and a linear-attention-style factorization so that nonlocal interactions can be modeled with practical \(O(N)\) cost at fixed reciprocal basis size [2510.13055].

## 1. Concept and problem setting

RSA is motivated by a limitation shared by many modern MLIPs. Descriptor models such as SOAP/ACE and equivariant graph neural networks such as NequIP or MACE are local by construction: atoms communicate only inside a cutoff radius \(r_{\mathrm{cut}}\), and with \(L\) message-passing layers the receptive field grows only semi-locally, roughly as \(L\,r_{\mathrm{cut}}\) [2510.13055]. This locality assumption is often adequate in homogeneous bulk settings, but it becomes problematic for surfaces, interfaces, dimers, ions, layered materials, polar liquids, and defected systems, where energetics and forces depend on correlations extending well beyond any practical local cutoff [2510.13055, 2211.16684].

The underlying physical rationale is that long-range interactions are smooth in real space and therefore concentrated in low-frequency Fourier modes. Periodic boundary conditions are also naturally represented in reciprocal space through reciprocal lattice vectors, mirroring the logic of Ewald summation: decompose interactions into a short-range part, which local MLIPs already model well, and a long-range part, which is more efficiently expressed as a sum over reciprocal-space modes [2510.13055]. RSA therefore introduces a global Fourier-space communication channel into a local or semi-local MLIP while preserving differentiability and translational invariance [2510.13055].

A closely related precursor is the reciprocal space neural network, which likewise augments local interatomic models with a global reciprocal-space channel based on a structure factor \(S(\mathbf{k},\mathbf r)\) and learned mode-dependent weighting in \(k\)-space [2211.16684]. That earlier work is conceptually aligned with RSA in its use of reciprocal-space global interaction modeling, but it is not an attention model in the transformer sense because it does not define learned queries, keys, and values [2211.16684]. This distinction matters: RSA preserves the spectral/global intuition of reciprocal-space modeling while explicitly casting the nonlocal module as reciprocal-space attention [2510.13055].

## 2. Mathematical formulation in reciprocal space

RSA is explicitly inspired by the reciprocal-space form of Ewald decomposition. For a Coulombic interaction,
\[
V(r)=v_{\mathrm{SR}}(r)+v_{\mathrm{LR}}(r)=\frac{\operatorname{erfc}\!\left(\frac{r}{\sqrt{2}\sigma}\right)}{r}+\frac{\operatorname{erf}\!\left(\frac{r}{\sqrt{2}\sigma}\right)}{r},
\]
where \(\sigma\) is the screening width [2510.13055]. In classical Ewald form, the long-range energy of a neutral periodic system is
\[
E_{\mathrm{LR}}=\frac{2\pi}{V}\sum_{\mathbf{k}\neq \mathbf{0}}\frac{e^{-k^{2}\sigma^{2}/2}}{k^{2}}|S(\mathbf{k})|^2,
\]
with
\[
S(\mathbf{k})=\sum_{n=1}^N \tilde q_n e^{i\mathbf{k}\cdot \mathbf r_n}.
\]
RSA adopts the structure-factor logic but replaces predefined charges \(\tilde q_n\) by learned atomwise features and replaces the analytic reciprocal kernel by a learned attention kernel [2510.13055].

Starting from scalar atom features \(\mathbf h_m\in\mathbb R^H\), RSA constructs
\[
\mathbf Q_m=\mathbf h_m\mathbf W_Q,\qquad
\mathbf K_m=\mathbf h_m\mathbf W_K,\qquad
\mathbf V_m=\mathbf h_m\mathbf W_V,
\]
with \(\mathbf W_Q,\mathbf W_K,\mathbf W_V\in\mathbb R^{H\times D}\), and in the implementation \(D=H\) [2510.13055]. Queries and keys are passed through a feature map \(\phi\), analogous to linear attention; the invariant MACE implementation uses SiLU [2510.13055]. Geometry and periodicity enter through the Fourier positional encoding
\[
\mathrm{FPE}_{\mathbf{k}}(x,\mathbf r_m)=x\,e^{i\mathbf{k}\cdot \mathbf r_m},
\]
giving
\[
\tilde{\mathbf Q}_m=\operatorname{FPE}\!\left(\mathbf r_m,\phi(\mathbf Q_m)\right),\qquad
\tilde{\mathbf K}_m=\operatorname{FPE}\!\left(\mathbf r_m,\phi(\mathbf K_m)\right).
\]
The values \(\mathbf V_m\) remain in feature space in the presented formulation [2510.13055].

A key algebraic identity is obtained by defining the complex inner product with conjugation,
\[
\langle \mathbf Q,\mathbf K\rangle=\mathbf Q^T\bar{\mathbf K},
\]
so that
\[
\big\langle \mathbf Q_m e^{i\mathbf{k}\cdot \mathbf r_m},\mathbf K_n e^{i\mathbf{k}\cdot \mathbf r_n}\big\rangle
=
\langle \mathbf Q_m,\mathbf K_n\rangle e^{i\mathbf{k}\cdot(\mathbf r_m-\mathbf r_n)}.
\]
This yields a reciprocal-space analogue of relative positional encoding: the dependence is on relative displacements \(\mathbf r_m-\mathbf r_n\), not on absolute positions [2510.13055]. For a lattice translation \(\mathbf T\),
\[
e^{i\mathbf{k}\cdot[(\mathbf r_m-\mathbf r_n)+\mathbf T]}=e^{i\mathbf{k}\cdot(\mathbf r_m-\mathbf r_n)},
\]
provided \(\mathbf k\) is a reciprocal lattice vector, so periodicity is automatic [2510.13055].

Ignoring softmax normalization, the quadratic reciprocal-space attention output is written as
\[
\operatorname{RSA}_m(\mathbf Q,\mathbf K,\mathbf V)
=
\sum_{\mathbf{k}\neq 0}\sum_{n=1}^N
\langle \mathbf Q_m,\mathbf K_n\rangle
e^{i\mathbf{k}\cdot(\mathbf r_m-\mathbf r_n)}
\mathbf V_n.
\]
The paper makes the analogy to the reciprocal-space long-range potential explicit:
\[
V_m^{\mathrm{LR}}
=
\frac{2\pi}{V}\sum_{\mathbf{k}\neq 0}\sum_{n=1}^N
\frac{e^{-k^2\sigma^2/2}}{k^2}\,\tilde q_n\,e^{i\mathbf{k}\cdot(\mathbf r_m-\mathbf r_n)}.
\]
In Ewald theory, charges weighted by the reciprocal kernel generate a smooth global field; in RSA, learned features weighted by learned query–key compatibility generate a smooth global message [2510.13055].

## 3. Architectural integration and computational properties

RSA is implemented as an add-on to an existing local or semi-local MLIP. In the reported experiments, the backbone is MACE. Each interaction layer contains two parallel channels: a standard short-range message-passing block and an RSA long-range block [2510.13055]. Given atom embeddings \(\mathbf h_m^{(t)}\) at layer \(t\),
\[
\mathbf M_{m,\mathrm{nl}}^{(t+1)}=\mathrm{RSA}\!\big(\mathbf H^{(t)},\boldsymbol{\delta}\big)_m,
\]
\[
\mathbf h_{m,\mathrm{nl}}^{(t+1)}=f_{\mathrm{upd}}\!\big(\mathbf h_m^{(t)},\mathbf M_{m,\mathrm{nl}}^{(t+1)}\big),
\]
where \(\mathbf H^{(t)}=(\mathbf h_1^{(t)},\dots,\mathbf h_N^{(t)})\) and \(\boldsymbol{\delta}=(\mathbf r_1,\dots,\mathbf r_N)\) [2510.13055]. Conceptually, the short-range path learns rapidly varying local chemistry, whereas RSA contributes slowly varying, cell-wide context.

The computationally important step is the linear-scaling factorization:
\[
\mathrm{RSA}_m(\mathbf Q,\mathbf K,\mathbf V)\simeq
\sum_{\mathbf{k}\neq 0}
\mathbf w_{\mathbf k}\,
\mathrm{FPE}(\phi(\mathbf Q_m),\mathbf r_m)^T
\left[
\sum_{n=1}^N
\mathrm{FPE}(\phi(\mathbf K_n),\mathbf r_n)\mathbf V_n^T
\right],
\]
with Ewald-inspired weights
\[
\mathbf w_{\mathbf k}=\frac{e^{-k^2\sigma^2/2}}{k^2}.
\]
The term in brackets is a \(\mathbf k\)-resolved key–value cache accumulated once over all atoms. For a fixed set of reciprocal modes, the operation becomes linear in \(N\): one pass builds the cache, and one pass contracts each atom’s rotated query against it [2510.13055].

This low-frequency truncation is both a computational device and a physical inductive bias. Long-range interactions are dominated by slowly varying components, so the most relevant information lies at small \(|\mathbf k|\), and the Ewald-like factor \(e^{-k^2\sigma^2/2}/k^2\) further emphasizes low-\(k\) modes while suppressing high-\(k\) oscillations [2510.13055]. A plausible implication is that RSA is not simply extending receptive field length, but reshaping the inductive bias of the model toward smooth global fields.

The energy model remains atomistic and differentiable. RSA augments the interaction layers of MACE, and the final readout maps updated atom embeddings to per-atom energy contributions that sum to the total energy [2510.13055]. Training is end to end on energies and forces with
\[
\mathcal L=
\frac{1}{|\mathcal D|}\sum_{\mathcal S\in\mathcal D}\left|\hat E^{(\mathcal S)}-E^{(\mathcal S)}\right|
+
\frac{\lambda}{|\mathcal D|}\sum_{\mathcal S\in\mathcal D}\frac{1}{|\mathcal S|}\sum_{i\in\mathcal S}
\left\|\hat{\mathbf F}^{(\mathcal S)}_i-\mathbf F_i^{(\mathcal S)}\right\|_1.
\]
Because the model is differentiable with respect to positions, forces are obtained as \(\mathbf F_i=-\partial E/\partial \mathbf r_i\), and the framework is energy–force consistent [2510.13055].

## 4. Physical interpretation, invariances, and relation to earlier reciprocal-space models

RSA can be understood as a learned reciprocal-space field theory attached to a local atomistic model [2510.13055]. Unlike charge-augmented methods that introduce explicit predicted charges as intermediate observables, RSA does not commit to a charge partitioning scheme. The stated rationale is that charges are not unique ab initio observables, partitioning schemes disagree, and locally predicted charges cannot capture genuinely nonlocal response such as long-range charge transfer or collective screening unless extra machinery is added [2510.13055]. Instead, RSA projects learned atom features onto reciprocal modes and couples them globally.

Periodic boundary conditions are handled natively because \(\mathbf k\) lives on the reciprocal lattice and position enters only through \(e^{i\mathbf{k}\cdot\mathbf r}\) [2510.13055]. The exclusion of \(\mathbf k=0\) is inherited from the Ewald analogy and avoids the divergent zero mode; in electrostatics that corresponds to neutral-cell behavior [2510.13055]. This reciprocal construction directly targets long-wavelength dielectric and polarization phenomena, which live in the low-\(k\) regime [2510.13055].

The reciprocal space neural network of 2022 provides a closely related, non-attention formulation based on the structure factor
\[
S(\mathbf{k},\mathbf r)=\sum_j q_j e^{-i\mathbf{k}\cdot \mathbf r_j},
\]
and a learned reciprocal-space long-range energy
\[
E_{\mathrm{long}}=\sum_{\mathbf k}\mathrm{FCN}(|\mathbf k|)\,|S(\mathbf{k},\mathbf r)|^2
\]
[2211.16684]. That model emphasizes Euclidean symmetry and cell-choice invariance, and its reciprocal descriptor is globally pooled over all atoms and all sampled reciprocal modes [2211.16684]. The formal connection to RSA is that both approaches use reciprocal vectors as global spectral indices and both encode nonlocal interaction through Fourier-phase interference across the full structure. The main difference is architectural: RSA introduces explicit query–key–value projections and a linear-attention factorization, whereas the reciprocal space neural network uses learned spectral filtering over structure factors without pairwise attention scores [2211.16684, 2510.13055].

The 2025 RSA work also notes a route to rotationally equivariant generalization via the plane-wave expansion
\[
e^{i\mathbf{k}\cdot \mathbf r}
=
4\pi\sum_{\ell'=0}^\infty\sum_{m'=-\ell'}^{\ell'}
i^{\ell'}j_{\ell'}(kr)\,Y_{\ell' m'}(\hat{\mathbf r})\,Y_{\ell' m'}^*(\hat{\mathbf k}),
\]
and proposes augmenting rank-\(\ell\) spherical tensors by
\[
\tilde T_m^{(\ell)}(\mathbf r,\mathbf k)=e^{i\mathbf{k}\cdot \mathbf r}\,T_m^{(\ell)}(\mathbf r),
\]
noting that irreducible tensor rank is preserved if RSA does not mix \(\ell\)-channels [2510.13055]. In the present implementation, however, the model is restricted to invariant scalar channels, \(\ell_{\max}=0\) [2510.13055].

## 5. Empirical evidence across benchmark classes

The empirical case for RSA is built around systems where local cutoffs are known to fail. In an \(\mathrm{S_N2}\) reaction complex involving fluoride and iodide, both SR-MACE and LR-MACE use two message-passing layers and \(r_{\mathrm{cut}}=5~\text{\AA}\), so the total receptive field is about \(10~\text{\AA}\), while LR-MACE additionally uses RSA with \(\sigma=5~\text{\AA}\) [2510.13055]. Along the reaction coordinate, the short-range model saturates to a constant energy once the ion–molecule distance exceeds its receptive field, whereas LR-MACE tracks the DFT potential energy surface across the full separation range, including the long-distance tail [2510.13055].

The dimer benchmarks extend this observation to charged and polar dimers in a periodic \(30~\text{\AA}\) cubic box, including systems derived from the BioFragment Database, a recomputed CP dimer curve with PBE0 + many-body dispersion, and the water dimer compared against SPC/E with explicit long-range Coulomb interactions [2510.13055]. In both CP and water dimer curves, LR-MACE follows the reference across all intermolecular separations, especially in the long-range tail, whereas SR-MACE flattens beyond the cutoff [2510.13055]. The emphasis is on asymptotic shape rather than a single summary metric.

For periodic systems, the paper reports a toy gas of random charges in a cubic periodic box with 128 atoms, consisting of 64 charges of \(+1e\) and 64 of \(-1e\), interacting via Coulomb plus the repulsive part of Lennard-Jones [2510.13055]. With a \(5~\text{\AA}\) cutoff and two message-passing layers for the short-range part, and RSA using \(\sigma=5~\text{\AA}\), the reported mean absolute errors are:

| Model | Energy MAE | Force MAE |
|---|---:|---:|
| LR MACE | \(2.5\) meV/atom | \(71.6\) meV/\AA |
| SR MACE | \(3.0\) meV/atom | \(97.1\) meV/\AA |

For molten liquid NaCl, using 1014 configurations of 128 atoms split \(80/20\) for training and validation, both models use a \(6~\text{\AA}\) receptive field with a single message-passing layer, and LR-MACE adds RSA [2510.13055]. The reported MAEs are:

| Model | Energy MAE | Force MAE |
|---|---:|---:|
| LR MACE | \(6.8\) meV/atom | \(141.9\) meV/\AA |
| SR MACE | \(8.7\) meV/atom | \(175.1\) meV/\AA |

The phosphorene exfoliation benchmark targets dispersion-dominated interlayer binding. With short-range MACE using cutoff \(6~\text{\AA}\), two message-passing layers, and RSA with \(\sigma=5~\text{\AA}\), the RSA-augmented model reproduces the interlayer exfoliation energy curve and approaches DFT+MBD behavior without any explicit empirical dispersion term [2510.13055]. This is one of the clearest demonstrations that the reciprocal-space channel is not restricted to Coulomb-like behavior.

The bulk-water study is designed to probe collective dielectric physics. Models are trained on 1593 liquid-water configurations of 64 molecules generated with CP2K at the revPBE0-D3 DFT level; the short-range model uses two message-passing layers and \(r_{\mathrm{cut}}=6~\text{\AA}\), while LR-MACE adds RSA with \(\sigma=5~\text{\AA}\) [2510.13055]. Molecular dynamics is then run for 300 water molecules at 300 K and density \(1~\mathrm{g/mL}\), in the NVT ensemble with 1 fs time step, using 10 trajectories of at least 300 ps each [2510.13055]. The oxygen–oxygen radial distribution function \(g_{\mathrm{OO}}(r)\) is nearly identical for SR-MACE and LR-MACE and both agree well with experiment, indicating that local structure is already well captured by short-range forces [2510.13055]. By contrast, the longitudinal dipole-density correlation function \(\chi_{zz}(k)\) diverges between models at small \(k\): SR models show incomplete dipolar screening, while LR-MACE correctly reproduces the low-\(k\) behavior [2510.13055]. This benchmark is particularly significant because it tests long-wavelength collective response rather than only local structure or aggregate error.

The 2022 reciprocal space neural network reports complementary evidence in periodic crystals. On a model NaCl system with explicit Coulomb interaction, the reciprocal-space potential improves over pure DimeNet++ for reciprocal energy and, under a three-step training procedure, improves total-energy prediction for \(3\times3\times3\) training cells and also for \(6\times6\times6\) and defected \(6\times6\times6\) test systems [2211.16684]. On a large \(\mathrm{Ga}_x\mathrm{N}_y\) defect dataset, PaiNN plus reciprocal potential reduces both MAE and RMSE for energies and forces relative to PaiNN alone [2211.16684]. These results support the broader proposition that reciprocal-space global interaction channels are especially useful in ionic and defected systems.

## 6. Scope, limitations, and common misconceptions

RSA is not presented as a replacement for local atomistic modeling. Its intended use is additive or parallel: a strong local backbone such as MACE handles chemistry and short-range repulsion, while RSA supplies the missing long-range interaction channel [2510.13055]. This matters because some observables in uniform bulk systems are already well modeled by local or semi-local MLIPs; the water RDF result is the explicit example [2510.13055]. A common misconception is therefore that any reciprocal-space augmentation should improve every property. The reported evidence suggests instead that RSA is most consequential for disconnected fragments, ionic systems, layered materials, heterogeneous interfaces, and long-wavelength dielectric observables [2510.13055].

Several limitations are stated directly. The current implementation is restricted to orthogonal unit cells [2510.13055]. The learned representation may depend on the chosen \(\mathbf k\)-grid and therefore on training cell shapes and sizes, although the authors state that the dependence appears weak in their tests and that the model generalizes to moderately larger cells and denser \(\mathbf k\)-grids at inference with little loss [2510.13055]. While RSA scales linearly with atom count for a fixed number of reciprocal modes, there is overhead relative to a purely local backbone, and summing the full reciprocal lattice would recover direct Ewald-like cost rather than practical \(O(N)\) behavior [2510.13055]. The dependence on \(\sigma\), number of modes, and RSA depth is also not exhaustively benchmarked in the presented text [2510.13055].

Another misconception is to equate RSA with generic spectral pooling. The reciprocal space neural network shows that reciprocal-space global interaction can be implemented without attention by using structure factors and learned \(k\)-space filters [2211.16684]. RSA differs in that it maps a linear-scaling attention mechanism into Fourier space, uses learned query, key, and value projections, and constructs atomwise nonlocal messages through reciprocal-phase-modulated compatibility scores [2510.13055]. It is therefore more precise to describe RSA as a reciprocal-space attention mechanism rather than merely a reciprocal descriptor.

A final source of ambiguity is terminological. “RSA” is used in unrelated literatures, including a 2020 paper on radiostereometric analysis in which RSA denotes a clinical-imaging methodology rather than reciprocal-space attention [2010.15178]. In that work, “total rotation” is analyzed as a small-angle approximation to the helical angle in rigid-body kinematics [2010.15178]. This acronym collision is purely nominal and does not indicate technical overlap.

Taken together, the available literature defines RSA as a reciprocal-space, linear-attention module for long-range atomistic learning under periodic boundary conditions, with explicit ties to Ewald theory, Fourier positional encoding, and low-frequency spectral inductive bias [2510.13055]. The broader research trajectory, including reciprocal-space neural global interaction models, suggests that reciprocal-space representations provide a principled route for incorporating full-system information into MLIPs when local cutoffs and finite-depth message passing are insufficient [2211.16684].

Source: https://www.emergentmind.com/topics/reciprocal-space-attention-rsa