Fourier Feature Networks: A Spectral Approach

• Fourier Feature Networks are neural architectures that integrate sinusoidal or Fourier-domain embeddings, enriching the input representation and mitigating spectral bias.
• They alter the effective kernel of MLPs to enable efficient approximation of high-frequency, oscillatory, and multiscale functions in various scientific and imaging applications.
• Key architectural patterns include fixed random embeddings, deterministic boundary-aware mappings, and trainable frequency adaptations that improve convergence, robustness, and accuracy.

A Fourier Feature Network is a neural architecture in which sinusoidal basis functions, Fourier-domain representations, or explicitly spectral feature transforms are inserted into the learning pipeline so that the model operates on a frequency-rich representation rather than only on raw coordinates or conventional hidden activations. In the cited literature, the designation spans coordinate-based multilayer perceptrons with Fourier embeddings, perturbation-conditioned optical predictors, Fourier-parameterized message-passing modules, Fourier-domain scattering constructions, and Fourier-aware contour models. The term therefore functions less as the name of a single canonical architecture than as a family of related spectral design patterns organized around sine, cosine, or Fourier-domain structure (Tancik et al., 2020, Jandrell et al., 27 Aug 2025, Balestriero et al., 2017).

1. Definitions and scope of the term

The most widely cited formulation is the coordinate-based network introduced in “Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains,” where an input point $v \in \mathbb{R}^d$ is mapped to a trigonometric feature vector

$$\gamma(v)=\{a_1 \cos(2\pi \mathbf b_1^\top v),\; a_1 \sin(2\pi \mathbf b_1^\top v),\; \ldots,\; a_m \cos(2\pi \mathbf b_m^\top v),\; a_m \sin(2\pi \mathbf b_m^\top v)\},$$

and the embedded representation is then processed by a standard MLP $f(\gamma(v);\theta)$. In that usage, a Fourier Feature Network is a coordinate MLP whose representational power is altered primarily by the input embedding rather than by a specialized deep backbone (Tancik et al., 2020).

A second major usage appears in physics-informed learning. “On the eigenvector bias of Fourier feature networks” defines a Fourier feature network as a fully connected neural network preceded by a fixed random coordinate embedding,

$\gamma(x)=\big[\cos(Bx),\ \sin(Bx)\big],$

with $B$ sampled i.i.d. from a Gaussian distribution with scale parameter $\sigma$. In that setting, Fourier features are treated as a fixed coordinate embedding layer whose purpose is to alter the effective NTK spectrum of the downstream PINN without adding trainable parameters in the embedding itself (Wang et al., 2020).

Other papers use the term more specifically for domain-adapted architectures. In perturbed multimode-fibre prediction, the Fourier Feature Network (FNET) is a shallow MLP that receives a perturbation $\tilde d$ together with a hand-designed sinusoidal map

$$\gamma(\tilde d)=\big[\cos(\omega_1 \tilde d),\sin(\omega_1 \tilde d),\dots,\cos(\omega_M \tilde d),\sin(\omega_M \tilde d)\big],$$

so that learning becomes a basis-combination problem rather than direct regression of a highly oscillatory complex-valued field (Jandrell et al., 27 Aug 2025). In graph collaborative filtering, FourierKAN-GCF replaces the interaction transform in NGCF-style message passing with a Fourier series transform

$$\phi_F(\mathbf{x})=\sum_{i=1}^{d}\sum_{k=1}^{g} \left(\cos(k\mathbf{x}_i)\cdot a_{ik}+\sin(k\mathbf{x}_i)\cdot b_{ik}\right),$$

which is Fourier-feature-based in functional form even though the paper frames it as a Fourier Kolmogorov–Arnold Network (Xu et al., 2024).

Formulation	Defining mechanism	Representative use
Coordinate Fourier-feature MLP	Sinusoidal embedding of raw coordinates	Low-dimensional regression, graphics, tomography (Tancik et al., 2020)
Fourier-feature PINN	Fixed random embedding before fully connected backbone	Multiscale PDEs and inverse problems (Wang et al., 2020)
Optical FNET	Fixed perturbation-dependent sinusoidal basis plus shallow MLP	Perturbed optical field prediction (Jandrell et al., 27 Aug 2025)
Fourier-KAN transform	Trainable sine/cosine series in feature transformation	Graph collaborative filtering (Xu et al., 2024)
Fourier scattering network	Entire cascade implemented in Fourier domain	Deterministic invariant extraction (Balestriero et al., 2017)

This diversity is not merely terminological. It reflects a shared architectural principle: Fourier features can be injected at the input, inside message passing, inside hidden activations, or directly in the signal-processing domain.

2. Spectral rationale and kernel viewpoint

The central theoretical motivation is spectral bias. Standard MLPs tend to fit low-frequency structure first and struggle on high-frequency targets, especially in low-dimensional continuous domains and in oscillatory PDEs. In the NTK regime, training dynamics are governed by the eigenvalues of the kernel matrix; if $K=Q^\top \Lambda Q$, then error components aligned with larger eigenvalues decay faster. For ordinary fully connected networks, the dominant eigen-directions correspond to lower-frequency modes, which explains why sharp detail, rapid oscillation, and multiscale components are learned slowly or not at all within feasible training time (Tancik et al., 2020, Wang et al., 2020).

Fourier feature embeddings alter that regime by changing the effective kernel seen by the MLP. For the embedded representation, the induced kernel becomes

$$\gamma(v)=\{a_1 \cos(2\pi \mathbf b_1^\top v),\; a_1 \sin(2\pi \mathbf b_1^\top v),\; \ldots,\; a_m \cos(2\pi \mathbf b_m^\top v),\; a_m \sin(2\pi \mathbf b_m^\top v)\},$$0

which is stationary, depending on coordinate differences rather than absolute location. In the formulation of (Tancik et al., 2020), this turns the network’s effective NTK into a stationary kernel with tunable bandwidth, making it better matched to dense low-dimensional domains such as images, volumes, and continuous fields (Tancik et al., 2020).

The PINN analysis in (Wang et al., 2020) sharpens this interpretation by recasting spectral bias as NTK eigenvector bias. In a one-dimensional toy setting, a two-layer bias-free network with Fourier features yields a kernel $$\gamma(v)=\{a_1 \cos(2\pi \mathbf b_1^\top v),\; a_1 \sin(2\pi \mathbf b_1^\top v),\; \ldots,\; a_m \cos(2\pi \mathbf b_m^\top v),\; a_m \sin(2\pi \mathbf b_m^\top v)\},$$1, whose eigenfunctions satisfy

$$\gamma(v)=\{a_1 \cos(2\pi \mathbf b_1^\top v),\; a_1 \sin(2\pi \mathbf b_1^\top v),\; \ldots,\; a_m \cos(2\pi \mathbf b_m^\top v),\; a_m \sin(2\pi \mathbf b_m^\top v)\},$$2

with nonzero eigenfunctions

$$\gamma(v)=\{a_1 \cos(2\pi \mathbf b_1^\top v),\; a_1 \sin(2\pi \mathbf b_1^\top v),\; \ldots,\; a_m \cos(2\pi \mathbf b_m^\top v),\; a_m \sin(2\pi \mathbf b_m^\top v)\},$$3

and eigenvalues

$$\gamma(v)=\{a_1 \cos(2\pi \mathbf b_1^\top v),\; a_1 \sin(2\pi \mathbf b_1^\top v),\; \ldots,\; a_m \cos(2\pi \mathbf b_m^\top v),\; a_m \sin(2\pi \mathbf b_m^\top v)\},$$4

Larger Fourier scale $$\gamma(v)=\{a_1 \cos(2\pi \mathbf b_1^\top v),\; a_1 \sin(2\pi \mathbf b_1^\top v),\; \ldots,\; a_m \cos(2\pi \mathbf b_m^\top v),\; a_m \sin(2\pi \mathbf b_m^\top v)\},$$5 makes large-$$\gamma(v)=\{a_1 \cos(2\pi \mathbf b_1^\top v),\; a_1 \sin(2\pi \mathbf b_1^\top v),\; \ldots,\; a_m \cos(2\pi \mathbf b_m^\top v),\; a_m \sin(2\pi \mathbf b_m^\top v)\},$$6 features more likely, thereby inducing higher-frequency NTK eigenvectors and narrowing eigenvalue gaps. The same paper also stresses the failure mode: if $$\gamma(v)=\{a_1 \cos(2\pi \mathbf b_1^\top v),\; a_1 \sin(2\pi \mathbf b_1^\top v),\; \ldots,\; a_m \cos(2\pi \mathbf b_m^\top v),\; a_m \sin(2\pi \mathbf b_m^\top v)\},$$7 is too large, the model can overfit and interpolate training points with spurious oscillations rather than recover the target function (Wang et al., 2020).

A more application-specific rationale appears in the optical FNET literature. There, Fourier features do not primarily appear as a kernel trick; they reframe a hard regression problem. Standard MLPs are described as unsatisfactory because polar outputs introduce $$\gamma(v)=\{a_1 \cos(2\pi \mathbf b_1^\top v),\; a_1 \sin(2\pi \mathbf b_1^\top v),\; \ldots,\; a_m \cos(2\pi \mathbf b_m^\top v),\; a_m \sin(2\pi \mathbf b_m^\top v)\},$$8 phase discontinuities, while Cartesian outputs remain smooth but vary too rapidly with perturbation. By supplying known sinusoidal bases of the perturbation directly, the network is required only to learn a linear combination of basis functions, which produces a much smoother optimization landscape for highly oscillatory optical responses (Jandrell et al., 27 Aug 2025).

3. Architectural patterns

A first pattern is the fixed random input embedding. In the basic and still influential construction, entries of $$\gamma(v)=\{a_1 \cos(2\pi \mathbf b_1^\top v),\; a_1 \sin(2\pi \mathbf b_1^\top v),\; \ldots,\; a_m \cos(2\pi \mathbf b_m^\top v),\; a_m \sin(2\pi \mathbf b_m^\top v)\},$$9 are sampled from $f(\gamma(v);\theta)$0, the map $f(\gamma(v);\theta)$1 is held fixed, and only the downstream MLP is trained. The multiscale extension replaces a single bandwidth with several random feature maps $f(\gamma(v);\theta)$2 at different $f(\gamma(v);\theta)$3, while the spatio-temporal extension separates spatial and temporal embeddings $f(\gamma(v);\theta)$4 and $f(\gamma(v);\theta)$5, processes them with separate hidden stacks, and merges them by pointwise multiplication before the final readout (Wang et al., 2020).

A second pattern is the boundary-aware deterministic embedding. For homogeneous Neumann conditions on $f(\gamma(v);\theta)$6, “Hard-constraining Neumann boundary conditions in physics-informed neural networks via Fourier feature embeddings” uses

$f(\gamma(v);\theta)$7

or the multi-frequency cosine map

$f(\gamma(v);\theta)$8

with integer $f(\gamma(v);\theta)$9. Because each cosine feature has zero derivative at $\gamma(x)=\big[\cos(Bx),\ \sin(Bx)\big],$0 and $\gamma(x)=\big[\cos(Bx),\ \sin(Bx)\big],$1, the chain rule makes the derivative of the network output vanish at the boundaries automatically. In that design, Fourier features are not only a spectral prior but also a hard architectural constraint (Straub et al., 1 Apr 2025).

A third pattern is trainable frequency adaptation. In multiscale lubrication simulation, the proposed MLNN inserts a Fourier feature layer before a 5-layer, 100-neuron-per-layer sigmoid PINN and treats the frequencies $\gamma(x)=\big[\cos(Bx),\ \sin(Bx)\big],$2 as trainable parameters rather than fixed samples. The frequencies are initialized from Gaussian distributions with $\gamma(x)=\big[\cos(Bx),\ \sin(Bx)\big],$3, with 30 frequency parameters per scale. The stated motivation is that rough-surface lubrication contains unknown and mixed-frequency content that fixed feature embeddings may not match well (Tang et al., 2024).

A fourth pattern is explicit per-feature gating or hybrid activation. “Robust Fourier Neural Networks” inserts a diagonal layer immediately after a symmetrized Fourier embedding $\gamma(x)=\big[\cos(Bx),\ \sin(Bx)\big],$4, using the two-layer form

$\gamma(x)=\big[\cos(Bx),\ \sin(Bx)\big],$5

and argues that the diagonal parameterization induces sparse, noise-robust Fourier feature learning under layer-wise projected SGD (Jeong et al., 2024). “Kolmogorov-Arnold Fourier Networks” replaces spline-based KAN functions by trainable random Fourier features and combines them with a hybrid activation

$\gamma(x)=\big[\cos(Bx),\ \sin(Bx)\big],$6

initialized with $\gamma(x)=\big[\cos(Bx),\ \sin(Bx)\big],$7 and $\gamma(x)=\big[\cos(Bx),\ \sin(Bx)\big],$8 for $\gamma(x)=\big[\cos(Bx),\ \sin(Bx)\big],$9, so that low-frequency smooth modeling dominates early and Fourier corrections strengthen later in training (Zhang et al., 9 Feb 2025).

A fifth pattern applies Fourier expansion not to raw coordinates but to learned latent variables. In IFeF-PINN, a feedforward network first produces latent features $B$0, after which random Fourier features are applied in latent space: $B$1 The paper explicitly distinguishes this from earlier methods that Fourier-encode the input coordinates only; here the enhanced basis is adaptive because the latent representation is learned (Wu et al., 22 Oct 2025).

4. Scientific computing and PDE solution

Fourier Feature Networks have been especially prominent in scientific machine learning because PDE solutions often contain multi-scale, oscillatory, or variable-frequency structure. In the PINN setting, the multi-scale Fourier feature architecture of (Wang et al., 2020) substantially outperforms plain fully connected PINNs on oscillatory benchmarks. On the 1D Poisson problem with exact solution $B$2, a plain PINN fails badly even after $B$3 iterations, whereas the MFF model with $B$4 and $B$5 reaches a relative $B$6 error of about $B$7. On the 1D heat equation with initial condition $B$8, the ST-MFF model with spatial $B$9 and temporal $\sigma$0 achieves a relative $\sigma$1 error of about $\sigma$2. On the 1D wave equation, the paper also reports that the architecture alone does not remove optimization pathologies; combining ST-MFF with the adaptive weights strategy from Wang et al. yields a relative $\sigma$3 error of $\sigma$4. In the inverse 2D Gray–Scott problem, the same framework recovers $\sigma$5 versus the true $\sigma$6, and $\sigma$7 versus the true $\sigma$8 (Wang et al., 2020).

Domain decomposition provides a different Fourier-feature route to high-frequency PDEs. F-D3M decomposes the domain into overlapping subdomains and trains a local Multi Fourier Feature Network on each subdomain, using the augmented feature map

$\sigma$9

Boundary and interface conditions are enforced analytically through the ansatz

$\tilde d$0

so local training uses only the PDE residual. Reported relative errors are $\tilde d$1 for Global-FCN, $\tilde d$2 for Global-MFFNet, and $\tilde d$3 for F-D3M on a 1D diffusion problem; on a 2D Helmholtz problem, the corresponding errors are $\tilde d$4, $\tilde d$5, and $\tilde d$6 for an $\tilde d$7 decomposition (Li et al., 2022).

The Neumann-constraint construction of (Straub et al., 1 Apr 2025) combines architectural exactness with multiscale representation. On the 1D heat equation with homogeneous Neumann boundary conditions, the cosine embedding removes the boundary-loss term entirely for the hard-constrained models. With fixed wall-clock time, the paper reports relative improvements over the best vanilla PINN of $\tilde d$8 for low frequency, $\tilde d$9 for a 3rd-order polynomial, $$\gamma(\tilde d)=\big[\cos(\omega_1 \tilde d),\sin(\omega_1 \tilde d),\dots,\cos(\omega_M \tilde d),\sin(\omega_M \tilde d)\big],$$0 for a 4th-order polynomial, $$\gamma(\tilde d)=\big[\cos(\omega_1 \tilde d),\sin(\omega_1 \tilde d),\dots,\cos(\omega_M \tilde d),\sin(\omega_M \tilde d)\big],$$1 for high-frequency data, and $$\gamma(\tilde d)=\big[\cos(\omega_1 \tilde d),\sin(\omega_1 \tilde d),\dots,\cos(\omega_M \tilde d),\sin(\omega_M \tilde d)\big],$$2 for multiscale data when larger Fourier embeddings are used. Average iteration times are $$\gamma(\tilde d)=\big[\cos(\omega_1 \tilde d),\sin(\omega_1 \tilde d),\dots,\cos(\omega_M \tilde d),\sin(\omega_M \tilde d)\big],$$3 ms for the vanilla PINN, $$\gamma(\tilde d)=\big[\cos(\omega_1 \tilde d),\sin(\omega_1 \tilde d),\dots,\cos(\omega_M \tilde d),\sin(\omega_M \tilde d)\big],$$4 ms for the existing derivative-based hard-constraint method, and $$\gamma(\tilde d)=\big[\cos(\omega_1 \tilde d),\sin(\omega_1 \tilde d),\dots,\cos(\omega_M \tilde d),\sin(\omega_M \tilde d)\big],$$5 ms for the Fourier-based hard-constrained method (Straub et al., 1 Apr 2025).

In rough-surface lubrication, the trainable-frequency MLNN addresses a PDE regime for which the paper states that no PINN methods had previously been reported. For smooth, sinusoidal, textured, and random Gaussian rough surfaces, the model is compared against FEM. Reported maximum-pressure errors are $$\gamma(\tilde d)=\big[\cos(\omega_1 \tilde d),\sin(\omega_1 \tilde d),\dots,\cos(\omega_M \tilde d),\sin(\omega_M \tilde d)\big],$$6, $$\gamma(\tilde d)=\big[\cos(\omega_1 \tilde d),\sin(\omega_1 \tilde d),\dots,\cos(\omega_M \tilde d),\sin(\omega_M \tilde d)\big],$$7, $$\gamma(\tilde d)=\big[\cos(\omega_1 \tilde d),\sin(\omega_1 \tilde d),\dots,\cos(\omega_M \tilde d),\sin(\omega_M \tilde d)\big],$$8, and $$\gamma(\tilde d)=\big[\cos(\omega_1 \tilde d),\sin(\omega_1 \tilde d),\dots,\cos(\omega_M \tilde d),\sin(\omega_M \tilde d)\big],$$9, while load-carrying-capacity errors are $$\phi_F(\mathbf{x})=\sum_{i=1}^{d}\sum_{k=1}^{g} \left(\cos(k\mathbf{x}_i)\cdot a_{ik}+\sin(k\mathbf{x}_i)\cdot b_{ik}\right),$$0, $$\phi_F(\mathbf{x})=\sum_{i=1}^{d}\sum_{k=1}^{g} \left(\cos(k\mathbf{x}_i)\cdot a_{ik}+\sin(k\mathbf{x}_i)\cdot b_{ik}\right),$$1, $$\phi_F(\mathbf{x})=\sum_{i=1}^{d}\sum_{k=1}^{g} \left(\cos(k\mathbf{x}_i)\cdot a_{ik}+\sin(k\mathbf{x}_i)\cdot b_{ik}\right),$$2, and $$\phi_F(\mathbf{x})=\sum_{i=1}^{d}\sum_{k=1}^{g} \left(\cos(k\mathbf{x}_i)\cdot a_{ik}+\sin(k\mathbf{x}_i)\cdot b_{ik}\right),$$3, respectively. The comparison with a fixed-frequency Fourier feature network is especially strong: for the rough-surface case, the trainable model reports $$\phi_F(\mathbf{x})=\sum_{i=1}^{d}\sum_{k=1}^{g} \left(\cos(k\mathbf{x}_i)\cdot a_{ik}+\sin(k\mathbf{x}_i)\cdot b_{ik}\right),$$4 maximum-pressure error and $$\phi_F(\mathbf{x})=\sum_{i=1}^{d}\sum_{k=1}^{g} \left(\cos(k\mathbf{x}_i)\cdot a_{ik}+\sin(k\mathbf{x}_i)\cdot b_{ik}\right),$$5 load-carrying-capacity error, whereas the fixed-frequency baseline reports $$\phi_F(\mathbf{x})=\sum_{i=1}^{d}\sum_{k=1}^{g} \left(\cos(k\mathbf{x}_i)\cdot a_{ik}+\sin(k\mathbf{x}_i)\cdot b_{ik}\right),$$6 and $$\phi_F(\mathbf{x})=\sum_{i=1}^{d}\sum_{k=1}^{g} \left(\cos(k\mathbf{x}_i)\cdot a_{ik}+\sin(k\mathbf{x}_i)\cdot b_{ik}\right),$$7 (Tang et al., 2024).

A related but analytically solved variant is GFF-PIELM, which replaces the usual ELM activation with cosine neurons carrying neuron-specific frequency coefficients $$\phi_F(\mathbf{x})=\sum_{i=1}^{d}\sum_{k=1}^{g} \left(\cos(k\mathbf{x}_i)\cdot a_{ik}+\sin(k\mathbf{x}_i)\cdot b_{ik}\right),$$8: $$\phi_F(\mathbf{x})=\sum_{i=1}^{d}\sum_{k=1}^{g} \left(\cos(k\mathbf{x}_i)\cdot a_{ik}+\sin(k\mathbf{x}_i)\cdot b_{ik}\right),$$9 The paper proposes choosing the frequency interval by monitoring the distribution of the analytically solved output weights $K=Q^\top \Lambda Q$0. On a 1D Poisson example, refining the interval to $K=Q^\top \Lambda Q$1 yields MSE $K=Q^\top \Lambda Q$2 and relative $K=Q^\top \Lambda Q$3 error $K=Q^\top \Lambda Q$4. Across five case studies and ten numerical examples, the method is reported to improve predictive accuracy substantially over vanilla PIELM without additional training-time cost, with both models training in about 9 seconds under the default settings (Ren et al., 14 Oct 2025).

5. Imaging, wave physics, recommendation, and segmentation

Outside PDE solution, Fourier Feature Networks have been used as compact inductive biases for inverse problems and oscillatory physical systems. In free-breathing cardiac MRI reconstruction, FMLP models the complex image as a continuous function of space and time,

$K=Q^\top \Lambda Q$5

and uses separate spatial and temporal Fourier embeddings: $K=Q^\top \Lambda Q$6 Here $K=Q^\top \Lambda Q$7, $K=Q^\top \Lambda Q$8, both sampled i.i.d. from $K=Q^\top \Lambda Q$9 and fixed, producing a $$\gamma(v)=\{a_1 \cos(2\pi \mathbf b_1^\top v),\; a_1 \sin(2\pi \mathbf b_1^\top v),\; \ldots,\; a_m \cos(2\pi \mathbf b_m^\top v),\; a_m \sin(2\pi \mathbf b_m^\top v)\},$$00-dimensional feature vector that is processed by a 7-layer, 512-neuron ReLU MLP. The method achieves reconstruction quality on par with or slightly better than t-DIP and superior quality relative to NIK and KFMLP, although at higher computational cost (Kunz et al., 2023).

The optical FNET of (Jandrell et al., 27 Aug 2025) is deliberately small and specialized. It predicts the transmission-matrix column of a multimode fibre under mechanical compression $$\gamma(v)=\{a_1 \cos(2\pi \mathbf b_1^\top v),\; a_1 \sin(2\pi \mathbf b_1^\top v),\; \ldots,\; a_m \cos(2\pi \mathbf b_m^\top v),\; a_m \sin(2\pi \mathbf b_m^\top v)\},$$01, fixing the input field to $$\gamma(v)=\{a_1 \cos(2\pi \mathbf b_1^\top v),\; a_1 \sin(2\pi \mathbf b_1^\top v),\; \ldots,\; a_m \cos(2\pi \mathbf b_m^\top v),\; a_m \sin(2\pi \mathbf b_m^\top v)\},$$02 and predicting output modal weights in a 20-mode basis, which yields a 40-neuron real-valued output after stacking real and imaginary parts. The Fourier feature map uses a linear progression of frequencies

$$\gamma(v)=\{a_1 \cos(2\pi \mathbf b_1^\top v),\; a_1 \sin(2\pi \mathbf b_1^\top v),\; \ldots,\; a_m \cos(2\pi \mathbf b_m^\top v),\; a_m \sin(2\pi \mathbf b_m^\top v)\},$$03

and the backbone has one hidden layer of 20 neurons with hardtanh, for a total of 60 neurons versus 360 in the baseline MLP. Using 18 training samples and 18 unseen test samples, the final reconstructed field achieves a mean complex Pearson correlation of $$\gamma(v)=\{a_1 \cos(2\pi \mathbf b_1^\top v),\; a_1 \sin(2\pi \mathbf b_1^\top v),\; \ldots,\; a_m \cos(2\pi \mathbf b_m^\top v),\; a_m \sin(2\pi \mathbf b_m^\top v)\},$$04, compared with $$\gamma(v)=\{a_1 \cos(2\pi \mathbf b_1^\top v),\; a_1 \sin(2\pi \mathbf b_1^\top v),\; \ldots,\; a_m \cos(2\pi \mathbf b_m^\top v),\; a_m \sin(2\pi \mathbf b_m^\top v)\},$$05 for the baseline MLP, while using 85% fewer parameters (Jandrell et al., 27 Aug 2025).

In recommendation, FourierKAN-GCF inserts a trainable Fourier series transform into the interaction branch of NGCF-style message passing: $$\gamma(v)=\{a_1 \cos(2\pi \mathbf b_1^\top v),\; a_1 \sin(2\pi \mathbf b_1^\top v),\; \ldots,\; a_m \cos(2\pi \mathbf b_m^\top v),\; a_m \sin(2\pi \mathbf b_m^\top v)\},$$06 with an analogous item update. The authors’ ablation argues that removing $$\gamma(v)=\{a_1 \cos(2\pi \mathbf b_1^\top v),\; a_1 \sin(2\pi \mathbf b_1^\top v),\; \ldots,\; a_m \cos(2\pi \mathbf b_m^\top v),\; a_m \sin(2\pi \mathbf b_m^\top v)\},$$07 tends to help, while removing the whole interaction term hurts, especially on denser data. Replacing the transform by $$\gamma(v)=\{a_1 \cos(2\pi \mathbf b_1^\top v),\; a_1 \sin(2\pi \mathbf b_1^\top v),\; \ldots,\; a_m \cos(2\pi \mathbf b_m^\top v),\; a_m \sin(2\pi \mathbf b_m^\top v)\},$$08 yields the best reported performance among the compared graph collaborative filtering methods, including $$\gamma(v)=\{a_1 \cos(2\pi \mathbf b_1^\top v),\; a_1 \sin(2\pi \mathbf b_1^\top v),\; \ldots,\; a_m \cos(2\pi \mathbf b_m^\top v),\; a_m \sin(2\pi \mathbf b_m^\top v)\},$$09 and $$\gamma(v)=\{a_1 \cos(2\pi \mathbf b_1^\top v),\; a_1 \sin(2\pi \mathbf b_1^\top v),\; \ldots,\; a_m \cos(2\pi \mathbf b_m^\top v),\; a_m \sin(2\pi \mathbf b_m^\top v)\},$$10 on MOOC, and $$\gamma(v)=\{a_1 \cos(2\pi \mathbf b_1^\top v),\; a_1 \sin(2\pi \mathbf b_1^\top v),\; \ldots,\; a_m \cos(2\pi \mathbf b_m^\top v),\; a_m \sin(2\pi \mathbf b_m^\top v)\},$$11 and $$\gamma(v)=\{a_1 \cos(2\pi \mathbf b_1^\top v),\; a_1 \sin(2\pi \mathbf b_1^\top v),\; \ldots,\; a_m \cos(2\pi \mathbf b_m^\top v),\; a_m \sin(2\pi \mathbf b_m^\top v)\},$$12 on Amazon Games (Xu et al., 2024).

In ultrasound segmentation, FFPN treats contours rather than pixel masks as the primary prediction target. Closed contours are represented by elliptic Fourier descriptors,

$$\gamma(v)=\{a_1 \cos(2\pi \mathbf b_1^\top v),\; a_1 \sin(2\pi \mathbf b_1^\top v),\; \ldots,\; a_m \cos(2\pi \mathbf b_m^\top v),\; a_m \sin(2\pi \mathbf b_m^\top v)\},$$13

with $$\gamma(v)=\{a_1 \cos(2\pi \mathbf b_1^\top v),\; a_1 \sin(2\pi \mathbf b_1^\top v),\; \ldots,\; a_m \cos(2\pi \mathbf b_m^\top v),\; a_m \sin(2\pi \mathbf b_m^\top v)\},$$14, giving $$\gamma(v)=\{a_1 \cos(2\pi \mathbf b_1^\top v),\; a_1 \sin(2\pi \mathbf b_1^\top v),\; \ldots,\; a_m \cos(2\pi \mathbf b_m^\top v),\; a_m \sin(2\pi \mathbf b_m^\top v)\},$$15 parameters per contour. The key architectural move is to align Fourier orders with FPN levels so that coefficients of similar scale and frequency are predicted at the same pyramid level. On the 2CH dataset, FFPN reports DICE $$\gamma(v)=\{a_1 \cos(2\pi \mathbf b_1^\top v),\; a_1 \sin(2\pi \mathbf b_1^\top v),\; \ldots,\; a_m \cos(2\pi \mathbf b_m^\top v),\; a_m \sin(2\pi \mathbf b_m^\top v)\},$$16, IoU $$\gamma(v)=\{a_1 \cos(2\pi \mathbf b_1^\top v),\; a_1 \sin(2\pi \mathbf b_1^\top v),\; \ldots,\; a_m \cos(2\pi \mathbf b_m^\top v),\; a_m \sin(2\pi \mathbf b_m^\top v)\},$$17, HD $$\gamma(v)=\{a_1 \cos(2\pi \mathbf b_1^\top v),\; a_1 \sin(2\pi \mathbf b_1^\top v),\; \ldots,\; a_m \cos(2\pi \mathbf b_m^\top v),\; a_m \sin(2\pi \mathbf b_m^\top v)\},$$18, Conf $$\gamma(v)=\{a_1 \cos(2\pi \mathbf b_1^\top v),\; a_1 \sin(2\pi \mathbf b_1^\top v),\; \ldots,\; a_m \cos(2\pi \mathbf b_m^\top v),\; a_m \sin(2\pi \mathbf b_m^\top v)\},$$19, memory $$\gamma(v)=\{a_1 \cos(2\pi \mathbf b_1^\top v),\; a_1 \sin(2\pi \mathbf b_1^\top v),\; \ldots,\; a_m \cos(2\pi \mathbf b_m^\top v),\; a_m \sin(2\pi \mathbf b_m^\top v)\},$$20 GB, and FPS $$\gamma(v)=\{a_1 \cos(2\pi \mathbf b_1^\top v),\; a_1 \sin(2\pi \mathbf b_1^\top v),\; \ldots,\; a_m \cos(2\pi \mathbf b_m^\top v),\; a_m \sin(2\pi \mathbf b_m^\top v)\},$$21; the refined FFPN-R reports DICE $$\gamma(v)=\{a_1 \cos(2\pi \mathbf b_1^\top v),\; a_1 \sin(2\pi \mathbf b_1^\top v),\; \ldots,\; a_m \cos(2\pi \mathbf b_m^\top v),\; a_m \sin(2\pi \mathbf b_m^\top v)\},$$22, IoU $$\gamma(v)=\{a_1 \cos(2\pi \mathbf b_1^\top v),\; a_1 \sin(2\pi \mathbf b_1^\top v),\; \ldots,\; a_m \cos(2\pi \mathbf b_m^\top v),\; a_m \sin(2\pi \mathbf b_m^\top v)\},$$23, and HD $$\gamma(v)=\{a_1 \cos(2\pi \mathbf b_1^\top v),\; a_1 \sin(2\pi \mathbf b_1^\top v),\; \ldots,\; a_m \cos(2\pi \mathbf b_m^\top v),\; a_m \sin(2\pi \mathbf b_m^\top v)\},$$24. On FH, FFPN-R reaches DICE $$\gamma(v)=\{a_1 \cos(2\pi \mathbf b_1^\top v),\; a_1 \sin(2\pi \mathbf b_1^\top v),\; \ldots,\; a_m \cos(2\pi \mathbf b_m^\top v),\; a_m \sin(2\pi \mathbf b_m^\top v)\},$$25 and HD $$\gamma(v)=\{a_1 \cos(2\pi \mathbf b_1^\top v),\; a_1 \sin(2\pi \mathbf b_1^\top v),\; \ldots,\; a_m \cos(2\pi \mathbf b_m^\top v),\; a_m \sin(2\pi \mathbf b_m^\top v)\},$$26 (Chen et al., 2023).

6. Interpretability, robustness, and conceptual boundaries

Fourier Feature Networks are not only generative or predictive models; they also support attribution and explanatory analysis in the spectral domain. “Fourier Feature Attribution: A New Efficiency Attribution Method” argues that deletion/insertion games are conceptually better aligned with Fourier coefficients than with pixels because zeroing a Fourier coefficient removes an additive signal component, whereas zeroing a pixel inserts a pulse-like signal. The proposed attribution score is based on an error-response signal,

$$\gamma(v)=\{a_1 \cos(2\pi \mathbf b_1^\top v),\; a_1 \sin(2\pi \mathbf b_1^\top v),\; \ldots,\; a_m \cos(2\pi \mathbf b_m^\top v),\; a_m \sin(2\pi \mathbf b_m^\top v)\},$$27

and the final score is a self-energy difference,

$$\gamma(v)=\{a_1 \cos(2\pi \mathbf b_1^\top v),\; a_1 \sin(2\pi \mathbf b_1^\top v),\; \ldots,\; a_m \cos(2\pi \mathbf b_m^\top v),\; a_m \sin(2\pi \mathbf b_m^\top v)\},$$28

On ImageNet, the paper reports that for ViT_B_32 only 8% of the Fourier features are required to preserve the original predictions for 80% of the samples; for ResNet50, about 10% preserve decisions for 70% of samples. The same study reports kurtosis values near $$\gamma(v)=\{a_1 \cos(2\pi \mathbf b_1^\top v),\; a_1 \sin(2\pi \mathbf b_1^\top v),\; \ldots,\; a_m \cos(2\pi \mathbf b_m^\top v),\; a_m \sin(2\pi \mathbf b_m^\top v)\},$$29 for Fourier attributions in intra-class concentration analysis and finds higher inter-class distinctiveness than spatial attribution methods (Liu et al., 2 Apr 2025).

Robustness under noise has produced a second line of work. The diagonal-layer construction of (Jeong et al., 2024) is motivated by the observation that ordinary Fourier embeddings may still yield high generalization error when labels or measurements are noisy. By replacing the first dense layer with per-coordinate gating, the network is argued to learn sparse Fourier features through implicit regularization. The theory relies on Chebyshev expansions of both the target link functions and the ReLU activation and shows that, under stated support-separation assumptions, the gradient aligns with the true Fourier coefficients. Empirically, the diagonal models activate far fewer Fourier modes than standard Fourier-embedded dense networks and generalize better on noisy synthetic and semi-synthetic data (Jeong et al., 2024).

The term also has a wider conceptual boundary than the input-embedding view alone suggests. The “Linear Time Complexity Deep Fourier Scattering Network” is a deterministic cascade of fixed wavelet filters, quadratic nonlinearities $$\gamma(v)=\{a_1 \cos(2\pi \mathbf b_1^\top v),\; a_1 \sin(2\pi \mathbf b_1^\top v),\; \ldots,\; a_m \cos(2\pi \mathbf b_m^\top v),\; a_m \sin(2\pi \mathbf b_m^\top v)\},$$30, and invariant summary statistics implemented entirely in the Fourier domain. It exploits high sparsity of wavelets in frequency, reports approximately 98% sparsity for large signals under realistic settings, and reaches true linear-time behavior in $$\gamma(v)=\{a_1 \cos(2\pi \mathbf b_1^\top v),\; a_1 \sin(2\pi \mathbf b_1^\top v),\; \ldots,\; a_m \cos(2\pi \mathbf b_m^\top v),\; a_m \sin(2\pi \mathbf b_m^\top v)\},$$31. On bird-song classification, the reported performance is 47.6% accuracy and 52.4% MAP using only deterministic features and a random forest classifier (Balestriero et al., 2017). This usage shows that some papers treat “Fourier network” as a broader spectral-processing architecture rather than a trainable sine/cosine positional encoding.

Several recurring design tensions emerge across the literature. Frequency-scale selection remains central: (Tancik et al., 2020) recommends sweeping $$\gamma(v)=\{a_1 \cos(2\pi \mathbf b_1^\top v),\; a_1 \sin(2\pi \mathbf b_1^\top v),\; \ldots,\; a_m \cos(2\pi \mathbf b_m^\top v),\; a_m \sin(2\pi \mathbf b_m^\top v)\},$$32 on a validation set, (Wang et al., 2020) stresses that mismatched scales can lead to slow convergence or overfitting, (Tang et al., 2024) introduces trainable frequencies to reduce manual tuning, and (Ren et al., 14 Oct 2025) proposes output-weight diagnostics for frequency-range initialization. Basis design is similarly unresolved: (Jandrell et al., 27 Aug 2025) uses hand-designed frequencies guided by observed phase wraps, (Straub et al., 1 Apr 2025) uses cosine-only embeddings chosen for boundary flatness, and (Wu et al., 22 Oct 2025) moves Fourier enhancement into latent space and turns output fitting into a convex regression stage for linear PDEs. A plausible implication is that “Fourier Feature Network” now denotes a research program as much as an architecture family: the common objective is to reshape representation and optimization around frequency structure, but the specific place where the spectral prior is injected remains highly task-dependent.