SV-SNN: Separated-Variable Spectral Neural Networks

• Separated-Variable Spectral Neural Networks (SV-SNN) are physics-informed models that use variable separation and adaptive Fourier features to accurately capture oscillatory, multiscale solutions in PDEs.
• The architecture decouples spatial and temporal complexities by employing independent univariate subnetworks and analytic spatial derivatives, thereby mitigating spectral bias common in standard PINNs.
• Empirical results demonstrate that SV-SNN delivers 1–3 orders of magnitude improvement in accuracy with fewer parameters and dramatically faster training on benchmarks like heat, Helmholtz, and Navier-Stokes equations.

Searching arXiv for the cited works and related papers on SV-SNN, SNN, and separable Fourier methods. Separated-Variable Spectral Neural Networks (SV-SNN) are a physics-informed neural architecture for high-frequency partial differential equations that integrates separation of variables with adaptive spectral methods. In the reported formulation, multivariate solutions are represented as sums of products of univariate factors, with independent spatial and temporal subnetworks, adaptive Fourier spectral features with learnable frequency parameters, and an SVD-based framework that quantifies spectral bias through Jacobian singular values and effective rank. The method is designed for regimes in which standard PINNs under-resolve oscillatory structure, especially in high-frequency, multiscale, and geometrically nontrivial PDEs (Xiong et al., 1 Aug 2025).

1. Problem setting and motivation

SV-SNN targets PDEs whose solutions contain strong oscillations or multi-scale structure. The benchmark classes explicitly include 1D and 2D Heat equations with high spatial frequencies $\kappa = 20\pi, 100\pi, 500\pi$, 2D Helmholtz equations with large wavenumber $\kappa = 24\pi, 48\pi$, 2D Poisson equations with complex geometry and highly oscillatory source terms, 2D nonlinear elliptic equations, and Navier–Stokes equations in both the Taylor–Green vortex and steady flow around two cylinders settings. These are presented as regimes in which standard PINNs often fail (Xiong et al., 1 Aug 2025).

The central obstruction is spectral bias, also described as the frequency principle. In this account, common neural networks fit low-frequency components early and easily, but learn high-frequency components much more slowly and often not at all within practical training time. For oscillatory PDEs, that failure mode appears as accurate fitting of mean trends with missing oscillations, large $L^2$ errors, stiff loss landscapes, and slow or stalled convergence. The same discussion identifies an architectural source of the problem: in conventional PINNs, all spatial and temporal variables are fed jointly to a single MLP, keeping the parameter space high-dimensional and ill-conditioned, making high-order, high-frequency automatic differentiation numerically fragile, and providing no mechanism to align the model with known physical frequency scales such as a wavenumber.

Within that framing, SV-SNN is not merely a change of basis but a restructuring of the neural approximation problem. The separation of variables reduces joint-input coupling, and the spectral parameterization supplies explicit channels for oscillatory content. This suggests a closer alignment with classical high-frequency numerical analysis than with generic coordinate-based PINNs.

2. Separated-variable ansatz

The starting point is the classical separated representation

$$u(\mathbf{x}, t) \approx \sum_{n=1}^{N} c_n \left[ \prod_{j=1}^{d} X_n^{(j)}(x_j) \right] T_n(t),$$

which SV-SNN converts into a neural parameterization by replacing each univariate factor with a subnetwork. Two variants are distinguished. For elliptic and steady problems, the architecture uses a spatial spectral neural network,

$$u(\mathbf{x}; \Theta) = \sum_{n=1}^{N} c_n\, \Phi_n(\mathbf{x}; \Theta_n^{(s)}), \qquad \Phi_n(\mathbf{x}; \Theta_n^{(s)}) = \prod_{j=1}^{d} \Phi_n^{(j)}(x_j; \Theta_n^{(s,j)}).$$

For parabolic and hyperbolic problems, it uses a spatio-temporal spectral neural network,

$$u(\mathbf{x}, t; \Theta) = \sum_{n=1}^{N} c_n\, \Phi_n(\mathbf{x}; \Theta_n^{(s)})\, T_n(t; \Theta_n^{(t)}),$$

with the same spatial factorization.

The separated-variable structure is the defining feature. Each $\Phi_n^{(j)}$ is a univariate spatial spectral module acting on a single coordinate $x_j$, while each $T_n$ is a univariate temporal network. The coefficients $c_n$ are scalar modal weights. The resulting model

$\kappa = 24\pi, 48\pi$0

decouples spatial and temporal complexity and replaces a single map $\kappa = 24\pi, 48\pi$1 by products of one-dimensional approximants.

The stated consequence is a reduction of the curse of dimensionality. Rather than constructing a Cartesian spectral grid, SV-SNN uses a modal sum of products of 1D components. The paper characterizes the resulting parameter scaling as roughly $\kappa = 24\pi, 48\pi$2, linear in the dimension $\kappa = 24\pi, 48\pi$3 and in the number of modes and frequencies, rather than an exponential blow-up associated with a Cartesian Fourier grid. The same section connects this construction to classical eigenfunction expansions and spectral methods, but with learnable bases (Xiong et al., 1 Aug 2025).

3. Adaptive spectral modules and frequency initialization

The spatial modules are explicit trainable Fourier series. For each spatial coordinate,

$\kappa = 24\pi, 48\pi$4

The parameters are

$\kappa = 24\pi, 48\pi$5

The frequencies $\kappa = 24\pi, 48\pi$6 are trainable rather than fixed multiples of a base frequency, and the coefficients $\kappa = 24\pi, 48\pi$7, $\kappa = 24\pi, 48\pi$8 play the role of learned Fourier coefficients.

The temporal factors $\kappa = 24\pi, 48\pi$9 are small MLPs,

$L^2$0

The implementation notes describe these temporal subnetworks as usually shallow; in the heat and Navier–Stokes experiments, the reported choice is 4 layers with 10 neurons and $L^2$1 activation.

A key design element is adaptive Fourier feature initialization. For each spatial dimension $L^2$2, the method defines a characteristic frequency

$L^2$3

where the terms come from the Fourier transform of initial data, boundary condition spectra, forcing term spectra, eigenvalues or natural frequencies of the operator, and explicit frequency parameters appearing in the PDE. Frequencies are then initialized by a three-level scheme: $L^2$4 This initialization is described as covering low, mid, and high frequency ranges while concentrating many features around physically important frequencies. All $L^2$5 remain trainable during optimization.

The architecture is therefore spectral in a literal sense: the spatial part is an adaptive trigonometric expansion, and the separated-variable structure allocates those spectral modes coordinatewise. A plausible implication is that SV-SNN externalizes the frequency allocation problem that standard PINNs leave implicit inside a generic MLP.

4. Physics-informed objective and hybrid differentiation

SV-SNN is trained as a PINN with a composite initial-condition, PDE-residual, and boundary-condition loss: $L^2$6 The component losses are empirical squared errors on collocation sets. For example,

$L^2$7

with analogous expressions for $L^2$8 and $L^2$9. All later experiments set $$u(\mathbf{x}, t) \approx \sum_{n=1}^{N} c_n \left[ \prod_{j=1}^{d} X_n^{(j)}(x_j) \right] T_n(t),$$0.

The distinctive training mechanism is hybrid differentiation. Spatial derivatives are computed analytically from the Fourier representation, while temporal derivatives are computed by automatic differentiation on the temporal MLPs. For a separated spatial factor

$$u(\mathbf{x}, t) \approx \sum_{n=1}^{N} c_n \left[ \prod_{j=1}^{d} X_n^{(j)}(x_j) \right] T_n(t),$$1

any mixed derivative factorizes as

$$u(\mathbf{x}, t) \approx \sum_{n=1}^{N} c_n \left[ \prod_{j=1}^{d} X_n^{(j)}(x_j) \right] T_n(t),$$2

Because each $$u(\mathbf{x}, t) \approx \sum_{n=1}^{N} c_n \left[ \prod_{j=1}^{d} X_n^{(j)}(x_j) \right] T_n(t),$$3 is a sine-cosine expansion, the paper states that all spatial derivatives can be computed exactly. The stated purpose is to avoid numerical error accumulation and gradient pathologies associated with automatic differentiation of high-frequency spatial derivatives, while keeping the time-dependent part flexible.

The training workflow consists of pre-analysis to estimate characteristic frequencies, initialization of spectral and temporal parameters, sampling of IC, BC, and PDE collocation points, construction of the separated-variable network, evaluation of residuals with hybrid differentiation, and parameter updates with Adam using a cosine annealing schedule. The implementation details specify Adam with initial learning rate $$u(\mathbf{x}, t) \approx \sum_{n=1}^{N} c_n \left[ \prod_{j=1}^{d} X_n^{(j)}(x_j) \right] T_n(t),$$4, cosine annealing with decay factor $$u(\mathbf{x}, t) \approx \sum_{n=1}^{N} c_n \left[ \prod_{j=1}^{d} X_n^{(j)}(x_j) \right] T_n(t),$$5 every 500 epochs, 5,000 epochs for most problems, and 40,000 epochs for the challenging Poisson problem with complex source. Collocation points are sampled by Latin Hypercube Sampling or uniform grid for some initial conditions; for complex geometries, points are sampled in a bounding box and then points in holes are discarded (Xiong et al., 1 Aug 2025).

5. Jacobian-SVD theory of spectral bias

The theoretical account is built around Jacobians of the IC, PDE, and BC operators with respect to the parameter vector $$u(\mathbf{x}, t) \approx \sum_{n=1}^{N} c_n \left[ \prod_{j=1}^{d} X_n^{(j)}(x_j) \right] T_n(t),$$6. For each loss component $$u(\mathbf{x}, t) \approx \sum_{n=1}^{N} c_n \left[ \prod_{j=1}^{d} X_n^{(j)}(x_j) \right] T_n(t),$$7, the Jacobian is written as

$$u(\mathbf{x}, t) \approx \sum_{n=1}^{N} c_n \left[ \prod_{j=1}^{d} X_n^{(j)}(x_j) \right] T_n(t),$$8

with SVD

$$u(\mathbf{x}, t) \approx \sum_{n=1}^{N} c_n \left[ \prod_{j=1}^{d} X_n^{(j)}(x_j) \right] T_n(t),$$9

The effective rank at energy threshold $$u(\mathbf{x}; \Theta) = \sum_{n=1}^{N} c_n\, \Phi_n(\mathbf{x}; \Theta_n^{(s)}), \qquad \Phi_n(\mathbf{x}; \Theta_n^{(s)}) = \prod_{j=1}^{d} \Phi_n^{(j)}(x_j; \Theta_n^{(s,j)}).$$0 is defined by

$$u(\mathbf{x}; \Theta) = \sum_{n=1}^{N} c_n\, \Phi_n(\mathbf{x}; \Theta_n^{(s)}), \qquad \Phi_n(\mathbf{x}; \Theta_n^{(s)}) = \prod_{j=1}^{d} \Phi_n^{(j)}(x_j; \Theta_n^{(s,j)}).$$1

This quantity is used to formalize parameter space collapse: if $$u(\mathbf{x}; \Theta) = \sum_{n=1}^{N} c_n\, \Phi_n(\mathbf{x}; \Theta_n^{(s)}), \qquad \Phi_n(\mathbf{x}; \Theta_n^{(s)}) = \prod_{j=1}^{d} \Phi_n^{(j)}(x_j; \Theta_n^{(s,j)}).$$2, then most parameter directions are effectively inactive because their associated singular values are tiny. The gradient of each loss component expands in the right singular vectors,

$$u(\mathbf{x}; \Theta) = \sum_{n=1}^{N} c_n\, \Phi_n(\mathbf{x}; \Theta_n^{(s)}), \qquad \Phi_n(\mathbf{x}; \Theta_n^{(s)}) = \prod_{j=1}^{d} \Phi_n^{(j)}(x_j; \Theta_n^{(s,j)}).$$3

so directions with small singular values receive negligible updates. In the NTK view,

$$u(\mathbf{x}; \Theta) = \sum_{n=1}^{N} c_n\, \Phi_n(\mathbf{x}; \Theta_n^{(s)}), \qquad \Phi_n(\mathbf{x}; \Theta_n^{(s)}) = \prod_{j=1}^{d} \Phi_n^{(j)}(x_j; \Theta_n^{(s,j)}).$$4

and the residual dynamics satisfy

$$u(\mathbf{x}; \Theta) = \sum_{n=1}^{N} c_n\, \Phi_n(\mathbf{x}; \Theta_n^{(s)}), \qquad \Phi_n(\mathbf{x}; \Theta_n^{(s)}) = \prod_{j=1}^{d} \Phi_n^{(j)}(x_j; \Theta_n^{(s,j)}).$$5

The interpretation given is that high-frequency residuals correspond to small singular values or small NTK eigenvalues, so their errors decay very slowly.

SV-SNN is presented as altering this Jacobian spectrum in four ways: the separated-variable structure reduces redundancy and aligns parameters with physically meaningful modes; adaptive Fourier features allocate parameters directly to high-frequency sine and cosine modes; analytic spatial derivatives avoid numerical damping of high-frequency derivative information; and three-level frequency sampling ensures non-negligible representation of low and high frequencies from initialization. The empirical SVD analyses reported for the heat equation are striking: for $$u(\mathbf{x}; \Theta) = \sum_{n=1}^{N} c_n\, \Phi_n(\mathbf{x}; \Theta_n^{(s)}), \qquad \Phi_n(\mathbf{x}; \Theta_n^{(s)}) = \prod_{j=1}^{d} \Phi_n^{(j)}(x_j; \Theta_n^{(s,j)}).$$6, SV-SNN has $$u(\mathbf{x}; \Theta) = \sum_{n=1}^{N} c_n\, \Phi_n(\mathbf{x}; \Theta_n^{(s)}), \qquad \Phi_n(\mathbf{x}; \Theta_n^{(s)}) = \prod_{j=1}^{d} \Phi_n^{(j)}(x_j; \Theta_n^{(s,j)}).$$7 and $$u(\mathbf{x}; \Theta) = \sum_{n=1}^{N} c_n\, \Phi_n(\mathbf{x}; \Theta_n^{(s)}), \qquad \Phi_n(\mathbf{x}; \Theta_n^{(s)}) = \prod_{j=1}^{d} \Phi_n^{(j)}(x_j; \Theta_n^{(s,j)}).$$8, versus PINN’s 3 and 7; for $$u(\mathbf{x}; \Theta) = \sum_{n=1}^{N} c_n\, \Phi_n(\mathbf{x}; \Theta_n^{(s)}), \qquad \Phi_n(\mathbf{x}; \Theta_n^{(s)}) = \prod_{j=1}^{d} \Phi_n^{(j)}(x_j; \Theta_n^{(s,j)}).$$9, SV-SNN has 145 and 111, versus PINN’s 2 and 5. Similar patterns are reported for Helmholtz and nonlinear elliptic cases (Xiong et al., 1 Aug 2025).

6. Reported empirical performance

The reported empirical picture is consistent across oscillatory PDE classes. On the 1D heat equation with high-frequency initial condition $$u(\mathbf{x}, t; \Theta) = \sum_{n=1}^{N} c_n\, \Phi_n(\mathbf{x}; \Theta_n^{(s)})\, T_n(t; \Theta_n^{(t)}),$$0, SV-SNN uses 3,730 parameters at $$u(\mathbf{x}, t; \Theta) = \sum_{n=1}^{N} c_n\, \Phi_n(\mathbf{x}; \Theta_n^{(s)})\, T_n(t; \Theta_n^{(t)}),$$1 and achieves ReL2E $$u(\mathbf{x}, t; \Theta) = \sum_{n=1}^{N} c_n\, \Phi_n(\mathbf{x}; \Theta_n^{(s)})\, T_n(t; \Theta_n^{(t)}),$$2 and MAPE $$u(\mathbf{x}, t; \Theta) = \sum_{n=1}^{N} c_n\, \Phi_n(\mathbf{x}; \Theta_n^{(s)})\, T_n(t; \Theta_n^{(t)}),$$3, while the compared PINN uses 40,801 parameters and yields ReL2E $$u(\mathbf{x}, t; \Theta) = \sum_{n=1}^{N} c_n\, \Phi_n(\mathbf{x}; \Theta_n^{(s)})\, T_n(t; \Theta_n^{(t)}),$$4 and MAPE $$u(\mathbf{x}, t; \Theta) = \sum_{n=1}^{N} c_n\, \Phi_n(\mathbf{x}; \Theta_n^{(s)})\, T_n(t; \Theta_n^{(t)}),$$5. At $$u(\mathbf{x}, t; \Theta) = \sum_{n=1}^{N} c_n\, \Phi_n(\mathbf{x}; \Theta_n^{(s)})\, T_n(t; \Theta_n^{(t)}),$$6, SV-SNN uses 1,612 parameters and achieves ReL2E $$u(\mathbf{x}, t; \Theta) = \sum_{n=1}^{N} c_n\, \Phi_n(\mathbf{x}; \Theta_n^{(s)})\, T_n(t; \Theta_n^{(t)}),$$7, whereas the PINN uses 58,561 parameters and remains at ReL2E $$u(\mathbf{x}, t; \Theta) = \sum_{n=1}^{N} c_n\, \Phi_n(\mathbf{x}; \Theta_n^{(s)})\, T_n(t; \Theta_n^{(t)}),$$8. At $$u(\mathbf{x}, t; \Theta) = \sum_{n=1}^{N} c_n\, \Phi_n(\mathbf{x}; \Theta_n^{(s)})\, T_n(t; \Theta_n^{(t)}),$$9, SV-SNN uses 1,412 parameters and keeps ReL2E at $\Phi_n^{(j)}$0, while the PINN again remains near 1 (Xiong et al., 1 Aug 2025).

On 2D Helmholtz, the same pattern persists. For $\Phi_n^{(j)}$1, SV-SNN uses 2,322 parameters and reports ReL2E $\Phi_n^{(j)}$2, while the PINN gives ReL2E $\Phi_n^{(j)}$3. For $\Phi_n^{(j)}$4, SV-SNN uses 3,096 parameters and reports ReL2E $\Phi_n^{(j)}$5, with the PINN again near 1. Against stronger baselines on Helmholtz $\Phi_n^{(j)}$6, trained for 50,000 epochs versus 5,000 for SV-SNN, the reported AvgReL2E and training times are XPINN $\Phi_n^{(j)}$7, 7,965 s; FBPINN $\Phi_n^{(j)}$8, 14,256 s; FourierPINN $\Phi_n^{(j)}$9, 9,273 s; BsPINN $x_j$0, 13,127 s; and SV-SNN $x_j$1, 132.5 s. For $x_j$2, the paper reports a comparable gap, with SV-SNN near $x_j$3 and the best listed baseline near $x_j$4.

The gains extend beyond Helmholtz. For the nonlinear elliptic problem, SV-SNN uses 780 parameters and attains ReL2E $x_j$5, while the PINN reports ReL2E $x_j$6. For the Poisson problem with complex source term and $x_j$7, SV-SNN uses 1,212 parameters and reports ReL2E $x_j$8 and MAPE $x_j$9, while the PINN uses 20,601 parameters and reports ReL2E $T_n$0 and MAPE $T_n$1. For the Poisson problem on a complex geometry with $T_n$2, SV-SNN reports ReL2E $T_n$3, whereas the PINN remains at $T_n$4.

For incompressible flow, the Taylor–Green vortex experiment reports 2,688 parameters, final loss $T_n$5, and relative errors $T_n$6 for $T_n$7, $T_n$8 for $T_n$9, and $c_n$0 for $c_n$1. The PINN baseline, with 41,103 parameters, reports final loss $c_n$2 and relative errors approximately 0.685 for $c_n$3, 0.684 for $c_n$4, and 1.67 for $c_n$5. In steady Navier–Stokes flow around two cylinders, SV-SNN reports 404 parameters, total loss $c_n$6, and ReL2E $c_n$7 for $c_n$8 and $c_n$9 for $\kappa = 24\pi, 48\pi$00, while the PINN reports 7,953 parameters, total loss $\kappa = 24\pi, 48\pi$01, and ReL2E $\kappa = 24\pi, 48\pi$02 for $\kappa = 24\pi, 48\pi$03 and $\kappa = 24\pi, 48\pi$04 for $\kappa = 24\pi, 48\pi$05. The paper summarizes these outcomes as 1–3 orders of magnitude improvement in accuracy, more than 90% reduction in parameters, and about 60× speedup in wall-clock training on high-frequency Helmholtz problems (Xiong et al., 1 Aug 2025).

7. Related formulations, scope, and later comparisons

The label “spectral neural network” predates SV-SNN and refers to several distinct constructions. In graph-based manifold learning, an SNN is trained by minimizing

$\kappa = 24\pi, 48\pi$06

so that the network outputs approximate leading eigenvectors of a normalized graph operator. That formulation already has a latent separated-variable structure, because the network is analyzed as learning coordinates

$\kappa = 24\pi, 48\pi$07

separating a mode index from spatial dependence on $\kappa = 24\pi, 48\pi$08 (Li et al., 2023).

A different SNN line in matrix sensing parameterizes matrices by singular values and singular vectors rather than entries, applying nonlinearities only to singular values. Under its stated assumptions, the architecture is already separated-variable in the sense of fixed directions and learned magnitudes, and gradient flow converges to the interpolating solution of minimal nuclear norm. That work therefore supplies a distinct spectral-coordinate interpretation of separation, centered on singular-vector frames rather than PDE variables (Chu et al., 2024).

Frequency-separable decomposition also appears in dynamical-systems modeling. The Frequency-Separable Hamiltonian Neural Network (FS-HNN) decomposes an effective Hamiltonian into multiple components trained on data sampled at distinct temporal resolutions and recomposes them through a multiscale combiner. Its separation is temporal rather than spatial, but it is explicitly framed as frequency separation and as a way to mitigate spectral bias in multi-timescale Hamiltonian systems (Li et al., 6 Mar 2026).

The SV-SNN formulation for PDEs also comes with clear scope conditions. Its expressivity assumes that the solution is reasonably approximable by a low-rank separated expansion

$\kappa = 24\pi, 48\pi$09

If the true solution has intrinsically high separation rank or strong non-separable interactions, many modes may be required. The Fourier basis is described as ideal for smooth, periodic, or oscillatory solutions on simple domains; for nonperiodic boundaries, discontinuities, or strongly localized features, the paper identifies Chebyshev and wavelets as plausible alternatives not yet integrated. It also notes that the basis remains rectangular in coordinates even when geometry is handled through collocation, and that strongly nonlinear phenomena with shocks or discontinuities are not addressed (Xiong et al., 1 Aug 2025).

A later comparison introduces Multi-Scale Separable Fourier Neural Networks (MS-SFNN) and treats SV-SNN as a baseline specifically designed for high-frequency PDEs. On shared benchmarks, that paper reports substantially smaller errors for the random-feature, least-squares, analytically differentiated alternative; for example, on 2D Helmholtz with $\kappa = 24\pi, 48\pi$10, it reports $\kappa = 24\pi, 48\pi$11 for MS-SFNN versus $\kappa = 24\pi, 48\pi$12 for SV-SNN, and on the complex-source Poisson problem it reports $\kappa = 24\pi, 48\pi$13 versus $\kappa = 24\pi, 48\pi$14. The same comparison argues for per-dimension frequency scaling, analytic derivatives, and batched QR least-squares as key advantages, while still preserving the underlying separated-variable spectral viewpoint (Yang et al., 29 May 2026).