Frequency-Guided Physics-Informed Neural Networks

Updated 22 November 2025

Frequency-Guided Physics-Informed Neural Networks (FG-PINNs) are advanced architectures that embed frequency information to counteract spectral bias in solving high-frequency and multi-scale PDEs.
They employ strategies such as two-branch frequency decomposition, sinusoidal feature augmentation, and curriculum learning to accelerate convergence and enhance performance.
Empirical results demonstrate significant reductions in training time and error, with applications in wave propagation, convection-diffusion, and other complex PDE scenarios.

Frequency-Guided Physics-Informed Neural Networks (FG-PINNs) are a class of neural architectures and training methodologies developed to overcome the spectral bias and convergence limitations of standard PINNs when solving partial differential equations (PDEs) with high-frequency or multi-scale solutions. FG-PINNs employ explicit encoding of frequency information—through architectural, feature-mapping, or optimization strategies—to accelerate the learning of both high-frequency and low-frequency components, often by leveraging prior knowledge about the problem’s frequency content, introducing frequency-adaptive subnetworks, or augmenting the input layer with engineered sinusoidal features.

1. Motivation and Theoretical Foundation

Physics-Informed Neural Networks (PINNs) embed the residual of PDE operators into the loss during training. Despite their empirical success on smooth or low-frequency problems, PINNs are affected by spectral bias: fully-connected MLPs initialized in standard ways prioritize the learning of low-frequency solution components and exhibit very slow or unstable convergence for highly oscillatory targets. This spectral bias arises due to the small variance of initial input gradients and the inductive biases of common activation functions, as detailed in theoretical analyses of sinusoidal-feature PINNs (Wong et al., 2021).

FG-PINNs address this limitation by explicitly guiding the network’s representation toward the relevant frequency content—either by embedding frequency priors in the architecture, decomposing the target into high- and low-frequency branches, or incorporating feature maps (e.g., Fourier, Gabor, or transfer-learned) aligned to the physics. Theoretical analyses demonstrate that such feature enrichments can strictly enlarge the hypothesis space, allow for universal approximation with controlled error, and provide provable convergence improvements for linear PDEs (Wu et al., 22 Oct 2025).

2. Frequency-Guided Architectural Variants

Several distinct architectural prototypes of FG-PINNs have been proposed:

Two-Branch Frequency Decomposition: The network output is decomposed as $u_{nn}(x,t)=u_H(x,t)+u_L(x,t)$ , with $u_H$ a high-frequency subnetwork designed to rapidly capture oscillatory components—often using explicit frequency priors from the PDE’s source term or boundary/initial data—and $u_L$ as a standard feed-forward low-frequency branch. The high-frequency subnetwork may fuse the normalized source term directly into each hidden layer, using specialized attention-inspired modules to accelerate high-frequency learning (Zheng et al., 15 Nov 2025).
Feature-Based Frequency Encoding: Input features are augmented with sinusoidal mappings—e.g., $\phi(x)=\sin(2\pi \Omega x + b)$ with trainable or fixed frequencies—so that the network’s input gradients match the spectrum of the expected solution (Wong et al., 2021). Random Fourier Feature (RFF) layers, projecting latent representations into randomized Fourier bases, have also been shown to enable universal kernel approximation and strict mitigation of spectral bias (Wu et al., 22 Oct 2025). Gabor-filtered layers, which encode oscillatory and localized basis functions aligned to the dominant physical frequencies, provide strong performance on wave problems (Huang et al., 2023, Abedi et al., 24 Feb 2025).
Curriculum and Transfer-Learning Strategies: Instead of explicit architectural modification, the “frequency-guided” approach can be realized by staged training—first solving easy (low-frequency) problems and transferring weights to consecutively harder (higher-frequency) regimes (Mustajab et al., 2024). This exploits the spectral learning path of PINNs and enables rapid convergence even on problems where direct training fails.

3. Mathematical Formulation and Loss Structures

FG-PINNs employ the standard composite PINN loss,

$L(\theta) = \lambda_F L_F(\theta) + \lambda_B L_B(\theta) + \lambda_I L_I(\theta),$

where the physics (interior) loss $L_F$ , boundary loss $L_B$ , and initial loss $L_I$ , are computed at collocation points.

For two-branch frequency decomposition (Zheng et al., 15 Nov 2025), with $u_{nn}(x,t) = u_H(x,t;\theta_H) + u_L(x,t;\theta_L)$ , the residuals are: $r(x^i, t^i) = \mathcal{L}[u_H + u_L](x^i, t^i) - f(x^i, t^i),$

$r_b(x_b^i, t_b^i) = u_H + u_L(x_b^i, t_b^i) - h(x_b^i, t_b^i),$

$r_0(x_0^i) = u_H + u_L(x_0^i, 0) - g(x_0^i).$

Weighted squared errors with exponential or data-driven balancing are used to stabilize training, particularly when the source term $f$ has large amplitude.

Feature-encoded FG-PINNs (Wu et al., 22 Oct 2025, Wong et al., 2021) introduce basis parameterizations and RFF or sinusoidal feature layers. In the RFF case,

$\psi_D(x) = \frac{1}{\sqrt{D}} [\cos(2\pi B_D h_\omega(x)),\, \sin(2\pi B_D h_\omega(x))]$

is used to augment $h_\omega(x)$ , with a bi-level optimization scheme: inner-level coefficient regression using closed-form convex optimization, and outer-level gradient descent for the nonlinear subnetwork.

GaborPINN (Huang et al., 2023) and variants modulate every hidden layer with a Gabor filter of the input coordinates, matching frequency content to the physically dominant modes.

4. Empirical Performance and Benchmark Results

FG-PINNs consistently achieve superior accuracy and drastically accelerated convergence rates on high-frequency and multi-scale PDEs compared to standard PINNs and other baselines:

Method/Problem	Rel. $L^2$ error (high-freq)	Epochs to target error	Key features
FG-PINN (RFF) (Wu et al., 22 Oct 2025)	$O(10^{-3})$ to $O(10^{-5})$	$10^3$ – $10^4$	RFF features, bi-level
FG-PINN (decomp.) (Zheng et al., 15 Nov 2025)	$3\times10^{-4}$ to $2\times10^{-3}$	$2\times10^3$ – $6\times10^3$	Two-branch, source prior
GaborPINN (Huang et al., 2023)	<1% error (4–16 Hz Helmholtz)	$\sim 600$ –1,000 epochs	Gabor filters, initialization
Curriculum FG-PINN (Mustajab et al., 2024)	≤1%	$600$–2,500 (transfer)	staged frequency training
sf-PINN (Wong et al., 2021)	$2\times10^{-5}$ (Helmholtz)	$<0.5\times$ standard	learnable sinusoid features

All methods report order-of-magnitude reductions in training time, improved stability, and consistently lower errors, especially for tasks such as high-frequency wave propagation, convection-diffusion with multi-scale structure, and oscillatory ODEs and PDEs. Detailed hyperparameter guidelines specify spectral bandwidth, size of RFF/Gabor banks, and loss balancing.

5. Theoretical Analysis and Mitigation of Spectral Bias

FG-PINNs provide both empirical and theoretical guarantees of spectral bias mitigation. For RFF-based variants (Wu et al., 22 Oct 2025), the addition of a sufficiently rich Fourier feature basis expands the representable space to a universal RKHS, theoretically guaranteeing that all solution frequencies up to the feature bandwidth can be learned efficiently. Explicit convergence results are available for linear problems, including stationary-point convergence of the bi-level scheme.

In two-branch architectures with embedded priors (Zheng et al., 15 Nov 2025), the spectral bias of standard smooth-activation MLPs is counteracted by hardwiring the oscillatory structure into the network, enabling rapid exponential convergence of individual frequency modes. Sinusoidal feature mappings (Wong et al., 2021) are shown to maximize input gradient variability and thereby increase the probability of escaping flat local minima associated with low-frequency bias.

Ablation studies and FFT-band analysis confirm that FG-PINNs recover high-frequency spectral amplitudes, while standard PINNs persistently underestimate these components.

6. Practical Implementation and Design Considerations

Implementation of FG-PINNs entails careful selection of feature mapping dimensions, spectral bandwidth parameters, normalization strategies, and loss term weighting. For architectures with frequency priors, normalization of source terms and fusion weights are necessary to prevent gradient pathologies. In RFF and sinusoidal mapping approaches, the number of features must be matched to the number of collocation points to ensure convexity and stability.

Curriculum-based FG-PINNs (Mustajab et al., 2024) do not require architecture modifications, but transfer is most effective when the initial frequency or problem parameter enables quick convergence, and the increment between staged problems is moderate. Optimizer order (Adam followed by L-BFGS) and sample selection (Latin Hypercube or Sobol sequences) further impact performance.

FG-PINNs contrast with hybrid frameworks that alternate classical iterative solvers and standard PINNs (e.g., PINN-MG (Dong et al., 2024)), which conceptually leverage frequency separation but do not employ explicit frequency-guided encoding or spectral filtering. Feature-engineered PINNs such as SAFE-NET (Fazliani et al., 11 Feb 2025) and Fourier-attention architectures (Arni et al., 6 Oct 2025) share design philosophy with FG-PINNs, though the latter distinguish themselves via explicit frequency-informed decomposition, prior embedding, or bi-level optimization.

The principal distinction between FG-PINNs and earlier adaptive or fixed-feature approaches is the systematic use of problem or physics-based frequency information to guide the representation, as opposed to learning all features from scratch. This results in faster convergence, more reliable solution of high-frequency PDEs, and a tractable path toward scalability in multi-scale and physically rigorous simulations.

Key literature: (Zheng et al., 15 Nov 2025, Wu et al., 22 Oct 2025, Wong et al., 2021, Huang et al., 2023, Abedi et al., 24 Feb 2025, Mustajab et al., 2024).