Frequency Shift Physics-Informed Extreme Learning Machine for Solving High-Frequency Partial Differential Equations

Abstract: Solving partial differential equations (PDEs) with high-frequency solutions remains a central challenge in physics-informed machine learning due to spectral bias -- the tendency of neural networks to learn low-frequency components preferentially. This paper proposes a Frequency Shift Physics-Informed Extreme Learning Machine (FS-PIELM) framework that addresses this limitation through an additive mechanism for weight initialization. Rather than multiplying random weights by a scaling factor, the method translates the mean of the Gaussian weight distribution while keeping the variance fixed at unity, thereby avoiding the variance amplification inherent in scaling-based methods. Two variants are developed: FS-PIELM-L assigns independent frequency magnitudes to individual neurons, while FS-PIELM-G groups neurons for improved robustness. Theoretical analysis shows that the frequency variance under the proposed framework remains bounded and approaches unity regardless of target frequency, in contrast to the quadratic growth of conventional approaches. The method preserves the computational efficiency of extreme learning machines, requiring only a single linear solve. Experiments on seven benchmark problems spanning six equation types -- Helmholtz, wave, Poisson, Klein-Gordon, heat, and advection-diffusion -- on both regular and complex geometries show that the linear variant achieves the best accuracy in six of seven cases, with improvements of one to nearly five orders of magnitude over existing PIELM variants. The code and data accompanying this manuscript will be made publicly available at https://github.com/xgxgnpu/Physics-informed-vibe-coding/tree/main/FS-PIELM.

• The paper introduces a mean-shift physics-informed extreme learning machine that maintains unit variance to tightly control neuron frequencies for high-frequency PDEs.
• It compares traditional scaling methods with the new mean-shift approach, demonstrating up to five orders of magnitude error reduction on benchmark problems.
• The FS-PIELM framework is applied across elliptic, parabolic, and hyperbolic PDEs, showing remarkable stability and robustness even in complex geometries.

Frequency Shift Physics-Informed Extreme Learning Machine for High-Frequency PDEs

Introduction and Motivation

Solving PDEs with high-frequency solutions is a central challenge in PINNs and physics-informed ML due to strong spectral bias—parametric NNs systematically prioritize low-frequency components, with convergence for high-frequency features orders of magnitude slower. While approaches such as Fourier Feature Mappings and SIREN address this by directly encoding higher frequency information, they introduce frequency-scale hyperparameters with significant variance inflation for large target frequencies due to their multiplicative scaling of the input layer. This causes the effective frequency distribution to become highly dispersed as the scaling factor increases, fundamentally limiting expressive power and stability for high-frequency regimes.

The FS-PIELM framework introduces a new mechanism: instead of scaling the variance (as in classical Fourier and SIREN embeddings), the method shifts the mean of the Gaussian weight distribution for the single-hidden-layer ELM, maintaining fixed unit variance. This mean shift enables tight concentration of neuron effective frequencies around prescribed targets without quadratic growth in variance. The framework’s theoretical and empirical properties are exhaustively analyzed, with two implementation variants—FS-PIELM-L (neuron-wise) and FS-PIELM-G (group-wise)—and demonstrated on elliptic, parabolic, and hyperbolic PDEs with complex geometries and multi-scale spectral content.

Frequency Control Mechanism: Additive Mean Shift vs. Variance Scaling

Traditional frequency augmentation (e.g., GFF-PIELM, SIREN-PIELM) multiplies base random Gaussian weights by a scaling factor $\delta_m$. This quadratically increases the variance of weight magnitudes with $\delta_m^2$, causing the effective frequency in the cosine/sine basis to become widely scattered. In contrast, FS-PIELM generates each hidden-layer weight as

$$\mathbf{w}_m = \mu_m \mathbf{d}_m + \boldsymbol{\epsilon}_m,\quad \boldsymbol{\epsilon}_m \sim \mathcal{N}(0, I)$$

where $\mu_m$ is the desired mean magnitude and $\mathbf{d}_m$ is a random direction on the unit sphere. The variance remains unity for all $\mu_m$.

Figure 1: Comparison of frequency control mechanisms for scaling-based and mean-shift approaches; scaling-based variance grows quadratically, while mean-shift variance remains constant.

FS-PIELM Architecture and Implementation

The FS-PIELM network is a single-hidden-layer ELM with basis activations $$h_m(\mathbf{x}) = \cos(\mathbf{w}_m^T \mathbf{x} + b_m)$$, with the weights $\mathbf{w}_m$ sampled via the frequency shift mechanism. The mean parameters $\mu_m$ are linearly distributed between $\mu_{\min}$ and $\delta_m^2$0, ensuring spectral coverage over the desired frequency band.

• FS-PIELM-L: Allocates a unique $\delta_m^2$1 to each neuron.
• FS-PIELM-G: Neurons are assigned to $\delta_m^2$2 groups; group members share a common mean magnitude for enhanced robustness.

Derivative computation for enforcing PDE constraints via automatic or analytic differentiation is simplified by this mechanism, with the form $\delta_m^2$3 and Laplacian operations straightforward and stable even for large $\delta_m^2$4.

Figure 2: Schematic of FS-PIELM, illustrating mean-shift frequency assignment and adaptive spectral coverage across the hidden layer.

Theoretical Analysis: Spectral Concentration and Variance

The paper provides explicit statistical comparisons between scaling-based and shift-based frequency assignment:

• Scaling-based: For target frequency $\delta_m^2$5 in dimension $\delta_m^2$6, matching the mean requires $\delta_m^2$7, producing $\delta_m^2$8, which leads to severe spread for large $\delta_m^2$9.
• Mean-shift-based: $$\mathbf{w}_m = \mu_m \mathbf{d}_m + \boldsymbol{\epsilon}_m,\quad \boldsymbol{\epsilon}_m \sim \mathcal{N}(0, I)$$0, and variance remains bounded by unity for large $$\mathbf{w}_m = \mu_m \mathbf{d}_m + \boldsymbol{\epsilon}_m,\quad \boldsymbol{\epsilon}_m \sim \mathcal{N}(0, I)$$1. Thus, neurons are reliably allocated to desired frequencies without wasted capacity or instability.

This spectral concentration is critical for representing oscillatory or multi-scale solutions of high-frequency PDEs.

Figure 3: Empirical frequency distribution for basis functions showing quadratic spread with scaling and tight concentration under mean-shifting.

Numerical Results

Comprehensive benchmark experiments include high-wavenumber Helmholtz equations, time-varying wave and Klein-Gordon equations, multi-frequency and multi-scale Poisson/heat equations, and advection-diffusion/Helmholtz problems on irregular (Pacman, Panda) domains.

• FS-PIELM-L achieves the best accuracy in six of seven cases, with errors reduced by up to five orders of magnitude compared to GFF-PIELM and SIREN-PIELM. These reductions in error are stable across random seeds due to the bounded-variance frequency assignment.
• On the 2D high-frequency Helmholtz equation ($$\mathbf{w}_m = \mu_m \mathbf{d}_m + \boldsymbol{\epsilon}_m,\quad \boldsymbol{\epsilon}_m \sim \mathcal{N}(0, I)$$2), FS-PIELM-L reached a best relative $$\mathbf{w}_m = \mu_m \mathbf{d}_m + \boldsymbol{\epsilon}_m,\quad \boldsymbol{\epsilon}_m \sim \mathcal{N}(0, I)$$3 error of $$\mathbf{w}_m = \mu_m \mathbf{d}_m + \boldsymbol{\epsilon}_m,\quad \boldsymbol{\epsilon}_m \sim \mathcal{N}(0, I)$$4—over $$\mathbf{w}_m = \mu_m \mathbf{d}_m + \boldsymbol{\epsilon}_m,\quad \boldsymbol{\epsilon}_m \sim \mathcal{N}(0, I)$$5 lower than GFF-PIELM.
• In the multi-scale 1D Poisson equation (spectral ratio 15:1), FS-PIELM-L achieves machine-precision accuracy ($$\mathbf{w}_m = \mu_m \mathbf{d}_m + \boldsymbol{\epsilon}_m,\quad \boldsymbol{\epsilon}_m \sim \mathcal{N}(0, I)$$6), with $$\mathbf{w}_m = \mu_m \mathbf{d}_m + \boldsymbol{\epsilon}_m,\quad \boldsymbol{\epsilon}_m \sim \mathcal{N}(0, I)$$7 improvement in error stability/resilience compared to standard methods.
• For the most challenging Panda-shaped domain and irregular boundary, FS-PIELM-L maintains up to $$\mathbf{w}_m = \mu_m \mathbf{d}_m + \boldsymbol{\epsilon}_m,\quad \boldsymbol{\epsilon}_m \sim \mathcal{N}(0, I)$$8 lower errors, demonstrating robustness to both geometric complexity and wide spectral demands.

Parameter sweeps, illustrated in figures (e.g., Figure 4, Figure 5–Figure 6), show that FS-PIELM-L error landscapes are flatter and less sensitive to frequency bounds than scaling-based baselines, which often exhibit narrow optimal regions due to variance inflation.

Practical and Theoretical Implications

The central implication is that the mean-shifted random feature approach enables explicit, robust control of the spectral distribution of basis functions in PIELM. This makes high-frequency PDE solvers feasible and stable even for high wavenumbers, multi-frequency solutions, and complex geometries where scaling-based random baselines are fundamentally variance-limited.

Pragmatically, FS-PIELM retains the computational simplicity and closed-form learning of ELM-based PINN variants—matrix assembly and SVD are performed with no asymptotic additional cost. Since frequency assignment and coverage do not interact pathologically with increasing frequency, spectrum allocation can be matched exactly to problem requirements, providing a stepping stone to future adaptive or data-driven spectral selection algorithms. The results also point towards directions for convergence rate analysis of random feature models with shift-based parameterization in high-frequency PDE learning.

Outlook for Further Development

• Adaptive frequency range selection: The manual specification of $$\mathbf{w}_m = \mu_m \mathbf{d}_m + \boldsymbol{\epsilon}_m,\quad \boldsymbol{\epsilon}_m \sim \mathcal{N}(0, I)$$9 is currently required; integrating network or PDE-aware adaptivity could further automate deployment.
• Extension to nonlinear PDEs: Iterative/nonlinear contexts may require coupling of frequency assignment with convegence acceleration and theory.
• Analytical bounds and random feature theory: The boundedness of variance under mean-shifted sampling invites refined analysis for error/approximation rates in infinite width limits and high-dimensional PDEs.
• Physics-informed architectures: Integration with advanced physics-driven architectures—hierarchical Fourier, Jacobi/KAN, or divided domain PINNs—will be facilitated by precise spectrum targeting in FS-PIELM.

Conclusion

The FS-PIELM framework for high-frequency PDEs provides a variance-controlled, mean-shifted random feature assignment for ELM-based physics-informed networks. By tightly concentrating the spectral content of each neuron around prescribed target frequencies, the method achieves dramatic (several orders of magnitude) improvements in error and reproducibility for oscillatory and multi-frequency PDEs compared to established scaling-based random feature approaches. The mean-shift paradigm constitutes an essential advance for stable, efficient, and accurate mesh-free PDE solvers in computational science, and sets the stage for further theoretical and architectural developments in PINN and physics-informed ML (2607.01694).