Physics-Encoded Spectral Regression

• Physics-encoded spectral regression is a framework that embeds physical laws, such as PDEs and conservation principles, into spectral methods to yield interpretable and bias-minimized models.
• The approach leverages adaptive spectral bases, physics-informed loss functions, and operator modeling to efficiently capture both low- and high-frequency solution components.
• Applications include solving high-frequency PDEs, optical inversion, and hyperspectral imaging, demonstrating significant gains in accuracy, convergence, and computational efficiency.

Physics-encoded spectral regression encompasses a set of methodological frameworks in computational science and machine learning that explicitly integrate physical knowledge—typically through the structure of partial differential equations (PDEs), conservation laws, or physical measurement models—into regression algorithms using spectral or modal representations. These approaches address the need for physically consistent, interpretable, and highly accurate spectral approximations or regressions, especially in problems characterized by high-frequency content, ill-posedness, or challenging inversion. The core idea is to “encode” the physical constraints either into the functional basis, the neural architecture, the loss landscape, or the measurement model, and to use “spectral” (frequency or modal) decompositions as the preferred representational substrate. This enables robust, bias-minimized learning and estimation far beyond what is typically available to black-box or naïve regression. The physics-encoded spectral regression paradigm has found sharp recent formulations in neural PDE solvers, polynomial optics-based regression, Koopman operator learning, Gaussian process PDE inference, symbolic regression for remote sensing, and hyperspectral imaging.

1. Mathematical and Algorithmic Formulations

A canonical physics-encoded spectral regression framework is the Separated-Variable Spectral Neural Network (SV-SNN) for high-frequency PDEs (Xiong et al., 1 Aug 2025). Here, the solution $u(\mathbf{x},t)$ is represented as a (learnable) sum over separated modal products,

$$u(\mathbf x,t) = \sum_{n=1}^N c_n \Bigl[ \prod_{i=1}^d f_{i,n}(x_i) \Bigr]\, g_n(t)$$

where:

• Each $f_{i,n}(x_i)$ is a one-dimensional, adaptive Fourier feature, e.g.

$$f_{i,n}(x_i) = \sum_{k=1}^{K_i} \left[ a^{(i)}_{n,k} \sin(\omega^{(i)}_{n,k} x_i) + b^{(i)}_{n,k} \cos(\omega^{(i)}_{n,k} x_i) \right] + \beta^{(i)}_n$$

with all amplitudes, phases, and frequencies learned.

• The temporal network $g_n(t)$ is typically a small multilayer perceptron.
• All parameters are trained by minimizing a physics-informed residual loss encoding the PDE operator, boundary, and initial conditions.

Adaptive frequency selection in the spectral expansion allows the learned basis to efficiently resolve both low and high-frequency solution components, overcoming the severe spectral bias inherent in standard physics-informed neural networks (PINNs).

In optics-encoded learning systems, such as vortex-encoded spectral correlations (Perry et al., 2023), regression proceeds from a physically motivated measurement process:

• The object $\psi(x,y)$ is optically encoded (e.g., via a vortex phase mask), producing a sensor field whose intensity $\mathcal{X}$ is a polynomial function of the “unknown” input field.
• The regression task is to invert this polynomial relation, which—by construction—can be accomplished with a single matrix-multiplied linear layer if the polynomial degree is supported in the measurement vector:

$\widehat{\psi} = \mathcal{X} W^\top$

where $W$ is the learned weight matrix.

Koopman operator spectral approximation (Valva et al., 2024) leverages a variational eigenvalue problem for a physically regularized compact operator:

• The underlying dynamics are mapped to the Koopman generator $L f = F \cdot \nabla f$, then compactified and smoothed with a Markov kernel.
• Physics encoding appears in that all kernel derivatives are computed by automatic differentiation from the known dynamical vector field, yielding a Galerkin or kernelized eigenproblem with the structure inherited from dynamical flows.

Other variants include physics-informed Gaussian process regression for linear PDEs, where the observation operator itself encodes both the differential equation and any measurement/physical constraints (Pförtner et al., 2022), and hybrid neural-symbolic regression methods for physical image data, where the search space is restricted to expressions that satisfy explicit conservation laws or other physical criteria (Yu et al., 6 Jun 2025).

2. Physics Encoding in Spectral Regression

Physics encoding in these frameworks emerges at several architectural and procedural levels:

• Basis construction: SV-SNN and PhISM (Gawrysiak et al., 29 Aug 2025) build their spectral representations from physically plausible basis functions (adaptive Fourier features or smooth unimodal functions), reflecting known solution smoothness, modal separability, or spectral properties of the task. In Koopman and GP-PDE methods, the relevant test functionals and trial spaces are drawn from eigenfunctions of physical integral operators.
• Loss or constraint formulation: All residuals, regularizations, or fit criteria include direct terms derived from physical governing equations, translated into differentiable programmatic components. SV-SNN, PeSANet (Wan et al., 3 May 2025), and PhISM explicitly embed the residual of the PDE (or spectral mixing law) into their training objective.
• Operator and measurement modeling: In HSDiff (Romero et al., 23 Nov 2025), the measurement operator $A$ incorporates the physical optics (e.g., point spread functions, spectral responses), so the inverse problem solution remains consistent with the real forward physics.
• Symmetry or conservation laws: SatelliteFormula imposes divergence constraints to ensure any derived index satisfies conservation or other relevant physical invariances (Yu et al., 6 Jun 2025).
• Hard-coded or learnable stencils: PeSANet constrains the convolutional kernels intended to approximate spatial derivatives so that, in the data-rich limit, they exactly implement the intended physical operators.

3. Spectral Representations and Feature Adaptivity

The “spectral” attribute refers not merely to the use of standard Fourier or polynomial expansions, but to the integration of adaptive, learnable, or modality-matched spectral features in the learning process. The main instantiations are:

• Adaptive Fourier features: SV-SNN learns the frequencies in its spectral expansion, permitting high wavenumber components to be represented without bias.
• Self-attention in spectral space: PeSANet’s spectral-enhanced block implements self-attention among frequency components of the field, capturing long-range spatial dependencies and intermodal couplings efficiently (Wan et al., 3 May 2025).
• Polynomial basis inversion: In vortex optics (Perry et al., 2023), the physical propagation and encoding of the field results in polynomial regression in the Fourier domain.
• Skew-normal basis for hyperspectral modeling: PhISM fits reflectance curves as sums of smooth, parametric unimodal components, better matching the actual physics of reflectance spectra (Gawrysiak et al., 29 Aug 2025).

Adaptivity of the spectral bases is crucial for efficiently resolving complex, multi-scale, or highly oscillatory solution structures. This adaptivity also improves the conditioning of the regression or inversion problem, as seen in the effective flattening of Jacobian spectra in SV-SNN.

4. Theoretical and Diagnostic Frameworks

A distinguishing feature of the most recent physics-encoded spectral regression techniques is the systematic diagnosis and mitigation of spectral bias and ill-conditioning:

• Spectral bias quantification: The singular value decomposition (SVD) of the loss Jacobian in SV-SNN provides a rigorous framework for tracking how quickly each spectral component of the solution is learned during training. The effective rank metric $r^{\rm eff}$ at energy level $\eta$ quantifies the spread between well- and poorly-learned frequencies (Xiong et al., 1 Aug 2025).
• Condition number and convergence: The NTK block’s condition number directly impacts convergence rates of gradient flow for different spectral modes, demonstrating why flat singular spectra are desirable.
• Bayesian uncertainty quantification: Physics-informed GP regression (Pförtner et al., 2022) and HSDiff (Romero et al., 23 Nov 2025) naturally provide posterior credible intervals, reflecting unresolved error in both the noise and the spectral discretization.
• Duality-based information bounds: Convex program duality in model-free spectral reconstruction yields explicit, information-theoretically optimal bounds on smeared integrals of spectral density, providing certificates of incompleteness or completeness of the inversion (Lawrence, 2024).

These analysis tools not only inform model selection and tuning but also allow a principled comparison between physics-encoded spectral regression and black-box or naive spectral methods.

5. Applications and Empirical Performance

Physics-encoded spectral regression methods have demonstrated marked gains across a spectrum of high-impact applications:

• High-frequency PDE solving: SV-SNN achieves 1–3 orders of magnitude improvement in accuracy, with >90% reduction in parameter count and ≈60× acceleration over PINNs, on heat, Helmholtz, Poisson, and Navier–Stokes equations (Xiong et al., 1 Aug 2025). PeSANet outperforms FNO and other SOTA baselines in RMSE and long-horizon prediction for turbulent and reaction–diffusion systems (Wan et al., 3 May 2025).
• Compressed optics inversion: In vortex-encoded optics, polynomial regression paired with physics-encoded encoding yields efficient, interpretable, and highly accurate image reconstructions, with learned feature distillation that boosts shallow classifier performance to SOTA levels on canonical datasets (Perry et al., 2023).
• Koopman operator learning: Physics-informed kernel eigenvalue methods deliver asymptotically consistent extraction of coherent structures and spectral modes even for weak-mixing or chaotic nonlinear flows (Valva et al., 2024).
• Remote sensing and symbolic regression: SatelliteFormula discovers closed-form, physically plausible indices and scaling laws from multispectral imagery, outperforming black-box models in accuracy, efficiency, and interpretability (Yu et al., 6 Jun 2025).
• Hyperspectral imaging: Physics-informed autoencoders (PhISM) yield state-of-the-art soil parameter prediction and material clustering, with built-in interpretability of learned spectral shapes (Gawrysiak et al., 29 Aug 2025).
• Uncertainty-aware estimation: HSDiff demonstrates that optics-encoded spectral regression in a Bayesian framework leads to lower error, sharper uncertainty quantification, and well-calibrated credible intervals for hyperspectral image reconstruction (Romero et al., 23 Nov 2025).
• Model-free convex inversion: Lagrange duality gives mathematically guaranteed bounds on smeared spectral densities and real-time correlators in quantum Monte Carlo and statistical mechanics, without reliance on regularization heuristics (Lawrence, 2024).

Empirical results consistently show that the explicit integration of physical constraints and spectral adaptivity yields substantial improvements in expressivity, generalization, and interpretability compared to black-box approaches.

6. Interpretability, Generality, and Limitations

A salient advantage of physics-encoded spectral regression is interpretability:

• The learned spectral coefficients, physically indexed bases, or uncoverable formula trees transparently encode the structure and dominant scales of the solution, enabling downstream diagnostics and physical insight. PhISM and SatelliteFormula directly visualize basis contributions in material classification or environmental index prediction (Gawrysiak et al., 29 Aug 2025, Yu et al., 6 Jun 2025).
• Parameter-efficient representations allow for tracing the importance of specific frequencies or bands, with explicit reference to physical mechanisms (e.g., absorption lines, conservation laws).
• Convex duality and kernel eigenfunction approaches enable rigorous certification—providing information-theoretic guarantees for spectral inversion quality (Lawrence, 2024).

However, limitations remain. In highly nonlinear, high-dimensional, or data-scarce settings, spectral regression may confront computational challenges (e.g., eigenvalue problems with large kernels, dual programs with dense constraints), and systematic hyperparameter tuning (number of modes, degree of polynomial, kernel bandwidth, etc.) may be required. Physics encoding that is too rigid may restrict expressivity, whereas underencoding may forfeit error guarantees or fail to leverage physical priors fully. A plausible implication is that model selection and cross-validation strategies targeting the trade-off between physical fidelity and spectral expressivity will remain active areas for further research.

---

References:

• (Xiong et al., 1 Aug 2025) Separated-Variable Spectral Neural Networks: A Physics-Informed Learning Approach for High-Frequency PDEs
• (Valva et al., 2024) Physics-informed spectral approximation of Koopman operators
• (Pförtner et al., 2022) Physics-Informed Gaussian Process Regression Generalizes Linear PDE Solvers
• (Perry et al., 2023) Parameterized Learning and Distillation with Vortex-encoded Spectral Correlations
• (Wan et al., 3 May 2025) PeSANet: Physics-encoded Spectral Attention Network for Simulating PDE-Governed Complex Systems
• (Yu et al., 6 Jun 2025) SatelliteFormula: Multi-Modal Symbolic Regression from Remote Sensing Imagery for Physics Discovery
• (Romero et al., 23 Nov 2025) Uncertainty Quantification in HSI Reconstruction using Physics-Aware Diffusion Priors and Optics-Encoded Measurements
• (Lawrence, 2024) Model-free spectral reconstruction via Lagrange duality
• (Gawrysiak et al., 29 Aug 2025) Physics-Informed Spectral Modeling for Hyperspectral Imaging