LLM-PDESR: Robust PDE Discovery via Subdomain Weighted Residuals and LLM-Guided Symbolic Hypothesis Generation

Abstract: Discovering governing partial differential equations (PDEs) from noisy observational data is a fundamental challenge in scientific machine learning. Traditional symbolic regression (SR) methods often struggle to identify accurate equations within vast combinatorial search spaces, largely due to their inability to incorporate essential domain-specific prior knowledge. Furthermore, reliance on pointwise evaluations and discrete finite differences inherently amplifies high-frequency noise, creating deceptive fitness landscapes that derail the optimization process. To resolve these bottlenecks, we propose LLM-PDESR, a framework that integrates the structural hypothesis generation of LLMs with a mathematically rigorous evaluation environment. By employing C^4-continuous quintic splines for robust differentiation and subdomain weighted residuals as natural low-pass filters, our approach effectively mitigates the fitness landscape distortion that plagues existing methods. A Pareto-driven feedback loop then enables the LLM to iteratively refine candidate equations, balancing predictive accuracy with structural parsimony. We evaluate LLM-PDESR on 23 canonical PDEs and five structurally novel equations (including a multivariate system) specifically designed to preclude dataset memorization and test true discovery capabilities. Demonstrating real-world applicability, the framework successfully extracts a consistent structural skeleton for an interpretable 1D dynamical surrogate (1D-CACE) directly from noisy ERA5 reanalysis data. Extensive experiments and out-of-distribution testing confirm that LLM-PDESR significantly outperforms state-of-the-art methodologies in structural recovery, noise resilience, and the avoidance of spurious complexity and equation bloat.

• The paper introduces LLM-PDESR, a pipeline that integrates LLM-guided symbolic hypothesis generation with robust spline-based derivative estimation and Pareto-driven optimization for PDE discovery.
• It leverages C⁴ quintic B-splines and Subdomain Weighted Residuals to accurately recover symbolic PDE structures from noisy observational data.
• Empirical evaluations on canonical benchmarks and real-world cases demonstrate high structural recovery rates, outperforming traditional finite difference and symbolic regression approaches.

Robust Symbolic PDE Discovery with LLM-PDESR: Architecture, Evaluation, and Empirical Analysis

Introduction

LLM-PDESR introduces a novel methodology for autonomous discovery of governing partial differential equations (PDEs) from noisy observational data. The framework synergistically integrates LLM-based symbolic hypothesis generation, noise-robust parameter identification, and Pareto-driven evolutionary selection. The primary scientific motivation is to address fundamental bottlenecks ubiquitous in previous symbolic regression (SR) and neural PDE discovery approaches—namely, combinatorial explosion in hypothesis space, rigidity with respect to novel functional forms, and severe fitness landscape distortion due to noise amplification during derivative computation.

Figure 1: The LLM-PDESR architecture. The automated pipeline integrates three core modules: (1) LLM-Driven Hypothesis Generation, (2) Robust Parameter Identification via splines, SWR, and BFGS optimization, and (3) Pareto-Driven Evaluation providing dynamic feedback.

Methodology

LLM-Driven Hypothesis Generation

Unlike dictionary-based symbolic regression that confines search to predetermined libraries, LLM-PDESR leverages constrained prompting of LLMs to sample executable symbolic PDE skeletons. Candidate equations are represented as Python ASTs with parameter placeholders, ensuring functional validity and parameter identifiability. Prompt constraints enforce strict differentiability structure, allowable mathematical operations, and enforce context consistency to preclude generation of degenerate or unphysical terms.

Robust Parameter Identification: $\mathcal{C}^4$ Splines and SWR

A key advancement lies in the elimination of pointwise finite difference differentiation for spatial derivative estimation. Instead, $\mathcal{C}^4$ quintic B-spline interpolation is utilized, enabling consistent, high-order accurate, and noise-attenuated computation of up to fourth-order derivatives. This is followed by fitness evaluation via Subdomain Weighted Residuals (SWR): The residual between measured and candidate-predicted time derivatives is integrated over local subdomains against compactly supported smooth test functions, acting as a controllable low-pass filter that systematically attenuates high-frequency noise. Parameter optimization is then executed using BFGS, exploiting the smoothed loss landscape delivered by the SWR evaluation.

Pareto-Driven Evolutionary Feedback

LLM-PDESR institutes a rigorously multi-objective evolutionary loop. The Pareto front is constructed with axis metrics given by negative residual error (structural fitness) and symbolic complexity (AST node count). The LLM is then prompted with top frontier solutions in each generation, actively shaping its search heuristics toward interpretable, parsimonious, yet high-fidelity equations. Final selection operates within a logarithmic error tolerance, ensuring minimal-complexity models while safeguarding predictive error.

Empirical Evaluation

Benchmark PDEs: Noise Resilience

Evaluation across 23 canonical 1D PDEs demonstrates state-of-the-art structural recovery and parameter accuracy under escalating observational noise levels. At 0–20% noise, LLM-PDESR maintains near-unity symbolic recovery rates; baseline methods exhibit catastrophic degradation due to their reliance on finite differences and absence of intrinsic smoothing, as shown in the noise-robustness curves below.

Figure 2: Noise robustness on benchmark PDEs. Impact of varying observational noise on (a) structural success rate and (b) relative coefficient error.

Contrary to EqGPT, which depends on external data denoising, LLM-PDESR is applied to unprocessed, raw observations, preserving the integrity of high-fidelity physics while simultaneously isolating true operator structure from observation noise.

Structurally Novel Equations

Performance on five custom task-agnostic OOD PDEs highlights the main claim: LLM-PDESR generalizes symbolic discovery to equation classes fundamentally inaccessible to prior SR baselines. For rational, coupled, non-polynomial, and derivative-nested dynamics, it matches or approximates the true skeleton, with only the most numerically stiff/shock-prone systems (traffic flow) resisting recovery—a regime known to challenge all gradient-based identification methods.

Figure 3: Topography-Constrained Chemotaxis. LLM-PDESR accurately approximates macroscopic wave propagation and logistic saturation, providing a robust phenomenological surrogate without requiring the explicit $\cos(x)$ micro-scale forcing.

Figure 4: Morphogenesis. LLM-PDESR achieves perfect structural recovery and cross-section alignment. The discovered equation algebraically mirrors the true Michaelis-Menten fractional kinetics and accurately identifies the explicit spatial Gaussian injection.

Figure 5: Forced Quintic Swift-Hohenberg. The discovered skeleton successfully captures the fundamental $4^{\text{th}}$-order pattern-forming instability and correct wavelength selection, despite missing the explicit quintic term.

Figure 6: Traffic Flow. A transparent example of algorithmic failure. The numerical stiffness caused by the $\tanh(x)$ bottleneck prevents the framework from learning the correct localized flux, resulting in discovered dynamics that fail to reproduce the steep traffic shockwave.

Figure 7: Cross-Chemotactic Predator-Prey. LLM-PDESR perfectly isolates the cross-variable spatial coupling $(vu_x)_x$. The microscopic alignment in (c) visually confirms that the discovered equation faithfully forces the predator ($v$) to actively track the spatial gradient of the prey ($u$), with zero observable deviation from the ground truth.

Real-World Climate Data: Discovery of 1D-CACE

To validate real-world applicability, LLM-PDESR is applied to noisy ERA5 reanalysis atmospheric data. The framework autonomously reconstructs a coupled, interpretable 1D dynamical surrogate (1D-CACE) that consistently captures cross-wind momentum transfer and macro-geophysical closure terms essential for reduced-order modeling. Out-of-distribution cross-validation (across space, season, and hemisphere) empirically confirms structural invariance: the discovered skeleton is transferable, requiring only minimal adjustment of numerical coefficients to adapt to new physical regimes.

Figure 8: Hemispheric OOD predictions of 1D-CACE. The model robustly reconstructs South Indian Ocean dynamics (austral winter) despite severe geographical shifts.

Figure 9: In-Distribution Performance (North Pacific, Jan 1--10). The 1D-CACE skeleton reconstructs the chaotic wind tendency ($R^2=76.28\%$), successfully balancing macroscopic wave tracking with high-frequency fluctuations.

Figure 10: Spatial OOD Performance (West Pacific, Jan 1--10). The frozen skeleton adapts robustly to distinct oceanic forcings without numerical divergence ($R^2=60.97\%$), confirming the structural invariance of the discovered advection terms.

Figure 11: Temporal Drift Performance (North Pacific, Feb 1--10). Despite severe atmospheric phase shifts, the invariant skeleton reconstructs new weather tendencies ($\mathcal{C}^4$0), proving the extraction of genuine dynamical abstractions over statistical memorization.

Ablation Studies

Ablation confirms that every module is indispensably synergistic. Reverting to finite difference or abandoning SWR drastically reduces both symbolic recovery and parameter precision. Dropping Pareto selection results in persistent equation bloat—low error but structurally incorrect models—demonstrating the criticality of explicit multi-objective optimization.

Figure 12: Ablation study of the LLM-PDESR framework. Impact of removing core components on (a) structural success rate and (b) MSE distributions under noise-free conditions.

Implications and Future Directions

LLM-PDESR's architectural decoupling of symbolic synthesis from nonlinear parameterization provides a mechanism for leveraging semantic prior knowledge while remaining open to structurally novel equations and functional forms. The demonstrated empiricism confirms not only OOD recovery but also scalable applicability to multivariate and noisy real-world problems, surpassing the symbolic expressivity ceilings imposed by traditional library-based or purely neural surrogates.

However, some limitations persist. PDEs with extreme numerical stiffness, discontinuities, or gradient singularities are incompletely recoverable due to the reliance on smooth spline-based spatial reconstruction and optimization dynamics. The computational cost of LLM-in-the-loop inference remains substantial. Directions for future work include adaptive mesh refinement for high-gradient regimes, extension to unstructured multidimensional domains, and integration with parameter-efficient LLM variants to minimize overhead.

Conclusion

LLM-PDESR advances the frontier of automated scientific discovery by integrating hypothesis generation, noise-robust evaluation, and parsimonious selection in a unified, mathematically principled pipeline. The architecture achieves robust, interpretable PDE discovery under realistic physical noise, supports genuine OOD symbolic generalization, and provides a foundation for interpretable models in operational scientific modeling. The empirical results, especially in real-world dynamical systems, attest to the capacity of LLM-guided SR pipelines to transcend combinatorial and noise-robustness barriers inherent to earlier scientific ML approaches (2607.10546).