PhysGuard: Fisher-Guided Gradient Projection for Sim-to-Real Neural PDE Surrogates

Abstract: Neural operator models trained on simulation data often lose accuracy when applied to experimental measurements due to the sim-to-real gap. Standard fine-tuning with limited real data can reduce this gap, but it may also damage the core physics-relevant representations learned during pretraining. Although knowledge-preserving adaptation has been widely investigated in vision or language tasks, it remains unclear whether these methods are suitable for neural operators whose architectures and protected knowledge are fundamentally different. Neural operators need to preserve core-scale physical structures rather than semantic or visual features. We propose PhysGuard, a physics-preserving framework for accurate sim-to-real adaptation of neural operators. Specifically, PhysGuard uses the empirical Fisher Information Matrix computed on simulation data to identify physics-critical parameter directions, then restricts fine-tuning updates to directions that do not interfere with them. A layer-wise Gram-matrix formulation makes this efficient for models with millions of parameters, while an adaptive threshold automatically determines the protected subspace size. A spectral probe experiment shows that the dominant Fisher directions are strongly associated with low-frequency output structures. Experiments on benchmark across four neural operator architectures and different physical systems show that PhysGuard performs strongly on most evaluation metrics compared to baselines. The benefits are most evident under severe domain shift, where it reduces low-frequency error by up to 32\% compared to standard fine-tuning while maintaining adaptability. Our code is available at https://github.com/ZhouChaunge/PhysGuard.

• The paper presents a novel Fisher-guided method to identify and safeguard physics-critical subspaces in neural PDE surrogates, achieving up to 32% low-frequency error reduction.
• It details an efficient spectral probing technique using the empirical Fisher Information Matrix to align gradient updates with essential physical features during sim-to-real adaptation.
• Experiments on RealPDEBench show that PhysGuard significantly outperforms standard fine-tuning methods across multiple architectures and challenging flow scenarios.

PhysGuard: Fisher-Guided Gradient Projection for Sim-to-Real Neural PDE Surrogates

Introduction and Motivation

PhysGuard addresses the critical challenge of sim-to-real adaptation for neural operator surrogates of partial differential equations (PDEs). Neural operators trained exclusively on synthetic simulation data often experience significant performance drop when deployed on real-world experimental measurements due to distributional shift, noise, and unmodeled physical phenomena. Naïve fine-tuning with limited real data frequently leads to degradation of large-scale, low-frequency physical patterns, resulting in catastrophic forgetting of core physical knowledge. Existing knowledge-preservation methods from the vision and NLP domains, such as uniform parameter regularization (L$_2$-SP), importance weighting (EWC), and subspace-based fine-tuning, are suboptimal for neural PDE surrogates because the features to preserve are global physics-consistent structures, not semantic classes or visual features.

PhysGuard introduces a principled, efficient, and architecture-agnostic mechanism to identify and safeguard the physics-critical subspace within high-dimensional neural operator models. The method ensures that fine-tuning on real data adapts models to new conditions while explicitly protecting learned representations essential for capturing governing physical laws, quantified as the low-frequency components of model outputs.

Fisher-Guided Subspace Identification and Gradient Projection

PhysGuard's central mechanism is the use of the empirical Fisher Information Matrix (FIM), estimated from simulation data, to identify parameter directions most influential for physical prediction fidelity.

The approach is outlined as follows:

1. Empirical FIM Estimation: Per-layer, per-sample gradients from the simulation dataset are aggregated to form a matrix $G$ whose covariance yields the FIM, $F = \frac{1}{N} G^\top G$. Due to the high parameter dimension, a scalable Gram-matrix (kernel trick) formulation is employed, performing eigendecomposition on the much smaller $K = GG^\top$ (size $N \times N$).
2. Physics Subspace Construction: The leading eigenvectors of $F$ define directions in parameter space where the loss is most sensitive—these are the physics-critical directions. An adaptive threshold (default $\tau=0.9$) is used to select the number $k$ of dominant eigenvectors that cumulatively explain a large portion of the Fisher spectrum.
3. Spectral Probing: A novel spectral probe experimentally confirms that the FIM’s leading eigenvectors correspond to output perturbations dominating the low-frequency regime, i.e., precisely those carrying large-scale, system-governing physical structures.
4. Orthogonal Projection: During fine-tuning on experimental data, gradient updates are projected onto the orthogonal complement of the physics subspace. This is implemented per-layer and per-gradient step, optionally controlling the projection strength via a scalar $\alpha$, typically set to 1 for strict preservation.

   Figure 1: The FIM spectral probe demonstrates that output perturbations from top FIM eigenvectors predominantly affect low frequencies, unlike random parameter directions.

Experimental Evaluation

Setup

PhysGuard is evaluated on the RealPDEBench suite, which uniquely provides paired synthetic and experimentally measured data for three increasingly challenging scenarios: cylinder flow, controlled cylinder flow, and turbulent combustion. Four representative neural operator architectures (FNO, CNO, DeepONet, Transolver) spanning diverse parameterizations and inductive biases serve as testbeds.

Five adaptation strategies are compared: Pretrained (no adaptation), DFT (unconstrained direct fine-tuning), L$_2$-SP, EWC, and PhysGuard.

FIM–Physics Alignment

Spectral probe experiments validate that the physics-critical directions identified by the FIM are indeed those whose parameter perturbations induce low-frequency changes in the model output, as shown by near-total concentration of spectral energy in these modes.

Figure 2: Visual comparison of predicted fields across adaptation methods on three canonical scenarios, highlighting the superior recovery of large-scale structures by PhysGuard.

Quantitative Results and Analysis

Across 48 metric-architecture-scenario combinations, PhysGuard outperforms all baselines in the majority of cases, particularly in regimes involving strong sim-to-real domain shift. The most notable improvements are manifested in metrics directly indexing low-frequency (large-scale) error, with up to 32% reduction in these errors relative to standard fine-tuning. Architectures with strong low-frequency bias (e.g., FNO, CNO) especially benefit from PhysGuard, which acts to prevent overfitting to high-frequency noise in scarce real data.

Comprehensive results indicate that uniform regularization baselines (L$G$0-SP, diagonal EWC) are suboptimal, often constraining too many or the wrong parameter directions. PhysGuard’s subspace projection targets only those directions critical for physics preservation, as quantified by the FIM, leaving the remaining degrees of freedom for effective adaptation.

Figure 3: PhysGuard delivers consistent low-frequency error reduction and protects only a small fraction of parameter directions, maximizing both preservation and adaptability.

Qualitative Assessment

Qualitative field visualizations confirm that PhysGuard maintains sharp, physically meaningful structures such as vortex streets and coherent flow profiles (see Figure 2), where conventional baselines yield either over-smoothed or spatially degraded predictions, especially when the domain gap is substantial. Controlled cylinder flow—where the sim-to-real shift is minor—shows minimal differentiation across methods, indicating that PhysGuard does not impede beneficial adaptation when physics-critical directions are less involved.

Limitations and Applicability

PhysGuard's core assumption is a low-rank empirical FIM spectrum. When Fisher information is uniformly distributed (as with foundation-scale models like DPOT-S), the projection severely restricts adaptation capacity. This limitation is empirically corroborated: PhysGuard’s benefit diminishes or reverses when there is no dominant subspace (see Appendix, Section~\ref{app:dpot}). The method's efficacy is thus architecture- and scenario-dependent, and applicability to non-fluid PDEs remains to be established.

Implications and Future Directions

PhysGuard formalizes sim-to-real transfer for neural operators in an information-geometric framework, introducing an interpretable mechanism to balance physical knowledge preservation and data-driven adaptation. This method is practically important for SciML pipelines in computational fluid dynamics, weather modeling, and engineering design, where real-world data is scarce and model fidelity is paramount. The approach offers transparency, minimal hyperparameter overhead, and is compatible with modern differentiable operator architectures.

Looking ahead, two research directions are highlighted:

• Scalability to foundation-model-scale neural operators where the Fisher spectrum may not be low-rank, calling for adaptive or dynamic subspace mechanisms.
• Generalization and benchmarking beyond fluid mechanics, encouraging development of broader experimental PDE datasets and extensions to other physical domains and sensing modalities.

Conclusion

PhysGuard provides a principled, efficient, and effective solution for physics-preserving adaptation of neural operator PDE surrogates under sim-to-real distribution shift. By leveraging subspace projection informed by the empirical Fisher Information, it achieves strong empirical improvements in both low-frequency fidelity and overall predictive accuracy across multiple architectures and scenarios. While the low-rank Fisher assumption remains a constraint, PhysGuard establishes a rigorous foundation for future research on trustworthy and resource-efficient adaptation in scientific machine learning.

Figure 2: Predicted fields across methods and scenarios, with PhysGuard best aligning to experimental ground truth over both global and local flow features.

Figure 3: (a) Low-frequency error is strictly reduced by PhysGuard vs. unconstrained fine-tuning; (b) only a small portion of the parameter space is protected, enabling adaptation.