PhysGuard: Physics-Preserving Sim-to-Real Adaptation

• PhysGuard is a framework for sim-to-real adaptation that safeguards low-frequency, physics-critical structures by projecting gradients away from key Fisher directions.
• It leverages an empirical Fisher Information Matrix computed on simulation data to identify and protect parameter directions central to large-scale physical patterns.
• PhysGuard demonstrates significant improvements in RMSE and low-frequency error metrics across various neural operator architectures under severe domain shifts.

Searching arXiv for the specified paper and closely related context papers. [arXiv search] Query: (Zhou et al., 15 Jun 2026) PhysGuard is a physics-preserving framework for sim-to-real adaptation of neural operators for neural PDE surrogates. It is designed for settings in which models such as FNO, CNO, DeepONet, and Transolver are pretrained on large numerical simulation datasets and then deployed on experimental measurements that differ from simulations because of sensor noise, model mismatch, unmodeled physics, and other systematic shifts. The central premise is that unconstrained fine-tuning on limited real data can overwrite the low-frequency, large-scale structures learned during pretraining, even when those structures encode the governing physics. PhysGuard addresses this by using the empirical Fisher Information Matrix computed on simulation data to identify physics-critical parameter directions and by projecting real-data gradients onto the orthogonal complement of those directions during fine-tuning (Zhou et al., 15 Jun 2026).

1. Sim-to-real adaptation in neural operators

Neural operator models are trained to approximate PDE solution operators from simulation data, but their accuracy often degrades when they are evaluated on experimental measurements. The reported source of this sim-to-real gap includes sensor noise, model mismatch, unmodeled physics, and other systematic shifts. In this setting, direct fine-tuning on real data is the most direct adaptation strategy, yet it can damage the pretrained representations that control core-scale physical structure (Zhou et al., 15 Jun 2026).

For neural operators, the protected knowledge is not semantic or visual content but the low-frequency, large-scale structures of solution fields, including mean flow profiles, vortex streets, and coherent combustion zones. The reported argument is that simulation data are typically smooth and dominated by large-scale patterns, so the pretrained model’s sensitivity concentrates along parameter directions that control these low-frequency outputs. During fine-tuning, gradients from noisy or shifted real data may contain components along these same directions; if these components are not constrained, updates may degrade low-frequency fidelity while overfitting high-frequency fluctuations.

This framing distinguishes PhysGuard from knowledge-preserving adaptation methods developed for vision and language. In the description of the method, uniform penalties such as L2-SP and diagonal importance methods such as EWC are characterized as potentially too coarse for neural operators: they may either restrict all parameters equally or fail to capture correlated importance directions. A plausible implication is that the relevant invariants in operator learning reside in structured subspaces rather than in isolated coordinates.

2. Constrained formulation and Fisher-based identification of the protected subspace

PhysGuard considers a pretrained neural operator $f_\theta$ with pretrained parameters $\theta^*$, simulation dataset $D_{\mathrm{sim}} = \{(x_i,y_i)\}$, and real experimental dataset $D_{\mathrm{real}} = \{(\tilde{x}_j,\tilde{y}_j)\}$. The adaptation objective is described as fine-tuning on real data while preserving low-frequency output structure. The formulation is

$$\min_{\theta}\;\; \mathcal{L}_{\mathrm{real}}(\theta) \quad\text{s.t.}\quad \mathcal{L}_{\mathrm{low\text{-}f}}(\theta) \le \mathcal{L}_{\mathrm{low\text{-}f}}(\theta^*).$$

Rather than adding an auxiliary penalty, PhysGuard adopts a geometric restriction on update directions. The stated goal is to preserve low-frequency physics by restricting optimization to parameter directions along which the pretrained simulation loss is relatively insensitive (Zhou et al., 15 Jun 2026).

For a general probabilistic model $p(y|x;\theta)$, the Fisher Information is defined as

$$F(\theta) \;=\; \mathbb{E}_{(x,y)}\!\left[ \nabla_{\theta}\log p(y|x;\theta)\; \nabla_{\theta}\log p(y|x;\theta)^{\top} \right].$$

In the supervised regression setting with MSE-like losses, PhysGuard uses the empirical Fisher Information Matrix computed from per-sample gradients of the simulation loss at $\theta^*$:

$$g_i \;=\; \nabla_{\theta}\,\ell(\theta^*; x_i, y_i), \qquad G \;=\; \begin{bmatrix} g_1^{\top} \ \vdots \ g_N^{\top} \end{bmatrix} \in \mathbb{R}^{N \times d}, \qquad F \;=\; \frac{1}{N}\,G^{\top}G \in \mathbb{R}^{d \times d}.$$

The top eigenvectors of $F$ are interpreted as physics-critical parameter directions: perturbations along them strongly change the pretrained loss and thus the core physical outputs, whereas small-eigenvalue directions are treated as safer for adaptation. The reported implementation finds that $\theta^*$0 of the simulation data is sufficient for estimating this empirical Fisher structure.

3. Scalable Gram-matrix construction and adaptive subspace sizing

Direct eigendecomposition of $\theta^*$1 is infeasible when the parameter dimension $\theta^*$2 is large. PhysGuard therefore uses the identity that $\theta^*$3 and $\theta^*$4 share the same non-zero eigenvalues. It defines the Gram matrix

$\theta^*$5

and computes

$\theta^*$6

where $\theta^*$7 with $\theta^*$8. The corresponding eigenvectors in parameter space are recovered by

$\theta^*$9

This avoids explicit construction of the $D_{\mathrm{sim}} = \{(x_i,y_i)\}$0 Fisher matrix and reduces the eigendecomposition cost to $D_{\mathrm{sim}} = \{(x_i,y_i)\}$1 with $D_{\mathrm{sim}} = \{(x_i,y_i)\}$2 (Zhou et al., 15 Jun 2026).

The protected subspace size is selected by the cumulative Fisher fraction,

$D_{\mathrm{sim}} = \{(x_i,y_i)\}$3

The rule is to choose the smallest $D_{\mathrm{sim}} = \{(x_i,y_i)\}$4 such that $D_{\mathrm{sim}} = \{(x_i,y_i)\}$5, with $D_{\mathrm{sim}} = \{(x_i,y_i)\}$6 by default. The resulting basis is

$D_{\mathrm{sim}} = \{(x_i,y_i)\}$7

PhysGuard estimates this basis independently for each layer $D_{\mathrm{sim}} = \{(x_i,y_i)\}$8,

$D_{\mathrm{sim}} = \{(x_i,y_i)\}$9

The layer-wise spectral decomposition is summarized as

$D_{\mathrm{real}} = \{(\tilde{x}_j,\tilde{y}_j)\}$0

with

$D_{\mathrm{real}} = \{(\tilde{x}_j,\tilde{y}_j)\}$1

Across the reported models and scenarios, the protected ratio $D_{\mathrm{real}} = \{(\tilde{x}_j,\tilde{y}_j)\}$2 is typically $D_{\mathrm{real}} = \{(\tilde{x}_j,\tilde{y}_j)\}$3–$D_{\mathrm{real}} = \{(\tilde{x}_j,\tilde{y}_j)\}$4 with median $D_{\mathrm{real}} = \{(\tilde{x}_j,\tilde{y}_j)\}$5, so most directions remain unconstrained for adaptation.

For complex-valued parameters, including FNO spectral weights, the method concatenates real and imaginary gradient components,

$D_{\mathrm{real}} = \{(\tilde{x}_j,\tilde{y}_j)\}$6

and maps the projected gradient back to complex form afterward:

$D_{\mathrm{real}} = \{(\tilde{x}_j,\tilde{y}_j)\}$7

4. Gradient projection, update rule, and training procedure

During fine-tuning on real data, PhysGuard projects the gradient in each layer away from the physics-critical subspace. For layer $D_{\mathrm{real}} = \{(\tilde{x}_j,\tilde{y}_j)\}$8 with gradient $D_{\mathrm{real}} = \{(\tilde{x}_j,\tilde{y}_j)\}$9, the projected gradient is

$$\min_{\theta}\;\; \mathcal{L}_{\mathrm{real}}(\theta) \quad\text{s.t.}\quad \mathcal{L}_{\mathrm{low\text{-}f}}(\theta) \le \mathcal{L}_{\mathrm{low\text{-}f}}(\theta^*).$$0

where $$\min_{\theta}\;\; \mathcal{L}_{\mathrm{real}}(\theta) \quad\text{s.t.}\quad \mathcal{L}_{\mathrm{low\text{-}f}}(\theta) \le \mathcal{L}_{\mathrm{low\text{-}f}}(\theta^*).$$1 controls the protection strength. The fully projected case $$\min_{\theta}\;\; \mathcal{L}_{\mathrm{real}}(\theta) \quad\text{s.t.}\quad \mathcal{L}_{\mathrm{low\text{-}f}}(\theta) \le \mathcal{L}_{\mathrm{low\text{-}f}}(\theta^*).$$2 enforces orthogonal projection, while $$\min_{\theta}\;\; \mathcal{L}_{\mathrm{real}}(\theta) \quad\text{s.t.}\quad \mathcal{L}_{\mathrm{low\text{-}f}}(\theta) \le \mathcal{L}_{\mathrm{low\text{-}f}}(\theta^*).$$3 recovers standard fine-tuning. Equivalently, with projection operator

$$\min_{\theta}\;\; \mathcal{L}_{\mathrm{real}}(\theta) \quad\text{s.t.}\quad \mathcal{L}_{\mathrm{low\text{-}f}}(\theta) \le \mathcal{L}_{\mathrm{low\text{-}f}}(\theta^*).$$4

the update is

$$\min_{\theta}\;\; \mathcal{L}_{\mathrm{real}}(\theta) \quad\text{s.t.}\quad \mathcal{L}_{\mathrm{low\text{-}f}}(\theta) \le \mathcal{L}_{\mathrm{low\text{-}f}}(\theta^*).$$5

The method is described as architecture-agnostic and as introducing no auxiliary losses or delicate hyperparameters (Zhou et al., 15 Jun 2026).

The procedure has two phases. In the one-time offline phase, per-layer simulation gradients are collected at $$\min_{\theta}\;\; \mathcal{L}_{\mathrm{real}}(\theta) \quad\text{s.t.}\quad \mathcal{L}_{\mathrm{low\text{-}f}}(\theta) \le \mathcal{L}_{\mathrm{low\text{-}f}}(\theta^*).$$6, stacked into $$\min_{\theta}\;\; \mathcal{L}_{\mathrm{real}}(\theta) \quad\text{s.t.}\quad \mathcal{L}_{\mathrm{low\text{-}f}}(\theta) \le \mathcal{L}_{\mathrm{low\text{-}f}}(\theta^*).$$7, converted into Gram matrices $$\min_{\theta}\;\; \mathcal{L}_{\mathrm{real}}(\theta) \quad\text{s.t.}\quad \mathcal{L}_{\mathrm{low\text{-}f}}(\theta) \le \mathcal{L}_{\mathrm{low\text{-}f}}(\theta^*).$$8, eigendecomposed, and truncated according to $$\min_{\theta}\;\; \mathcal{L}_{\mathrm{real}}(\theta) \quad\text{s.t.}\quad \mathcal{L}_{\mathrm{low\text{-}f}}(\theta) \le \mathcal{L}_{\mathrm{low\text{-}f}}(\theta^*).$$9, optionally with $p(y|x;\theta)$0, after which the normalized bases $p(y|x;\theta)$1 are stored. In the online phase, each fine-tuning step computes real-data gradients, projects them layer-wise, and passes the projected gradients to the optimizer.

The reported computational profile is as follows: gradient collection requires $p(y|x;\theta)$2 operations across all parameters; Gram matrix construction requires $p(y|x;\theta)$3 and is implemented in chunks; eigendecomposition costs $p(y|x;\theta)$4 and is stated to be negligible for $p(y|x;\theta)$5–$p(y|x;\theta)$6; mapping back to parameter space requires $p(y|x;\theta)$7. Subspace estimation is performed once offline and takes approximately $p(y|x;\theta)$8–$p(y|x;\theta)$9 minutes across architectures, whereas per-step projection adds less than $$F(\theta) \;=\; \mathbb{E}_{(x,y)}\!\left[ \nabla_{\theta}\log p(y|x;\theta)\; \nabla_{\theta}\log p(y|x;\theta)^{\top} \right].$$0 ms on the reported setup. The description states that this remains tractable even for $$F(\theta) \;=\; \mathbb{E}_{(x,y)}\!\left[ \nabla_{\theta}\log p(y|x;\theta)\; \nabla_{\theta}\log p(y|x;\theta)^{\top} \right].$$1M-parameter operators because $$F(\theta) \;=\; \mathbb{E}_{(x,y)}\!\left[ \nabla_{\theta}\log p(y|x;\theta)\; \nabla_{\theta}\log p(y|x;\theta)^{\top} \right].$$2.

5. Spectral interpretation and low-frequency physics

A central empirical claim of PhysGuard is that dominant Fisher directions align with low-frequency physical structure. This is evaluated through a spectral probe. For a unit direction $$F(\theta) \;=\; \mathbb{E}_{(x,y)}\!\left[ \nabla_{\theta}\log p(y|x;\theta)\; \nabla_{\theta}\log p(y|x;\theta)^{\top} \right].$$3 in layer $$F(\theta) \;=\; \mathbb{E}_{(x,y)}\!\left[ \nabla_{\theta}\log p(y|x;\theta)\; \nabla_{\theta}\log p(y|x;\theta)^{\top} \right].$$4, the method perturbs parameters via $$F(\theta) \;=\; \mathbb{E}_{(x,y)}\!\left[ \nabla_{\theta}\log p(y|x;\theta)\; \nabla_{\theta}\log p(y|x;\theta)^{\top} \right].$$5 with $$F(\theta) \;=\; \mathbb{E}_{(x,y)}\!\left[ \nabla_{\theta}\log p(y|x;\theta)\; \nabla_{\theta}\log p(y|x;\theta)^{\top} \right].$$6, evaluates outputs on held-out real validation samples, computes $$F(\theta) \;=\; \mathbb{E}_{(x,y)}\!\left[ \nabla_{\theta}\log p(y|x;\theta)\; \nabla_{\theta}\log p(y|x;\theta)^{\top} \right].$$7, and measures the low-frequency energy fraction after applying a $$F(\theta) \;=\; \mathbb{E}_{(x,y)}\!\left[ \nabla_{\theta}\log p(y|x;\theta)\; \nabla_{\theta}\log p(y|x;\theta)^{\top} \right].$$8D FFT:

$$F(\theta) \;=\; \mathbb{E}_{(x,y)}\!\left[ \nabla_{\theta}\log p(y|x;\theta)\; \nabla_{\theta}\log p(y|x;\theta)^{\top} \right].$$9

with the low-frequency band defined as the lowest third of the radial wavenumber range, $\theta^*$0 (Zhou et al., 15 Jun 2026).

The reported finding is that top-$\theta^*$1 FIM eigenvectors produce output perturbations concentrated almost entirely at low frequencies, with $\theta^*$2. Across the top-$\theta^*$3 eigenvectors, low-frequency dominance persists, whereas random directions spread energy more uniformly. This supports the interpretation that dominant Fisher directions encode large-scale physics and that removing fine-tuning gradient components along these directions preferentially preserves low-frequency structure.

A plausible implication is that PhysGuard operationalizes a physically meaningful decomposition of parameter space: directions of high simulation-loss sensitivity correspond to macro-scale field organization, while lower-sensitivity directions provide capacity for domain adaptation. This interpretation remains tied to the reported spectra and probe outcomes rather than to a general theorem about all neural operators.

6. Experimental evaluation across architectures and domain shifts

The reported experiments are conducted on RealPDEBench scenarios: Cylinder Flow, Controlled Cylinder, and Turbulent Combustion. The architectures are FNO ($\theta^*$4M), CNO ($\theta^*$5M), DeepONet ($\theta^*$6M), and Transolver ($\theta^*$7M). The compared baselines are Pretrained, Direct Fine-Tuning (DFT), L2-SP, EWC, and PhysGuard with $\theta^*$8, $\theta^*$9, and $$g_i \;=\; \nabla_{\theta}\,\ell(\theta^*; x_i, y_i), \qquad G \;=\; \begin{bmatrix} g_1^{\top} \ \vdots \ g_N^{\top} \end{bmatrix} \in \mathbb{R}^{N \times d}, \qquad F \;=\; \frac{1}{N}\,G^{\top}G \in \mathbb{R}^{d \times d}.$$0 of simulations. Evaluation uses RMSE, $$g_i \;=\; \nabla_{\theta}\,\ell(\theta^*; x_i, y_i), \qquad G \;=\; \begin{bmatrix} g_1^{\top} \ \vdots \ g_N^{\top} \end{bmatrix} \in \mathbb{R}^{N \times d}, \qquad F \;=\; \frac{1}{N}\,G^{\top}G \in \mathbb{R}^{d \times d}.$$1, frequency-domain fRMSE, and Low-f band error (Zhou et al., 15 Jun 2026).

Category	Items	Details
Scenarios	Cylinder Flow; Controlled Cylinder; Turbulent Combustion	Large, small, and intermediate domain shift, respectively
Architectures	FNO; CNO; DeepONet; Transolver	$$g_i \;=\; \nabla_{\theta}\,\ell(\theta^*; x_i, y_i), \qquad G \;=\; \begin{bmatrix} g_1^{\top} \ \vdots \ g_N^{\top} \end{bmatrix} \in \mathbb{R}^{N \times d}, \qquad F \;=\; \frac{1}{N}\,G^{\top}G \in \mathbb{R}^{d \times d}.$$2M, $$g_i \;=\; \nabla_{\theta}\,\ell(\theta^*; x_i, y_i), \qquad G \;=\; \begin{bmatrix} g_1^{\top} \ \vdots \ g_N^{\top} \end{bmatrix} \in \mathbb{R}^{N \times d}, \qquad F \;=\; \frac{1}{N}\,G^{\top}G \in \mathbb{R}^{d \times d}.$$3M, $$g_i \;=\; \nabla_{\theta}\,\ell(\theta^*; x_i, y_i), \qquad G \;=\; \begin{bmatrix} g_1^{\top} \ \vdots \ g_N^{\top} \end{bmatrix} \in \mathbb{R}^{N \times d}, \qquad F \;=\; \frac{1}{N}\,G^{\top}G \in \mathbb{R}^{d \times d}.$$4M, and $$g_i \;=\; \nabla_{\theta}\,\ell(\theta^*; x_i, y_i), \qquad G \;=\; \begin{bmatrix} g_1^{\top} \ \vdots \ g_N^{\top} \end{bmatrix} \in \mathbb{R}^{N \times d}, \qquad F \;=\; \frac{1}{N}\,G^{\top}G \in \mathbb{R}^{d \times d}.$$5M parameters
Baselines and metrics	Pretrained; DFT; L2-SP; EWC; PhysGuard	RMSE, $$g_i \;=\; \nabla_{\theta}\,\ell(\theta^*; x_i, y_i), \qquad G \;=\; \begin{bmatrix} g_1^{\top} \ \vdots \ g_N^{\top} \end{bmatrix} \in \mathbb{R}^{N \times d}, \qquad F \;=\; \frac{1}{N}\,G^{\top}G \in \mathbb{R}^{d \times d}.$$6, fRMSE, Low-f band error

The key quantitative result is that PhysGuard ranks first on $$g_i \;=\; \nabla_{\theta}\,\ell(\theta^*; x_i, y_i), \qquad G \;=\; \begin{bmatrix} g_1^{\top} \ \vdots \ g_N^{\top} \end{bmatrix} \in \mathbb{R}^{N \times d}, \qquad F \;=\; \frac{1}{N}\,G^{\top}G \in \mathbb{R}^{d \times d}.$$7 of $$g_i \;=\; \nabla_{\theta}\,\ell(\theta^*; x_i, y_i), \qquad G \;=\; \begin{bmatrix} g_1^{\top} \ \vdots \ g_N^{\top} \end{bmatrix} \in \mathbb{R}^{N \times d}, \qquad F \;=\; \frac{1}{N}\,G^{\top}G \in \mathbb{R}^{d \times d}.$$8 metric–architecture–scenario combinations. The strongest benefits appear under severe domain shift. On Cylinder Flow, PhysGuard reduces low-frequency error by up to $$g_i \;=\; \nabla_{\theta}\,\ell(\theta^*; x_i, y_i), \qquad G \;=\; \begin{bmatrix} g_1^{\top} \ \vdots \ g_N^{\top} \end{bmatrix} \in \mathbb{R}^{N \times d}, \qquad F \;=\; \frac{1}{N}\,G^{\top}G \in \mathbb{R}^{d \times d}.$$9 versus DFT for FNO, CNO, and DeepONet. Relative reductions from Pretrained to PhysGuard are reported as $F$0 for FNO, $F$1 for CNO, $F$2 for Transolver, and $F$3 for DeepONet.

Performance varies with shift severity. On Cylinder Flow, PhysGuard surpasses DFT and the regularization baselines in $F$4 and Low-f metrics. On Controlled Cylinder, where the sim-to-real gap is small, all methods perform similarly and PhysGuard matches or slightly improves over DFT. On Turbulent Combustion, PhysGuard remains competitive with narrower margins and still improves $F$5 and Low-f relative to DFT.

The qualitative visualizations reported in the paper are consistent with these metrics. For Cylinder Flow with DeepONet, PhysGuard reconstructs alternating vortex cores with sharper boundaries than DFT and EWC and attains the best Low-f metrics. For Controlled Cylinder with FNO, all methods closely track ground truth. For Turbulent Combustion with FNO, all methods improve over Pretrained, while PhysGuard yields slightly sharper localized high-intensity zones.

The baseline comparisons are also specific. DFT can improve aggregate error while increasing Low-f error, as reported for CNO on Cylinder Flow. L2-SP is described as potentially over-restricting adaptation along non-critical directions and under-protecting critical ones. EWC is described as unable to capture correlations among parameter directions and as generally remaining close to DFT. GPM/GEM-style methods are contrasted with PhysGuard because their subspace construction is based on representation features across tasks, whereas PhysGuard uses loss-level Fisher structure on a single simulation dataset to preserve physics-critical low-frequency structures.

7. Practical use, limitations, and prospective extensions

The reported application recipe requires a simulation dataset, used once to estimate Fisher subspaces, and a real experimental dataset, used for projected fine-tuning. The guidance is to use $F$6 of simulation samples, compute per-layer gradients at $F$7, build $F$8 per layer, eigendecompose, select $F$9 via $\theta^*$00, optionally cap $\theta^*$01, and store $\theta^*$02 for reuse across downstream tasks. Fine-tuning can be combined with any optimizer, including SGD and AdamW, by applying the projection after the backward pass and before the optimizer step. The default setting is $\theta^*$03, with softer protection in the range $\theta^*$04–$\theta^*$05 suggested for benign shifts (Zhou et al., 15 Jun 2026).

The principal limitation identified in the paper is the low-rank Fisher assumption. Although it is observed across FNO, CNO, DeepONet, and Transolver, not all operator architectures are reported to exhibit clear low-rank Fisher spectra. The cited negative case is DPOT-S, described as a foundation-scale operator with an almost uniform Fisher spectrum across attention heads and blocks. Under $\theta^*$06, the protected subspace in this case becomes too large, removing most gradient signal, and PhysGuard underperforms DFT and EWC on some DPOT-S tasks.

The scope of evaluation is another limitation. Experiments are restricted to fluid-mechanics scenarios in RealPDEBench, described as currently the only public benchmark pairing numerical and experimental data. Generalization to other PDE families, sensing modalities, and experimental setups is therefore left unresolved. The paper also notes that under extreme domain shifts, real-data gradients may lie mostly within the critical subspace, in which case strict projection can overly limit adaptation; reducing $\theta^*$07 or using adaptive $\theta^*$08 per layer is proposed as a mitigation.

Prospective extensions listed in the paper include adaptive $\theta^*$09 scheduling based on real-data validation feedback, layer-wise weighting for uneven Fisher distributions, combination with parameter-efficient adapters such as LoRA in the free subspace, and extension to foundation-scale operators through block-wise Fisher models or curvature-aware preconditioning. The accompanying codebase is available at https://github.com/ZhouChaunge/PhysGuard.

Taken together, PhysGuard is presented as a Fisher-guided gradient projection method for preserving physics-critical parameter directions learned from simulation while adapting to real measurements. Its reported contribution is to convert the preservation of low-frequency physical structure into an explicit subspace constraint in parameter space, with scalable layer-wise estimation and minimal online overhead, and to demonstrate consistent gains across multiple neural operator architectures and sim-to-real regimes (Zhou et al., 15 Jun 2026).