DeltaPhi: Residual Learning in Neural PDE Solvers

• DeltaPhi is a framework for PDE solving that predicts the residual between a pending trajectory and an auxiliary trajectory, redefining operator learning.
• The method transforms training through residual mapping on paired input-output fields, effectively mitigating overfitting and bias in low-data and biased distribution scenarios.
• Empirical results show up to 50% performance gains in irregular domain problems, underscoring its applicability in engineering and scientific computing.

DeltaPhi refers to the Physical Trajectory Residual Learning paradigm for data-driven partial differential equation (PDE) solving via neural operators. Unlike classical neural operator approaches which directly regress solution fields given input conditions, DeltaPhi formulates PDE surrogate modeling as a residual prediction task: the model learns to estimate the physical residual between a pending trajectory and a known, similar auxiliary trajectory. This design addresses shortcomings in model generalization when data is scarce, resolutions are low, or the underlying data distribution is biased. DeltaPhi operates by transforming both data and learning objective, leveraging similarity-based auxiliary inputs, and implementing residual mapping atop standard @@@@1@@@@ (Yue et al., 2024).

1. Conceptual Foundation: Physical Trajectory Residual Learning

DeltaPhi reinterprets operator learning for PDEs by focusing on residuals between solution trajectories. Instead of learning the mapping $\mathcal{G}_\theta:\mathcal{A}\to\mathcal{U}$ (with $\mathcal{A}$ input fields and $\mathcal{U}$ output fields), DeltaPhi constructs the residual operator mapping $$\mathcal{G}^{(\Delta)}:(\mathcal{A},\mathcal{A}')\to \mathcal{U}-\mathcal{U}'$$, where $\mathcal{A}'$ is an auxiliary input and $\mathcal{U}'$ its solution. This explicit difference learning mitigates overfitting and bias, ensuring the surrogate model captures physical dynamics more robustly. Training is performed on pairs $(a_i,a_{k_i})$ drawn from the dataset $\mathcal{T}$, with targets $u_i - u_{k_i}$. At inference, the predicted residual is summed with $u_{k_i}$ to recover $u_i$.

2. Residual Operator Formulation and Data Transformation

Let the training set $\mathcal{T}=\{(a_i, u_i)\}$ consist of input-output field pairs. DeltaPhi generates a residual training set $$\mathcal{T}^{(res)}=\{((a_i,a_{k_i}),u_i-u_{k_i})\}$$. The learning objective becomes

$$\ell = \sum_{i=1}^N L\left( \mathcal{G}^{(\Delta)}_\theta(a_i, a_{k_i}), u_i-u_{k_i} \right),$$

where $L(\cdot,\cdot)$ is an appropriate field-wise loss and $(a_{k_i}, u_{k_i})$ is the selected auxiliary sample (see Section 4). This methodology is compatible with various operator architectures: Fourier Neural Operator (FNO), NORM, FFNO, CFNO, Galerkin operators, Graph Neural Operators (GNOT), and others, requiring only input expansion and residual target specification.

3. Surrogate Model Architecture: Residual-Enhanced Neural Operators

The architecture in DeltaPhi preserves the backbone of conventional neural operator networks, with the critical adjustment of concatenating the auxiliary trajectory to the primary input. Specifically,

$v_0 = P(a_i \oplus a_{k_i}),$

where $\oplus$ denotes channel-wise concatenation and $P$ is the input projector. The output mapping is: $\hat{u}_i = Q(v_l) + u_{k_i},$ with $Q(\cdot)$ the output projector and $v_l$ the deepest latent representation. The surrogate thus learns to approximate the physical residual rather than the field itself. This implementation leverages the universal approximation properties of neural operators, while residual formulation improves label space coverage and reduces non-physical regression bias.

4. Auxiliary Input Selection and Physical Priors

A distinctive aspect of DeltaPhi is the non-parametric integration of physical priors via auxiliary trajectories. The selection of the auxiliary index $k_i$ for each $a_i$ is governed by a similarity retrieval function: $k_i = \mathcal{F}^{(ret)}(a_i, \mathcal{T}),$ commonly based on cosine similarity,

$$\text{Sim}(a_i, a_j) = 1 - \frac{a_i \cdot a_j}{\|a_i\| \|a_j\|}.$$

During training, $a_{k_i}$ is chosen randomly from the top $K$ most similar samples (with $K$ often set to 20), while at inference, the single most similar sample is used. The auxiliary inputs may be further customized, e.g., using only the latest time step for time-dependent tasks or including the full auxiliary solution $u_{k_i}$ and similarity scores directly. This enhances optimization by injecting physical context and guiding the surrogate towards plausible solution space.

5. Empirical Results: Generalization and Robustness

DeltaPhi exhibits marked improvements over direct operator learning across diverse PDE problem settings and model architectures:

• Irregular domain problems such as Irregular Darcy Flow, Pipe Turbulence, Heat Transfer, Composite, and Blood Flow see performance gains up to 40–50% relative over direct learning formulations.
• Regular domain benchmarks including Darcy Flow and Navier–Stokes demonstrate universally improved accuracy when residual mapping is incorporated, even on time-series problems where input–output similarity is weak.
• Low-data and resolution transfer regimes: DeltaPhi's advantages scale with reduced training sample size — it mitigates overfitting, enables zero-shot generalization from coarse to fine grids, and produces less biased target distributions (as shown by 2D projections).
• Label distribution analysis: DeltaPhi's residual targets are shown empirically to cover a more favorable and less biased distribution compared to the original solution fields, which enhances learnability and generalization.

6. Applications and Implications in Data-Driven Scientific Computing

DeltaPhi is particularly effective in engineering, scientific simulation, and biomedical contexts where PDE solvers must extrapolate beyond limited or biased training sets, handle resolution variation, and integrate external physical knowledge:

• Turbulent fluid dynamics (Navier–Stokes, pipe turbulence)
• Heat transfer in heterogeneous domains
• Physical simulation in composite or biological systems The framework is compatible with any setting providing auxiliary or historical trajectory data, and can be extended with advanced retrieval (e.g., latent-space search) or memory mechanisms. Its non-parametric approach to prior integration and residual-centric label space provides a template for further advances in machine learning for PDEs.

A plausible implication is that DeltaPhi's residual learning paradigm may inform developments in broader scientific machine learning, such as inverse problems or high-dimensional regression, where data efficiency and physical validity are paramount. The method's capacity for improved generalization amid sparse data and its systematic injection of physical priors set a precedent for operator learning frameworks in computational physics, engineering, and data-driven modeling (Yue et al., 2024).